The proper way to fix this is to disable FDS. We should not need yet another option to control debug info. Dan On 2/28/14 4:13 AM, David Holmes wrote:

Hi,

As I put in the bug report this seems way too complicated. Seems to me all you need to do to get what you want is not use FDS and not strip the symbols from the binary. The former is trivial. The latter is more awkward as the strip policy stuff does not work as I would want it to work, but still doable.

David

On 28/02/2014 7:18 PM, Yasumasa Suenaga wrote:

...

Hi all,

Currently, configure script can accept --disable-debug-symbols and --disable-zip-debug-info as controlling to generate debug information. However, current makefiles cannot build ELF binaries which is contained debug information with "images" target.

Some Linux distros use RPM as package manager. RPM is built by rpmbuild command, it strips symbols and debug information during to generate rpm packages. https://fedoraproject.org/wiki/Packaging:Debuginfo

For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not contain debug information. Actual debug information is shipped by OpenJDK debuginfo package. This packaging is important when we have to debug JVM/JNI libraries. If we want to use debugging tools (GDB, SystemTap, etc...), they may requires debuginfo package. Debuggee (e.g. libjvm.so) points libjvm.so.debug which is located at sub directory in /usr/lib/debug . It is defined in ELF section (.note.gnu.build-id) of libjvm.so . However libjvm.so.debug does not contain valid debug information. We need to access libjvm.debuginfo.debug at same location.

When we think to build OpenJDK rpm packages, we have to build ELF binaries which contain all debug information. we should not to separate debug information.

Thus I want to add a variable "SEPARATED_DEBUGINFO_FILES" . If we pass "SEPARATED_DEBUGINFO_FILES=false" to make command, "make" does not execute objcopy command for generating debuginfo binaries.

If we build rpm packages, we also have to add "STRIP_POLICY=no_strip" to arguments of make command. Or ELF binaries will be stripped.

I've uploaded webrev for this enhancement. This is separated 3-part: top of forest, hotspot, jdk http://cr.openjdk.java.net/~ysuenaga/JDK-8036003/

Please review it and sponsoring!

Thanks,

Yasumasa

P.S. I tried to add SEPARATED_DEBUGINFO_FILES as a configure option like "--disable-separated-debug-info" . I ran configure which is located at jdk9/dev directory after I ran autoconf and common/autoconf/autogen.sh. However it failed.

I guess this is caused by changeset as below. JDK-8035495: Improvements in autoconf integration http://hg.openjdk.java.net/jdk9/dev/rev/6e29cd9ac2b4

This changes add "CHECKME" option to configure script. However, this changes affects "configure" only. It should change "configure.ac" .

On 1/03/2014 1:38 AM, Yasumasa Suenaga wrote:

...

The proper way to fix this is to disable FDS.

Does this mean I have to pass --disable-debug-symbols to configure ? I've added comment to JDK-8036003, I think if we pass --disable-debug-symbols to configure, gcc/g++ is not passed -g option. "-g" is needed to generate debuginfo.

There are three pieces to all of this: 1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy) It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Also there may well be differences between how things are handled on the JDK side and hotspot side. I will try to look closer if I get a chance but my time is limited at the moment. David

Thanks,

Yasumasa

On 2014/02/28 23:04, Daniel D. Daugherty wrote:

...

The proper way to fix this is to disable FDS. We should not need yet another option to control debug info.

Dan

On 2/28/14 4:13 AM, David Holmes wrote:

...

Hi,

As I put in the bug report this seems way too complicated. Seems to me all you need to do to get what you want is not use FDS and not strip the symbols from the binary. The former is trivial. The latter is more awkward as the strip policy stuff does not work as I would want it to work, but still doable.

David

On 28/02/2014 7:18 PM, Yasumasa Suenaga wrote:

...

Hi all,

Currently, configure script can accept --disable-debug-symbols and --disable-zip-debug-info as controlling to generate debug information. However, current makefiles cannot build ELF binaries which is contained debug information with "images" target.

Some Linux distros use RPM as package manager. RPM is built by rpmbuild command, it strips symbols and debug information during to generate rpm packages. https://fedoraproject.org/wiki/Packaging:Debuginfo

For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not contain debug information. Actual debug information is shipped by OpenJDK debuginfo package. This packaging is important when we have to debug JVM/JNI libraries. If we want to use debugging tools (GDB, SystemTap, etc...), they may requires debuginfo package. Debuggee (e.g. libjvm.so) points libjvm.so.debug which is located at sub directory in /usr/lib/debug . It is defined in ELF section (.note.gnu.build-id) of libjvm.so . However libjvm.so.debug does not contain valid debug information. We need to access libjvm.debuginfo.debug at same location.

When we think to build OpenJDK rpm packages, we have to build ELF binaries which contain all debug information. we should not to separate debug information.

Thus I want to add a variable "SEPARATED_DEBUGINFO_FILES" . If we pass "SEPARATED_DEBUGINFO_FILES=false" to make command, "make" does not execute objcopy command for generating debuginfo binaries.

If we build rpm packages, we also have to add "STRIP_POLICY=no_strip" to arguments of make command. Or ELF binaries will be stripped.

I've uploaded webrev for this enhancement. This is separated 3-part: top of forest, hotspot, jdk http://cr.openjdk.java.net/~ysuenaga/JDK-8036003/

Please review it and sponsoring!

Thanks,

Yasumasa

P.S. I tried to add SEPARATED_DEBUGINFO_FILES as a configure option like "--disable-separated-debug-info" . I ran configure which is located at jdk9/dev directory after I ran autoconf and common/autoconf/autogen.sh. However it failed.

I guess this is caused by changeset as below. JDK-8035495: Improvements in autoconf integration http://hg.openjdk.java.net/jdk9/dev/rev/6e29cd9ac2b4

This changes add "CHECKME" option to configure script. However, this changes affects "configure" only. It should change "configure.ac" .

On Mar 1 2014, at 06:07 , Yasumasa Suenaga <yasu@ysfactory.dip.jp> wrote:

Hi David,

...

1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy)

It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward.

Currently, 1. depends 2. If FDS set to disable, debug symbols (-g) are not generated. SEPARATED_DEBUGINFO_FILES which my patch provides can separate them.

I think David's list (kudos) is very helpful as I haven't seen the requirements articulated this clearly anywhere else. Some comments: - Are there important tools that cannot deal with external debuginfo files? If we can put debuginfo into external files why would we ever want unstripped binaries? Is strip policy really necessary? Strip policy would seem to only be necessary if there are tools which can't handle external debuginfo. Assuming that everything can use external debuginfo then the policy would be to strip everything. Do I understand correctly that rpmbuild can only deal with unstripped binaries and generates the stripped rpm package and debuginfo package. It sounds kind of strange that find-debuginfo.sh wouldn't be able to use already generated debuginfo files. - I would expect that the flow is something like an extended version of https://blogs.oracle.com/dbx/entry/creating_separate_debug_info : 1. Compile source files with some form of "-g" 2. Create separate debug files for object files. 3. Strip object files. 4. Add gnu_debuglink to object files. 5. Link library or executable which should carry along links to debuginfo. 6a. Debugging, testing, profiling, etc. by developers. 6b. Packaging for program bits and debuginfo. 7. Testing and Use. Hopefully I didn't get any of those steps in the wrong order. What are the "you-cant-get-there-from-here" gaps and breakdowns from implementing this process? - Whether for the FDS debug symbols or RPM debuginfo packaging it seems that the default debug level isn't quite right. I believe that in both cases what's wanted is abbreviated debug info, mostly function names and line number tables, for building stack traces. This is not the debug info that is currently generated. There are four basic alternatives for levels of debug info: 1. (-g0) No debug info whatsoever. Only exported functions and globals will have names. 2. (-g1) Abbreviated debug info. All functions have names and line number information. This seems to be what is needed by both FDS and RPM debuginfo files to produce nice stack traces. 3. (-g) Default debugging info. Adds macro expansions and local variable names. Suitable for source level debugging. 4. (-g3 or -gdwarf-4 -fvar-tracking-assignments). Full debugging info. Most suitable for source level debugging including debugging of optimized code. It is not clear that anyone would want to build the entire product with this option. Compilations are currently done with a mix of -g, -g1, and -gstabs. It would be good to rationalize this somewhat and provide a mechanism for developers to tweak generated debugging output for files or components. - Eventual alignment with http://gcc.gnu.org/wiki/DebugFission on some platforms?

So I change:

1. Separating to add "-g" option to compiler from executing objcopy. 2. jdk/make/Images.gmk regards STRIP_POLICY.

As I mentioned earlier, this issue is related to RPM. So I hope to fix it before JDK8 GA is released.

This won't happen (at least not for Java 8u0). Java 8 is already late in it's final candidate stage. It is too late for the initial Java 8 release to consider this magnitude of change. In any event, since the Java 8 rampdown began back in November, any change would first have to be applied to Java 9 and then approved for backport to a Java 8 or an update release (and it is also possibly too late for 8u20). Inability to include your suggested change in Java 8 or 8u20 is in no way a rejection of the ideas or contribution, it's merely a normal consequence of timelines and process. Mike

On 03/01/2014 11:08 PM, Mike Duigou wrote:

Do I understand correctly that rpmbuild can only deal with unstripped binaries and generates the stripped rpm package and debuginfo package.

Exactly. OpenJDK generating separate debuginfo files is very inconvenient for RPM builds.

It sounds kind of strange that find-debuginfo.sh wouldn't be able to use already generated debuginfo files.

The OpenJDK build can't have any idea of what debuginfo files the distro runtime requires, and indeed this may change. There is more than one mechanism for finding debuginfo. Separating debuginfo is not something that we want the OpenJDK build to do, and the most useful thing that we could have is a switch to turn all of OpenJDK's stripping and separate debuginfo off. I expect the same applies to all distros. Andrew.

I think we're having a problem with not all replies making it to all the aliases. Yasumasa's reply to David below that Mike is replying to did not arrive on any of the aliases that I'm on... Folks need to remember to reply to all of the aliases... On 3/1/14 4:08 PM, Mike Duigou wrote:

On Mar 1 2014, at 06:07 , Yasumasa Suenaga <yasu@ysfactory.dip.jp> wrote:

...

Hi David,

...

1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy)

It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Currently, 1. depends 2. If FDS set to disable, debug symbols (-g) are not generated.

Bit of history here. When FDS is disabled, the build system is supposed to go back to what was there prior to the FDS project. On Linux, one of the debug configs didn't actually have any debugging flags enabled (debug or fastdebug, I don't remember which). So when I implemented FDS, disabling FDS meant no debug info. Bug in the original? You betcha.

...

SEPARATED_DEBUGINFO_FILES which my patch provides can separate them. I think David's list (kudos) is very helpful as I haven't seen the requirements articulated this clearly anywhere else.

Some comments:

- Are there important tools that cannot deal with external debuginfo files?

Not that I'm aware of. There have been bugs in the way gobjcopy is extracting the debuginfo into the separate files, but no complete failures as far as I know...

If we can put debuginfo into external files why would we ever want unstripped binaries?

We deliver minimal stripped binaries so that can have stack traces with some amount of information in them when the debuginfo files are not available. We have not yet figured out how to implement minimal stripping on Windows so Windows hs_err_pid files have no symbols in them when the debuginfo bundles are not present.

Is strip policy really necessary?

Yes. STRIP_POLICY allows folks to configure what they want for the build that they are doing. It is also meant to map to the Oracle policy where we were granted a waiver to deliver minimal symbols in FDS project.

Strip policy would seem to only be necessary if there are tools which can't handle external debuginfo. Assuming that everything can use external debuginfo then the policy would be to strip everything.

No you don't want to strip everything. See above aboutminimal info in stack traces.

Do I understand correctly that rpmbuild can only deal with unstripped binaries and generates the stripped rpm package and debuginfo package. It sounds kind of strange that find-debuginfo.sh wouldn't be able to use already generated debuginfo files.

- I would expect that the flow is something like an extended version of https://blogs.oracle.com/dbx/entry/creating_separate_debug_info : 1. Compile source files with some form of "-g" 2. Create separate debug files for object files. 3. Strip object files. 4. Add gnu_debuglink to object files. 5. Link library or executable which should carry along links to debuginfo. 6a. Debugging, testing, profiling, etc. by developers. 6b. Packaging for program bits and debuginfo. 7. Testing and Use.

Hopefully I didn't get any of those steps in the wrong order. What are the "you-cant-get-there-from-here" gaps and breakdowns from implementing this process?

- Whether for the FDS debug symbols or RPM debuginfo packaging it seems that the default debug level isn't quite right. I believe that in both cases what's wanted is abbreviated debug info, mostly function names and line number tables, for building stack traces. This is not the debug info that is currently generated.

It's STRIP_POLICY=min_strip that leaves the minimal debug info attached to the binaries and the more complete debug info in left in the debuginfo file(s). And I'll disagree about only wanting abbreviated debug info in the separate debuginfo files.

There are four basic alternatives for levels of debug info: 1. (-g0) No debug info whatsoever. Only exported functions and globals will have names. 2. (-g1) Abbreviated debug info. All functions have names and line number information. This seems to be what is needed by both FDS and RPM debuginfo files to produce nice stack traces. 3. (-g) Default debugging info. Adds macro expansions and local variable names. Suitable for source level debugging. 4. (-g3 or -gdwarf-4 -fvar-tracking-assignments). Full debugging info. Most suitable for source level debugging including debugging of optimized code. It is not clear that anyone would want to build the entire product with this option.

Compilations are currently done with a mix of -g, -g1, and -gstabs. It would be good to rationalize this somewhat and provide a mechanism for developers to tweak generated debugging output for files or components.

There is a whole boatload of history behind using STABS or DWARF or -g1 and usually (at least in HotSpot) there are comments in the Makefiles explaining the sometimes arcane reasons why these choices were made... On Linux there is a special option DEBUG_BINARIES that is used to specify '-g' for everything. I believe the magic incantation for making RPM happy is something like: DEBUG_BINARIES=true FULL_DEBUG_SYMBOLS=0 STRIP_POLICY=none ALT_OBJCOPY=/does_not_exist The combination of FULL_DEBUG_SYMBOLS=0 and ALT_OBJCOPY=/does_not_exist disables FDS for all build configs and reverts to pre-FDS make logic. STRIP_POLICY=none says don't do any stripping. DEBUG_BINARIES=true says ignore all the other logic about which debug options and just do '-g'. I've accidentally broken DEBUG_BINARIES at least once, but I usually get pinged by the Red Hat folks when I do... Dan

- Eventual alignment with http://gcc.gnu.org/wiki/DebugFission on some platforms?

...

So I change:

1. Separating to add "-g" option to compiler from executing objcopy. 2. jdk/make/Images.gmk regards STRIP_POLICY.

As I mentioned earlier, this issue is related to RPM. So I hope to fix it before JDK8 GA is released. This won't happen (at least not for Java 8u0). Java 8 is already late in it's final candidate stage. It is too late for the initial Java 8 release to consider this magnitude of change. In any event, since the Java 8 rampdown began back in November, any change would first have to be applied to Java 9 and then approved for backport to a Java 8 or an update release (and it is also possibly too late for 8u20).

Inability to include your suggested change in Java 8 or 8u20 is in no way a rejection of the ideas or contribution, it's merely a normal consequence of timelines and process.

Mike

On 3/3/14 12:11 PM, Omair Majid wrote:

Hi,

* Daniel D. Daugherty <daniel.daugherty@oracle.com> [2014-03-03 09:57]:

...

On 3/1/14 4:08 PM, Mike Duigou wrote:

...

If we can put debuginfo into external files why would we ever want unstripped binaries? We deliver minimal stripped binaries so that can have stack traces with some amount of information in them when the debuginfo files are not available. We have not yet figured out how to implement minimal stripping on Windows so Windows hs_err_pid files have no symbols in them when the debuginfo bundles are not present. Am I reading it right that when tools (like RPM) strip the debug information from these binaries, hotspot will break completely when it tries to generate a stack trace on crash?

Not quite. If you follow this process: - use gobjcopy to copy debug info from libjvm.so -> libjvm.debuginfo - use gobjcopy to tell libjvm.so that debug info is found in libjvm.debuginfo - strip libjvm.so (I don't remember the strip everything option) then you can still get symbols in your crashes as long as you have BOTH libjvm.so and libjvm.debuginfo. However, if you are in a situation where you don't have libjvm.debuginfo, then you do not get symbols in your crashes. When I did the Full Debug Symbols (FDS) project, I copied complete debug info from libjvm.so -> libjvm.debuginfo and left minimal debug info in libjvm.so so that you could still get symbols on crashes on Linux and Solaris when the libjvm.debuginfo files were not present. Dan

Thanks, Omair

Hi David,

Moving forward I think we would need to expose the strip-policy via a configure option and have that pass STRIP_POLICY through to both hotspot and the Images target.

I want you to do so :-) Additionally,

...

...

1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy)

I think that FDS should be separated 1 from 2. That is more useful for building OpenJDK.

That said, as per the above I think you can achieve what you want by enabling FDS but set STRIP_POLICY to none, and set POST_STRIP_CMD to empty.

I tried: $ make images STRIP_POLICY=no_strip POST_STRIP_CMD="" I was able to get binaries which is included debug information. Thanks, Yasumasa On 2014/03/03 13:39, David Holmes wrote:

Hi,

On 2/03/2014 12:07 AM, Yasumasa Suenaga wrote:

...

Hi David,

...

1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy)

It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward.

Currently, 1. depends 2. If FDS set to disable, debug symbols (-g) are not generated.

Okay. That would seem potentially inconvenient but not really critical. You simply enable FDS (the mere existence of the debuginfo files "shouldn't" cause a problem). Then you only need to address #3 - strip-policy.

...

SEPARATED_DEBUGINFO_FILES which my patch provides can separate them.

...

Also there may well be differences between how things are handled on the JDK side and hotspot side.

libjvm, libjsig. libsaproc are built with each makefiles in hotspot/make . Other native library is built with make/common/NativeCompilation.gmk .

Additionally strip command is executed in "image" target which is defined in jdk/make/Images.gmk . This "strip" ignores STRIP_POLICY . Thus ELF binaries which are contained in JDK/JRE images are stripped when we execute "images" or "all" target.

strip is also applied to the JVM libraries as part of the hotspot build. What occurs in the JDK "images" is additional to that.The hotspot build honors the STRIP_POLICY.

It appears that STRIP_POLICY is not considered in the new build, though it was for the old. But perhaps I'm missing it somewhere in the autoconf settings? That aside it seems that you should be able to set POST_STRIP_CMD to "" avoid stripping.

Moving forward I think we would need to expose the strip-policy via a configure option and have that pass STRIP_POLICY through to both hotspot and the Images target.

...

So I change:

1. Separating to add "-g" option to compiler from executing objcopy. 2. jdk/make/Images.gmk regards STRIP_POLICY.

As I mentioned earlier, this issue is related to RPM. So I hope to fix it before JDK8 GA is released.

As Mike said there is no chance of any changes to this getting into 8 GA. But if these are your own builds I'm not sure that is an issue anyway. That said, as per the above I think you can achieve what you want by enabling FDS but set STRIP_POLICY to none, and set POST_STRIP_CMD to empty.

Thanks, David

...

Thanks,

Yasumasa

On 2014/03/01 8:47, David Holmes wrote:

...

On 1/03/2014 1:38 AM, Yasumasa Suenaga wrote:

...

...

The proper way to fix this is to disable FDS.

Does this mean I have to pass --disable-debug-symbols to configure ? I've added comment to JDK-8036003, I think if we pass --disable-debug-symbols to configure, gcc/g++ is not passed -g option. "-g" is needed to generate debuginfo.

There are three pieces to all of this:

1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy)

It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward.

Also there may well be differences between how things are handled on the JDK side and hotspot side.

I will try to look closer if I get a chance but my time is limited at the moment.

David

...

Thanks,

Yasumasa

On 2014/02/28 23:04, Daniel D. Daugherty wrote:

...

The proper way to fix this is to disable FDS. We should not need yet another option to control debug info.

Dan

On 2/28/14 4:13 AM, David Holmes wrote:

...

Hi,

As I put in the bug report this seems way too complicated. Seems to me all you need to do to get what you want is not use FDS and not strip the symbols from the binary. The former is trivial. The latter is more awkward as the strip policy stuff does not work as I would want it to work, but still doable.

David

On 28/02/2014 7:18 PM, Yasumasa Suenaga wrote: > Hi all, > > > Currently, configure script can accept --disable-debug-symbols and > --disable-zip-debug-info as controlling to generate debug information. > However, current makefiles cannot build ELF binaries which is contained > debug information with "images" target. > > Some Linux distros use RPM as package manager. > RPM is built by rpmbuild command, it strips symbols and debug information > during to generate rpm packages. > https://fedoraproject.org/wiki/Packaging:Debuginfo > > For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . > libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not > contain debug information. Actual debug information is shipped by OpenJDK > debuginfo package. > This packaging is important when we have to debug JVM/JNI libraries. > If we want to use debugging tools (GDB, SystemTap, etc...), they may requires > debuginfo package. Debuggee (e.g. libjvm.so) points libjvm.so.debug which is > located at sub directory in /usr/lib/debug . It is defined in ELF section > (.note.gnu.build-id) of libjvm.so . However libjvm.so.debug does not contain > valid debug information. We need to access libjvm.debuginfo.debug at same location. > > > When we think to build OpenJDK rpm packages, we have to build ELF binaries > which contain all debug information. we should not to separate debug information. > > > Thus I want to add a variable "SEPARATED_DEBUGINFO_FILES" . > If we pass "SEPARATED_DEBUGINFO_FILES=false" to make command, "make" does not > execute objcopy command for generating debuginfo binaries. > > If we build rpm packages, we also have to add "STRIP_POLICY=no_strip" to arguments > of make command. Or ELF binaries will be stripped. > > > I've uploaded webrev for this enhancement. > This is separated 3-part: top of forest, hotspot, jdk > http://cr.openjdk.java.net/~ysuenaga/JDK-8036003/ > > Please review it and sponsoring! > > > Thanks, > > Yasumasa > > > P.S. > I tried to add SEPARATED_DEBUGINFO_FILES as a configure option like > "--disable-separated-debug-info" . > I ran configure which is located at jdk9/dev directory after I ran autoconf > and common/autoconf/autogen.sh. However it failed. > > I guess this is caused by changeset as below. > JDK-8035495: Improvements in autoconf integration > http://hg.openjdk.java.net/jdk9/dev/rev/6e29cd9ac2b4 > > This changes add "CHECKME" option to configure script. However, this changes > affects "configure" only. It should change "configure.ac" . >

On 3/3/14 2:49 PM, Omair Majid wrote:

* David Holmes <david.holmes@oracle.com> [2014-02-28 18:48]:

...

There are three pieces to all of this:

1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy)

It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Any chance we could clean up the names too? It's not obvious to me why FDS means 'generating debuginfo files'.

FDS stands for Full Debug Symbols and is defined that way in quite a few Makefiles... We just call it FDS for short...

What do folks think of modifying the OPENJDK build to use this as the settings for the options above by default.

1. Yes 2. No 3. No

Whatever we do here has be architecturallyconsistent across the various OpenJDK platforms. The above settings would not work well on non-Linux platforms so I'm not in favor of doing that. However, I don't have a problem with providing support for config'ing a build to work with the downstream Linux repos...

...

Also there may well be differences between how things are handled on the JDK side and hotspot side. Is there any chance of this becoming consistent between hotspot and JDK? It would be much nicer if concepts like strip-policy were consistent between hotspot and the rest of the JDK.

When I did the Full Debug Symbols (FDS) project, I made it architecturally consistent between the HotSpot and JDK sides. Since the new build system has arrived, something of the consistency has been broken and I haven't been back on the JDK side to see what happened... Dan

Thanks, Omair

On 2014-03-18 02:19, Andrew Hughes wrote:

Do we need more than just the following three alternatives?

#1. No debugging information at all. #2. Debugging information left in the original binaries. #3. Debugging information stripped from the binaries and zipped in separate files.

It sounds to me like Oracle want #3, while distros want #2 and I imagine some end users may just want #1 for a faster, smaller build.

I'm completely thinking along your lines here. I think we should focus on the end result, not the specific implementations details. I've been thinking about exposing a configure option like this: --with-debug-symbols=none (for #1) --with-debug-symbols=internal (for #2) --with-debug-symbols=zipped (for #3) but perhaps with better names. I also imagine there might be a use case for yet another option, #4. Debuggin information stripped from the binaries, but not zipped expressed like this --with-debug-symbols=external And then this would be the only user interface needed, and the rest of the current set of not-really-clear flags can be hidden inside the configure script. For compatibility reasons, the default would be --with-debug-symbols=zipped (which is called "FDS" here), just as we currently do, but distributions would just need to add a --with-debug-symbols=internal to get what they want. /Magnus

On 18/03/2014 6:59 PM, Magnus Ihse Bursie wrote:

On 2014-03-18 02:19, Andrew Hughes wrote:

...

Do we need more than just the following three alternatives?

#1. No debugging information at all. #2. Debugging information left in the original binaries. #3. Debugging information stripped from the binaries and zipped in separate files.

It sounds to me like Oracle want #3, while distros want #2 and I imagine some end users may just want #1 for a faster, smaller build.

I'm completely thinking along your lines here. I think we should focus on the end result, not the specific implementations details. I've been thinking about exposing a configure option like this:

--with-debug-symbols=none (for #1) --with-debug-symbols=internal (for #2) --with-debug-symbols=zipped (for #3)

I don't think this quite works as there are other variations not captured here. Rather than "zipped" it should just be "external". Whether the debuginfo files are zipped or not is then an additional build time option. Additionally we still have to be able to control the degree of stripping that is carried out. It doesn't make sense to me to try and enumerate all possible combinations as top-level configure options. Further, as Dan was saying, this doesn't capture the overlap between "internal" and "external" because we still leave some symbols internal even when creating the debuginfo file. So I don't think this proposed categorization works. We still have three aspects: - generating symbols into the object files - stripping symbols from the final binaries - saving symbols into an external form Each of which can potentally be varied (of course if you don't generate symbols in the obj files stripping and saving are non issues). David -----

but perhaps with better names. I also imagine there might be a use case for yet another option, #4. Debuggin information stripped from the binaries, but not zipped

expressed like this --with-debug-symbols=external

And then this would be the only user interface needed, and the rest of the current set of not-really-clear flags can be hidden inside the configure script.

For compatibility reasons, the default would be --with-debug-symbols=zipped (which is called "FDS" here), just as we currently do, but distributions would just need to add a --with-debug-symbols=internal to get what they want.

/Magnus

On 21/03/2014 6:14 PM, Magnus Ihse Bursie wrote:

...

I don't think this quite works as there are other variations not captured here. Rather than "zipped" it should just be "external". Whether the debuginfo files are zipped or not is then an additional build time option. Additionally we still have to be able to control the degree of stripping that is carried out. It doesn't make sense to me to try and enumerate all possible combinations as top-level configure options.

Further, as Dan was saying, this doesn't capture the overlap between "internal" and "external" because we still leave some symbols internal even when creating the debuginfo file.

So I don't think this proposed categorization works. We still have three aspects: - generating symbols into the object files - stripping symbols from the final binaries - saving symbols into an external form

Each of which can potentally be varied (of course if you don't generate symbols in the obj files stripping and saving are non issues).

But they are not independent on each other!

Unless you generate debug symbols, you can't strip them, nor save them elsewhere. So generating debug symbols (your item #1) is a prerequisite for the rest of your items.

Yes I just said that. :)

And while, technically, you can save symbols externally, and at the same time keep them in the binary, that does not make sense. So, in a practical sense, you are going to do #2 if you are going to do #3.

But you can vary what is kept internally independent of what is made external.

And you can't zip external symbols unless you create a non-zipped version. And if you zip them, it does not make sense to keep the non-zipped version.

zip vs non-zip is just an issue of disk space. It is not a fundamental configuration choice, just a variation on how external symbols are packaged.

And yes, we do not strip all debug information when creating external debug info. But there seems to be no real use case (not even for IcedTea, as it turned out) to want a completely stripped binary, I don't think that's worth discussing much. That's just part of how the external debuginfo scheme works.

Umm we fully strip all our binaries in embedded.

Can you give a more concrete example of the variations of your "aspects" that you think my suggested solution would not capture?

I think I already have. There aren't tree simple choices here, there are three aspects, each of which could have different variants. If I could draw a flow chart easily I would but basically if you generate debug symbols you can then select what symbols are kept internally (the strip policy) and what are put externally (objcopy options), then for the external symbol file you can choose zipped or unzipped. Multiple inter-connected choices, not just three (or four if you add zipped) David

/Magnus

On 21/03/2014 7:36 PM, Dmitry Samersoff wrote:

David,

In practice, we don't have to much to keep internally.

There are no reason to copy out some of .debug_* sections but keep other ones.

I'm not familiar with exactly what the different strip options do but the point is this is covered by the strip-policy.

So we have a matrix:

(a) Strip mode:

1. full 2. keep symbols 3. keep symbols and .debug_*

(b) Copy mode:

1. Copy all to ext file 2. Copy none to ext file

(c) Compression mode:

1. none 2. per-section compression, SHF_GNU_COMPRESSED [1] 3. zip entire file

Where *a2 + b2 + c2* have perfect sense,

So now your compression mode may apply to either the external file or the original? That's even more permutations.

*a1 + b1 + c3* have no sense etc.

Why does full strip + copy-all + zip make no sense? It is exactly what we do with embedded builds. (Naturally you have to copy before you strip) David -----

-Dmitry

Refs: 1. https://sourceware.org/bugzilla/show_bug.cgi?id=11819 http://gcc.gnu.org/ml/gcc-patches/2013-04/msg01780.html

-Dmitry

On 2014-03-21 12:57, David Holmes wrote:

...

On 21/03/2014 6:14 PM, Magnus Ihse Bursie wrote:

...

...

I don't think this quite works as there are other variations not captured here. Rather than "zipped" it should just be "external". Whether the debuginfo files are zipped or not is then an additional build time option. Additionally we still have to be able to control the degree of stripping that is carried out. It doesn't make sense to me to try and enumerate all possible combinations as top-level configure options.

Further, as Dan was saying, this doesn't capture the overlap between "internal" and "external" because we still leave some symbols internal even when creating the debuginfo file.

So I don't think this proposed categorization works. We still have three aspects: - generating symbols into the object files - stripping symbols from the final binaries - saving symbols into an external form

Each of which can potentally be varied (of course if you don't generate symbols in the obj files stripping and saving are non issues).

But they are not independent on each other!

Unless you generate debug symbols, you can't strip them, nor save them elsewhere. So generating debug symbols (your item #1) is a prerequisite for the rest of your items.

Yes I just said that. :)

...

And while, technically, you can save symbols externally, and at the same time keep them in the binary, that does not make sense. So, in a practical sense, you are going to do #2 if you are going to do #3.

But you can vary what is kept internally independent of what is made external.

...

And you can't zip external symbols unless you create a non-zipped version. And if you zip them, it does not make sense to keep the non-zipped version.

zip vs non-zip is just an issue of disk space. It is not a fundamental configuration choice, just a variation on how external symbols are packaged.

...

And yes, we do not strip all debug information when creating external debug info. But there seems to be no real use case (not even for IcedTea, as it turned out) to want a completely stripped binary, I don't think that's worth discussing much. That's just part of how the external debuginfo scheme works.

Umm we fully strip all our binaries in embedded.

...

Can you give a more concrete example of the variations of your "aspects" that you think my suggested solution would not capture?

I think I already have. There aren't tree simple choices here, there are three aspects, each of which could have different variants.

If I could draw a flow chart easily I would but basically if you generate debug symbols you can then select what symbols are kept internally (the strip policy) and what are put externally (objcopy options), then for the external symbol file you can choose zipped or unzipped.

Multiple inter-connected choices, not just three (or four if you add zipped)

David

...

/Magnus

On 2014-03-21 10:36, Dmitry Samersoff wrote:

(c) Compression mode:

1. none 2. per-section compression, SHF_GNU_COMPRESSED [1] 3. zip entire file

Is 2 something we're doing? I couldn't find any references to it in the code. Or is it something we're planning to do? /Magnus

On 2014-03-21 09:57, David Holmes wrote:

...

And while, technically, you can save symbols externally, and at the same time keep them in the binary, that does not make sense. So, in a practical sense, you are going to do #2 if you are going to do #3.

But you can vary what is kept internally independent of what is made external.

While that technically is correct, not all combinations does make sense.

...

And you can't zip external symbols unless you create a non-zipped version. And if you zip them, it does not make sense to keep the non-zipped version.

zip vs non-zip is just an issue of disk space. It is not a fundamental configuration choice, just a variation on how external symbols are packaged.

So can we agree that for external symbols, there are three ways to deal with them: no external symbols unzipped external symbols zipped external symbols ?

...

Can you give a more concrete example of the variations of your "aspects" that you think my suggested solution would not capture?

I think I already have. There aren't tree simple choices here, there are three aspects, each of which could have different variants.

If I could draw a flow chart easily I would but basically if you generate debug symbols you can then select what symbols are kept internally (the strip policy) and what are put externally (objcopy options), then for the external symbol file you can choose zipped or unzipped.

Multiple inter-connected choices, not just three (or four if you add zipped)

And then there's the aspect that got this bug started, that you need to compile with -g to be able to have any useful symbols at all. While technically you can compile without -g and then run strip, it's a combination that does not make sense. In the end, it all boils down to a few sensible ways to handle all debug information. 1) The Oracle way -- copying debug information into an external format (that by default is zipped), and stripping the binary as much as reasonable. 2) The community way -- keep everything in the binary, and not creating a separate external format. 3) The hardcore way -- ignore debug information alltogether. Instead of providing a lots of handles and knobs to configure, that will allow the user to configure invalid (e.g. zip-external=true, produce-external=false), or meaningless (compile-with-debug-flag=yes, strip=everything, produce-external=false), I'd rather provide a high-level option (or two, if it's not possible to condense into a single option) that allows the user to express the intent. And for all practical purposes, it's going to be set to either "oracle mode" or "community mode". /Magnus

----- Original Message -----

On 3/17/14 7:19 PM, Andrew Hughes wrote:

...

----- Original Message -----

...

On 3/3/14 2:49 PM, Omair Majid wrote:

...

* David Holmes <david.holmes@oracle.com> [2014-02-28 18:48]:

...

There are three pieces to all of this:

1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy)

It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Any chance we could clean up the names too? It's not obvious to me why FDS means 'generating debuginfo files'. FDS stands for Full Debug Symbols and is defined that way in quite a few Makefiles... We just call it FDS for short...

At least to me, Full Debug Symbols suggests #1 (i.e. that symbols are generated in the binaries), not #2. That's why it sounded so odd to me when you suggested turning it off, when we discussed this before.

It's also not clear why you'd want a situation where #3 would be turned off, but not #2, as you end up with two copies of the debug symbols. That's the problem I believe we have with our builds; we can turn the stripping off, but then we end up with duplicate debug information.

Do we need more than just the following three alternatives?

#1. No debugging information at all. #2. Debugging information left in the original binaries. #3. Debugging information stripped from the binaries and zipped in separate files.

The way Oracle does it is complete debug info in the separate files and partial debug info left in the original binaries so you get symbol names in stack traces for those cases where the full debug info bundles are not available.

I'm not clear whether we need a 4th alternative or if #3 covers this case.

The intent was for #3 to cover this case (i.e. whatever Oracle does now) and for #2 to be what the GNU/Linux distributions want (i.e. binaries with all debuginfo generated and left intact so they can do their own stripping).

Dan

...

It sounds to me like Oracle want #3, while distros want #2 and I imagine some end users may just want #1 for a faster, smaller build.

-- Andrew :) Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) PGP Key: 248BDC07 (https://keys.indymedia.org/) Fingerprint = EC5A 1F5E C0AD 1D15 8F1F 8F91 3B96 A578 248B DC07

On 2014-03-18 19:25, Andrew Haley wrote:

On 03/18/2014 06:22 PM, Andrew Hughes wrote:

...

The intent was for #3 to cover this case (i.e. whatever Oracle does now) and for #2 to be what the GNU/Linux distributions want (i.e. binaries with all debuginfo generated and left intact so they can do their own stripping). Mmm, but maybe that will break things when debuginfo isn't installed. In fact, I already know that it has broken some things. So it's not ideal.

Which case is it that will break? #3 or #2? What would an ideal solution be for you? /Magnus

----- Original Message -----

On 03/19/2014 01:51 PM, Magnus Ihse Bursie wrote:

...

On 2014-03-18 19:25, Andrew Haley wrote:

...

On 03/18/2014 06:22 PM, Andrew Hughes wrote:

...

The intent was for #3 to cover this case (i.e. whatever Oracle does now) and for #2 to be what the GNU/Linux distributions want (i.e. binaries with all debuginfo generated and left intact so they can do their own stripping). Mmm, but maybe that will break things when debuginfo isn't installed. In fact, I already know that it has broken some things. So it's not ideal.

Which case is it that will break? #3 or #2? What would an ideal solution be for you?

I think the problem was that jstack and jmap didn't work until the debuginfo package was installed. This tells me that the debuginfo stripping in our Linux build system is too aggressive for this purpose.

Yes, https://bugzilla.redhat.com/show_bug.cgi?id=1010786 The distro builds strip too much and remove the needed symtab from libjvm.so.

I think that's something we must fix ourselves. What we really need from OpenJDK is a way to build with complete debuginfo in both binaries and jarfiles.

This was my intent with #2. The jstack/jmap issue needs to be fixed by stripping less debuginfo post-build on libjvm.so.

Andrew.

-- Andrew :) Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) PGP Key: 248BDC07 (https://keys.indymedia.org/) Fingerprint = EC5A 1F5E C0AD 1D15 8F1F 8F91 3B96 A578 248B DC07

On 3/18/14 12:22 PM, Andrew Hughes wrote:

----- Original Message -----

...

On 3/17/14 7:19 PM, Andrew Hughes wrote:

...

----- Original Message -----

...

On 3/3/14 2:49 PM, Omair Majid wrote:

...

* David Holmes <david.holmes@oracle.com> [2014-02-28 18:48]:

...

There are three pieces to all of this:

1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy)

It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Any chance we could clean up the names too? It's not obvious to me why FDS means 'generating debuginfo files'. FDS stands for Full Debug Symbols and is defined that way in quite a few Makefiles... We just call it FDS for short...

At least to me, Full Debug Symbols suggests #1 (i.e. that symbols are generated in the binaries), not #2. That's why it sounded so odd to me when you suggested turning it off, when we discussed this before.

It's also not clear why you'd want a situation where #3 would be turned off, but not #2, as you end up with two copies of the debug symbols. That's the problem I believe we have with our builds; we can turn the stripping off, but then we end up with duplicate debug information.

Do we need more than just the following three alternatives?

#1. No debugging information at all. #2. Debugging information left in the original binaries. #3. Debugging information stripped from the binaries and zipped in separate files. The way Oracle does it is complete debug info in the separate files and partial debug info left in the original binaries so you get symbol names in stack traces for those cases where the full debug info bundles are not available.

I'm not clear whether we need a 4th alternative or if #3 covers this case.

The intent was for #3 to cover this case (i.e. whatever Oracle does now) and for #2 to be what the GNU/Linux distributions want (i.e. binaries with all debuginfo generated and left intact so they can do their own stripping).

Very cool. Thanks for the clarification. Dan

...

Dan

...

It sounds to me like Oracle want #3, while distros want #2 and I imagine some end users may just want #1 for a faster, smaller build.

* Andrew Haley <aph@redhat.com> [2014-03-03 04:43]:

On 02/28/2014 09:18 AM, Yasumasa Suenaga wrote:

...

For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not contain debug information. Actual debug information is shipped by OpenJDK debuginfo package.

That's a bug in Fedora's build. We should fix it.

This should fix it: http://pkgs.fedoraproject.org/cgit/java-1.8.0-openjdk.git/commit/?id=1734315... Thanks, Omair -- PGP Key: 66484681 (http://pgp.mit.edu/) Fingerprint = F072 555B 0A17 3957 4E95 0056 F286 F14F 6648 4681

On 5/04/2014 12:55 AM, Yasumasa Suenaga wrote:

Hi all,

...

This should fix it: http://pkgs.fedoraproject.org/cgit/java-1.8.0-openjdk.git/commit/?id=1734315...

I updated OpenJDK8 to java-1.8.0-openjdk-1.8.0.0-0.34.b132.fc20 . However, debuginfo files are not contained ELF sections for debugging. (I checked libjvm.so.debug and libnio.so.debug with "readelf -S")

According to SPEC file of OpenJDK8, following options are passed to "make": ----------------- make \ SCTP_WERROR= \ DEBUG_BINARIES=true \ FULL_DEBUG_SYMBOLS=0 \ STRIP_POLICY=none \ ALT_OBJCOPY=/does_not_exist \ LOG=trace \ all -----------------

I ran "grep" with DEBUG_BINARIES in jdk makefiles, however I could not find it. At least, DEBUG_BINARIES does not affect to jdk sources, and also does not affect to hotspot sources.

DEBUG_BINARIES is used in the hotspot makefiles. ./linux/makefiles/gcc.make ifeq ($(DEBUG_BINARIES), true) CFLAGS += -g else David -----

I've succeeded to make binaries which are contained debuginfo as following:

http://mail.openjdk.java.net/pipermail/build-dev/2014-March/012037.html $ make images STRIP_POLICY=no_strip POST_STRIP_CMD=""

I guess that we should run "make" above options to avoid this issue in currently.

Thanks,

Yasumasa

On 03/04/2014 04:01 AM, Omair Majid wrote:

...

* Andrew Haley <aph@redhat.com> [2014-03-03 04:43]:

...

On 02/28/2014 09:18 AM, Yasumasa Suenaga wrote:

...

For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not contain debug information. Actual debug information is shipped by OpenJDK debuginfo package. That's a bug in Fedora's build. We should fix it. This should fix it: http://pkgs.fedoraproject.org/cgit/java-1.8.0-openjdk.git/commit/?id=1734315...

Thanks, Omair