JDK-8036003: Add variable not to separate debug information.

[Daniel D. Daugherty]

I think we're having a problem with not all replies making it
to all the aliases. Yasumasa's reply to David below that Mike
is replying to did not arrive on any of the aliases that I'm
on... Folks need to remember to reply to all of the aliases...


On 3/1/14 4:08 PM, Mike Duigou wrote:
> On Mar 1 2014, at 06:07 , Yasumasa Suenaga <yasu at ysfactory.dip.jp> wrote:
>
>> Hi David,
>>
>>> 1. Generating debug symbols in the binaries (via gcc -g or whatever)
>>> 2. Generating debuginfo files (zipped or not) (FDS)
>>> 3. Stripping debug symbols from the binaries (strip-policy)
>>>
>>> It may be that we don't have clean separation between them, and if so
>>> that should be fixed, but I don't see the current proposal as the way
>>> forward.
>> Currently, 1. depends 2. If FDS set to disable, debug symbols (-g) are not
>> generated.

Bit of history here. When FDS is disabled, the build system is
supposed to go back to what was there prior to the FDS project.
On Linux, one of the debug configs didn't actually have any
debugging flags enabled (debug or fastdebug, I don't remember
which). So when I implemented FDS, disabling FDS meant no
debug info. Bug in the original? You betcha.


>> SEPARATED_DEBUGINFO_FILES which my patch provides can separate them.
> I think David's list (kudos) is very helpful as I haven't seen the requirements articulated this clearly anywhere else.
>
> Some comments:
>
> - Are there important tools that cannot deal with external debuginfo files?

Not that I'm aware of. There have been bugs in the way gobjcopy
is extracting the debuginfo into the separate files, but no
complete failures as far as I know...


>  If we can put debuginfo into external files why would we ever want unstripped binaries?

We deliver minimal stripped binaries so that can have stack traces
with some amount of information in them when the debuginfo files
are not available. We have not yet figured out how to implement
minimal stripping on Windows so Windows hs_err_pid files have no
symbols in them when the debuginfo bundles are not present.


>  Is strip policy really necessary?

Yes. STRIP_POLICY allows folks to configure what they want
for the build that they are doing. It is also meant to map
to the Oracle policy where we were granted a waiver to
deliver minimal symbols in FDS project.


> Strip policy would seem to only be necessary if there are tools which can't handle external debuginfo. Assuming that everything can use external debuginfo then the policy would be to strip everything.

No you don't want to strip everything. See above aboutminimal
info in stack traces.


> Do I understand correctly that rpmbuild can only deal with unstripped binaries and generates the stripped rpm package and debuginfo package. It sounds kind of strange that find-debuginfo.sh wouldn't be able to use already generated debuginfo files.
>
> - I would expect that the flow is something like an extended version of https://blogs.oracle.com/dbx/entry/creating_separate_debug_info :
>   1.  Compile source files with some form of "-g"
>   2.  Create separate debug files for object files.
>   3.  Strip object files.
>   4.  Add gnu_debuglink to object files.
>   5.  Link library or executable which should carry along links to debuginfo.
>   6a. Debugging, testing, profiling, etc. by developers.
>   6b. Packaging for program bits and debuginfo.
>   7.  Testing and Use.
>
> Hopefully I didn't get any of those steps in the wrong order. What are the "you-cant-get-there-from-here" gaps and breakdowns from implementing this process?
>
> - Whether for the FDS debug symbols or RPM debuginfo packaging it seems that the default debug level isn't quite right. I believe that in both cases what's wanted is abbreviated debug info, mostly function names and line number tables, for building stack traces. This is not the debug info that is currently generated.

It's STRIP_POLICY=min_strip that leaves the minimal
debug info attached to the binaries and the more
complete debug info in left in the debuginfo file(s).
And I'll disagree about only wanting abbreviated debug
info in the separate debuginfo files.


> There are four basic alternatives for levels of debug info:
> 1. (-g0)  No debug info whatsoever.
>    Only exported functions and globals will have names. 
> 2. (-g1)  Abbreviated debug info. 
>    All functions have names and line number information. This seems to be what is needed by both FDS and RPM debuginfo files to produce nice stack traces. 
> 3. (-g)   Default debugging info. 
>    Adds macro expansions and local variable names. Suitable for source level debugging.
> 4. (-g3 or -gdwarf-4 -fvar-tracking-assignments). Full debugging info. 
>    Most suitable for source level debugging including debugging of optimized code. It is not clear that anyone would want to build the entire product with this option.
>
> Compilations are currently done with a mix of -g, -g1, and -gstabs. It would be good to rationalize this somewhat and provide a mechanism for developers to tweak generated debugging output for files or components.

There is a whole boatload of history behind using STABS or
DWARF or -g1 and usually (at least in HotSpot) there are
comments in the Makefiles explaining the sometimes arcane
reasons why these choices were made...

On Linux there is a special option DEBUG_BINARIES that is
used to specify '-g' for everything. I believe the magic
incantation for making RPM happy is something like:

DEBUG_BINARIES=true
FULL_DEBUG_SYMBOLS=0
STRIP_POLICY=none
ALT_OBJCOPY=/does_not_exist

The combination of FULL_DEBUG_SYMBOLS=0 and ALT_OBJCOPY=/does_not_exist
disables FDS for all build configs and reverts to pre-FDS make logic.
STRIP_POLICY=none says don't do any stripping. DEBUG_BINARIES=true says
ignore all the other logic about which debug options and just do '-g'.

I've accidentally broken DEBUG_BINARIES at least once, but I usually
get pinged by the Red Hat folks when I do...

Dan



> - Eventual alignment with http://gcc.gnu.org/wiki/DebugFission on some platforms?
>
>> So I change:
>>
>> 1. Separating to add "-g" option to compiler from executing objcopy.
>> 2. jdk/make/Images.gmk regards STRIP_POLICY.
>>
>>
>> As I mentioned earlier, this issue is related to RPM.
>> So I hope to fix it before JDK8 GA is released.
> This won't happen (at least not for Java 8u0). Java 8 is already late in it's final candidate stage. It is too late for the initial Java 8 release to consider this magnitude of change. In any event, since the Java 8 rampdown began back in November, any change would first have to be applied to Java 9 and then approved for backport to a Java 8 or an update release (and it is also possibly too late for 8u20).  
>
> Inability to include your suggested change in Java 8 or 8u20 is in no way a rejection of the ideas or contribution, it's merely a normal consequence of timelines and process.
>
> Mike