8234624: jstack mixed mode should refer DWARF

Sat Nov 23 08:59:35 UTC 2019

Hi Volker,

I pushed new patch to submit repo [1] which includes fallback code if .eh_frame does not exist.
So your guess might correct.

However, according to 2.7 Stack Unwind Algorithm in System V Application Binary Interface AMD64
Architecture Processor Supplement [2], we need to use DWARF in .eh_frame or .debug_frame for
stack unwinding.
So I think .eh_frame section should be included in ELF binaries for AMD64.
In fact, libjvm.so in JDKs from jdk.java.net includes it.

The patch [1] passed serviceability/sa tests except TestJhsdbJstackMixed.java and ClhsdbPstack.java  .
They seem to be timeout. They might need more time to complete with my patch.
If we can extend timeout value, I want to try it.

I want to know which configure options are passed in Mach5 on submit repo, and all binaries in
submit repo have .eh_frame section.
(I know it would be configured to slowdebug, but that's all...)

Of course, I also want to know stdout of `jhsdb jstack` on the failure tests :)

Thanks,

Yasumasa

[1] https://hg.openjdk.java.net/jdk/submit/rev/c3334c661fdf
[2] https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf

On 2019/11/23 17:14, Volker Simonis wrote:
> Just a wild guess, but maybe after your changes, the debug symbols are required and not found any more? This could be caused by the fact that Mach5 builds with different settings compared to you? See https://hg.openjdk.java.net/jdk-updates/jdk9u/raw-file/tip/common/doc/building.html#native-debug-symbols for the different variants.
> 
> You could try to see if you can reproduce the failure locally by building --with-native-debug-symbols=none, external,zipped
> 
> Yasumasa Suenaga <suenaga at oss.nttdata.com <mailto:suenaga at oss.nttdata.com>> schrieb am Sa., 23. Nov. 2019, 05:24:
> 
>     David, Chris,
> 
>     Can you share the result of this test?
> 
>         mach5-one-ysuenaga-JDK-8234624-1-20191123-0234-6938325
> 
>     It failed on TestJhsdbJstackMixed.java and ClhsdbPstack.java .
> 
> 
>     Thanks,
> 
>     Yasumasa
> 
> 
>     On 2019/11/23 10:39, Yasumasa Suenaga wrote:
>      > On 2019/11/23 1:52, Chris Plummer wrote:
>      >> Hi Yasumasa,
>      >>
>      >> Start with the following code in HotSpotAgent.java:
>      >>
>      >>          catch (NoSuchSymbolException e) {
>      >>              throw new DebuggerException("Doesn't appear to be a HotSpot VM (could not find symbol \"" +
>      >>              e.getSymbol() + "\" in remote process)");
>      >>          }
>      >>
>      >> Fix it to include "e" as the cause of the DebuggerException. Then the exception backtrace that David included below will be a bit more useful.
>      >
>      > Thank you for the advise, Chris!
>      > But I cannot access Mach 5 result because I'm not an Oracle employee...
>      >
>      > I'm not sure I can get root cause from the email from submit repo.
>      >
>      >
>      > yasumasa
>      >
>      >
>      >> thanks,
>      >>
>      >> Chris
>      >>
>      >>
>      >> On 11/22/19 12:55 AM, Yasumasa Suenaga wrote:
>      >>> Thanks David!
>      >>>
>      >>> Hmm... my slowdebug build works fine on my laptop.
>      >>> I will investigate more.
>      >>>
>      >>>
>      >>> Thanks,
>      >>>
>      >>> Yasumasa
>      >>>
>      >>>
>      >>> On 2019/11/22 17:08, David Holmes wrote:
>      >>>> On 22/11/2019 5:42 pm, Yasumasa Suenaga wrote:
>      >>>>> Hi all,
>      >>>>>
>      >>>>> I tried to get mixed stack via `jhsdb jstack --mixed`, but I couldn't.
>      >>>>> (See JBS for details)
>      >>>>>
>      >>>>> https://bugs.openjdk.java.net/browse/JDK-8234624
>      >>>>>
>      >>>>> I think it is caused by DWARF. AMD64 needs DWARF for stack unwinding, but SA does not handle it.
>      >>>>> So I created a patch. It works fine on my Fedora 31 x64 box, but it failed on submit repo.
>      >>>>>
>      >>>>> http://hg.openjdk.java.net/jdk/submit/rev/f97745e0af75
>      >>>>>
>      >>>>> Failed test was linux-x64-debug, and it is due to "gHotSpotVMTypes" was not found.
>      >>>>> I wonder why it failed, and why my serviceability/sa tests (with fastdebug build) was succeeded.
>      >>>>> Can you share details for this test? mach5-one-ysuenaga-JDK-8234624-20191122-0630-6909161
>      >>>>
>      >>>> I can't really shed any light on it, there were lots of failures - see below for example. The issue is with the VM that was being inspected but there's no output from that VM.
>      >>>>
>      >>>> David
>      >>>>   -----
>      >>>>
>      >>>> ----------System.out:(10/413)----------
>      >>>> Starting TestUniverse
>      >>>> Started LingeredApp with G1GC and pid 31111
>      >>>> Starting clhsdb against 31111
>      >>>> [2019-11-22T07:03:42.836056Z] Gathering output for process 31133
>      >>>> [2019-11-22T07:03:44.395452Z] Waiting for completion for process 31133
>      >>>> [2019-11-22T07:03:44.395815Z] Waiting for completion finished for process 31133
>      >>>> hsdb> Command not valid until attached to a VM
>      >>>> hsdb>
>      >>>> 'Heap Parameters' missing from stdout/stderr
>      >>>>
>      >>>> ----------System.err:(53/3915)----------
>      >>>> Command line: ['/opt/mach5/mesos/work_dir/jib-master/install/2019-11-22-0629473.suenaga.source/linux-x64-debug.jdk/jdk-14/fastdebug/bin/java' '-XX:+UnlockExperimentalVMOptions' '-XX:+UseG1GC' '-cp' '/opt/mach5/mesos/work_dir/slaves/6e54f4af-e606-43b0-80ce-0a482a5988b6-S156/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/0454c404-a309-4896-bf31-90b9636056fa/runs/eed41e19-a725-491b-9ddd-c380024cedc9/testoutput/test-support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/2/serviceability/sa/TestUniverse.d:/opt/mach5/mesos/work_dir/slaves/6e54f4af-e606-43b0-80ce-0a482a5988b6-S156/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/0454c404-a309-4896-bf31-90b9636056fa/runs/eed41e19-a725-491b-9ddd-c380024cedc9/testoutput/test-support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/2/test/lib' 'jdk.test.lib.apps.LingeredApp' '918bf6a8-d3df-4fd1-bdca-13fc399c67f3.lck' ]
>      >>>> Attaching to process 31111, please wait...
>      >>>> Unable to connect to process ID 31111:
>      >>>>
>      >>>> Doesn't appear to be a HotSpot VM (could not find symbol "gHotSpotVMTypes" in
>      >>>> remote process)
>      >>>> sun.jvm.hotspot.debugger.DebuggerException: Doesn't appear to be a HotSpot VM (could not find symbol "gHotSpotVMTypes" in remote process)
>      >>>>      at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.setupVM(HotSpotAgent.java:413)
>      >>>>      at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.go(HotSpotAgent.java:306)
>      >>>>      at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.attach(HotSpotAgent.java:141)
>      >>>>      at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.attachDebugger(CLHSDB.java:180)
>      >>>>      at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.run(CLHSDB.java:61)
>      >>>>      at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.main(CLHSDB.java:40)
>      >>>>      at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runCLHSDB(SALauncher.java:270)
>      >>>>      at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:406)
>      >>>>   stdout: [ Command not valid until attached to a VM
>      >>>> ];
>      >>>>   stderr: [ Command not valid until attached to a VM
>      >>>> ]
>      >>>>   exitValue = -1
>      >>>>
>      >>>>   LingeredApp stdout: [];
>      >>>>   LingeredApp stderr: []
>      >>>>   LingeredApp exitValue = 0
>      >>>> java.lang.RuntimeException: 'Heap Parameters' missing from stdout/stderr
>      >>
>      >>
> 