8234624: jstack mixed mode should refer DWARF

David Holmes david.holmes at oracle.com
Sun Nov 24 23:12:09 UTC 2019


On 23/11/2019 2:24 pm, Yasumasa Suenaga wrote:
> David, Chris,
> 
> Can you share the result of this test?
> 
>    mach5-one-ysuenaga-JDK-8234624-1-20191123-0234-6938325
> 
> It failed on TestJhsdbJstackMixed.java and ClhsdbPstack.java .

I don't know what to make of the result. For TestJhsdbJstackMixed it 
timed out. There is a stack dump in the log which for the most part is 
quite normal e.g.

----------------- 8449 -----------------
"NoFramePointerJNIFib" #13 prio=5 tid=0x00007f7224674000 nid=0x2101 
runnable [0x00007f71f54d9000]
    java.lang.Thread.State: RUNNABLE
    JavaThread state: _thread_in_native
0x00007f722d2d26c0	fib + 0x40
----------------- 8438 -----------------
"Common-Cleaner" #12 daemon prio=8 tid=0x00007f722461a800 nid=0x20f6 in 
Object.wait() [0x00007f71f5af8000]
    java.lang.Thread.State: TIMED_WAITING (on object monitor)
    JavaThread state: _thread_blocked
0x00007f722ce4cde2	__pthread_cond_timedwait + 0x132
0x00007f722c030fa4	ObjectSynchronizer::wait(Handle, long, Thread*) + 0x84
0x00007f722b9097fd	JVM_MonitorWait + 0x11d
0x00007f720c927dbe	<interpreter> method entry point (kind = native)
0x00007f720c91f0b3	* java.lang.Object.wait(long) bci:0 (Interpreted frame)
0x00007f720c91ee00	* java.lang.ref.ReferenceQueue.remove(long) bci:59 
line:155 (Interpreted frame)
0x00007f720c91f0f8	* jdk.internal.ref.CleanerImpl.run() bci:65 line:148 
(Interpreted frame)
0x00007f720c91f0b3	* java.lang.Thread.run() bci:11 line:832 (Interpreted 
frame)
0x00007f720c9159ca	<StubRoutines>
0x00007f722b7af34c	JavaCalls::call_helper(JavaValue*, methodHandle 
const&, JavaCallArguments*, Thread*) + 0x6ac
0x00007f722b7ac3ce	JavaCalls::call_virtual(JavaValue*, Klass*, Symbol*, 
Symbol*, JavaCallArguments*, Thread*) + 0x33e
0x00007f722b7ac5ea	JavaCalls::call_virtual(JavaValue*, Handle, Klass*, 
Symbol*, Symbol*, Thread*) + 0xca
0x00007f722b905f07	thread_entry(JavaThread*, Thread*) + 0x127
0x00007f722c09dde6	JavaThread::thread_main_inner() + 0x226
0x00007f722c0a34c6	Thread::call_run() + 0xf6
0x00007f722bdd57be	thread_native_entry(Thread*) + 0x10e
0x00007f722ce48ea5	start_thread + 0xc5

but after normal thread output we get to

----------------- 8406 -----------------
0x00007f722ce4a017	pthread_join + 0xa7
0x00007f722d474050		????????
0x00007f722d474050		????????
< repeats unknown number of times due to output log overflow>
0x00007f722d474050		????????
0x00007f722d474050		????????

DEBUG: [0x00007f722d2d26c0	fib + 0x40]
DEBUG: Test triggered interesting condition.
DEBUG: Test PASSED.

---

The ClhsdbPstack.java generated a core dump but no hs_err file. It also 
had the ????? stack dump

0x00007f882d400050		????????
0x00007f882d400050		????????
0x00007f882d400050		????????
0x00007f882d400050		????????
];
  stderr: []
  exitValue = 134

  LingeredApp stdout: [];
  LingeredApp stderr: []
  LingeredApp exitValue = 0
java.lang.RuntimeException: Test ERROR java.lang.RuntimeException: 
Expected to get exit value of [0]

David
-----


> 
> Thanks,
> 
> Yasumasa
> 
> 
> On 2019/11/23 10:39, Yasumasa Suenaga wrote:
>> On 2019/11/23 1:52, Chris Plummer wrote:
>>> Hi Yasumasa,
>>>
>>> Start with the following code in HotSpotAgent.java:
>>>
>>>          catch (NoSuchSymbolException e) {
>>>              throw new DebuggerException("Doesn't appear to be a 
>>> HotSpot VM (could not find symbol \"" +
>>>              e.getSymbol() + "\" in remote process)");
>>>          }
>>>
>>> Fix it to include "e" as the cause of the DebuggerException. Then the 
>>> exception backtrace that David included below will be a bit more useful.
>>
>> Thank you for the advise, Chris!
>> But I cannot access Mach 5 result because I'm not an Oracle employee...
>>
>> I'm not sure I can get root cause from the email from submit repo.
>>
>>
>> yasumasa
>>
>>
>>> thanks,
>>>
>>> Chris
>>>
>>>
>>> On 11/22/19 12:55 AM, Yasumasa Suenaga wrote:
>>>> Thanks David!
>>>>
>>>> Hmm... my slowdebug build works fine on my laptop.
>>>> I will investigate more.
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Yasumasa
>>>>
>>>>
>>>> On 2019/11/22 17:08, David Holmes wrote:
>>>>> On 22/11/2019 5:42 pm, Yasumasa Suenaga wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> I tried to get mixed stack via `jhsdb jstack --mixed`, but I 
>>>>>> couldn't.
>>>>>> (See JBS for details)
>>>>>>
>>>>>>    https://bugs.openjdk.java.net/browse/JDK-8234624
>>>>>>
>>>>>> I think it is caused by DWARF. AMD64 needs DWARF for stack 
>>>>>> unwinding, but SA does not handle it.
>>>>>> So I created a patch. It works fine on my Fedora 31 x64 box, but 
>>>>>> it failed on submit repo.
>>>>>>
>>>>>>    http://hg.openjdk.java.net/jdk/submit/rev/f97745e0af75
>>>>>>
>>>>>> Failed test was linux-x64-debug, and it is due to 
>>>>>> "gHotSpotVMTypes" was not found.
>>>>>> I wonder why it failed, and why my serviceability/sa tests (with 
>>>>>> fastdebug build) was succeeded.
>>>>>> Can you share details for this test? 
>>>>>> mach5-one-ysuenaga-JDK-8234624-20191122-0630-6909161
>>>>>
>>>>> I can't really shed any light on it, there were lots of failures - 
>>>>> see below for example. The issue is with the VM that was being 
>>>>> inspected but there's no output from that VM.
>>>>>
>>>>> David
>>>>>   -----
>>>>>
>>>>> ----------System.out:(10/413)----------
>>>>> Starting TestUniverse
>>>>> Started LingeredApp with G1GC and pid 31111
>>>>> Starting clhsdb against 31111
>>>>> [2019-11-22T07:03:42.836056Z] Gathering output for process 31133
>>>>> [2019-11-22T07:03:44.395452Z] Waiting for completion for process 31133
>>>>> [2019-11-22T07:03:44.395815Z] Waiting for completion finished for 
>>>>> process 31133
>>>>> hsdb> Command not valid until attached to a VM
>>>>> hsdb>
>>>>> 'Heap Parameters' missing from stdout/stderr
>>>>>
>>>>> ----------System.err:(53/3915)----------
>>>>> Command line: 
>>>>> ['/opt/mach5/mesos/work_dir/jib-master/install/2019-11-22-0629473.suenaga.source/linux-x64-debug.jdk/jdk-14/fastdebug/bin/java' 
>>>>> '-XX:+UnlockExperimentalVMOptions' '-XX:+UseG1GC' '-cp' 
>>>>> '/opt/mach5/mesos/work_dir/slaves/6e54f4af-e606-43b0-80ce-0a482a5988b6-S156/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/0454c404-a309-4896-bf31-90b9636056fa/runs/eed41e19-a725-491b-9ddd-c380024cedc9/testoutput/test-support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/2/serviceability/sa/TestUniverse.d:/opt/mach5/mesos/work_dir/slaves/6e54f4af-e606-43b0-80ce-0a482a5988b6-S156/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/0454c404-a309-4896-bf31-90b9636056fa/runs/eed41e19-a725-491b-9ddd-c380024cedc9/testoutput/test-support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/2/test/lib' 
>>>>> 'jdk.test.lib.apps.LingeredApp' 
>>>>> '918bf6a8-d3df-4fd1-bdca-13fc399c67f3.lck' ]
>>>>> Attaching to process 31111, please wait...
>>>>> Unable to connect to process ID 31111:
>>>>>
>>>>> Doesn't appear to be a HotSpot VM (could not find symbol 
>>>>> "gHotSpotVMTypes" in
>>>>> remote process)
>>>>> sun.jvm.hotspot.debugger.DebuggerException: Doesn't appear to be a 
>>>>> HotSpot VM (could not find symbol "gHotSpotVMTypes" in remote process)
>>>>>      at 
>>>>> jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.setupVM(HotSpotAgent.java:413) 
>>>>>
>>>>>      at 
>>>>> jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.go(HotSpotAgent.java:306) 
>>>>>
>>>>>      at 
>>>>> jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.attach(HotSpotAgent.java:141) 
>>>>>
>>>>>      at 
>>>>> jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.attachDebugger(CLHSDB.java:180) 
>>>>>
>>>>>      at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.run(CLHSDB.java:61)
>>>>>      at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.main(CLHSDB.java:40)
>>>>>      at 
>>>>> jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runCLHSDB(SALauncher.java:270) 
>>>>>
>>>>>      at 
>>>>> jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:406)
>>>>>   stdout: [ Command not valid until attached to a VM
>>>>> ];
>>>>>   stderr: [ Command not valid until attached to a VM
>>>>> ]
>>>>>   exitValue = -1
>>>>>
>>>>>   LingeredApp stdout: [];
>>>>>   LingeredApp stderr: []
>>>>>   LingeredApp exitValue = 0
>>>>> java.lang.RuntimeException: 'Heap Parameters' missing from 
>>>>> stdout/stderr
>>>
>>>


More information about the serviceability-dev mailing list