8234624: jstack mixed mode should refer DWARF
Yasumasa Suenaga
suenaga at oss.nttdata.com
Mon Nov 25 04:54:22 UTC 2019
Thanks David!
I tweaked my patch, and it passed all tests on submit repo (mach5-one-ysuenaga-JDK-8234624-2-20191125-0350-6967731).
I will send review request.
Yasumasa
On 2019/11/25 8:12, David Holmes wrote:
> On 23/11/2019 2:24 pm, Yasumasa Suenaga wrote:
>> David, Chris,
>>
>> Can you share the result of this test?
>>
>> mach5-one-ysuenaga-JDK-8234624-1-20191123-0234-6938325
>>
>> It failed on TestJhsdbJstackMixed.java and ClhsdbPstack.java .
>
> I don't know what to make of the result. For TestJhsdbJstackMixed it timed out. There is a stack dump in the log which for the most part is quite normal e.g.
>
> ----------------- 8449 -----------------
> "NoFramePointerJNIFib" #13 prio=5 tid=0x00007f7224674000 nid=0x2101 runnable [0x00007f71f54d9000]
> java.lang.Thread.State: RUNNABLE
> JavaThread state: _thread_in_native
> 0x00007f722d2d26c0 fib + 0x40
> ----------------- 8438 -----------------
> "Common-Cleaner" #12 daemon prio=8 tid=0x00007f722461a800 nid=0x20f6 in Object.wait() [0x00007f71f5af8000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> JavaThread state: _thread_blocked
> 0x00007f722ce4cde2 __pthread_cond_timedwait + 0x132
> 0x00007f722c030fa4 ObjectSynchronizer::wait(Handle, long, Thread*) + 0x84
> 0x00007f722b9097fd JVM_MonitorWait + 0x11d
> 0x00007f720c927dbe <interpreter> method entry point (kind = native)
> 0x00007f720c91f0b3 * java.lang.Object.wait(long) bci:0 (Interpreted frame)
> 0x00007f720c91ee00 * java.lang.ref.ReferenceQueue.remove(long) bci:59 line:155 (Interpreted frame)
> 0x00007f720c91f0f8 * jdk.internal.ref.CleanerImpl.run() bci:65 line:148 (Interpreted frame)
> 0x00007f720c91f0b3 * java.lang.Thread.run() bci:11 line:832 (Interpreted frame)
> 0x00007f720c9159ca <StubRoutines>
> 0x00007f722b7af34c JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*) + 0x6ac
> 0x00007f722b7ac3ce JavaCalls::call_virtual(JavaValue*, Klass*, Symbol*, Symbol*, JavaCallArguments*, Thread*) + 0x33e
> 0x00007f722b7ac5ea JavaCalls::call_virtual(JavaValue*, Handle, Klass*, Symbol*, Symbol*, Thread*) + 0xca
> 0x00007f722b905f07 thread_entry(JavaThread*, Thread*) + 0x127
> 0x00007f722c09dde6 JavaThread::thread_main_inner() + 0x226
> 0x00007f722c0a34c6 Thread::call_run() + 0xf6
> 0x00007f722bdd57be thread_native_entry(Thread*) + 0x10e
> 0x00007f722ce48ea5 start_thread + 0xc5
>
> but after normal thread output we get to
>
> ----------------- 8406 -----------------
> 0x00007f722ce4a017 pthread_join + 0xa7
> 0x00007f722d474050 ????????
> 0x00007f722d474050 ????????
> < repeats unknown number of times due to output log overflow>
> 0x00007f722d474050 ????????
> 0x00007f722d474050 ????????
>
> DEBUG: [0x00007f722d2d26c0 fib + 0x40]
> DEBUG: Test triggered interesting condition.
> DEBUG: Test PASSED.
>
> ---
>
> The ClhsdbPstack.java generated a core dump but no hs_err file. It also had the ????? stack dump
>
> 0x00007f882d400050 ????????
> 0x00007f882d400050 ????????
> 0x00007f882d400050 ????????
> 0x00007f882d400050 ????????
> ];
> stderr: []
> exitValue = 134
>
> LingeredApp stdout: [];
> LingeredApp stderr: []
> LingeredApp exitValue = 0
> java.lang.RuntimeException: Test ERROR java.lang.RuntimeException: Expected to get exit value of [0]
>
> David
> -----
>
>
>>
>> Thanks,
>>
>> Yasumasa
>>
>>
>> On 2019/11/23 10:39, Yasumasa Suenaga wrote:
>>> On 2019/11/23 1:52, Chris Plummer wrote:
>>>> Hi Yasumasa,
>>>>
>>>> Start with the following code in HotSpotAgent.java:
>>>>
>>>> catch (NoSuchSymbolException e) {
>>>> throw new DebuggerException("Doesn't appear to be a HotSpot VM (could not find symbol \"" +
>>>> e.getSymbol() + "\" in remote process)");
>>>> }
>>>>
>>>> Fix it to include "e" as the cause of the DebuggerException. Then the exception backtrace that David included below will be a bit more useful.
>>>
>>> Thank you for the advise, Chris!
>>> But I cannot access Mach 5 result because I'm not an Oracle employee...
>>>
>>> I'm not sure I can get root cause from the email from submit repo.
>>>
>>>
>>> yasumasa
>>>
>>>
>>>> thanks,
>>>>
>>>> Chris
>>>>
>>>>
>>>> On 11/22/19 12:55 AM, Yasumasa Suenaga wrote:
>>>>> Thanks David!
>>>>>
>>>>> Hmm... my slowdebug build works fine on my laptop.
>>>>> I will investigate more.
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Yasumasa
>>>>>
>>>>>
>>>>> On 2019/11/22 17:08, David Holmes wrote:
>>>>>> On 22/11/2019 5:42 pm, Yasumasa Suenaga wrote:
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I tried to get mixed stack via `jhsdb jstack --mixed`, but I couldn't.
>>>>>>> (See JBS for details)
>>>>>>>
>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8234624
>>>>>>>
>>>>>>> I think it is caused by DWARF. AMD64 needs DWARF for stack unwinding, but SA does not handle it.
>>>>>>> So I created a patch. It works fine on my Fedora 31 x64 box, but it failed on submit repo.
>>>>>>>
>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/f97745e0af75
>>>>>>>
>>>>>>> Failed test was linux-x64-debug, and it is due to "gHotSpotVMTypes" was not found.
>>>>>>> I wonder why it failed, and why my serviceability/sa tests (with fastdebug build) was succeeded.
>>>>>>> Can you share details for this test? mach5-one-ysuenaga-JDK-8234624-20191122-0630-6909161
>>>>>>
>>>>>> I can't really shed any light on it, there were lots of failures - see below for example. The issue is with the VM that was being inspected but there's no output from that VM.
>>>>>>
>>>>>> David
>>>>>> -----
>>>>>>
>>>>>> ----------System.out:(10/413)----------
>>>>>> Starting TestUniverse
>>>>>> Started LingeredApp with G1GC and pid 31111
>>>>>> Starting clhsdb against 31111
>>>>>> [2019-11-22T07:03:42.836056Z] Gathering output for process 31133
>>>>>> [2019-11-22T07:03:44.395452Z] Waiting for completion for process 31133
>>>>>> [2019-11-22T07:03:44.395815Z] Waiting for completion finished for process 31133
>>>>>> hsdb> Command not valid until attached to a VM
>>>>>> hsdb>
>>>>>> 'Heap Parameters' missing from stdout/stderr
>>>>>>
>>>>>> ----------System.err:(53/3915)----------
>>>>>> Command line: ['/opt/mach5/mesos/work_dir/jib-master/install/2019-11-22-0629473.suenaga.source/linux-x64-debug.jdk/jdk-14/fastdebug/bin/java' '-XX:+UnlockExperimentalVMOptions' '-XX:+UseG1GC' '-cp' '/opt/mach5/mesos/work_dir/slaves/6e54f4af-e606-43b0-80ce-0a482a5988b6-S156/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/0454c404-a309-4896-bf31-90b9636056fa/runs/eed41e19-a725-491b-9ddd-c380024cedc9/testoutput/test-support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/2/serviceability/sa/TestUniverse.d:/opt/mach5/mesos/work_dir/slaves/6e54f4af-e606-43b0-80ce-0a482a5988b6-S156/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/0454c404-a309-4896-bf31-90b9636056fa/runs/eed41e19-a725-491b-9ddd-c380024cedc9/testoutput/test-support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/2/test/lib' 'jdk.test.lib.apps.LingeredApp' '918bf6a8-d3df-4fd1-bdca-13fc399c67f3.lck' ]
>>>>>> Attaching to process 31111, please wait...
>>>>>> Unable to connect to process ID 31111:
>>>>>>
>>>>>> Doesn't appear to be a HotSpot VM (could not find symbol "gHotSpotVMTypes" in
>>>>>> remote process)
>>>>>> sun.jvm.hotspot.debugger.DebuggerException: Doesn't appear to be a HotSpot VM (could not find symbol "gHotSpotVMTypes" in remote process)
>>>>>> at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.setupVM(HotSpotAgent.java:413)
>>>>>> at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.go(HotSpotAgent.java:306)
>>>>>> at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.attach(HotSpotAgent.java:141)
>>>>>> at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.attachDebugger(CLHSDB.java:180)
>>>>>> at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.run(CLHSDB.java:61)
>>>>>> at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.main(CLHSDB.java:40)
>>>>>> at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runCLHSDB(SALauncher.java:270)
>>>>>> at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:406)
>>>>>> stdout: [ Command not valid until attached to a VM
>>>>>> ];
>>>>>> stderr: [ Command not valid until attached to a VM
>>>>>> ]
>>>>>> exitValue = -1
>>>>>>
>>>>>> LingeredApp stdout: [];
>>>>>> LingeredApp stderr: []
>>>>>> LingeredApp exitValue = 0
>>>>>> java.lang.RuntimeException: 'Heap Parameters' missing from stdout/stderr
>>>>
>>>>
More information about the serviceability-dev
mailing list