: PING: RFR: 8234624: jstack mixed mode should refer DWARF
Yasumasa Suenaga
suenaga at oss.nttdata.com
Wed Mar 11 06:48:26 UTC 2020
On 2020/03/11 15:20, David Holmes wrote:
> On 11/03/2020 4:03 pm, Yasumasa Suenaga wrote:
>> Thanks David!
>>
>> Can you share native backtrace?
>> (Did /opt/core.sh collect it?)
>
> There is a core file but I can't process it, sorry.
Can you share entire of hs_err log and libsaproc.so on this test?
I cannot reproduce the crash on my laptop.
Yasumasa
> David
> -----
>
>>
>> Yasumasa
>>
>>
>> On 2020/03/11 14:59, David Holmes wrote:
>>> Hi Yasumasa,
>>>
>>> Partial hs_err info below.
>>>
>>> David
>>> -----
>>>
>>> #
>>> # A fatal error has been detected by the Java Runtime Environment:
>>> #
>>> # SIGSEGV (0xb) at pc=0x00007fdf2000e87c, pid=29798, tid=29800
>>> #
>>> # JRE version: Java(TM) SE Runtime Environment (15.0) (fastdebug build 15-internal+0-2020-03-11-0447267.suenaga.source)
>>> # Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 15-internal+0-2020-03-11-0447267.suenaga.source, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64)
>>> # Problematic frame:
>>> # C [libsaproc.so+0x487c] DwarfParser::process_dwarf(unsigned long)+0x2c
>>> #
>>> # Core dump will be written. Default location: Core dumps may be processed with "/opt/core.sh %p" (or dumping to /opt/mach5/mesos/work_dir/slaves/90726e33-be99-4e27-9d68-25dad266ef13-S5982/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/5f116aa5-2fc4-44e1-8808-1284111132ed/runs/908d4a0c-e2df-4273-bb74-b899888c3b6a/testoutput/test-support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/scratch/0/core.29798)
>>> #
>>> # If you would like to submit a bug report, please visit:
>>> # https://bugreport.java.com/bugreport/crash.jsp
>>> # The crash happened outside the Java Virtual Machine in native code.
>>> # See problematic frame for where to report the bug.
>>> #
>>>
>>> --------------- S U M M A R Y ------------
>>>
>>> Command Line:
>>> -Denv.class.path=/opt/mach5/mesos/work_dir/slaves/90726e33-be99-4e27-9d68-25dad266ef13-S5982/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/5f116aa5-2fc4-44e1-8808-1284111132ed/runs/908d4a0c-e2df-4273-bb74-b899888c3b6a/testoutput/test-support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/0/serviceability/sa/TestJhsdbJstackMixed.d:/opt/mach5/mesos/work_dir/jib-master/install/2020-03-11-0447267.suenaga.source/src.full/open/test/hotspot/jtreg/serviceability/sa:/opt/mach5/mesos/work_dir/slaves/90726e33-be99-4e27-9d68-25dad266ef13-S5982/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/5f116aa5-2fc4-44e1-8808-1284111132ed/runs/908d4a0c-e2df-4273-bb74-b899888c3b6a/testoutput/test-support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/0/test/lib:/opt/mach5/mesos/work_dir/jib-master/install/2020-03-11-0447267.suenaga.source/src.full/open/test/lib:/opt/mach5/mesos/work_dir/jib-master/install/jtreg/5.0/b01/bundles/jtreg_bin-5.0.zip/jtreg/lib/javatest.jar:/opt/mach5/mesos/work_dir/jib-master/install/jtreg/5.0/b01/bundles/jtreg_bin-5.0.zip/jtreg/lib/jtreg.jar
>>> -Dapplication.home=/opt/mach5/mesos/work_dir/jib-master/install/2020-03-11-0447267.suenaga.source/linux-x64-debug.jdk/jdk-15/fastdebug -Xms8m -Djdk.module.main=jdk.hotspot.agent jdk.hotspot.agent/sun.jvm.hotspot.SALauncher jstack --mixed --pid 29770
>>>
>>> Time: Wed Mar 11 05:20:57 2020 UTC elapsed time: 3.927809 seconds (0d 0h 0m 3s)
>>>
>>> --------------- T H R E A D ---------------
>>>
>>> Current thread (0x00007fdf5c032000): JavaThread "main" [_thread_in_native, id=29800, stack(0x00007fdf63a9e000,0x00007fdf63b9f000)]
>>>
>>> Stack: [0x00007fdf63a9e000,0x00007fdf63b9f000], sp=0x00007fdf63b9d190, free space=1020k
>>> Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
>>> C [libsaproc.so+0x487c] DwarfParser::process_dwarf(unsigned long)+0x2c
>>> j sun.jvm.hotspot.debugger.linux.amd64.DwarfParser.processDwarf0(J)V+0 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.debugger.linux.amd64.DwarfParser.processDwarf(Lsun/jvm/hotspot/debugger/Address;)V+7 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.debugger.linux.amd64.LinuxAMD64CFrame.getTopFrame(Lsun/jvm/hotspot/debugger/linux/LinuxDebugger;Lsun/jvm/hotspot/debugger/Address;Lsun/jvm/hotspot/debugger/ThreadContext;)Lsun/jvm/hotspot/debugger/linux/amd64/LinuxAMD64CFrame;+38 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.debugger.linux.LinuxCDebugger.topFrameForThread(Lsun/jvm/hotspot/debugger/ThreadProxy;)Lsun/jvm/hotspot/debugger/cdbg/CFrame;+116 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.tools.PStack.run(Ljava/io/PrintStream;Lsun/jvm/hotspot/debugger/Debugger;)V+125 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.tools.PStack.run(Ljava/io/PrintStream;)V+11 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.tools.PStack.run()V+4 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.tools.JStack.run()V+55 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.tools.Tool.startInternal()V+87 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.tools.Tool.start([Ljava/lang/String;)I+359 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.tools.Tool.execute([Ljava/lang/String;)V+4 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.tools.JStack.runWithArgs([Ljava/lang/String;)V+99 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.SALauncher.runJSTACK([Ljava/lang/String;)V+58 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.SALauncher$$Lambda$3.accept(Ljava/lang/Object;)V+4 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.SALauncher.main([Ljava/lang/String;)V+158 jdk.hotspot.agent at 15-internal
>>> v ~StubRoutines::call_stub
>>> V [libjvm.so+0xc2291c] JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0x6ac
>>> V [libjvm.so+0xd31970] jni_invoke_static(JNIEnv_*, JavaValue*, _jobject*, JNICallType, _jmethodID*, JNI_ArgumentPusher*, Thread*) [clone .isra.140] [clone .constprop.263]+0x370
>>> V [libjvm.so+0xd36202] jni_CallStaticVoidMethod+0x222
>>> C [libjli.so+0x4bed] JavaMain+0xbcd
>>> C [libjli.so+0x80a9] ThreadJavaMain+0x9
>>>
>>> Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
>>> j sun.jvm.hotspot.debugger.linux.amd64.DwarfParser.processDwarf0(J)V+0 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.debugger.linux.amd64.DwarfParser.processDwarf(Lsun/jvm/hotspot/debugger/Address;)V+7 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.debugger.linux.amd64.LinuxAMD64CFrame.getTopFrame(Lsun/jvm/hotspot/debugger/linux/LinuxDebugger;Lsun/jvm/hotspot/debugger/Address;Lsun/jvm/hotspot/debugger/ThreadContext;)Lsun/jvm/hotspot/debugger/linux/amd64/LinuxAMD64CFrame;+38 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.debugger.linux.LinuxCDebugger.topFrameForThread(Lsun/jvm/hotspot/debugger/ThreadProxy;)Lsun/jvm/hotspot/debugger/cdbg/CFrame;+116 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.tools.PStack.run(Ljava/io/PrintStream;Lsun/jvm/hotspot/debugger/Debugger;)V+125 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.tools.PStack.run(Ljava/io/PrintStream;)V+11 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.tools.PStack.run()V+4 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.tools.JStack.run()V+55 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.tools.Tool.startInternal()V+87 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.tools.Tool.start([Ljava/lang/String;)I+359 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.tools.Tool.execute([Ljava/lang/String;)V+4 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.tools.JStack.runWithArgs([Ljava/lang/String;)V+99 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.SALauncher.runJSTACK([Ljava/lang/String;)V+58 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.SALauncher$$Lambda$3.accept(Ljava/lang/Object;)V+4 jdk.hotspot.agent at 15-internal
>>> j sun.jvm.hotspot.SALauncher.main([Ljava/lang/String;)V+158 jdk.hotspot.agent at 15-internal
>>> v ~StubRoutines::call_stub
>>>
>>> siginfo: si_signo: 11 (SIGSEGV), si_code: 2 (SEGV_ACCERR), si_addr: 0x00007fded5076b79
>>>
>>> Register to memory mapping:
>>>
>>> RAX=0x00007f7e4dfe3229 is an unknown value
>>> RBX=0x00007fdf5c4d7080 points into unknown readable memory: 80 23 07 d4 de 7f 00 00
>>> RCX=0x00007fded4072380 points into unknown readable memory: 2f 75 73 72 2f 6c 69 62
>>> RDX=0x00007fded4076b85 points into unknown readable memory: 01 00 00
>>> RSP=0x00007fdf63b9d190 is pointing into the stack for thread: 0x00007fdf5c032000
>>> RBP=0x00007fdf63b9d1b0 is pointing into the stack for thread: 0x00007fdf5c032000
>>> RSI=0x0000000000000004 is an unknown value
>>> RDI=0x00007fdf5c4d7080 points into unknown readable memory: 80 23 07 d4 de 7f 00 00
>>> R8 =0x000000000146c380 points into unknown readable memory: 02 00 00 00 00 00 00 00
>>> R9 =0x00007fded4076b79 points into unknown readable memory: 7a 52 00 01 78 10 01
>>> R10=0x00000000ffffffff is an unknown value
>>> R11=0x000000000100527a is an unknown value
>>> R12=0x00007fded5076b79 is an unknown value
>>> R13=0x00007f7da2f8e68a is an unknown value
>>> R14=0x00007f7dbdf62b1d is an unknown value
>>> R15=0x00007fdf5c032000 is a thread
>>>
>>>
>>> Registers:
>>> RAX=0x00007f7e4dfe3229, RBX=0x00007fdf5c4d7080, RCX=0x00007fded4072380, RDX=0x00007fded4076b85
>>> RSP=0x00007fdf63b9d190, RBP=0x00007fdf63b9d1b0, RSI=0x0000000000000004, RDI=0x00007fdf5c4d7080
>>> R8 =0x000000000146c380, R9 =0x00007fded4076b79, R10=0x00000000ffffffff, R11=0x000000000100527a
>>> R12=0x00007fded5076b79, R13=0x00007f7da2f8e68a, R14=0x00007f7dbdf62b1d, R15=0x00007fdf5c032000
>>> RIP=0x00007fdf2000e87c, EFLAGS=0x0000000000010206, CSGSFS=0x002b000000000033, ERR=0x0000000000000004
>>> TRAPNO=0x000000000000000e
>>>
>>> Top of Stack: (sp=0x00007fdf63b9d190)
>>> 0x00007fdf63b9d190: 00007fdf209d0980 0000000000000000
>>> 0x00007fdf63b9d1a0: 00007fdf209d0980 00007fdf63b9d258
>>> 0x00007fdf63b9d1b0: 00007fdf63b9d228 00007fdf44778dbe
>>> 0x00007fdf63b9d1c0: 000000000146c380 00007fdf5c032000
>>>
>>> Instructions: (pc=0x00007fdf2000e87c)
>>> 0x00007fdf2000e77c: 89 43 18 4d 85 f6 75 0f eb 2a 66 2e 0f 1f 84 00
>>> 0x00007fdf2000e78c: 00 00 00 00 48 89 c2 48 8d 42 01 48 89 43 08 80
>>> 0x00007fdf2000e79c: 78 ff 00 78 ef 48 8d 42 02 48 89 43 08 0f b6 42
>>> 0x00007fdf2000e7ac: 01 88 43 10 48 c7 43 28 00 00 00 00 4c 89 e1 48
>>> 0x00007fdf2000e7bc: 89 df 31 f6 48 b8 07 00 00 00 10 00 00 00 c6 43
>>> 0x00007fdf2000e7cc: 3c 00 48 c7 c2 ff ff ff ff 48 89 43 14 48 c7 43
>>> 0x00007fdf2000e7dc: 30 00 00 00 00 c7 43 38 00 00 00 00 e8 13 fb ff
>>> 0x00007fdf2000e7ec: ff 4c 89 6b 08 48 83 c4 18 5b 41 5c 41 5d 41 5e
>>> 0x00007fdf2000e7fc: 41 5f 5d c3 83 e7 40 0f 84 63 ff ff ff 48 c7 c2
>>> 0x00007fdf2000e80c: ff ff ff ff 48 d3 e2 49 09 d0 e9 51 ff ff ff 90
>>> 0x00007fdf2000e81c: 0f 1f 40 00 0f b6 47 10 83 e0 07 3c 02 74 0a 76
>>> 0x00007fdf2000e82c: 1b 3c 03 74 04 3c 04 75 17 48 8b 57 08 8b 02 48
>>> 0x00007fdf2000e83c: 83 c2 04 48 89 57 08 c3 0f 1f 40 00 84 c0 74 e9
>>> 0x00007fdf2000e84c: 31 c0 c3 90 55 41 ba ff ff ff ff 48 89 e5 41 56
>>> 0x00007fdf2000e85c: 41 55 49 89 f5 41 54 53 48 8b 07 48 89 fb 4c 8b
>>> 0x00007fdf2000e86c: a0 28 11 00 00 eb 09 0f 1f 44 00 00 4c 89 63 08
>>> 0x00007fdf2000e87c: 41 8b 04 24 4d 8d 4c 24 04 4c 89 4b 08 4c 39 d0
>>> 0x00007fdf2000e88c: 75 0a 49 8b 44 24 04 4d 8d 4c 24 0c 45 8b 19 4d
>>> 0x00007fdf2000e89c: 8d 24 01 49 8d 41 04 48 89 43 08 45 85 db 74 cc
>>> 0x00007fdf2000e8ac: 48 89 df e8 8c f9 ff ff 48 8b 13 41 89 c6 4c 03
>>> 0x00007fdf2000e8bc: b2 18 11 00 00 e8 5a ff ff ff 89 c0 4c 01 f0 4c
>>> 0x00007fdf2000e8cc: 39 e8 76 a8 4d 39 ee 77 a3 44 89 da 4c 89 ce e8
>>> 0x00007fdf2000e8dc: 90 fd ff ff 48 8b 43 08 31 c9 31 ff 48 83 c0 01
>>> 0x00007fdf2000e8ec: 0f 1f 40 00 48 89 43 08 0f b6 70 ff 49 89 c0 48
>>> 0x00007fdf2000e8fc: 83 c0 01 48 89 f2 83 e2 7f 48 d3 e2 83 c1 07 48
>>> 0x00007fdf2000e90c: 09 d7 40 84 f6 78 dd 4c 01 c7 4c 89 e1 4c 89 ea
>>> 0x00007fdf2000e91c: 4c 89 f6 48 89 7b 08 48 89 df e8 d5 f9 ff ff 5b
>>> 0x00007fdf2000e92c: 31 c0 41 5c 41 5d 41 5e 5d c3 66 2e 0f 1f 84 00
>>> 0x00007fdf2000e93c: 00 00 00 00 55 48 89 e5 41 54 53 48 81 ec d0 00
>>> 0x00007fdf2000e94c: 00 00 48 89 b5 48 ff ff ff 48 89 95 50 ff ff ff
>>> 0x00007fdf2000e95c: 48 89 8d 58 ff ff ff 4c 89 85 60 ff ff ff 4c 89
>>> 0x00007fdf2000e96c: 8d 68 ff ff ff 84 c0 74 23 0f 29 85 70 ff ff ff
>>>
>>>
>>> On 11/03/2020 3:52 pm, Yasumasa Suenaga wrote:
>>>> Hi Kevin,
>>>>
>>>> I saw 2 errors on submit repo (mach5-one-ysuenaga-JDK-8234624-5-20200311-0209-9358475).
>>>> So I tweaked my patch, but I saw the crash again (mach5-one-ysuenaga-JDK-8234624-5-20200311-0448-9361448).
>>>>
>>>> Last change on submit repo is here:
>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8234624/webrev.05-2/
>>>>
>>>> Can you share details on submit repo?
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Yasumasa
>>>>
>>>>
>>>> On 2020/03/11 11:07, Yasumasa Suenaga wrote:
>>>>> Hi Kevin,
>>>>>
>>>>> I guess first program header in the libraries which are on your machine has exec flag (you can check it with `readelf -l`).
>>>>> So I tweaked my patch (initial value of exec_start and exec_end set to -1) in new webrev.
>>>>>
>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8234624/webrev.05/
>>>>>
>>>>> This webrev contains the fix for your comment (typo and DW_CFA_advance_loc4).
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Yasumasa
>>>>>
>>>>>
>>>>> On 2020/03/11 8:53, Kevin Walls wrote:
>>>>>> Hi -
>>>>>>
>>>>>> In testing I wasn't seeing any of the Dwarf code triggered.
>>>>>>
>>>>>> With LIBSAPROC_DEBUG set I'm getting the "Could not find executable section in" for lots of / maybe all the libraries...
>>>>>>
>>>>>> src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.c
>>>>>>
>>>>>> if (fill_instr_info(newlib)) {
>>>>>> if (!read_eh_frame(ph, newlib)) {
>>>>>>
>>>>>> fill_instr_info is failing, and we never get to read_eh_frame().
>>>>>>
>>>>>> output like:
>>>>>>
>>>>>> libsaproc DEBUG: [0] vaddr = 0x0, memsz = 0xaba4, filesz = 0xaba4
>>>>>> libsaproc DEBUG: Could not find executable section in /lib/x86_64-linux-gnu/libnss_nis-2.27.so
>>>>>>
>>>>>> (similar for all libraries).
>>>>>>
>>>>>> fill_instr fails if:
>>>>>>
>>>>>> if ((lib->exec_start == 0L) || (lib->exec_end == 0L))
>>>>>>
>>>>>> ...but isn't exec_start relative to the library address? It's the value of ph->vaddr and it is often zero.
>>>>>>
>>>>>> I added some booleans and did:
>>>>>>
>>>>>> 185 if ((lib->exec_start == 0L) || (lib->exec_start > ph->p_vaddr)) {
>>>>>> 186 lib->exec_start = ph->p_vaddr;
>>>>>> 187 found_start =true;
>>>>>> 188 }
>>>>>>
>>>>>> (similarly for end) and only failed if:
>>>>>>
>>>>>> 201 if (!found_start || !found_end) {
>>>>>> 202 return false;
>>>>>>
>>>>>> ...and now it's better. I go from:
>>>>>>
>>>>>> ----------------- 3306 -----------------
>>>>>> 0x00007f75824acd2d __GI___pthread_timedjoin_ex + 0x17d
>>>>>>
>>>>>> to:
>>>>>>
>>>>>> ----------------- 31127 -----------------
>>>>>> 0x00007fa284d78d2d __GI___pthread_timedjoin_ex + 0x17d
>>>>>> 0x00007fa2857aaf2d CallJavaMainInNewThread + 0xad
>>>>>> 0x00007fa2857a74ed ContinueInNewThread + 0x4d
>>>>>> 0x00007fa2857a8c49 JLI_Launch + 0x1529
>>>>>> 0x000055af1b78db1c main + 0x11c
>>>>>>
>>>>>>
>>>>>> Thanks
>>>>>> Kevin
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 10/03/2020 12:36, Yasumasa Suenaga wrote:
>>>>>>
>>>>>>> Hi Kevin,
>>>>>>>
>>>>>>> Thanks for your comment!
>>>>>>>
>>>>>>> On 2020/03/10 18:58, Kevin Walls wrote:
>>>>>>>> Hi Yasumasa ,
>>>>>>>>
>>>>>>>> The changes build OK for me in the latest jdk, and things still work.
>>>>>>>> I have not yet seen the dwarf usage in action: I've tried a couple of different systems and so far have not reproduced the problem, i.e. jstack has not failed on native frames.
>>>>>>>>
>>>>>>>> I may need more recent basic libraries, will look again for somewhere where the problem happens and get back to you as I really want to run the changes.
>>>>>>>
>>>>>>> You can see the problem with JShell.
>>>>>>> Some Java frames would not be seen in mixed jstack.
>>>>>>>
>>>>>>>
>>>>>>>> I have mostly minor other comments which don't need a new webrev, some just comments for the future:
>>>>>>>>
>>>>>>>> src/jdk.hotspot.agent/linux/native/libsaproc/dwarf.cpp:
>>>>>>>>
>>>>>>>> DW_CFA_nop - shouldn't this continue instead of return?
>>>>>>>> (It may "never" happen, but a nop could appear within some other instructions?)
>>>>>>>
>>>>>>> DW_CFA_nop is used for padding, so we can ignore (return immediately) it.
>>>>>>>
>>>>>>>
>>>>>>>> DW_CFA_remember_state: a minor typo in the comment, "DW_CFA_remenber_state".
>>>>>>>
>>>>>>> I will fix it.
>>>>>>>
>>>>>>>
>>>>>>>> We handle DW_CFA_advance_loc, and _loc1 and _loc2, but not DW_CFA_advance_loc4. I thought that was odd, but maybe addresses in these tables never increase by 4-byte amounts, would this mean a lot of code on one line. 8-)
>>>>>>>> So maybe it's never used in practice, if you think it's unnecessary no problem, maybe a comment, or add it for robustness.
>>>>>>>
>>>>>>> I will add DW_CFA_advance_loc4.
>>>>>>>
>>>>>>>
>>>>>>>> General-purpose methods like read_leb128(), get_entry_length(), get_decoded_value() specifically update the _buf pointer in this DwarfParser.
>>>>>>>>
>>>>>>>> DwarfParser::process_dwarf() moves _buf.
>>>>>>>> It calls process_cie() which reads, moves _buf and restores it to the original position, then we read augmentation_length from where _buf is.
>>>>>>>> I'm not sure if that's wrong, or if I just need to read again about the CIE/etc layout.
>>>>>>>>
>>>>>>>> I don't really want to suggest making the code pass around a current _buf for the invocation of these general purpose methods, but just wanted to comment that if these get used more widely that might become necessary.
>>>>>>>
>>>>>>> I saw GDB and binutils source for creating this patch.
>>>>>>> They seems to process similar code because we need to calculate DWARF instructions one-by-one to get the value which relates to specified PC.
>>>>>>>
>>>>>>>
>>>>>>>> Similarly in future, if this DWARF support code became used more widely, it might want to move to an
>>>>>>>> OS-neutral directory? It's odd to label it as Linux-specific.
>>>>>>>
>>>>>>> Windows does not use DWARF at least, it uses another feature.
>>>>>>>
>>>>>>> https://urldefense.com/v3/__https://docs.microsoft.com/en-us/cpp/build/exception-handling-x64?view=vs-2019__;!!GqivPVa7Brio!J801oKj34Q7f-4SzAWGKL67e6Xq2yMlV6f01eqp_fqqhqgKktCBiUi2RUaTtALtpRg$
>>>>>>> I'm not sure other platforms (Solaris, macOS) uses DWARF.
>>>>>>> If DWARF is used in them, I can move DWARF related code to posix directory.
>>>>>>>
>>>>>>>
>>>>>>>> src/jdk.hotspot.agent/linux/native/libsaproc/dwarf.hpp:
>>>>>>>> Thanks for changing "can_parsable" which was in the earlier version. 8-)
>>>>>>>>
>>>>>>>>
>>>>>>>> These are just comments to mainly say it looks good, and somebody else out there has read it.
>>>>>>>> I will look for a system that shows the problem, and get back to you again!
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Yasumasa
>>>>>>>
>>>>>>>
>>>>>>>> Many thanks
>>>>>>>> Kevin
>>>>>>>>
>>>>>>>> On 27/02/2020 05:13, Yasumasa Suenaga wrote:
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> webrev.03 cannot be applied to current jdk/jdk due to 8239224 and 8239462 changes (they updated copyright year).
>>>>>>>>> So I modified webrev (only copyright year changes) to be able to apply to current jdk/jdk.
>>>>>>>>> Could you review it?
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8234624/webrev.04/
>>>>>>>>>
>>>>>>>>> I need one more reviewer to push.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Yasumasa
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 2020/02/17 13:07, Yasumasa Suenaga wrote:
>>>>>>>>>> PING: Could you review it?
>>>>>>>>>>
>>>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8234624
>>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8234624/webrev.03/
>>>>>>>>>>
>>>>>>>>>> This change has been already reviewed by Serguei.
>>>>>>>>>> I need one more reviewer to push.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>> Yasumasa
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 2020/02/03 1:37, Yasumasa Suenaga wrote:
>>>>>>>>>>> PING: Could you reveiw this change?
>>>>>>>>>>>
>>>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8234624
>>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8234624/webrev.03/
>>>>>>>>>>>
>>>>>>>>>>> I believe this change helps troubleshooter to fight to postmortem analysis.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>>
>>>>>>>>>>> Yasumasa
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 2020/01/19 3:16, Yasumasa Suenaga wrote:
>>>>>>>>>>>> PING: Could you review it?
>>>>>>>>>>>>
>>>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8234624
>>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8234624/webrev.03/
>>>>>>>>>>>>
>>>>>>>>>>>> I updated webrev. I discussed with Serguei in off list, and I refactored webrev.02 .
>>>>>>>>>>>> It has passed tests on submit repo (mach5-one-ysuenaga-JDK-8234624-4-20200118-1353-8149549).
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>
>>>>>>>>>>>> Yasumasa
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 2019/12/15 10:51, Yasumasa Suenaga wrote:
>>>>>>>>>>>>> Hi Serguei,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for your comment!
>>>>>>>>>>>>> I refactored LinuxCDebugger and LinuxAMD64CFrame in new webrev.
>>>>>>>>>>>>> Also I fixed to free lib->eh_frame.data in libproc_impl.c as Dmitry said.
>>>>>>>>>>>>>
>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8234624/webrev.02/
>>>>>>>>>>>>>
>>>>>>>>>>>>> This change has been passed all tests on submit repo (mach5-one-ysuenaga-JDK-8234624-3-20191214-1527-7538487).
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yasumasa
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 2019/12/14 10:02, serguei.spitsyn at oracle.com wrote:
>>>>>>>>>>>>>> Hi Yasumasa,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This is nice move in general.
>>>>>>>>>>>>>> Thank you for working on this!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8234624/webrev.01/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/linux/LinuxCDebugger.java.frames.html
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 96 long libptr = dbg.findLibPtrByAddress(pc); 97 if (libptr == 0L) { // Java frame 98 Address rbp = context.getRegisterAsAddress(AMD64ThreadContext.RBP); 99 if (rbp == null) { 100 return null; 101 } 102 return new LinuxAMD64CFrame(dbg, rbp, pc, null); 103 } else { // Native frame 104 DwarfParser dwarf; 105 try { 106 dwarf = new DwarfParser(libptr); 107 } catch (DebuggerException e) { 108 Address rbp = context.getRegisterAsAddress(AMD64ThreadContext.RBP); 109 if (rbp == null) { 110 return null; 111 } 112 return new LinuxAMD64CFrame(dbg, rbp, pc, null); 113 } 114 dwarf.processDwarf(pc); 115 Address cfa = ((dwarf.getCFARegister() == AMD64ThreadContext.RBP) && 116 !dwarf.isBPOffsetAvailable()) 117 ? context.getRegisterAsAddress(AMD64ThreadContext.RBP) 118 : context.getRegisterAsAddress(dwarf.getCFARegister()) 119 .addOffsetTo(dwarf.getCFAOffset()); 120 if (cfa == null) { 121 return null; 122 } 123 return new LinuxAMD64CFrame(dbg, cfa, pc, dwarf); 124 }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'd suggest to simplify the logic by refactoring to something like below:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> long libptr = dbg.findLibPtrByAddress(pc);
>>>>>>>>>>>>>> Address cfa = context.getRegisterAsAddress(AMD64ThreadContext.RBP); // Java frame
>>>>>>>>>>>>>> DwarfParser dwarf = null;
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> if (libptr != 0L) { // Native frame
>>>>>>>>>>>>>> try {
>>>>>>>>>>>>>> dwarf = new DwarfParser(libptr);
>>>>>>>>>>>>>> dwarf.processDwarf(pc);
>>>>>>>>>>>>>> Address cfa = ((dwarf.getCFARegister() == AMD64ThreadContext.RBP) &&
>>>>>>>>>>>>>> !dwarf.isBPOffsetAvailable())
>>>>>>>>>>>>>> ? context.getRegisterAsAddress(AMD64ThreadContext.RBP)
>>>>>>>>>>>>>> : context.getRegisterAsAddress(dwarf.getCFARegister())
>>>>>>>>>>>>>> .addOffsetTo(dwarf.getCFAOffset());
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> } catch (DebuggerException e) { // bail out to Java frame case
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>> if (cfa == null) {
>>>>>>>>>>>>>> return null;
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>> return new LinuxAMD64CFrame(dbg, cfa, pc, dwarf);
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8234624/webrev.01/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/linux/amd64/LinuxAMD64CFrame.java.frames.html
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 58 long ofs = useDwarf ? dwarf.getReturnAddressOffsetFromCFA()
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Better to rename 'ofs' => 'offs'.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 77 nextCFA = nextCFA.addOffsetTo(- nextDwarf.getBasePointerOffsetFromCFA());
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Extra space after '-' sign.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 71 private Address getNextCFA(DwarfParser nextDwarf, ThreadContext context) {
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It feels like the logic has to be somehow refactored/simplified as
>>>>>>>>>>>>>> several typical fragments appears in slightly different contexts.
>>>>>>>>>>>>>> But it is not easy to understand what it is.
>>>>>>>>>>>>>> Could you, please, add some comments to key places explaining this logic.
>>>>>>>>>>>>>> Then I'll check if it is possible to make it a little bit simpler.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 109 private CFrame javaSender(ThreadContext context) { 110 Address nextCFA; 111 Address nextPC; 112 113 nextPC = getNextPC(false); 114 if (nextPC == null) { 115 return null; 116 } 117 118 DwarfParser nextDwarf = null; 119 long libptr = dbg.findLibPtrByAddress(nextPC); 120 if (libptr != 0L) { // Native frame 121 try { 122 nextDwarf = new DwarfParser(libptr); 123 } catch (DebuggerException e) { 124 nextCFA = getNextCFA(null, context); 125 return (nextCFA == null) ? null : new LinuxAMD64CFrame(dbg, nextCFA, nextPC, null); 126 } 127 nextDwarf.processDwarf(nextPC); 128 } 129 130 nextCFA = getNextCFA(nextDwarf, context); 131 return (nextCFA == null) ? null 132 : new LinuxAMD64CFrame(dbg, nextCFA, nextPC, nextDwarf); 133 }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The above can be simplified if a DebuggerException can not be thrown from processDwarf(nextPC):
>>>>>>>>>>>>>> private CFrame javaSender(ThreadContext context) {
>>>>>>>>>>>>>> Address nextPC = getNextPC(false);
>>>>>>>>>>>>>> if (nextPC == null) {
>>>>>>>>>>>>>> return null;
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>> long libptr = dbg.findLibPtrByAddress(nextPC);
>>>>>>>>>>>>>> DwarfParser nextDwarf = null;
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> if (libptr != 0L) { // Native frame
>>>>>>>>>>>>>> try {
>>>>>>>>>>>>>> nextDwarf = new DwarfParser(libptr);
>>>>>>>>>>>>>> nextDwarf.processDwarf(nextPC);
>>>>>>>>>>>>>> } catch (DebuggerException e) { // Bail out to Java frame
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>> Address nextCFA = getNextCFA(nextDwarf, context);
>>>>>>>>>>>>>> return (nextCFA == null) ? null : new LinuxAMD64CFrame(dbg, nextCFA, nextPC, nextDwarf);
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 135 public CFrame sender(ThreadProxy thread) { 136 ThreadContext context = thread.getContext(); 137 138 if (dwarf == null) { // Java frame 139 return javaSender(context); 140 } 141 142 Address nextPC = getNextPC(true); 143 if (nextPC == null) { 144 return null; 145 } 146 147 Address nextCFA; 148 DwarfParser nextDwarf = dwarf; 149 if (!dwarf.isIn(nextPC)) { 150 long libptr = dbg.findLibPtrByAddress(nextPC); 151 if (libptr == 0L) { 152 // Next frame might be Java frame 153 nextCFA = getNextCFA(null, context); 154 return (nextCFA == null) ? null : new LinuxAMD64CFrame(dbg, nextCFA, nextPC, null); 155 } 156 try { 157 nextDwarf = new DwarfParser(libptr); 158 } catch (DebuggerException e) { 159 nextCFA = getNextCFA(null, context); 160 return (nextCFA == null) ? null : new LinuxAMD64CFrame(dbg, nextCFA, nextPC, null); 161 } 162 } 163 164 nextDwarf.processDwarf(nextPC); 165 nextCFA = getNextCFA(nextDwarf, context); 166 return (nextCFA == null) ? null : new
>>>>>>>>>>>>>> LinuxAMD64CFrame(dbg, nextCFA, nextPC, nextDwarf); 167 }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This one can be also simplified a little:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> public CFrame sender(ThreadProxy thread) {
>>>>>>>>>>>>>> ThreadContext context = thread.getContext();
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> if (dwarf == null) { // Java frame
>>>>>>>>>>>>>> return javaSender(context);
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>> Address nextPC = getNextPC(true);
>>>>>>>>>>>>>> if (nextPC == null) {
>>>>>>>>>>>>>> return null;
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>> DwarfParser nextDwarf = null;
>>>>>>>>>>>>>> if (!dwarf.isIn(nextPC)) {
>>>>>>>>>>>>>> long libptr = dbg.findLibPtrByAddress(nextPC);
>>>>>>>>>>>>>> if (libptr != 0L) {
>>>>>>>>>>>>>> try {
>>>>>>>>>>>>>> nextDwarf = new DwarfParser(libptr);
>>>>>>>>>>>>>> nextDwarf.processDwarf(nextPC);
>>>>>>>>>>>>>> } catch (DebuggerException e) { // Bail out to Java frame
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>> Address nextCFA = getNextCFA(nextDwarf, context);
>>>>>>>>>>>>>> return (nextCFA == null) ? null : new LinuxAMD64CFrame(dbg, nextCFA, nextPC, nextDwarf);
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Finally, it looks like just one method could replace both
>>>>>>>>>>>>>> sender(ThreadProxy thread) and javaSender(ThreadContext context):
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> private CFrame commonSender(ThreadProxy thread) {
>>>>>>>>>>>>>> ThreadContext context = thread.getContext();
>>>>>>>>>>>>>> Address nextPC = getNextPC(false);
>>>>>>>>>>>>>> if (nextPC == null) {
>>>>>>>>>>>>>> return null;
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>> DwarfParser nextDwarf = null;
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> long libptr = dbg.findLibPtrByAddress(nextPC);
>>>>>>>>>>>>>> if (dwarf == null || !dwarf.isIn(nextPC)) {
>>>>>>>>>>>>>> long libptr = dbg.findLibPtrByAddress(nextPC);
>>>>>>>>>>>>>> if (libptr != 0L) {
>>>>>>>>>>>>>> try {
>>>>>>>>>>>>>> nextDwarf = new DwarfParser(libptr);
>>>>>>>>>>>>>> nextDwarf.processDwarf(nextPC);
>>>>>>>>>>>>>> } catch (DebuggerException e) { // Bail out to Java frame
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>> Address nextCFA = getNextCFA(nextDwarf, context);
>>>>>>>>>>>>>> return (nextCFA == null) ? null : new LinuxAMD64CFrame(dbg, nextCFA, nextPC, nextDwarf);
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm still reviewing the dwarf parser files.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Serguei
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 11/28/19 4:39 AM, Yasumasa Suenaga wrote:
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I refactored LinuxAMD64CFrame.java . It works fine in serviceability/sa tests and
>>>>>>>>>>>>>>> all tests on submit repo (mach5-one-ysuenaga-JDK-8234624-2-20191128-0928-7059923).
>>>>>>>>>>>>>>> Could you review new webrev?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8234624/webrev.01/
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The diff from previous webrev is here:
>>>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/4bc47efbc90b
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yasumasa
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 2019/11/25 14:08, Yasumasa Suenaga wrote:
>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Please review this change:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8234624
>>>>>>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8234624/webrev.00/
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> According to 2.7 Stack Unwind Algorithm in System V Application Binary Interface AMD64
>>>>>>>>>>>>>>>> Architecture Processor Supplement [1], we need to use DWARF in .eh_frame or .debug_frame
>>>>>>>>>>>>>>>> for stack unwinding.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> As JDK-8022183 said, omit-frame-pointer is enabled by default since GCC 4.6, so system
>>>>>>>>>>>>>>>> library (e.g. libc) might be compiled with this feature.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> However `jhsdb jstack --mixed` does not do so, it uses base pointer register (RBP).
>>>>>>>>>>>>>>>> So it might be lack of stack frames.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I guess JDK-8219201 is caused by same issue.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Yasumasa
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [1] https://urldefense.com/v3/__https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf__;!!GqivPVa7Brio!J801oKj34Q7f-4SzAWGKL67e6Xq2yMlV6f01eqp_fqqhqgKktCBiUi2RUaQusmjOqA$
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
More information about the serviceability-dev
mailing list