alternate pns() debug function

Thu Nov 9 06:55:16 UTC 2017

On Thu, Nov 9, 2017 at 7:52 AM, Chris Plummer <chris.plummer at oracle.com>
wrote:

> On 11/8/17 10:24 PM, Thomas Stüfe wrote:
>
> Hi Chris,
>
> I went accidentally off-list with my last response yesterday. I continue
> on-list.
>
> On Thu, Nov 9, 2017 at 2:27 AM, Chris Plummer <chris.plummer at oracle.com>
> wrote:
>
>> Hi Thomas,
>>
>> <snip>
>
>>
>>> Interesting. What did you see what the debugger would not show you?
>>>
>>> (gdb) call pns2()
>>>
>>> "Executing pns2"
>>> Native frames: (J=compiled Java code, A=aot compiled Java code,
>>> j=interpreted, Vv=VM code, C=native code)
>>> C  0x00007ffff5d5e7df
>>> C  0xcccccccccccccccc
>>> C  0xcccccccccccccccc
>>>
>>> (gdb) bt
>>> #0  0x00007fffd79eef5c in ?? ()
>>> #1  0x00007ffff73d79a5 in SafeFetch32 (errValue=2748,
>>> adr=0xabc0000000000abc) at /local/ws/jdk10/jdk10-hs.clean
>>> /open/src/hotspot/share/runtime/stubRoutines.hpp:462
>>> #2  test_safefetch32 () at /local/ws/jdk10/jdk10-hs.clean
>>> /open/src/hotspot/share/runtime/stubRoutines.cpp:244
>>> #3  StubRoutines::initialize2 () at /local/ws/jdk10/jdk10-hs.clean
>>> /open/src/hotspot/share/runtime/stubRoutines.cpp:365
>>> #4  0x00007ffff73d80ba in stubRoutines_init2 () at
>>> /local/ws/jdk10/jdk10-hs.clean/open/src/hotspot/share/runtim
>>> e/stubRoutines.cpp:374
>>> #5  0x00007ffff6bc2fa3 in init_globals () at
>>> /local/ws/jdk10/jdk10-hs.clean/open/src/hotspot/share/runtim
>>> e/init.cpp:150
>>> #6  0x00007ffff7474453 in Threads::create_vm (args=args at entry
>>> =0x7ffff5d5ee60, canTryAgain=canTryAgain at entry=0x7ffff5d5ed8f) at
>>> /local/ws/jdk10/jdk10-hs.clean/open/src/hotspot/share/runtim
>>> e/thread.cpp:3616
>>> #7  0x00007ffff6d19d0b in JNI_CreateJavaVM_inner (args=0x7ffff5d5ee60,
>>> penv=0x7ffff5d5ee58, vm=0x7ffff5d5ee50) at /local/ws/jdk10/jdk10-hs.clean
>>> /open/src/hotspot/share/prims/jni.cpp:3938
>>> #8  JNI_CreateJavaVM (vm=0x7ffff5d5ee50, penv=0x7ffff5d5ee58,
>>> args=0x7ffff5d5ee60) at /local/ws/jdk10/jdk10-hs.clean
>>> /open/src/hotspot/share/prims/jni.cpp:4033
>>> #9  0x00007ffff7dd89d1 in InitializeJVM () at
>>> /local/ws/jdk10/jdk10-hs.clean/open/src/java.base/share/
>>> native/libjli/java.c:1478
>>> #10 JavaMain () at /local/ws/jdk10/jdk10-hs.clean
>>> /open/src/java.base/share/native/libjli/java.c:411
>>> #11 0x00000034f8a07a51 in start_thread () from /lib64/libpthread.so.0
>>> #12 0x00000034f86e893d in clone () from /lib64/libc.so.6
>>>
>>> Actually would have been more useful to compare pns2() with pns()
>> instead of a gdb backtrace:
>>
>> (gdb) call pns($sp, $rbp, $pc)
>>
>> "Executing pns"
>> Native frames: (J=compiled Java code, A=aot compiled Java code,
>> j=interpreted, Vv=VM code, C=native code)
>> V  [libjvm.so+0x1677a07]  StubRoutines::initialize2()+0x737
>> V  [libjvm.so+0xe62fa3]  init_globals()+0xf3
>> V  [libjvm.so+0x1714453]  Threads::create_vm(JavaVMInitArgs*,
>> bool*)+0x2e3
>> V  [libjvm.so+0xfb9d0b]  JNI_CreateJavaVM+0x9b
>> C  [libjli.so+0x39d1]  JavaMain+0x81
>> C  [libpthread.so.0+0x7a51]
>>
>> I think gdb is making use of inlining info from the .debuginfo file since
>> it has a lot of extra frames.
>>
>>
> I still do not understand - here, the debugger shows me more than both
> pns() and pns2(). Especially, in this case, the real crash point, including
> file/line number. Is that not what you want? So, why do you need to invoke
> pns/pns2() ? Note that this is just curiosity, I still think pns/pns2() can
> be valuable for dumping stacks within the hotspot.
>
> Well pns2() is not even an option within the debugger. I believe the
> advantage of pns() is that it includes java frames.
>
>
> <snip>
>
>>
>>>> On Windows x64, we do not have frame pointers, so we cannot walk the
>>>> frame VM-Style, so os::platform_print_native_stack() is the only way
>>>> to print a stack.
>>>>
>>>> Yes, and I got this to work Windows/x64, but unfortunately no symbols
>>>> are printed.
>>>>
>>>
>>> When you let the Vm crash with e.g. -XX:ErrorHandlerTest=14, does the
>>> resulting hs-err file show a correct callstack with symbols? If not, what
>>> does the dbghelp.dll loader say (the one line below the loaded libraries
>>> section)?
>>>
>>> I'll need to give this a try, but I suspect it will not include symbols.
>>> I recall that you need to run windows stack traces through some special
>>> tool that converts addresses to symbols.
>>>
>>
>> I do not think this is true, at least not to my knowledge. Dbghelp.dll
>> must be installed on the system - it usually is - and you need the pdb
>> files laying beside the binaries. The latter depends on how you build.
>> -with-native-debug-symbols=external will place the pdb files beside the
>> binaries, but =zipped (or so, I am away from my machine atm) will zip them.
>> In zipped form they are unusable and they need to be unzipped before they
>> can be used. At least jvm.pdb, which must be located beside jvm.dll.
>>
>> This might be the issue. I'm using our build+test infrastructure to do
>> this testing, so I'm not in control of how it's built or deployed. I've
>> downloaded the binary bundle used for the testing so I could have a look,
>> and I see that it does not include the symbols. However, there is a
>> separate bundle with just the symbols. I'm unsure if it's being installed
>> for the testing. I'll ask around.
>>
>
>> I am curious about this because I recently changed the coding responsible
>> for the symbol resolution on windows. I‘d like to understand why you do not
>> see symbols, because in our tests everything works.
>>
>> The is from the hs_err file when running TestOnError.java, which uses
>> -XX:ErrorHandlerTest=12:
>>
>> Stack: [0x00000038a0f40000,0x00000038a1040000],  sp=0x00000038a103f740,
>> free space=1021k
>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
>> code)
>> V  [jvm.dll+0xc3255a]
>> V  [jvm.dll+0x7a6198]
>> V  [jvm.dll+0x7a8e8f]
>> C  [java.exe+0x3609]
>> C  [java.exe+0xe83f]
>> C  [java.exe+0xe9e6]
>> C  [KERNEL32.DLL+0x13d2]
>> C  [ntdll.dll+0x154f4]
>>
>> thanks,
>>
>> Chris
>>
>
> Yes, this may be a case of missing infos. I had a discussion with Coleen
> some weeks ago offlist in the context of testing the changes for
> https://bugs.openjdk.java.net/browse/JDK-8185712. I had to jtreg tests
> disable tests which ran nicely here at SAP but would not yield callstacks
> at Oracle. At that time I was wondering if there was something wrong with
> my code but now I am quite certain you guys either remove the debug info
> after building or you build with zipped debug info (which is the default on
> all platforms but AIX). Or both. I'll ask on build.dev.
>
> I've confirmed that it is a known issue that debug symbols are not
> included in our testing (or at least that is the case on Windows). pns2()
> is showing symbols on linux, but that might just be the result of having
> pretty much all the symbols exported, which allows lookup with dladdr().
>
> thanks,
>
> Chris
>

Thanks for confirming that!

..Thomas

>
> Thanks, Thomas
>
>
>