Fwd: Java stack unwinder for Linux + gdb7.10+

Tue May 31 16:25:31 UTC 2016

Mikael,

> I saw it, tried it and it worked after replacing a bunch of
> -        except Exception, arg:
> +        except Exception as arg:

Python 2.x (pre 2.7) requires Exception, arg
Python 3.x requires Exception as arg

Python 2.7 allow both syntax.

-Dmitry

On 2016-05-31 16:13, Mikael Gerdin wrote:
> Hi Andrew,
> 
> On 2016-05-31 15:09, Andrew Dinn wrote:
>> I probably should have sent this here initially rather than to jdk9-dev.
>> Anyway, I hope subscribers to this list will find this interesting.
> 
> I saw it, tried it and it worked after replacing a bunch of
> -        except Exception, arg:
> +        except Exception as arg:
> 
> and adjusting the sources to account for jdk 9 changes which I don't yet
> have in my project workspace.
> 
> I think it might make more sense to put this in the hotspot repo,
> similar to the serviceability agent, since it's functionality is tied to
> a particular revision of the sources.
> 
> /Mikael
> 
>>
>> regards,
>>
>>
>> Andrew Dinn
>> -----------
>>
>> -------- Forwarded Message --------
>> Subject: Java stack unwinder for Linux + gdb7.10+
>> Date: Tue, 31 May 2016 12:02:50 +0100
>> From: Andrew Dinn <adinn at redhat.com>
>> To: jdk9-dev at openjdk.java.net <jdk9-dev at openjdk.java.net>
>> CC: code-tools-dev at openjdk.java.net
>>
>> What's this all about?
>> ----------------------
>>
>> Recently, I have been working on implementing support for unwinding Java
>> stacks in gdb using the new python stack unwinder API provided from gdb
>> 7.10.1 onwards. I have managed to provide a pure python solution i.e.
>> one that that needs no modifications to the VM itself. So, if you use
>> gdb to debug operation of the JVM then this implementation will allow
>> you to use the bt command as $DEITY (and, indeed, the gdb team) intended.
>>
>> Attached at the bottom of this note is a before and after showing how
>> backtraces look without/with this package installed.
>>
>> Note that the unwinder works when using gdb to debug either a live
>> process or a core file.
>>
>> Also, note that at present gdb only uses python unwinders to display
>> frame info underneath the bt command. I am working with the gdb devs to
>> see if they can extend the use of the unwinder to other situations e.g.
>> to display the current frame when single-stepping.
>>
>> Can I try it out?
>> -----------------
>>
>> Yes, please do!
>>
>> Indeed, I am posting this to jdk9-dev because I want as many people as
>> possible to try out the unwinder and provide feedback on how it works.
>> So, please use it and respond -- positive or negative I would love to
>> hear what you think. I'm happy to hear offline or on this list. You
>> choose whether or not you think others on this list might want to hear
>> your comments.
>>
>> How do I obtain/use it?
>> -----------------------
>>
>> The unwinder code is available from the following hg repo
>>
>>    http://icedtea.classpath.org/people/adinn/unwinder
>>
>> See the README in that repo for details of how to install and use it
>> (gdb will auto-load it once you have configured things correctly)
>>
>> To summarize some important details:
>>
>> The extension requires gdb 7.10+ and you there are two unwinder
>> implementations, one for jdk8 and one for jdk9.
>>
>>    The jdk8 unwinder should work for any jdk8 built from the jdk8u repo.
>> In particular, it will work for /product releases/ available with your
>> Linux distro so long as the distro supports installation of minimal
>> debuginfo for those releases. For example, if you are using RHEL or
>> fedora then you can simply debuginfo-install the debug info
>> corresponding to your jdk8 java package and then install the uinwinder
>> under /etc/alternatives/java_sdk_1.8.0. If you want to use it with your
>> own jdk8 build then you need to build a debug release which includes a
>> libjvm.debuginfo product (well, I guess you could always hack the
>> product make process so it generates libjvm.debuginfo ).
>>
>>    The jdk9 unwinder will only work for your own debug builds and even
>> then only for builds which include the  b119 tag. It doesn't work for
>> product builds because  ... well, there are no product builds available
>> from any distros just yet which, a fortiori, means there are no product
>> builds where you can install minimal debug info. Unfortunately, that
>> includes the currently available early access program builds which don't
>> ship libjvm.debuginfo.
>>
>> Ok, so why also the b119 restriction? That's because between b118 and
>> b119 the stack and code heap layouts changed and this required the
>> unwinder code to change accordingly. An older version of the unwinder
>> did work with b118 and earlier builds but tempus fugit, hey ho ....
>>
>>
>> How does it do that?
>> --------------------
>>
>> For those who are curious the python code is your friend (apologies for
>> my python noobness -- I mean, eeeuurrrggghhh, python :-). Basically, the
>> unwinder works without change to the JVM because it understands a few
>> details of how:
>>
>>    the code cache is laid out
>>    code blobs/methods are organised
>>    bci-to-line number maps are organized
>>    code address-to-bci maps are organized
>>
>> So, essentially, it uses the gdb inferior access functions to poke
>> around in JVM memory, allowing it to work out what the state of the
>> stack is and what method, file, bci and line number a specific
>> instruction address corresponds to. That includes using gdb to resolve a
>> few symbols in order to identify memory layouts and a few important
>> global addresses.
>>
>> This dependency has been kept to a bare minimum so that the unwinder
>> will be as resilient as possible to changes in the JVM.
>>
>> Can we have this in OpenJDK?
>> ----------------------------
>>
>> The current code is LGPL licensed. I'd be very happy to contribute it to
>> OpenJDK as part of the code tools set for JDK developers to use and Red
>> Hat would be very happy to maintain it once included. Ideally, Red Hat
>> would like to ship the tool as part of OpenJDK product releases for use
>> in any distro where minimal debug info is available with the product
>> release. We see great benefit in the opportunity it provides for support
>> staff to print stack traces from core files (clearly, shipping with
>> OpenJDK would help to ensure that changes in the JVM and required
>> changes in the python code were able to be kept in sync).
>>
>> I have copied this note to the code-tools-dev list on the assumption
>> that it is the correct place to open up discussion of whether it
>> could/should be provided as part of OpenJDK. If not can someone please
>> let me know where else I should start that discussion.
>>
>> regards,
>>
>>
>> Andrew Dinn
>> -----------
>> Senior Principal Software Engineer
>> Red Hat UK Ltd
>> Registered in England and Wales under Company Registration No. 03798903
>> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander
>>
>> ----- 8< -------- 8< -------- 8< -------- 8< -------- 8< -------- 8< ---
>>
>> Normal gdb backtrace for java thread fails ot unwind through Java frames
>>
>> (gdb) thread 16
>> [Switching to thread 16 (Thread 0x7fffa1fff700 (LWP 4930))]
>> #0 0x00007fffe1113484 in ?? ()
>> (gdb) bt 3
>> #0  0x00007fffe1113484 in ?? ()
>> #1  0x00007fffd3c00c18 in ?? ()
>> #2  0x0000000000000000 in ?? ()
>>
>> Backtrace with the unwinder installed
>>
>> (gdb) thread 2
>> [Switching to thread 2 (Thread 0x7ffff7fcc700 (LWP 4906))]
>> #0  0x00007ffff79b6540 in pthread_cond_wait@@GLIBC_2.3.2 ()
>>     from /lib64/libpthread.so.0
>> (gdb) bt 10
>> #0  0x00007ffff79b6540 in pthread_cond_wait@@GLIBC_2.3.2 ()
>>      at /lib64/libpthread.so.0
>> #1  0x00007ffff64ae82d in os::PlatformEvent::park() (this=0x7ffff0019700)
>>      at
>> /home/adinn/redhat/openjdk/jdk9/hs/hotspot/src/os/linux/vm/os_linux.cpp:5582
>>
>> #2  0x00007ffff6486f59 in ObjectMonitor::wait(long, bool, Thread*)
>> (this=0x7fff90008b80, millis=0, interruptible=true,
>> __the_thread__=0x7ffff0018800)
>>      at
>> /home/adinn/redhat/openjdk/jdk9/hs/hotspot/src/share/vm/runtime/objectMonitor.cpp:1490
>>
>> #3  0x00007ffff66518cb in ObjectSynchronizer::wait(Handle, long,
>> Thread*) (obj=..., millis=0, __the_thread__=0x7ffff0018800)
>>      at
>> /home/adinn/redhat/openjdk/jdk9/hs/hotspot/src/share/vm/runtime/synchronizer.cpp:496
>>
>> #4  0x00007ffff618fa54 in JVM_MonitorWait(JNIEnv*, jobject, jlong)
>> (env=0x7ffff0018a28, handle=0x7ffff7fcb738, ms=0)
>>      at
>> /home/adinn/redhat/openjdk/jdk9/hs/hotspot/src/share/vm/prims/jvm.cpp:614
>> #5  0x00007fffd8924f7d in [interpreted: bc = 0]
>> java.lang.Object.wait(long) ()
>>      at java/lang/Object.java
>> #6  0x00007fffd89034a3 in [interpreted: bc = 38]
>> java.lang.Thread.join(long) () at java/lang/Thread.java:1352
>> #7  0x00007fffd89034a3 in [interpreted: bc = 2]
>> java.lang.Thread.join() ()
>>      at java/lang/Thread.java:1426
>> #8  0x00007fffd89034a3 in [interpreted: bc = 35]
>> HelloV.main(java.lang.String) () at HelloV.java:80
>> #9  0x00007fffd88f89f1 in StubRoutines (1) ()
>> #10 0x00007ffff6109bb3 in JavaCalls::call_helper(JavaValue*,
>> methodHandle const&, JavaCallArguments*, Thread*)
>> (result=0x7ffff7fcbd30, method=..., args=0x7ffff7fcbc10,
>> __the_thread__=0x7ffff0018800)
>>      at
>> /home/adinn/redhat/openjdk/jdk9/hs/hotspot/src/share/vm/runtime/javaCalls.cpp:409
>>
>> (gdb) thread 26
>> [Switching to thread 26 (Thread 0x7fffa1fff700 (LWP 4930))]
>> #0  0x00007fffe03cb664 in ?? ()
>> (gdb) bt 5
>> #0  0x00007fffe03cb664 in [compiled offset = 0x3a4] HelloV$1.run() ()
>>      at HelloV.java:63
>> #1  0x00007fffd88f89f1 in StubRoutines (1) ()
>> #2  0x00007ffff6109bb3 in JavaCalls::call_helper(JavaValue*,
>> methodHandle const&, JavaCallArguments*, Thread*)
>> (result=0x7fffa1ffecc0, method=..., args=0x7fffa1ffec00,
>> __the_thread__=0x7ffff0471800)
>>      at
>> /home/adinn/redhat/openjdk/jdk9/hs/hotspot/src/share/vm/runtime/javaCalls.cpp:409
>>
>> #3  0x00007ffff64ad409 in os::os_exception_wrapper(void (*)(JavaValue*,
>> methodHandle const&, JavaCallArguments*, Thread*), JavaValue*,
>> methodHandle const&, JavaCallArguments*, Thread*) (f=0x7ffff610951a
>> <JavaCalls::call_helper(JavaValue*, methodHandle const&,
>> JavaCallArguments*, Thread*)>, value=0x7fffa1ffecc0, method=...,
>> args=0x7fffa1ffec00, thread=0x7ffff0471800)
>>      at
>> /home/adinn/redhat/openjdk/jdk9/hs/hotspot/src/os/linux/vm/os_linux.cpp:5128
>>
>> #4  0x00007ffff6109502 in JavaCalls::call(JavaValue*, methodHandle
>> const&, JavaCallArguments*, Thread*) (result=0x7fffa1ffecc0, method=...,
>> args=0x7fffa1ffec00, __the_thread__=0x7ffff0471800)
>>      at
>> /home/adinn/redhat/openjdk/jdk9/hs/hotspot/src/share/vm/runtime/javaCalls.cpp:298
>>
>> #5  0x00007ffff610877b in JavaCalls::call_virtual(JavaValue*,
>> KlassHandle, Symbol*, Symbol*, JavaCallArguments*, Thread*)
>> (result=0x7fffa1ffecc0, spec_klass=..., name=0x7ffff4031338,
>> signature=0x7ffff4033220, args=0x7fffa1ffec00,
>> __the_thread__=0x7ffff0471800)
>>      at
>> /home/adinn/redhat/openjdk/jdk9/hs/hotspot/src/share/vm/runtime/javaCalls.cpp:193
>>
>> (gdb)
>>
>>
>>
>>

-- 
Dmitry Samersoff
Oracle Java development team, Saint Petersburg, Russia
* I would love to change the world, but they won't give me the source code.