What does BufferBlob::Interpreter in a JVM crash log mean

[Martin Hare Robertson]

Thanks for the tip. I should have mentioned that I already spotted that option. However, I am concerned that disabling biased locking will dramatically impact perf. 

I am also really keen to understand what the BufferBlob::Interpreter line actually means. 

On 14 Nov 2011, at 17:33, Vitaly Davidovich <vitalyd at gmail.com> wrote:

> You can try disabling biased locking, -XX:-UseBiasedLocking, as a possible workaround.
> 
> On Nov 14, 2011 11:01 AM, "Martin Hare Robertson" <mchr3k at gmail.com> wrote:
> This mailing list looks like the best bet to get an answer to my question. However, please let me know if I should be posting this somewhere more appropriate.
> I am investigating a JVM crash which happens occasionally in my application. The hs_err file contains the following details about the crash.
> 
> #  SIGSEGV (0xb) at pc=0x065e68f4, pid=20208, tid=570166160
> 
> 
> 
> #
> # Java VM: Java HotSpot(TM) Server VM (10.0-b23 mixed mode linux-x86)
> 
> 
> 
> ...
> 
> # Problematic frame:
> # V  [libjvm.so+0x5e68f4]
> 
> 
> 
> ...
> 
> Current thread (0x099ea800):  JavaThread "Thread-315" daemon [_thread_in_vm, id=25782, stack(0x21fa3000,0x21fc1000)]
> 
> 
> 
> ...
> 
> vm_info: Java HotSpot(TM) Server VM (10.0-b23) for linux-x86 JRE (1.6.0_07-b06), built on Jun 10 2008 01:20:15 by "java_re" with gcc 3.2.1-7a (J2SE release)
> 
> 
> 
> So this tells me that the JVM hit a segfault when running some Java code. The error log also contains information about the stack of the thread which crashed.
> 
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
> 
> 
> 
> V  [libjvm.so+0x5e68f4]
> 
> V  [libjvm.so+0x1c054f]
> 
> V  [libjvm.so+0x1bfef2]
> 
> V  [libjvm.so+0x1bf57f]
> 
> V  [libjvm.so+0x592495]
> 
> V  [libjvm.so+0x365c4e]
> 
> v  ~BufferBlob::Interpreter
> v  ~BufferBlob::Interpreter
> 
> v  ~BufferBlob::Interpreter
> v  ~BufferBlob::Interpreter
> 
> v  ~BufferBlob::Interpreter
> 
> Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
> 
> 
> 
> v  ~BufferBlob::Interpreter
> v  ~BufferBlob::Interpreter
> 
> v  ~BufferBlob::Interpreter
> v  ~BufferBlob::Interpreter
> 
> v  ~BufferBlob::Interpreter
> J  org.myapp.AppClass.getBytes()Lorg/myapp/ByteHolder;
> 
> 
> 
> I have used GDB to connect to the core file from the crash and get more detail about the stack. This gives me the following output.
> 
> #5  <signal handler called>
> 
> #6  0x065e68f4 in interpretedVFrame::monitors() const ()
> 
> 
> 
>    from /usr/java/jdk1.6.0_07/jre/lib/i386/server/libjvm.so
> 
> 
> 
> #7  0x061c054f in get_or_compute_monitor_info(JavaThread*) ()
> 
>    from /usr/java/jdk1.6.0_07/jre/lib/i386/server/libjvm.so
> 
> 
> 
> #8  0x061bfef2 in revoke_bias(oopDesc*, bool, bool, JavaThread*) ()
> 
> 
> 
>    from /usr/java/jdk1.6.0_07/jre/lib/i386/server/libjvm.so
> 
> 
> 
> #9  0x061bf57f in BiasedLocking::revoke_and_rebias(Handle, bool, Thread*) ()
> 
> 
> 
>    from /usr/java/jdk1.6.0_07/jre/lib/i386/server/libjvm.so
> 
> 
> 
> #10 0x06592495 in ObjectSynchronizer::fast_enter(Handle, BasicLock*, bool, Thread*) ()
> 
> 
> 
>    from /usr/java/jdk1.6.0_07/jre/lib/i386/server/libjvm.so
> 
> 
> 
> #11 0x06365c4e in InterpreterRuntime::monitorenter(JavaThread*, BasicObjectLock*) ()
> 
> 
> 
>    from /usr/java/jdk1.6.0_07/jre/lib/i386/server/libjvm.so
> 
> 
> 
> This shows that the six libjvm.so frames listed in the original bug report were related to grabbing a Java lock. However, I can't find any code within org.myapp.AppClass.getBytes() which uses any locks.
> 
> ___
> 
> 1) What do the BufferBlob::Interpreter lines in the stack mean? 
> 
> - Are these Java stack frames? 
> 
> - JVM stack frames? 
> 
> - Is it possible to work out what was being called in these stack frames?
> 
> ___
> 
> NOTE: Please don't suggest that I try to switch to a newer Hotspot JVM. I rely on the CMS collector and none of the more recent V1.6 Hotspot JVMs have been stable enough with the CMS collector in the testing which I have done.
> 
> org.myapp.AppClass.getBytes() reads from a DataInputStream. This could produce the following stack trace:
> AppClass.getBytes()
> AppClass.readByte()
> 
> DataInputStream.readByte()
> SocketInputStream.read()
> 
> SocketInputStream.read(byte[],int,int)
> 
> PlainSocketImpl.aquireFD()
> This final method grabs a lock. This could be the source of the eventual call into the JVM code listed above. This stack above has the convenient feature that there are 5 Java stack frames below getBytes(). This would match up neatly with the 5 lines of BufferBlob::Interpreter in the list of "Java frames".
> 
> ___
> 
> 2) Could these 5 BufferBlob::Interpreter frames correspond to the Java stack trace I identified above? Is it possible that the 5 lines of BufferBlob::Interpreter under the "Native frames" section refer to the same stack frames as the same lines under the "Java frames" section?
> 
> ___
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/attachments/20111114/8f64de5c/attachment.html