JVM hangs beyond recovery

Fri Apr 16 16:30:16 PDT 2010

You may want to use kill -3 to this process and it should print the 
thread dump to standard out for the java process.
You can also use jstack <pid> from $JAVA_HOME/bin to achieve the same 
dump.  jstack has an -m option which will show native frames as well.
Both of these have lock deadlock detection so might show you where your 
Java program is hung.

Coleen

Y. Srinivas Ramakrishna wrote:
> On 04/16/10 15:51, Stas Oskin wrote:
>   
>> Hi.
>>
>> [Make sure to provide them with the JDK version string.]
>>
>> Right, totally forgot.
>> The Java version is:
>> java version "1.6.0_16"
>> Java(TM) SE Runtime Environment (build 1.6.0_16-b01)
>> Java HotSpot(TM) 64-Bit Server VM (build 14.2-b01, mixed mode)
>>     
>
> I suggest using 6u18 (with -XX:-ReduceInitialCardMarks as in release
> notes; this is independent of the hang you are seeing) to see
> if the hang reproduces; just on the off-chance this may have been
> fixed in the interim (i must state that such a hang does not sound
> familiar to me though). Or if you have a reproducible test
> case that would be best. If you have a support contract (and even
> if you don't), you should probably file a bug (through support, if
> you have a support contract), including the test case.
>
>   
>> Running on Linux x86_64.
>>
>>
>> The most import point in my opinion, is that JVM both compiles the Java 
>> code in two concurrent threads, and runs JNI code at a same time. So I 
>> would if this might be what causing the hangs?
>>     
>
> I don't think so. The compiler threads have stopped for the safepoint
> (see below), and (JNI) native code (outside of the JVM) is allowed to run.
>
> Thread 29 (Thread 0x40fe6940 (LWP 20859)):
> #0  0x0000003c7160ad09 in pthread_cond_wait@@GLIBC_2.3.2 ()
>     from /lib64/libpthread.so.0
> #1  0x00002ba3704859f7 in os::PlatformEvent::park ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #2  0x00002ba37045e6f9 in Monitor::ILock ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #3  0x00002ba37045ee30 in Monitor::lock_without_safepoint_check ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #4  0x00002ba3704efbfa in SafepointSynchronize::block ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #5  0x00002ba37056c89b in JavaThread::check_safepoint_and_suspend_for_native_trans () from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #6  0x00002ba37012f642 in ThreadStateTransition::transition_from_native ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #7  0x00002ba37015833b in BufferBlob::create ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #8  0x00002ba37016bbd9 in Compile::init_scratch_buffer_blob ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #9  0x00002ba37048ab2c in Compile::Output ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #10 0x00002ba370170d4d in Compile::Code_Gen ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #11 0x00002ba37016d364 in Compile::Compile ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #12 0x00002ba370101dee in C2Compiler::compile_method ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #13 0x00002ba370176028 in CompileBroker::invoke_compiler_on_method ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #14 0x00002ba3701759c6 in CompileBroker::compiler_thread_loop ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #15 0x00002ba3705711e9 in compiler_thread_entry ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #16 0x00002ba37056acb1 in JavaThread::run ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #17 0x00002ba37048626f in java_start ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #18 0x0000003c71606617 in start_thread () from /lib64/libpthread.so.0
> #19 0x0000003c70ad3c2d in clone () from /lib64/libc.so.6
>
> Thread 28 (Thread 0x416ab940 (LWP 20860)):
> #0  0x0000003c7160ad09 in pthread_cond_wait@@GLIBC_2.3.2 ()
>     from /lib64/libpthread.so.0
> #1  0x00002ba3704859f7 in os::PlatformEvent::park ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #2  0x00002ba37045e6f9 in Monitor::ILock ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #3  0x00002ba37045ee30 in Monitor::lock_without_safepoint_check ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #4  0x00002ba3704efbfa in SafepointSynchronize::block ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #5  0x00002ba37056c89b in JavaThread::check_safepoint_and_suspend_for_native_trans () from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #6  0x00002ba37012b066 in ciObject::klass ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #7  0x00002ba370441fb8 in LoadNode::Value ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #8  0x00002ba3704b814b in PhaseIterGVN::transform_old ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #9  0x00002ba3704b7cad in PhaseIterGVN::optimize ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #10 0x00002ba37042bbc9 in PhaseMacroExpand::expand_macro_nodes ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #11 0x00002ba37017062e in Compile::Optimize ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #12 0x00002ba37016d313 in Compile::Compile ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #13 0x00002ba370101dee in C2Compiler::compile_method ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #14 0x00002ba370176028 in CompileBroker::invoke_compiler_on_method ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #15 0x00002ba3701759c6 in CompileBroker::compiler_thread_loop ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #16 0x00002ba3705711e9 in compiler_thread_entry ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #17 0x00002ba37056acb1 in JavaThread::run ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #18 0x00002ba37048626f in java_start ()
>     from /usr/java/jdk1.6.0_16/jre/lib/amd64/server/libjvm.so
> #19 0x0000003c71606617 in start_thread () from /lib64/libpthread.so.0
> #20 0x0000003c70ad3c2d in clone () from /lib64/libc.so.6
>
>
> I am not familiar with this part of the runtime or its debugging,
> so I will copy the alias and let someone more familiar with such
> issues to help you. (Recall also the suggestions for the
> [Print]Safepoint[Statistics|[Timeout*]] flags in previous email
> to help with debugging.)
>
> hope that helps; 'bye!
> -- ramki
>
>   
>> Thanks again.
>>     
>
>   