Need help on debugging JVM crash

[Andrew Haley]

On 3/3/20 5:39 PM, Sundara Mohan M wrote:
>> Questions
>> 1. When i looked at source code for printing stack trace i see followinghttps://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L696
>> (Prints native stack trace)https://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L718
>> (printing Java thread stack trace if it is involved in GC crash)
>>   a. How do you know this java thread was involved in jvm crash?

The top thread -- the first in the file -- is the one that crashed.

>> When GC processes thread stack as root, the java thread first was
>> recorded. This is why at crash, the java thread was printed out.
>>
>>   b. Can i assume the java thread printed after native stack trace was the
>> culprit?

Certainly not.

>> Please check this thread stack frames, when GC is doing marking work, I
>> think, it encountered a bad oop. Check:
>>
>> If it is a compiled frame, if so, it may related to compiled code.
>>
>>   c. Since i am seeing the same frame (~RuntimeStub::_new_array_Java, J
>> 54174 c2 ch.qos.logback.classic.spi.ThrowableProxy.<init>..) but different
>> stack trace in both crashes can this be the root cause?
>>
>> It is a C2 compiled frame. The bad oop could be a result of compiler.
>>
> Actually the top two frame are always same in different crashes
> v ~RuntimeStub::_new_array_Java
> J 54174 c2
> ch.qos.logback.classic.spi.ThrowableProxy.<init>(Ljava/lang/Throwable;)V
> (207 bytes) @ 0x00007f6687d92678 [0x00007f6687d8c700+0x0000000000005f78]
> In this case do you think JVM code(frame 1) or C2 compiler code(frame 2)
> might be issue?

Probably not. My money would be on a bad library using Unsafe to do
something unwise. But there are many other possibilities.

> Is there any way to identify that and what kind of debug flags/settings
> might give us this information?
>
>> It also needs detail debug information to make the conclusion.
>>
> Do you think any of the information dumped in hs_err* file might give us
> more info (like registers content/Instructions/core file)?
>
> Can you please let me know what additional details might help to make the
> conclusion? Also how to get those information?

Let's see the complete hs_err file.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671