C2 Crash in Chunk/Arena - but with fine registers?

[Bernd Eckenfels]

Hello,

I have the Problem of randomly crashing OpenJDK JVMs (in my case Azul Zulu 8). Azul support was trying to help me, but the Problem seems to be elusive.

The below happens in the C2 compiler while VM warmup (but very sporadic), but I expect it to be a more General Problem, so I am hoping somebody with Debugger black belt  can explain what we see.

One Thing we noticed is, that if you look at the crashdump, the PC and the Register Content do not match up:

Stack with Debugging Symbols:

# jvm!Chunk::operator new
# jvm!Arena::grow+0x3c
# jvm!OopFlow::make+0xb4
# jvm!Compile::BuildOopMaps+0x381
# jvm!Compile::Output+0x3b4
# jvm!Compile::Code_Gen+0x422
# jvm!Compile::Compile+0xbfe
# jvm!C2Compiler::compile_method+0xa2
# jvm!CompileBroker::invoke_compiler_on_method+0x488
# jvm!CompileBroker::compiler_thread_loop+0x2ef
# jvm!JavaThread::thread_main_inner+0xaf
# jvm!java_start+0x8a
# msvcr100!_callthreadstartex+0x17
# msvcr100!_threadstartex+0x7f
# kernel32!BaseThreadInitThunk+0xd
# ntdll!RtlUserThreadStart+0x1d

Operation under PC

# jvm!Chunk::operator new+0 [zulu-src\hotspot\src\share\vm\memory\allocation.cpp @ 375]
# 00000000`64fb9180 48895c2410      mov     qword ptr [rsp+10h],rbx

But the Registers in the exception record suggest they do not Point into page Zero/address 0x40:

# EXCEPTION_RECORD:  (.exr -1)
# ExceptionAddress: 0000000064fb9180 (jvm!Chunk::operator new)
# ExceptionCode: c0000005 (Access violation)
# ExceptionFlags: 00000000
# NumberParameters: 2
#  Parameter[0]: 0000000000000000
#  Parameter[1]: 0000000000000040
# Attempt to read from address 0000000000000040

# Registers:
# RBX=0x000000002ec17da0
# RSP=0x000000003025c4b8 is pointing into the stack for thread: 0x000000002ee08800

Did you ever seen that? Is that a Problem of the Dump or can that be some microcode Problems related to Spectre (I feel it happens more often since BIOS update).


Rest of the error log.

Current thread (0x000000002ee08800):  JavaThread "C2 CompilerThread0" daemon [_thread_in_native, id=2212, stack(0x0000000030160000,0x0000000030260000)]

siginfo: ExceptionCode=0xc0000005, reading address 0x0000000000000040

Registers:
RAX=0x0000000000000000, RBX=0x000000002ec17da0, RCX=0x0000000000000010, RDX=0x0000000000000000
RSP=0x000000003025c4b8, RBP=0x00000000000007b0, RSI=0x0000000000007fd8, RDI=0x0000000032976e10
R8 =0x0000000000007fd8, R9 =0x0000000000000002, R10=0x0000000000000010, R11=0x0000000000000002
R12=0x00000000000000f5, R13=0x000000003025eb00, R14=0x0000000032785930, R15=0x0000000030d824a0
RIP=0x0000000064fb9180, EFLAGS=0x0000000000010297

Top of Stack: (sp=0x000000003025c4b8)
0x000000003025c4b8:   0000000064fb970c 00000000329777f0
0x000000003025c4c8:   0000000000000030 00000000000000f4
0x000000003025c4d8:   000000003025c740 00000000000007b0
0x000000003025c4e8:   0000000065334e64 000000002ec17da0
0x000000003025c4f8:   000000003297e470 000000003297e660
0x000000003025c508:   000000003025eb00 0000000000000008
0x000000003025c518:   000000003025eb00 0000000030d823b0
0x000000003025c528:   00000000653360c1 0000000031288f50
0x000000003025c538:   000000003025c5a8 000000003297e440
0x000000003025c548:   00000000325ff120 000000002ec17da0
0x000000003025c558:   00000000325ff120 000000002ec17da0
0x000000003025c568:   00000000000001e8 00000000000007a0
0x000000003025c578:   0000000000000080 000000002ec17da0
0x000000003025c588:   0000000032aedb10 0000000000000046
0x000000003025c598:   000000002ee08800 000000002f007ab0
0x000000003025c5a8:   00000000654bc3a8 000000003025eb00 

Instructions: (pc=0x0000000064fb9180)
0x0000000064fb9160:   00 00 e8 29 ff ff ff 48 8b 0d 8a 8d 71 00 ba 05
0x0000000064fb9170:   00 00 00 48 83 c4 28 e9 14 ff ff ff cc cc cc cc
0x0000000064fb9180:   48 89 5c 24 10 57 48 83 ec 60 48 8d 59 0f 8b fa
0x0000000064fb9190:   48 83 e3 f0 49 03 d8 49 81 f8 d8 00 00 00 0f 84 


Register to memory mapping:

RAX=0x0000000000000000 is an unknown value
RBX=0x000000002ec17da0 is an unknown value
RCX=0x0000000000000010 is an unknown value
RDX=0x0000000000000000 is an unknown value
RSP=0x000000003025c4b8 is pointing into the stack for thread: 0x000000002ee08800
RBP=0x00000000000007b0 is an unknown value
RSI=0x0000000000007fd8 is an unknown value
RDI=0x0000000032976e10 is an unknown value
R8 =0x0000000000007fd8 is an unknown value
R9 =0x0000000000000002 is an unknown value
R10=0x0000000000000010 is an unknown value
R11=0x0000000000000002 is an unknown value
R12=0x00000000000000f5 is an unknown value
R13=0x000000003025eb00 is pointing into the stack for thread: 0x000000002ee08800
R14=0x0000000032785930 is an unknown value
R15=0x0000000030d824a0 is an unknown value


Stack: [0x0000000030160000,0x0000000030260000],  sp=0x000000003025c4b8,  free space=1009k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [jvm.dll+0xa9180]
V  [jvm.dll+0xa970c]
V  [jvm.dll+0x424e64]
V  [jvm.dll+0x4260c1]
V  [jvm.dll+0x4cb8c4]
V  [jvm.dll+0x442192]
V  [jvm.dll+0x44492e]
V  [jvm.dll+0x427952]
V  [jvm.dll+0x82148]
V  [jvm.dll+0x82a7f]
V  [jvm.dll+0x21e16f]
V  [jvm.dll+0x26f9ba]
C  [msvcr100.dll+0x21d9f]
C  [msvcr100.dll+0x21e3b]
C  [KERNEL32.DLL+0x16ad]
C  [ntdll.dll+0x74629]

I also have the minidump if needed (I dont have the Symbols Right now)

Gruss
Bernd
-- 
http://bernd.eckenfels.net