Tips for diagnosing C2 compile crashes?

[Vladimir Kozlov]

I recalled this discussion and since I pushed related changes yesterday 
I want to comment. Better later than never :)

To get the same compilation you need to get the same profiling counters 
information and inlining. And it is difficult with big programs.

An other problem is, even if you can reproduce the problem with product 
version of VM, there is additional difficulties to reproduce and 
investigate the problem in debug VM.

In JDK8 we added new feature ReplayCompiles. When VM crashes in JIT 
compiler it dumps replay_pid123.log file which contains information to 
replay the compilation.

And I added inlining information to replay data with my changes for 
8028468 yesterday into jdk9 Hotspot and I hope to backport it into one 
of 8 updates.

This should help in situation like this. Note, you need to build debug 
VM to replay compilation. But it should not be a problem since Hotspot 
sources are open.

http://openjdk.java.net/

Even if you don't have replay compilation tool (jdk7), it is good idea 
to build fastdebug VM and try to reproduce the problem with it. Debug VM 
has a lot of asserts which may help.

Crash in build_loop_late_post() may happen for a lot of reasons which 
produce incorrect ideal IR graph and cause the crash. That is why it is 
so common.

Regards,
Vladimir

On 7/26/13 2:10 AM, Volker Simonis wrote:
> On Thu, Jul 25, 2013 at 4:33 PM, Jørgen Austvik
> <Jorgen.Austvik at kantega.no> wrote:
>> Hi,
>>
>> we have a problem where C2 SIGSEGVs in PhaseIdealLoop::build_loop_late_post(Node*) seen in 1.7.0_17-b02 and 1.7.0_25-b15 on linux-amd64. When I google sigsegv build_loop_late_post, I see that I am not alone (play framework community seems to be hit with some fascinating cargo cult workarounds).
>>
>> V  [libjvm.so+0x6835b4]  PhaseIdealLoop::build_loop_late_post(Node*)+0x154
>> V  [libjvm.so+0x683b1c]  PhaseIdealLoop::build_loop_late(VectorSet&, Node_List&, Node_Stack&)+0x13c
>> V  [libjvm.so+0x68ab07]  PhaseIdealLoop::build_and_optimize(bool, bool)+0x7d7
>> V  [libjvm.so+0x3b805a]  Compile::Optimize()+0xbea
>> V  [libjvm.so+0x3b8f5c]  Compile::Compile(ciEnv*, C2Compiler*, ciMethod*, int, bool, bool)+0xdac
>> V  [libjvm.so+0x32ae52]  C2Compiler::compile_method(ciEnv*, ciMethod*, int)+0x142
>>
>> It always crashes (on different linux distributions/machines/JVM versions) on the same method (from jetty-util-8.1.8.v20121106.jar):
>>
>> Current CompileTask:
>> C2:687410906   66 %           org.eclipse.jetty.util.URIUtil::canonicalPath @ 26 (633 bytes)
>>
>
> The "%" indicates that it is an "OnStackReplacment" compilation
>
>> Since this was a public static method I naively thought I could reproduce the crash with a simple for-loop, the same JVM on the same host with the same library version - and with -XX:+PrintCompilation I could see that the method was compiled, but it didn't crash.
>>
>> OK, so maybe the method had to be in its "natural environment" I thought, and started up the server process that uses Jetty, and ran our standard performance test on it. It crashed beautifully with hs_err and core the first time I ran it, so I wanted to see if it could be reproduces. I started again, saw that canonicalPath() was compiled, but it doesn't crash. I have tried to run the load against the server while restarting it every 2 minutes to trigger compilations, and it doesn't crash, so I think the crashes must be depending on how the compilation happens (the statistical information collected could trigger different compilations?).
>>
>
> Is it compiled as OSR again (i.e. does PrintCompilation still report
> "%") when it does not crash?
>
>> Q: Is there any way to trigger different compilations of that method so that I can try to figure out which compilation setting/methods that causes the SIGSEGV?
>>
>
> You could try to play around with the following options:
>
>    product_pd(intx, CompileThreshold,                                        \
>            "number of interpreted method invocations before (re-)compiling") \
>                                                                              \
>    product_pd(intx, BackEdgeThreshold,                                       \
>            "Interpreter Back edge threshold at which an OSR compilation
> is invoked")\
>
> For x86 they have the following default values:
>
> define_pd_global(intx, CompileThreshold,             10000);
> define_pd_global(intx, BackEdgeThreshold,            100000);
>
> Another problem may be that the inlining is different. In the case
> where it does not crash you can see what has been inlined with the
> dignostic option "-XX:+PrintInlining" option. Unfortunately you won't
> see what’s different for the crashing case because it will crash
> before it can print that information:)
>
>> Best Regards
>> Jørgen Austvik
>>
>> --
>> Kantega AS
>> Erling Skakkes gt. 25, 7013 Trondheim
>> E-post: jorgen.austvik at kantega.no
>> Mobil: +47 90 19 78 86
>> Sentralbord: +47 73 80 68 00
>> Fax: +47 73 80 68 01
>> www.kantega.no
>>
>>
>>
>>