JVM Crashes after few hours

Wed Nov 23 08:38:23 UTC 2016

On 23.11.2016 09:24, Agarwal, Rajesh wrote:
> How can we get the core file?

A core file is automatically generated by the VM if a crash occurs. According to the hs_err file you provided:
"# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again"

So you need to enable core dumping on your machine.

> And is it possible to download the JDK 8 fastdebug VM for linux 64 bit? If yes, please provide the link.

Unfortunately, it's not possible to download a fastdebug VM. You would have to build it on your own or go through the official bug reporting process on http://bugreport.java.com.

Thanks,
Tobias

> 
> Thanks
> Rajesh Agarwal
> 
> 
> -----Original Message-----
> From: Tobias Hartmann [mailto:tobias.hartmann at oracle.com] 
> Sent: Wednesday, November 23, 2016 1:50 PM
> To: Agarwal, Rajesh <Rajesh.Agarwal at snapon.com>; kirk at kodewerk.com
> Cc: hotspot-compiler-dev at openjdk.java.net
> Subject: Re: JVM Crashes after few hours
> 
> 
> On 23.11.2016 03:59, Agarwal, Rajesh wrote:
>> Apart from fastdebug VM, how can be debug the root cause of such issue? Do you think using Java Flight Recorder will help?
> 
> Since this is a bug in the VM, the Java Flight Recorder does not help. We would need a reproducer or at least a core file to investigate. Running with a fastdebug VM may also trigger an assert and generates a replay compilation file.
> 
> Best regards,
> Tobias
> 
>> Thanks
>> Rajesh Agarwal
>>
>>
>> -----Original Message-----
>> From: Tobias Hartmann [mailto:tobias.hartmann at oracle.com]
>> Sent: Wednesday, November 23, 2016 12:08 AM
>> To: Agarwal, Rajesh <Rajesh.Agarwal at snapon.com>; kirk at kodewerk.com
>> Cc: hotspot-compiler-dev at openjdk.java.net
>> Subject: Re: JVM Crashes after few hours
>>
>>
>> On 22.11.2016 19:18, Agarwal, Rajesh wrote:
>>> We are able to reproduce the issue everytime during load testing process of our app.
>>>
>>> Do you see any link with kryo which uses sun.misc.unsafe class for serializing the Java objects?
>>
>> It's very hard to say. The problem with JDK-8050079 was that the customers framework changed the common class hierarchy by replacing java.lang.Object and that triggered a bug in the compiler. This looks very similar so I would guess that this is caused by some unusual class hierarchy.
>>
>> If your time allows, you could try to build and run with a fastdebug VM build and check if we hit an assert. Also, a replay compilation file should be generated.
>>
>> Just out of curiosity, is there a specific reason you are using -XX:-ReduceInitialCardMarks?
>>
>> Best regards,
>> Tobias
>>
>>>
>>> Thanks
>>> Rajesh Agarwal
>>>
>>>
>>>
>>>
>>> On Tue, Nov 22, 2016 at 11:17 PM +0530, "Tobias Hartmann" <tobias.hartmann at oracle.com <mailto:tobias.hartmann at oracle.com>> wrote:
>>>
>>> Sorry, I got confused by the version string. Seems like you are using JDK 8u112 which includes the fix for 8050079.
>>>
>>> Do you happen to have a reproducer for this?
>>>
>>> Best regards,
>>> Tobias
>>>
>>> On 22.11.2016 18:25, Tobias Hartmann wrote:
>>>> Hi,
>>>>
>>>> the stack trace looks like:
>>>> https://bugs.openjdk.java.net/browse/JDK-8050079
>>>>
>>>> This bug was fixed in 8u40 and 9. Could you please check if you are able to reproduce the problem with 8u40 or later?
>>>>
>>>> Thanks,
>>>> Tobias
>>>>
>>>> On 22.11.2016 16:39, Agarwal, Rajesh wrote:
>>>>> Attached is the complete log. 
>>>>>
>>>>> Thanks
>>>>> Rajesh Agarwal
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: kirk at kodewerk.com [mailto:kirk at kodewerk.com]
>>>>> Sent: Tuesday, November 22, 2016 8:55 PM
>>>>> To: Agarwal, Rajesh <Rajesh.Agarwal at snapon.com>
>>>>> Cc: hotspot-compiler-dev at openjdk.java.net
>>>>> Subject: Re: JVM Crashes after few hours
>>>>>
>>>>> Hi,
>>>>>
>>>>> Can we see the rest of the log? This view of the log has cut off important information.
>>>>>
>>>>> Kind regards,
>>>>> Kirk Pepperdine
>>>>>
>>>>>> On Nov 22, 2016, at 8:43 AM, Agarwal, Rajesh <Rajesh.Agarwal at snapon.com> wrote:
>>>>>>
>>>>>> The JVM either crashes or hangs after running for few hours with following error.
>>>>>>
>>>>>> #
>>>>>> # A fatal error has been detected by the Java Runtime Environment:
>>>>>> #
>>>>>> #  SIGSEGV (0xb) at pc=0x00007f93aa58ed32, pid=24048,
>>>>>> tid=0x00007f9368a68700 # # JRE version: Java(TM) SE Runtime 
>>>>>> Environment (8.0_112-b15) (build 1.8.0_112-b15) # Java VM: Java
>>>>>> HotSpot(TM) 64-Bit Server VM (25.112-b15 mixed mode linux-amd64 
>>>>>> compressed oops) # Problematic frame:
>>>>>> # V  [libjvm.so+0x640d32]
>>>>>> InstanceKlass::find_method_index(Array<Method*>*, Symbol*, 
>>>>>> Symbol*, bool, bool)+0x42 # # Failed to write core dump. Core 
>>>>>> dumps have been disabled. To enable core dumping, try "ulimit -c unlimited"
>>>>>> before starting Java again # # If you would like to submit a bug report, please visit:
>>>>>> #   http://bugreport.java.com/bugreport/crash.jsp
>>>>>> #
>>>>>>
>>>>>> ---------------  T H R E A D  ---------------
>>>>>>
>>>>>> Current thread (0x00007f93a4a50800):  JavaThread "C2 CompilerThread0" 
>>>>>> daemon [_thread_in_vm, id=24067,
>>>>>> stack(0x00007f9368968000,0x00007f9368a69000)]
>>>>>>
>>>>>> siginfo: si_signo: 11 (SIGSEGV), si_code: 2 (SEGV_ACCERR), si_addr: 
>>>>>> 0x00007f9326bd3218
>>>>>>
>>>>>> vm_info: Java HotSpot(TM) 64-Bit Server VM (25.112-b15) for
>>>>>> linux-amd64 JRE (1.8.0_112-b15), built on Sep 22 2016 21:10:53 by 
>>>>>> "java_re" with gcc 4.3.0 20080428 (Red Hat 4.3.0-8)
>>>>>>
>>>>>> VM Arguments:
>>>>>> jvm_args: -Xms2560m -Xmx2560m -XX:-UseAdaptiveSizePolicy
>>>>>> -XX:NewRatio=2 -XX:SurvivorRatio=2 -XX:TargetSurvivorRatio=80 
>>>>>> -Xnoclassgc -verbose:gc -XX:+PrintGCDateStamps 
>>>>>> -XX:+HeapDumpOnOutOfMemoryError -XX:-ReduceInitialCardMarks 
>>>>>> -XX:+UseG1GC -XX:MaxGCPauseMillis=400 -XX:CICompilerCount=2 
>>>>>> -XX:+UnlockDiagnosticVMOptions -XX:+LogCompilation Thanks Rajesh 
>>>>>> Agarwal
>>>>>>
>>>>>>
>>>>>