[9] RFR for JDK-8155004: CrashOnOutOfMemoryError doesn't work for OOM caused by inability to create threads'

[Shafi Ahmad]

Hi Mikael,

Thanks for reviewing it. 

Once VM is initialized, following are two OOME scenarios:
 1) OOME due to unavailability of java memory [Mainly due to java application].
 2) OOME due to unavailability of native memory.

Please correct me if I am wrong, for 1) java.lang.OutOfMemoryError is correct.

Consider the following scenarios:
1) Let there is java application which uses JNI code and inside JNI code there is native memory allocation/free and we hit OOME. 
2) Let there is java application which uses JNI code and inside JNI code there is memory leak error and due to this OOME situation occurs.
3) We use jvm option Xms and  -Xmx in such a way that the available native memory is very less and VM hit OOME.

I am not sure above scenario is feasible in JVM or not but if any of the above scenario is possible in VM then should we consider it as OOME due java application or not?
I consider case 1) and 2) as OOME due to java application and added code for java.lang.OutOfMemoryError inside report_vm_out_of_memory.

My  assumption of OOME once VM is initialized completely is due to java application[directly or indirectly] may not hold true always. 
-XX:+CrashOnOutOfMemoryError is mostly for debugging purpose so I added the related code change inside report_vm_out_of_memory. 
Yes, I must not use ' java.lang.OutOfMemoryError'  for such case. 

Please let me know whether I should remove the code change inside  report_vm_out_of_memory or keep it by adding appropriate reason of OutOfMemoryError.

Regards,
Shafi

> -----Original Message-----
> From: Mikael Gerdin
> Sent: Monday, October 10, 2016 7:30 PM
> To: Shafi Ahmad; hotspot-runtime-dev at openjdk.java.net
> Subject: Re: [9] RFR for JDK-8155004: CrashOnOutOfMemoryError doesn't
> work for OOM caused by inability to create threads'
> 
> Hi,
> 
> On 2016-10-10 09:24, Shafi Ahmad wrote:
> > Hi All,
> >
> > Please review the simple change for the fix of bug '' JDK-8155004:
> CrashOnOutOfMemoryError doesn't work for OOM caused by inability to
> create threads'.
> >
> > Summary:
> > In the current implementation there are few scenarios where we are not
> obeying the jvm option -XX:+CrashOnOutOfMemoryError.
> > While I was analysis this issue I found there are two jvm state where OOM
> can happen:
> >  1.  OOM during VM initialization - as per our internal discussion for this case
> it is not worth for dumping core file, so this is left as it is.
> >  2.  OOM once VM is initialized - For this scenario most of the place code is
> already added but few place corresponding code changes are missing so this
> change covers it.
> >
> > Webrev link: http://cr.openjdk.java.net/~shshahma/8155004/webrev.00/
> 
> 
> There is a lot of confusion in the VM code with the term "out of memory
> error".
> In some places it refers to code throwing a java.lang.OutOfMemoryError and
> expecting running java code to be able to potentially catch that Error and
> continue running.
> 
> In other places, such as callers of report_vm_out_of_memory, the situation
> is much more dire and the calling thread may not even be a JavaThread and
> as such cannot "throw" an exception.
> report_vm_out_of_memory is only invoked through the macro
> vm_exit_out_of_memory, which of course implies that the condition is fatal
> and we are about to terminate the JVM process altogether.
> 
> I think that it's incorrect to call code related to java.lang.OutOfMemoryError
> in report_vm_out_of_memory since the condition may not even be
> correlated with Java level application behavior.
> 
> /Mikael
> 
> > Jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-8155004
> >
> > Testing: jprt and jtreg (on Linux x86_64)
> >
> > Regards,
> > Shafi
> >