[9] RFR for JDK-8155004: CrashOnOutOfMemoryError doesn't work for OOM caused by inability to create threads'

[David Holmes]

Hi Shafi,

On 10/10/2016 11:59 PM, Mikael Gerdin wrote:
> Hi,
>
> On 2016-10-10 09:24, Shafi Ahmad wrote:
>> Hi All,
>>
>> Please review the simple change for the fix of bug '' JDK-8155004:
>> CrashOnOutOfMemoryError doesn't work for OOM caused by inability to
>> create threads'.
>>
>> Summary:
>> In the current implementation there are few scenarios where we are not
>> obeying the jvm option -XX:+CrashOnOutOfMemoryError.
>> While I was analysis this issue I found there are two jvm state where
>> OOM can happen:
>>  1.  OOM during VM initialization - as per our internal discussion for
>> this case it is not worth for dumping core file, so this is left as it
>> is.
>>  2.  OOM once VM is initialized - For this scenario most of the place
>> code is already added but few place corresponding code changes are
>> missing so this change covers it.
>>
>> Webrev link: http://cr.openjdk.java.net/~shshahma/8155004/webrev.00/
>
>
> There is a lot of confusion in the VM code with the term "out of memory
> error".
> In some places it refers to code throwing a java.lang.OutOfMemoryError
> and expecting running java code to be able to potentially catch that
> Error and continue running.
>
> In other places, such as callers of report_vm_out_of_memory, the
> situation is much more dire and the calling thread may not even be a
> JavaThread and as such cannot "throw" an exception.
> report_vm_out_of_memory is only invoked through the macro
> vm_exit_out_of_memory, which of course implies that the condition is
> fatal and we are about to terminate the JVM process altogether.
>
> I think that it's incorrect to call code related to
> java.lang.OutOfMemoryError in report_vm_out_of_memory since the
> condition may not even be correlated with Java level application behavior.

I totally agree with Mikael. A call to report_java_out_of_memory should 
only be made on a code path that will throw an OOME.

There is a lot of contention over how things like HeapDumpOnOutOfMemory 
and CrashOnOutOfMemory should behave given the various reasons why we 
can run out of memory. I see little point in doing a heap dump, for 
example, if we did not exhaust the heap. I think there are a lot of 
issues with this mechanism and the placement of some of the calls to 
report_java_out_of_memory are questionable (eg should it come before or 
after posting JVMTI resource exhaustion events? should it come before or 
after vm initialization checks? [I think after, but that isn't always so!]).

In the context of this fix the change to jvm.cpp, in JVM_StartThread, is 
acceptable. And it addresses the request made by the bug submitter.

Thanks,
David

> /Mikael
>
>> Jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-8155004
>>
>> Testing: jprt and jtreg (on Linux x86_64)
>>
>> Regards,
>> Shafi
>>