RFR: JDK-8155004: CrashOnOutOfMemoryError doesn't work for OOM caused by inability to create threads

[Matthias Baesken]

On Tue, 20 Apr 2021 10:52:26 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> Greetings,
> 
> this is an old issue and I'd like to fix it. If we fail to create a java thread due to platform limitations we throw an OOM. But we fail to honor the various xxxOnOutOfMemoryError switches.
> 
> The fix is very straightforward. 
> 
> If fixes 
> - CrashOnOutOfMemoryError
> - ExitOnOutOfMemoryError
> - HeapDumpOnOutOfMemoryError
> - and, in theory "OnOutOfMemoryError=<user comand>".
> 
> the latter only in theory because most of the times whatever prevented the thread to start up will also prevent the fork needed to get the user command running.
> 
> One remaining question, maybe for a future RFE, is how we want to handle native threads creation error. AFAICS currently, failing to create a native thread may or may not result in a fatal shutdown, a log output, or just be ignored, depending on the thread. 
> 
> If `...OnOutOfMemoryError` is specified, should native thread creation failure be handled the same way as a java thread?
> 
> - No if I take the option name literally, since there is no OOM involved
> - Yes if I take into account what these switches are actually used for - analysis or quick shutdown of a JVM inside a container in case of an OOM. Since it is completely random which thread is hit by the limit.
> 
> Thanks, Thomas

Marked as reviewed by mbaesken (Reviewer).

Hello , the description of the flags in globals.hpp says :

ExitOnOutOfMemoryError  
JVM exits on the first occurrence of an out-of-memory error thrown from JVM

CrashOnOutOfMemoryError 
JVM aborts, producing an error log and core/mini dump, on the first occurrence of an out-of-memory error thrown from JVM

So I think the patch Thomas did is fine, it improves the situation where customers expect an exit/crash,  when the flag is set accordingly. But they do not get it at the moment.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3586