RFR: JDK-8155004: CrashOnOutOfMemoryError doesn't work for OOM caused by inability to create threads

[Thomas Stuefe]

On Tue, 20 Apr 2021 10:52:26 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> Greetings,
> 
> this is an old issue and I'd like to fix it. If we fail to create a java thread due to platform limitations we throw an OOM. But we fail to honor the various xxxOnOutOfMemoryError switches.
> 
> The fix is very straightforward. 
> 
> If fixes 
> - CrashOnOutOfMemoryError
> - ExitOnOutOfMemoryError
> - HeapDumpOnOutOfMemoryError
> - and, in theory "OnOutOfMemoryError=<user comand>".
> 
> the latter only in theory because most of the times whatever prevented the thread to start up will also prevent the fork needed to get the user command running.
> 
> One remaining question, maybe for a future RFE, is how we want to handle native threads creation error. AFAICS currently, failing to create a native thread may or may not result in a fatal shutdown, a log output, or just be ignored, depending on the thread. 
> 
> If `...OnOutOfMemoryError` is specified, should native thread creation failure be handled the same way as a java thread?
> 
> - No if I take the option name literally, since there is no OOM involved
> - Yes if I take into account what these switches are actually used for - analysis or quick shutdown of a JVM inside a container in case of an OOM. Since it is completely random which thread is hit by the limit.
> 
> Thanks, Thomas

Hi David,

(snip)

> > p.s. I do not understand your quotes around "bug", nor your hesitance to fix this. I have seen quite a number of desperate setups customers have around crashy or OOMy JVMs, to analyse and to restart quickly. Anything in that area we can do helps.
> 
> My hesitance is because the intent of these flags is to provide a
> response for Java heap exhaustion, not to provide a response to every
> place that java.lang.OutOfMemoryError may be thrown. You've picked this
> one case of native thread exhaustion to add-in but that just makes it
> inconsistent. Why should that case be handled and not all the others? It
> is unfortunate that there is more than one way to read OutOfMemoryError,
> but it should be read as out-of-Java-heap-memory-error. We have had bug
> reports when someone uses this flag and their code does "throw new
> OutOfMemoryError() and they want to know why the VM didn't crash!
> 
> I wouldn't object to a general broadening of what this flag applies to
> but I do object to cherry-picking a single case, resulting in the flag
> have no well-defined meaning at all.
> 

Okay, I understand your concern now better.

If you do not object in principle to broadening of ..OnOutOfMemoryError to cover more cases, but just to the fact that this RFE only handles this one case of resource exhaustion, I think we have common ground. Both Yasumasa and me are fully in favour of making the ..OnOutOfMemoryError coverage as complete as possible. I would prefer to do this one RFE at a time, since that way we can put in work when we have free cycles, and these smaller patches are more easy to backport.

Today ..OnOutOfMemoryError is invoked on OOM for both heap- and metaspace exhaustion, and AFAICS has been this way since at least jdk 11. So it is not consistently applied to java heap ooms only.

> And per your latest email perhaps we need new generic flags for
> resource-exhaustion.

I'd like that very much. But let me make an argument for fixing the ..OnOutOfMemoryError first, where fixing means making them apply to as many cases of thrown OOMs as possible:

The public does not know when these hooks fire and reasonably assume they fire for all instances of OOM, regardless the cause. To my knowledge we never told them differently. Since it is difficult enough to educate them about new options - look at AbortVMOnException, not many seem to know that one - I'd go with the flow and let the switches do what the public always thought they would do. There are a lot of StackOverFlow articles describing these switches, for instance.

Cheers, Thomas

> 
> Cheers,
> David
> -----
>

-------------

PR: https://git.openjdk.java.net/jdk/pull/3586