(Preliminary) RFC 7038914: VM could throw uncaught OOME in ReferenceHandler thread

[Alan Bateman]

On 30/04/2013 15:57, Thomas Schatzl wrote:
> Hi all,
>
>    the webrev at http://cr.openjdk.java.net/~tschatzl/7038914/webrev/
> presents a first stab at the CR "7038914: VM could throw uncaught OOME
> in ReferenceHandler thread".
>
> The problem is that under very heavy memory pressure, there is the
> reference handler throws an exception with the message "Exception:
> java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in
> thread "Reference Handler".
>
> The change improves handling of out-of-memory conditions in the
> ReferenceHandler thread. Instead of crashing the thread, and then
> disabling reference processing, it catches this exception and continues.
It's surprising to heard that the Reference Handler thread failed with 
OOME. I wouldn't expect anything in this code path to throw OOME, except 
maybe in fast-path for sun.misc.Cleaner but that will abort the VM be it 
fails. The enqueue method that you override in the test to provoke this 
is package-private so it's unlikely that the test or whatever that 
resulted in this bug report is doing that.

So I'm again this proposed change, rather I'm just trying to understand 
how it happened. Is there instrumentation involved by any chance? It the 
OOME something other than "java heap" or do we know?

-Alan.