JVMTI OOM handling when arrays / objects are too large

[Alan Bateman]

Martin Buchholz wrote:
> The resolution of this is still uncertain.
> We have one approval I think,
> but all of the maintainers have expressed reluctance at making this 
> change.
> I continue to agree with Jeremy and be in favor of making this change,
> but it's no big deal either way.
> Perhaps Alan's opinion is authoritative, since he was the author.
> Alan, perhaps you should give us an actual VETO, and we will shut up.
>
> Martin
The JVM TI ResourceExhausted event is intended to be sent to agents when 
a resource is exhausted. I would suggest we don't change this because 
this isn't really a resource exhaustion case.

The OnOutOfMemoryError option was intended to help with troubleshooting 
so we're really just talking about extending it to help diagnose 
attempts to allocate arrays larger than the VM limit. I don't see a 
problem with that, esp. as the option name hints that the actions are 
invoked when OOME is about to be thrown. Just on that, I don't think 
OOME thrown by JNI/library code is handled. Also, looking at it now, it 
might be useful to provide the data collection commands with the reason 
message (as %e or %r for example, like the pid is made available as %p). 
I don't wish to extend the scope of the patch but just mentioning it as 
something that might help avoid a wild goose chase.

I think we mostly agree that the heap dump isn't useful for this case 
but it would be inconsistent when both XX options are used. Furthermore, 
when used together the heap dump is generated before the data collection 
commands are run. For example, you might run a script that automatically 
compressed the heap dump. So these options are somewhat connected and 
need to work consistently.

Hopefully this helps. I can review the patch but as I'm not working in 
this area on a daily basis, so it would be best to have a reviewer from 
the hotspot team (and I assume you'll need someone to push this through 
JPRT).

-Alan.