[13] RFR(S): 8226536: Catch OOM from deopt that fails rematerializing objects

[Nils Eliasson]

Hi,

This is a patch of GarbageUtils.java used by a bunch of GC tests.

This issue was diagnosed by Stefan Karlsson and passed to the compiler team.

The core problem is that the test relies on successfully catching OOM 
errors. But the OOM causes a deopt, and the deopt must rematerialize 
some objects before passing the execution on to the interpreter to run 
the catch-block. If the rematerialization can't allocate, it will pass 
the OOM on, without having run the catch-block.

With this patch I fix the tests by wrapping the faulting code with an 
extra layer of try-catch-OOM. It relies on not inlining the inner 
method, because then it would be part of the compiled frame being 
deoptimized.

In general this is a problem for any kind of deopt when we are out of 
heap. Another bug will be opened to keep tracking this issue.

Webrev:http://cr.openjdk.java.net/~neliasso/8226536/webrev.02/ 
<http://cr.openjdk.java.net/~neliasso/8226536/webrev.02/>

Bug: https://bugs.openjdk.java.net/browse/JDK-8226536

// Nils