Preliminary RFR(M):8016057: OOME crash in vm/mlvm/meth/stress/jdi/breakpointInCompiledCode

[Albert Noll]

Hi all,

I have worked on a solution that deals with out-of-memory-errors (OOMEs) 
that can occur in C2
on Solaris. The current patch should give you an idea of the solution. 
The final patch will still
need some work.
See below for a more detailed description of the path.

Many thanks in advance,
Albert

jbs: https://jbs.oracle.com/bugs/browse/JDK-8016057
webrev: http://cr.openjdk.java.net/~anoll/8016057/webrev.00/ 
<http://cr.openjdk.java.net/%7Eanoll/8016057/webrev.00/>

The current approach aims at avoiding OOMEs using two approaches:

1) Pre-allocate memory in arenas used by C2
Preallocation happens at two layers:
a) Arenas
   Arenas used by C2 preallocate memory. If "normal" memory allocation 
fails then (i) C2 is asked
   to stop compilation (ii) the preallocated memory is used to continue 
execution until the next if
   (failing()) call, which provides a safe path to abort compilation. In 
the current patch, the size of
   preallocated memory is 10% of the size of the arena. This strategy 
works well for the reported
   problem.
b) Chunk pool
   Each chunk pool preallocates one chunk that can be used as a backup.

This two-layer memory preallocation avoids frequent preallocations of 
small (<32K) chunks in arenas.

2) Change code in C2 to check for NULL pointers when no memory is available
Where easily possible, memory allocation are changed so that they return 
NULL. Checks against NULL are introduced.

Discussion and alternative solutions:
This patch does not completely solve OOMEs in C2. It rather reduces the 
chance of crashing. I.e., C2 can still crash if the preallocated memory 
is not sufficient to reach the next if (failing()) check. Another 
disadvantage is that C2 consumes additional memory that is not used in 
most cases.

A clean solution would require to check all memory allocations in C2 
against NULL and provide a safe bailout path. I think that this is a 
major change to C2 that should be discussed in detail.


I want to thank Niclas, Roland, Markus, Vladimir, and John for the 
discussions.

Best,
Albert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/attachments/20130814/e7d8700d/attachment-0001.html