Preliminary RFR(M):8016057: OOME crash in vm/mlvm/meth/stress/jdi/breakpointInCompiledCode

[Vladimir Kozlov]

Albert,

Since it is only manifesting on Solaris with 64-bit VM (compressed oops) could you try to run it with next flag and see 
if it helps?:

-XX:HeapBaseMinAddress=2g

Thanks,
Vladimir

PS: I don't like that you rewriting Allocation code which is used by whole VM. I would prefer to see Compiler's wrapper 
for allocation.

On 8/14/13 5:03 AM, Albert Noll wrote:
> Hi all,
>
> I have worked on a solution that deals with out-of-memory-errors (OOMEs) that can occur in C2
> on Solaris. The current patch should give you an idea of the solution. The final patch will still
> need some work.
> See below for a more detailed description of the path.
>
> Many thanks in advance,
> Albert
>
> jbs: https://jbs.oracle.com/bugs/browse/JDK-8016057
> webrev: http://cr.openjdk.java.net/~anoll/8016057/webrev.00/ <http://cr.openjdk.java.net/%7Eanoll/8016057/webrev.00/>
>
> The current approach aims at avoiding OOMEs using two approaches:
>
> 1) Pre-allocate memory in arenas used by C2
> Preallocation happens at two layers:
> a) Arenas
>    Arenas used by C2 preallocate memory. If "normal" memory allocation fails then (i) C2 is asked
>    to stop compilation (ii) the preallocated memory is used to continue execution until the next if
>    (failing()) call, which provides a safe path to abort compilation. In the current patch, the size of
>    preallocated memory is 10% of the size of the arena. This strategy works well for the reported
>    problem.
> b) Chunk pool
>    Each chunk pool preallocates one chunk that can be used as a backup.
>
> This two-layer memory preallocation avoids frequent preallocations of small (<32K) chunks in arenas.
>
> 2) Change code in C2 to check for NULL pointers when no memory is available
> Where easily possible, memory allocation are changed so that they return NULL. Checks against NULL are introduced.
>
> Discussion and alternative solutions:
> This patch does not completely solve OOMEs in C2. It rather reduces the chance of crashing. I.e., C2 can still crash if
> the preallocated memory is not sufficient to reach the next if (failing()) check. Another disadvantage is that C2
> consumes additional memory that is not used in most cases.
>
> A clean solution would require to check all memory allocations in C2 against NULL and provide a safe bailout path. I
> think that this is a major change to C2 that should be discussed in detail.
>
>
> I want to thank Niclas, Roland, Markus, Vladimir, and John for the discussions.
>
> Best,
> Albert