[10] RFR (M/L): 8137099: G1 needs to "upgrade" GC within the safepoint if it can't allocate during that safepoint to avoid OoME

[Thomas Schatzl]

Hi Paul,

  sorry for the late reply, there has been another issue keeping me
busy ;)

On Thu, 2017-12-07 at 00:44 +0000, Hohensee, Paul wrote:
> In attempt_allocation_humongous in g1CollectedHeap.cpp, there’s a
> note that says there’s code duplication between it and
> attempt_allocation_slow. There may be a case of version skew between
> the code at line 553 and the lack of similar code at line 972. If you
> don’t want to retry the humongous allocation attempt at that point,
> then the comment at line 968 should be updated.

  humongous allocation needs to be under the heap lock anyway, so we
just skip the inline allocation as it would be the same as the code at
the start of the loop.

I did add a comment, fixed some whitespace and had to move declarations
in the JNI c file to make it compile on some platforms:

New webrevs:
http://cr.openjdk.java.net/~tschatzl/8137099/webrev.0_to_1 (diff)
http://cr.openjdk.java.net/~tschatzl/8137099/webrev.1 (full)

There is some potential to do refactoring in the
attempt_allocation_slow/humongous methods (the part in "if
(should_try_gc)"), but I would like to do that in a different change.

Thanks,
  Thomas