RFR: 8260044: Parallel GC: Concurrent allocation after heap expansion may cause unnecessary full gc

[Kim Barrett]

Please review this change to ParallelGC to avoid unnecessary full GCs when
concurrent threads attempt oldgen allocations during evacuation.

When a GC thread fails an oldgen allocation it expands the heap and retries
the allocation.  If the second allocation attempt fails then allocation
failure is reported to the caller, which can lead to a full GC.  But the
retried allocation could fail because, after expansion, some other thread
allocated enough of the available space that the retry fails.  This can
happen even though there is plenty of space available, if only that retry
were to perform another expansion.

Rather than trying to combine the allocation retry with the expansion (it's
not clear there's a way to do so without breaking invariants), we instead
simply loop on the allocation attempt + expand, until either the allocation
succeeds or the expand fails.  If some other thread "steals" space from the
expanding thread and causes its next allocation attempt to fail and do
another expansion, that's functionally no different from the expanding
thread succeeding and causing the other thread to fail allocation and do the
expand instead.

This change includes modifying PSOldGen::expand_to_reserved to return false
when there is no space available, where it previously returned true.  It's
not clear why it returned true; that seems wrong, but was harmless.  But it
must not do so with the new looping behavior for allocation, else it would
never terminate.

Testing:
mach5 tier1-3, tier5 (tier2-3, 5 do a lot of ParallelGC testing)

-------------

Commit messages:
 - retry failed allocation if expand succeeds

Changes: https://git.openjdk.java.net/jdk/pull/2309/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2309&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8260044
  Stats: 15 lines in 2 files changed: 5 ins; 0 del; 10 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2309.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2309/head:pull/2309

PR: https://git.openjdk.java.net/jdk/pull/2309