RFR (L): 8067433: Keep waste at end of regions for further allocation during GC

[Thomas Schatzl]

Hi all,

  can I get reviews for this change that decreases fragmentation
overhead during GC by retaining and reusing regions for allocations that
still contain "sufficient" empty space that have been allocated into
during GC?

If the application for some reason has a lot of medium size objects,
this change decreases the amount of used space during gc significantly.

The main change is to, instead of retiring a region when it is "full",
put it into a list of regions (RetainedRegionsList), and when there is a
new request to allocate, try to get new regions with that empty space
from that list first.

Some note about the implementation that may not be obvious (these notes
are in the sources too):

- we now need to track how much memory has been allocated from the start
of the GC in a particular region on a per-region basis instead of the
G1AllocRegion instance for young/old. That (and other changes) required
massaging the code a little.

- when a thread puts a region into the RetainedRegionsList and tries to
get a new region, other threads might still have references to that
region and still allocate into it at that point of time.
The code does not prevent that (by e.g. filling up the region with a
dummy object) to avoid some serialization on the freelist-lock, so at
the time when the code searches for a region in that list, it needs to
"atomically" allocate memory out of that selected region.
Because of this, when walking the regions, we might encounter "full"
regions that are automatically pruned from the list and retired.

- it implements a first-fit algorithm, as others did not really improve
the amount of total wasted space. This (imho) has to do with the fact
that most allocations G1 does are of the flexible kind, i.e. we want to
allocate a PLAB with a (large) desired word size, but actually only
requires to fill a few words for an object. I opted for simplicity here.
Note that the included test case is more a stress test as it does not
allocate any small object (i.e. that fits into a PLAB) and not
representative :)

- there is some small optimization where the code keeps track of a
conservative estimate of the largest free block in the list.

CR:
https://bugs.openjdk.java.net/browse/JDK-8067433
Webrev:
http://cr.openjdk.java.net/~tschatzl/8067433/webrev/
Testing:
jprt, dev-submit, rbt vm.gc runs, nosql, hadoop

Thanks,
  Thomas