RFR: Shortcut concurrent cycle when enough immediate garbage is reclaimed

[Aleksey Shipilev]

Meditating over performance data from our workloads, it struck me that we miss an obvious
optimization opportunity. In our final mark, we know the liveness more or less precisely
(conservatism is coming from SATB, but we don't care about it here). We exploit this by figuring out
the immediate garbage regions, and reclaiming them on the spot, without adding them to collection set.

Now, in many workloads, especially those following the generational hypothesis to the extreme, we
have lots and lots of immediately reclaimed regions (E.g. various SPECjvm workloads have around
70..95% of regions immediately reclaimable). We can exploit this further by shortcutting the
concurrent cycle when we already have a huge bang from immediate regions already.

In other words, if there is enough immediately reclaimed regions, skip concurrent evac and
update-refs. This saves cycles walking the heap in update-refs, saves pauses for update-refs, saves
enabling the write barriers and going to slow-paths for cset checks in WBs, etc.

Webrev:
 http://cr.openjdk.java.net/~shade/shenandoah/immediate-shortcut/webrev.01/

Example run with XmlTransform (has ~92% immediately reclaimable regions) -- cuts total pause time
~2.3x, the number of pauses ~1.7x, etc:
 http://cr.openjdk.java.net/~shade/shenandoah/immediate-shortcut/xmltransform-before.txt
 http://cr.openjdk.java.net/~shade/shenandoah/immediate-shortcut/xmltransform-after.txt

Testing: hotspot_gc_shenandoah

Thanks,
-Aleksey