RFR: Asynchronous region recycling

[Aleksey Shipilev]

http://cr.openjdk.java.net/~shade/shenandoah/async-recycle/webrev.02/

With large heaps that lead to large number of regions, we spend a lot of time (compared to the rest
of the pause) in recycling the regions. This gets worse when partial collections are enabled, and we
need to clean up the matrix then. We have done some optimizations there, but nothing beats making
recycling really asynchronous. Recent heap region work enables us to do this!

Brief tour of changes:

 *) Reclaim is now split into "trashing" (marking region Trash) and the actual "recycling" (marking
region Empty);

 *) Introduced "Concurrent Cleanup" phase that runs after Final Mark, Final Update Refs and Partial
to pick up asynchronous recycling. Folded "Concurrent Reset Bitmaps" into that phase.

 *) Verifier checks there are no lingering Trash regions between the phases/cycles. This would be
committed as separate changeset.

 *) recycle_trash() is the entry point for asyncronous recycling. It uses a few tricks to improve
performance. Notably, acquiring the locks very shortly to alleviate allocation stalls.

 *) Allocation paths make the attempt to recycle some trash regions on allocation failure. This is
needed in case we have acquired the heap lock before the recycler had chance to run.

 *) Minor changes in FreeSet to first check for region present, and then checking the size.
Otherwise duplicate regions blow the size assert first.

Running SPECjvm:serial on 100 GB heap:
  http://cr.openjdk.java.net/~shade/shenandoah/async-recycle/serial-before.txt
  http://cr.openjdk.java.net/~shade/shenandoah/async-recycle/serial-after.txt

Testing: hotspot_gc_shenandoah, benchmarks

Thanks,
-Aleksey