RFR: Adaptive/Traversal heuristics rewrite for allocation rate

[Aleksey Shipilev]

http://cr.openjdk.java.net/~shade/shenandoah/alloc-rate-heuristics/webrev.02/

Our current free-threshold based heuristics are good when heap is not over-occupied. When the heap
is over-occupied, it has a few positive feedback loops that make it runaway into back-to-back
cycles, never recovering even if heap occupancy goes down. It can be observed by free threshold
latching to one aggressive value and never going down. My attempts at resolving that in a simple
manner were not successful.

To add insult to injury, free-threshold is also indirect: it targets the free space with assuming
things about allocation rate, the GC time needed to collect it, etc. It all works well when workload
is very steady-state and not otherwise.

This rewrite makes the adaptive/traversal heuristics track the allocation rate and gc times, and
schedule the next GC when it knows allocations would deplete the free space while GC is running. It
also adjusts the "headroom" available based on the max CSet cap, has the reserve for absorbing
allocation spikes, and has negative feedback loop from degen/full GCs.

It does not regress our current benchmarks, but does not improve them either -- because, as said
above, current heuristics works well until it runs away. It should work much more reliably on
non-steady-state workloads.

Testing: tier3_gc_shenandoah, specjbb, specjvm

Thanks,
-Aleksey