RFR: Pack old evacuations tightly [v2]

[Kelvin Nilsen]

> This pull request includes several improvements to reduce the likelihood of evacuation failures and to reduce the frequency of old-gen collections.  Key contributions:
> 1. If a PLAB allocation fails, resize the the thread's preferred PLAB size to minimum size and retry the allocation request
> 2. Place a limit on how large the preferred PLAB size grows for individual threads.  Otherwise a small number of threads can consume the full promotion budget, leaving nothing left for the PLABs required by other threads.
> 3. If a PLAB promotion fails, and the remaining words within the PLAB is greater than minimum size, that means this is a request to promote a relatively large object.  In this case, do not retire the PLAB.  Instead, use a shared allocation to promote this large object.
> 4. If a PLAB promotion fails and the remaining words is less than the PLAB minimum size, try to replace the PLAB and then retry the PLAB allocation.  If this still fails, evacuate the object to young-gen using a GCLAB.  Don't try to promote with a shared allocation because that is too "expensive".
> 
> This commit has demonstrated fairly significant benefits on certain workloads.  For specjbb2015, we got a 17% improvement in critical jops, from 3093.847 to 3622.  For an extremem 2020 workload,
> we improved p99.999 for customer preparation processing by more than four fold, from 607,107 microseconds to 142,286 microseconds, decreasing the number of concurrent GCs from 99 to 37, old GCs from 12 to 7, full GCs from 3 to 0.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Improvements requested by reviewer

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/145/files
  - new: https://git.openjdk.org/shenandoah/pull/145/files/d5263dcb..0741d705

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=145&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=145&range=00-01

  Stats: 6 lines in 3 files changed: 0 ins; 0 del; 6 mod
  Patch: https://git.openjdk.org/shenandoah/pull/145.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah pull/145/head:pull/145

PR: https://git.openjdk.org/shenandoah/pull/145