RFR: Load balance remset scan [v2]

[Kelvin Nilsen]

> Prior to this change, the initial group of remembered set assignments was given to worker threads one entire region at a time.  We found that with large region sizes (e.g. 16 MiB and above), this resulted in too much imbalance in the work performed by individual threads.  A few threads assigned to scan 16 MiB regions with high density of "interesting pointers" were still scanning after all other worker threads finished their scanning efforts.
> 
> This change caps the maximum assignment size for worker threads at 4 MiB.  This results in better distribution of efforts between multiple concurrent threads.  With 13 worker threads and 16 MiB heap regions, we observe the following benefits on an Extremem workload (46_064 MiB heap size, 27_648 MiB new size):
> 
> Latency for customer preparation processing improved by 0.79% for P50, 2.26% for P95, 8.21% for p99, 28.17% for p99.9, 86.59% for p99.99, 86.77% for p99.999.  The p100 response improved only slightly, by 1.99%.
> 
> Average time for concurrent remembered set marking scan improved by 1.92%.  The average time for concurrent update refs time, which includes remembered set scanning, improved by 1.72%.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Add comment to explain ShenandoahRegionChunkIterator

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/173/files
  - new: https://git.openjdk.org/shenandoah/pull/173/files/9704b348..256d88da

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=173&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=173&range=00-01

  Stats: 11 lines in 1 file changed: 11 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/173.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah pull/173/head:pull/173

PR: https://git.openjdk.org/shenandoah/pull/173