RFR (S): Reset bitmaps in MarkCompact should use parallel threads

[Aleksey Shipilev]

Hi,

One of the things that bugs me: in scheduling (especially on large
machines), ramping up threads is latency-consuming. Even if all threads
started nicely, some kernels use the power saving strategies that
schedule cold threads on the same core first, and only then reschedule.
So if you are doing a high-perf batch job, it is generally a bad idea to
switch threads in the middle.

Enter MarkCompact. There, we do these steps in these threads:
  1. Concurrent:   Mark heap (reusing SHConcurrentMark)
  2. Parallel:     Calculate addrs, adjust pointers, copy objects
  3. VMThread (!): Reset prev bitmap
  4. Concurrent:   Reset current bitmap

This patch makes steps 3 and 4 in the Parallel thread pool:

http://cr.openjdk.java.net/~shade/shenandoah/markcompact-par-reset-bitmaps/webrev.01/

Aside: At some point, we'd have to see if we can decouple
SHConcurrentMark enough to use Parallel, not Concurrent, threads for
MarkCompact cycle.

Thanks,
-Aleksey