RFR: 8256265 G1: Improve parallelism in regions that failed evacuation [v2]

[Hamlin Li]

> Currently G1 assigns a thread per failed evacuated region. This can in effect serialize the whole process as often (particularly with region pinning) there is only one region to fix up.
> Try to improve the parallelsim when walking over the regions by 
> 
>  - first, split a region into tasks;
>  - then, process these task in parallel and load balance among GC threads;
>  - last, necessary cleanup
> 
> The perf test based on lastest implementation + JDK-8277736 shows that:
> 
>  - when ParallelGCThreads=32, when G1EvacuationFailureALotCSetPercent <= 50, the parallelism bring more benefit than regression;
> - when ParallelGCThreads=128, whatever G1EvacuationFailureALotCSetPercent is, the parallelism bring more benefit than regression;
> 
> For detailed perf test result, please check:
> 
>  - https://bugs.openjdk.java.net/secure/attachment/97227/parallel.evac.failure-threads.32.png
>  - https://bugs.openjdk.java.net/secure/attachment/97228/parallel.evac.failure-threads.128.png
>  
> NOTE: load balance part of code is almost same as G1ParScanThreadState, if necessary and feasible, consider to refactor this part into a shared code base.

Hamlin Li has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains ten commits:

 - Merge branch 'master' into parallelize-evac-failure
 - Adjust worker cost by a factor; initialize task queues set and terminator threads by active workers
 - Fix wrong merge
 - Merge with master
 - Remove and merge code of G1ParRemoveSelfForwardPtrsTask into RemoveSelfForwardPtrsTask
 - Fix crashes in ~G1GCParPhaseTimesTracker(), G1PreRemoveSelfForwardClosure::do_heap_region, G1CollectedHeap::par_iterate_regions_array()=>~StubRoutines::atomic entry points; Refine comments
 - Fix inconsistent length between task queues and terminator
 - Fix crash when heap verification; Fix compilation error; Refine comments
 - Initial commit

-------------

Changes: https://git.openjdk.java.net/jdk/pull/6627/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6627&range=01
  Stats: 1055 lines in 22 files changed: 698 ins; 280 del; 77 mod
  Patch: https://git.openjdk.java.net/jdk/pull/6627.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/6627/head:pull/6627

PR: https://git.openjdk.java.net/jdk/pull/6627