RFR: Optimizing best of two work stealing queue selection

[Zhengyu Gu]

Hi,

This optimization is based on this paper [1]. The idea is to keep last 
successfully stolen queue as one of best of two candidates.

My experiments showed that it only marginally improved successful 
stealing ratio, given the ratio is *very* high to begin with, north of 
95%, even under extremely unbalanced scenarios (e.g. concurrent threads 
= 3, parallel threads = 11, of course, based on Shenandoah 
ParallelClaimableQueueSet ).

However, it showed remarkable improvement on task termination:
I measured how often worker threads have to exit termination protocol 
and retry (failed termination)

Compiler.sunflow:

Before patch:
   Avg: failed termination 4.747   Worst case: 209

After patch:
   Avg: failed termination 0.513   Worst case: 11

Although, above result is from one particular run, but it is *very 
reproducible* and also with other benchmarks.


Webrev: 
http://cr.openjdk.java.net/~zgu/shenandoah/optimized_best_of_2/webrev.00/index.html

Hopefully, Aleksey and Thomas have better ways to measure the impact to 
overall termination.

Tony, I would appreciate it if you have any comments.

Test:
   tier3_gc_shenandoah


[1] Characterizing and Optimizing Hotspot Parallel Garbage
     Collection on Multicore Systems
     http://ranger.uta.edu/~jrao/papers/EuroSys18.pdf

Thanks,

-Zhengyu