RFR: Lazy parallel code cache iterator

[Aleksey Shipilev]

Hi,

The last time we improved code cache iterator with more parallelism, and it
improved the pause times. However, we had to do the eager preparation stage in
its constructor, which means we scan code cache during ShenandoahRP
construction, even if we skip code cache iteration later! Which means we have
redundant work during Init Mark and Final Update-Refs pauses.

The fix is to redo the iterator to be lazy again, while maintaining parallelism:
  http://cr.openjdk.java.net/~shade/shenandoah/codecache-lazyparallel/webrev.01/

Before/after on SPECjbb:
 http://cr.openjdk.java.net/~shade/shenandoah/codecache-lazyparallel/baseline.log
 http://cr.openjdk.java.net/~shade/shenandoah/codecache-lazyparallel/patched.log

Saves 3..8 ms per pause!

Testing: hotspot_gc_shenandoah, jcstress/fastdebug/aggressive

-Aleksey