RFR: Safepooint suspendible worker threads

[Roman Kennke]

Currently, our GC workers are unaffected by (non-GC) safepoints: they
happily carry on working their stuff. This is usually not a problem.
However, lately we found that concurrent code cache marking sometimes
barfs between the nmethod marking/sweeping sometimes steps in between
(during a non-GC safepoint). Also, heapdump seems to rely on the heap
holding still (naturally). We don't really know what other VM_Ops might
depend on the heap/GC holding still and the general assumption is that
nothing moves at a safepoint except the VMThread and the workers it spawns.

This change makes Shenandoah's GC worker threads suspend at safepoints.

It uses G1's (not G1-specific) SuspendibleThreadSet.

It requires some extra magic to coordinate with full-gc. Specifically,
when we check for cancelled heap, we must not yield when being
cancelled. In order to do so, we first CAS _cancelled_gc to
NOT_CANCELLED (prevents cancelling threads to bump it to CANCELLED while
we yield()), then yield() (check for safepoint and suspend).

I spared making ShenandoahConcurrentThread to take part in this dance.
I'll probably add that later, however it seems not important (and will
require some additional complicated co-ordination).

It's enabled by -XX:+ShenandoahSuspendibleWorkers and off by default.
Works like a charm for me on or off. We should check if/how it affects
performance.

Testing: hotspot_gc_shenandoah and specjvm

http://cr.openjdk.java.net/~rkennke/suspendibleworkers/webrev.00/
<http://cr.openjdk.java.net/%7Erkennke/suspendibleworkers/webrev.00/>

Roman