RFR: Consistenly order STS scope outside of OOM scope to avoid deadlocks

[Roman Kennke]

> On 10/16/2018 09:44 PM, Roman Kennke wrote:
>> We have observed very spurious deadlocks that involve the OOMEvacScope
>> and STSJoiner scope. It goes like this: Thread 1 enters OOM scope, but
>> did not yet enter the STS scope. Thread2 is inside both scopes and
>> OOM-s, cancelling evac and leaves the STS and attempts to leave the
>> OOM-scope, waiting for Thread 1. Apparently, another thread calls for a
>> (non-GC) safepoint, which makes Thread 1 block upon entering the STS
>> scope, in other words it will never leave the OOM scope.
>>
>> Ordering the STS-scope consistently outside of OOM-scope should solve
>> this problem: the OOM-leaver will not get blocked by threads trying to
>> get into STS first, and will thus always leave all relevant scopes.
>>
>> Testing: tier3_gc_shenandoah
>>
>> http://cr.openjdk.java.net/~rkennke/sts-oom-deadlock/webrev.00/
> 
> OK. I wonder if we should put the assert somewhere. For example, subclass STS with ShenandoahSTS and
> check that OOM scope is not yet acquired?
> 
> The patch seems to also miss previous STS change in Traversal, needs retesting, probably.
> 
> -Aleksey

All implemented here:
http://cr.openjdk.java.net/~rkennke/sts-oom-deadlock/webrev.01/

I opted to not subclass, but instead compose the super-object. This
seems more intuitive to me with destructors implicitley calling super,
but only when virtual (or not?), and didn't want to make superclass
destructor virtual, etc. Shouldn't make any practical difference, it's
only a scope-object.

Testing: Tier3_gc_shenandoah

Ok?