RFR: 8212933: Thread-SMR: requesting a VM operation whilst holding a ThreadsListHandle can cause deadlocks

[Robbin Ehn]

Hi, please review.

When the VM thread executes a handshake it uses different ThreadsLists during
the execution. A JavaThread that is armed for the handshake when it is already
in the exit path in VM will cancel the handshake. Even if the VM thread cannot
see this thread after the initial ThreadsList which where used for arming, the
handshake can progress when the exiting thread cancels the handshake.

But if a third thread takes a ThreadsList where the exiting JavaThread is 
present and tries to execute a VM operation, hence waiting on VM thread to 
finish the handshake, the JavaThread in the exit path can never reach the 
handshake cancellation point. VM thread cannot finishes the handshake and the 
third thread is stuck waiting on the VM thread.

To allow holding a ThreadsList when executing a VM operation we instead let the
VM thread use the same ThreadsList over the entire handshake making all armed
threads visible to the VM thread at all time. And if VM thread spots a 
terminated thread it will count that thread is already done by only clearing
it's operation.

Passes local stress testing, t1-5 and the deadlock is no longer reproduce-able.
Added a jtreg handshake + thread suspend test as a reproducer.

Issue: https://bugs.openjdk.java.net/browse/JDK-8212933
Code: http://cr.openjdk.java.net/~rehn/8212933/v1/webrev/

Thanks, Robbin