RFR: 8267079: Support async handshakes that can be executed by a remote thread [v2]

[Robbin Ehn]

On Tue, 18 May 2021 19:08:08 GMT, Man Cao <manc at openjdk.org> wrote:

>> Hi all,
>> 
>> Can I have reviews for this small refactoring change? It resolves a pending concern from [JDK-8238761](https://bugs.openjdk.java.net/browse/JDK-8238761), clarifies the code and allows more use case of async handshakes. See [JDK-8267079](https://bugs.openjdk.java.net/browse/JDK-8267079) for detailed description.
>> 
>> -Man
>
> Man Cao has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Added missing deallocation and renamed "remote" to "non-self".

"The goal is to support asynchronous handshake with all threads for JDK-8236485, and skipping threads that are blocked, without entering a safepoint in the requesting thread."

This line is just confusing, the requesting thread only enters a safepoint if it's a JavaThread.

- You don't need to skip blocked/native thread, those handshakes are finished really quickly on first iteration.
- If VM thread is busy doing starting safepoint, you can skip your epoch, since you just got it free via the safepoint.
  - Save the current safepoint counter and store it in the VM op.
  - Check that safepoint counter vs current counter in the VM ops doit_prologue()
  - If they do not match all threads have emitted the barrier. 

- All JavaThreads will safepoint before they perform any handshakes.
- If there is a safepoint, we should perform it was quick as possible, otherwise it will take longer for them to perform handshakes. 
- G1 concurrent threads are stopped before the safepoint: 

void G1CollectedHeap::safepoint_synchronize_begin() {
  SuspendibleThreadSet::synchronize();
}  



Not sure if this epoch sync is run by a thread part of SuspendibleThreadSet or not, but:
If it is; it is just prolonging the time to safepoint, which is a free epoch, thus finishing the epoch just wastes CPU.
If it is not; it is just hurting it's own performance, since handshakes are performed _after_ the safepoint.

So I do not see any reason why you can't use a synchronous handshakes.

If a Java thread does not do it's handshake in timely manner, there are generally be two issues:
- You don't have enough CPU, so the thread is not running. Using more CPU will just hurt in this case, so yielding is the correct thing todo.
- We are allocating and zeroing a large array, which is a GC issue. (fixed in ZGC)

If you have seem another case where you think a synchronous handshake takes to long, please let me know.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4005