RFR: 8267079: Support async handshakes that can be executed by a remote thread [v2]

[Man Cao]

On Tue, 18 May 2021 19:08:08 GMT, Man Cao <manc at openjdk.org> wrote:

>> Hi all,
>> 
>> Can I have reviews for this small refactoring change? It resolves a pending concern from [JDK-8238761](https://bugs.openjdk.java.net/browse/JDK-8238761), clarifies the code and allows more use case of async handshakes. See [JDK-8267079](https://bugs.openjdk.java.net/browse/JDK-8267079) for detailed description.
>> 
>> -Man
>
> Man Cao has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Added missing deallocation and renamed "remote" to "non-self".

I addressed the comments in the second commit.

For Robbin's question:
> The AsyncHandshakeClosure must still be on heap but when executed by a remote thread it looks like it's never deleted?

Great catch! Added delete in HandshakeState::try_process() similarly to HandshakeState::process_self_inner().

> If you need guarantee that the handshake is executed, you cannot use AsyncHandshakeClosure and since there is almost never a remote thread and the performance difference should be very slim?

I hope the reply to David could also answer this. I don't need a guarantee that the handshake is executed. I mainly need a safe approach to skip target threads that are blocked. Reusing the four if checks in HandshakeState::try_process() is a possible approach, and it would need async non-self executable handshake for HandshakeState::claim_handshake() to return true.

> To me it sounds like you want to customize the yield method or manually loop the processing of 'normal' handshakes?

Yes, that could work as well. I did find the "async" name confusing. If we support `Handshake::enqueue(handshake)` for 'normal' handshakes, it is basically an async submission of the 'normal' handshake. It also has the problem of how to deallocate the `HandshakeOperation` object, which is currently handled by `AsyncHandshakeOperation`.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4005