RFR 8234613: JavaThread can escape back to Java from an ongoing handshake

[coleen.phillimore at oracle.com]

This fix looks good to me, from your explanations offline and in the bug 
report.
Coleen

On 11/22/19 1:25 PM, Patricio Chilano wrote:
> Hi,
>
> This patch aims to address a current bug where, given the right 
> combination of handshakes and external suspend/resume, a JavaThread 
> can transition from a safe state back to Java without blocking for a 
> still-in-progress handshake. In the description of the bug I added an 
> example, tracing the state changes of the JavaThread as it goes 
> through the different transitions until it escapes the handshake. 
> Currently, the window of time for this issue to happen is so small 
> that we do not see actual failures running tests. Running test 
> SuspendAtExit.java and adding some small delay before restoring the 
> JavaThread state in java_suspend_self_with_safepoint_check() can 
> demonstrate the issue.
> The proposed fix is to check again if we have a pending/in-progress 
> handshake operation after executing ~ThreadInVMForHandshake().
>
> Tested with mach5, tiers1-6 on all platforms (Linux, macOS, Windows 
> and Solaris).
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8234613
> Webrev: http://cr.openjdk.java.net/~pchilanomate/8234613/v01/webrev/
>
> Thanks,
> Patricio