RFR: 8373366: HandshakeState should disallow suspend ops for disabler threads [v11]

[David Holmes]

On Thu, 15 Jan 2026 01:34:28 GMT, Serguei Spitsyn <sspitsyn at openjdk.org> wrote:

>> An asynchronous handshake operation (`ThreadSelfSuspensionHandshakeClosure`) can be installed when the target thread is not in a `MountUnmountDisabler` scope. But the target thread can enter such scope by the time the operation is self-processed by the target thread.
>> 
>> This is fixed by a small tweak in the function
>> `HandshakeOperation* HandshakeState::get_op_for_self(bool allow_suspend, bool check_async_exception)`.
>> The tweak is to skip a `HandshakeOperation` if `_handshakee->is_vthread_transition_disabler() == true`, so the same temporary suspension disabling mechanism would be used as for `_handshakee->is_disable_suspend() == true`.
>> 
>> All other changes are to move the `is_vthread_transition_disabler()` out of DEBUG to product.
>> 
>> Testing:
>>  - In progress: mach5 tiers 1-6
>
> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix trailing space in new test

I can't say that I understand what is happening in [JDK-8375362](https://bugs.openjdk.org/browse/JDK-8375362) but I do wonder if we need to get all the current VT issues resolved (in Loom repo?) before trying to update mainline. Maybe they are distinct issues but maybe there is some overlap.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/28740#issuecomment-3753029067