RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v2]

[Serguei Spitsyn]

> This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame.
> It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms.
> The deadlocking scenario is well described by Patricio in a bug report comment.
> In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections.
> 
> The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section.
> This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section.
> 
> Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified.
> 
> New test was developed by Patricio:
>  `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock`
> The test is very nice as it reliably in 100% reproduces the deadlock without the fix.
> The test is never failing with this fix.
> 
> Testing:
>  - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock`
>  - tested with mach5 tiers 1-6

Serguei Spitsyn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits:

 - Resolved merge conflict in VirtualThread.java
 - added @summary to new test SuspendWithInterruptLock.java
 - add new test SuspendWithInterruptLock.java
 - 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable

-------------

Changes: https://git.openjdk.org/jdk/pull/17011/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=17011&range=01
  Stats: 183 lines in 15 files changed: 178 ins; 0 del; 5 mod
  Patch: https://git.openjdk.org/jdk/pull/17011.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17011/head:pull/17011

PR: https://git.openjdk.org/jdk/pull/17011