RFR: 8298853: JvmtiVTMSTransitionDisabler should support disabling one virtual thread transitions [v4]

[Serguei Spitsyn]

On Fri, 23 Dec 2022 03:53:19 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> I see the problem. Good catch, thanks!
>
> Yep my bad in earlier review. The one case needs the same synchronization/coordination as the all case.

I've moved the `is_in_VTMS_transition()` from `JvmtiThreadState` to `java_lang_Thread (in javaClasses.cpp)`.  My initial implementation had this but at some point I decided to use `JvmtiThreadState::is_in_VTMS_transition()` for optimization purposes.
It introduced a problem as the `JvmtiThreadState` can be absent at the point where it is needed for sync.
I've also fixed race between `VTMS_transition_disable_for_one()` and `start_VTMS_transition()`. It was tested with the nanosleep delays and the assert from Patricio.
Also, one assert has been removed from `jvmtiEnvThreadState.cpp` as it became not always correct.

-------------

PR: https://git.openjdk.org/jdk/pull/11690