RFR: 8346727: JvmtiVTMSTransitionDisabler deadlock

[Chris Plummer]

On Thu, 9 Jan 2025 05:03:33 GMT, Serguei Spitsyn <sspitsyn at openjdk.org> wrote:

> This is a fix of one more deadlock issue related to `interruptLock` critical sections. When the `interruptLock` is hold by the target virtual thread it is unsafe to suspend or post JVMTI events. This update is to ignore the JVMTI events when the `interruptLock` is hold. It is additionally to the cases when the target JavaThread is in a `VTMS` transition. It is based on the existing mechanism disallowing JVMTI suspends while the `interruptLock` is hold. In order to support this the target JavaThread has the `_is_disable_suspend` bit.
> 
> Testing:
> - Ran mach5 tiers 1-6
> - There is no regression test for this issue as it is not hard to construct one. Originally, the issue was reported by JGroups which is using the `Async` profiler.

src/hotspot/share/prims/jvmtiExport.cpp line 1105:

> 1103: 
> 1104:   if (JavaThread::current()->hide_jvmti_events()) {
> 1105:     return false; // no events should be posted if thread is in VTMS alike transition

I don't think your use of "alike" is what you mean here. Did you mean "VTMS-like", or maybe something like "VTMS transition or similar state".

Also `hide_jvmti_events()` is a pretty self explanatory name already. Probably no comment is needed at the call sites, and instead you should just add a comment where `hide_jvmti_events()` is declared.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22997#discussion_r1909124132