RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v3]

[Alan Bateman]

On Thu, 14 Dec 2023 12:06:41 GMT, Serguei Spitsyn <sspitsyn at openjdk.org> wrote:

> Carrier thread also can be suspended when executing the "critical code". Why do you think it can't be a problem? Do you think the deadlocking scenario described in the bug report is not possible?

It's a different scenario. When mounting, the coordination of the interrupt status is done before the thread identity is changed. Similarly, when unmounting, the coordination is done after reverting the thread identity to the carrier. So if there is an agent randomly suspending threads when it shouldn't be an issue here. 

> > toggle_is_in_critical_section needs to detect reentrancy, it is otherwise too easy to  refactor the Java code, e.g. call threadState while holding the interrupt lock.
> 
> Is your concern a recursive `interruptLock` enter? I was also thinking if this scenario is possible, so a counter can be used instead of boolean.

Minimally an assert. A counter might be needed later.


> Okay. What about the Leonid's suggestion to name it `notifyJvmtiDisableSuspend()` ?

We have changes in the works that require pinning during some critical sections so I think I prefer to use that terminology. We can move the notification to JVMTI to the enter/leave methods.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/17011#issuecomment-1855748841