RFR: 8361103: java_lang_Thread::async_get_stack_trace does not properly protect JavaThread [v4]

[Serguei Spitsyn]

On Wed, 16 Jul 2025 07:33:08 GMT, Serguei Spitsyn <sspitsyn at openjdk.org> wrote:

>> Sorry for being late to the party. I still need to walk through all the comments but it seems you have already caught the main problem with virtual threads. Nothing protects them from being concurrently mounted/unmounted/remounted. It is why the `JvmtiVTMSTransitionDisabler` is used in the JVMTI code. But, please, note that the `JvmtiVTMSTransitionDisabler` mechanism is enabled only when there is a JVMTI agent. Otherwise, it has been disabled for scalability purposes to exclude potentially high performance overhead at the VTMS transition points.
>
> With `JvmtiVTMSTransitionDisabler` there is normally no need in the carrier thread protection. A virtual thread is either mounted or unmounted. There is no carrier thread when virtual thread is unmounted. If virtual thread is mounted and VTMS transitions are disabled then the carrier thread is blocked until VTMS transitions are enabled again and the execution control is returned to the carrier. But JVMTI is playing by the established rules and always using TLH to protect the JavaThread associated with the virtual thread and its carrier.

> @sspitsyn can you clarify this please. When you take a thread dump via the HotsotDiagnosticMXBean is there a JVMTI agent involved? Because that code uses the transition disabler.

Yes, we have this issue for `HotsotDiagnosticMXBean.dumpThreads` as Alan has pointed out. We are aware about it and will evaluate a couple of fixing approaches.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26119#discussion_r2212582215