RFR: 8361103: java_lang_Thread::async_get_stack_trace does not properly protect JavaThread [v4]

[David Holmes]

On Thu, 17 Jul 2025 07:51:46 GMT, Serguei Spitsyn <sspitsyn at openjdk.org> wrote:

>> With `JvmtiVTMSTransitionDisabler` there is normally no need in the carrier thread protection. A virtual thread is either mounted or unmounted. There is no carrier thread when virtual thread is unmounted. If virtual thread is mounted and VTMS transitions are disabled then the carrier thread is blocked until VTMS transitions are enabled again and the execution control is returned to the carrier. But JVMTI is playing by the established rules and always using TLH to protect the JavaThread associated with the virtual thread and its carrier.
>
>> @sspitsyn can you clarify this please. When you take a thread dump via the HotsotDiagnosticMXBean is there a JVMTI agent involved? Because that code uses the transition disabler.
> 
> Yes, we have this issue for `HotsotDiagnosticMXBean.dumpThreads` as Alan has pointed out. We are aware about it and will evaluate a couple of fixing approaches.

> Thread::getStackTrace handles this with a SUSPEND bit and loop.

Okay so that loop keeps retrying any time we get a null stacktrace from the VM, and unless we are extremely unlucky it should only take a couple of attempts to get an actual stack.

So that just leaves the issue of the carrier thread terminating, which is what this current fix was trying to address. So we need to ensure the carrier is protected by the TLH, which is what my other fix will address.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26119#discussion_r2218159540