[jdk19] RFR: 8289709: fatal error: stuck in JvmtiVTMSTransitionDisabler::disable_VTMS_transitions [v2]

[Alan Bateman]

On Sat, 9 Jul 2022 07:34:48 GMT, Serguei Spitsyn <sspitsyn at openjdk.org> wrote:

>> This is a test bug. The test should filter out non-tested threads to avoid generating such kind of deadlocks. 
>> 
>> In short, the deadlock dependencies are:
>> 
>> - The `Common-Cleaner` thread is executing the JVM TI agent `MethodEntry` event callback which grabbed the `agent_lock` raw monitor and calls JVM TI `GetFrameCount`. The `GetFrameCount` is blocked in disabling VTMS transitions because the `ForkJoinPool-1-worker-2` is at a mount (VTMS) transition.
>> - The `ForkJoinPool-1-worker-2` is at mount (VTMS) transition and blocked in `java.lang.ref.NativeReferenceQueue.poll()` when acquiring the `NativeReferenceQueue` lock which is held by the `Reference Handler` thread.
>> - The `Reference Handler` thread grabbed the `NativeReferenceQueue` lock and is entering the `signal()` method. It triggered a JVM TI `MethodEntry` event. The JVM TI agent `MethodEntry` event callback is blocked on grabbing the `agent_lock` raw monitor which is held by the `Common-Cleaner` thread.
>> 
>> Also, the `timeout=360 `is explicitly set to avoid frequent timeouts in locals runs. 
>> 
>> Testing: submitted mach5 job with 100 runs on 3 debug platforms.
>
> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision:
> 
>   adjusted max number of threads instead of adding explicit timeout

test/hotspot/jtreg/serviceability/jvmti/events/FramePop/framepop02/libframepop02.cpp line 75:

> 73:   jvmtiThreadInfo inf;
> 74:   const char* TEST_THREAD_NAME_BASE = "Test Thread";
> 75:   check_jvmti_status(jni, jvmti->GetThreadInfo(thr, &inf), "Error in GetThreadInfo.");

In passing, should isTestThread deallocate inf.name to avoid leaking memory?

-------------

PR: https://git.openjdk.org/jdk19/pull/129