RFR: 8337331: crash: pinned virtual thread will lead to jvm crash when running with the javaagent option [v2]

[Alan Bateman]

On Mon, 29 Jul 2024 11:26:06 GMT, Jiawei Tang <jwtang at openjdk.org> wrote:

>> test/hotspot/jtreg/serviceability/jvmti/vthread/VThreadTraceWithAgent/TestPinCaseWithTrace.java line 70:
>> 
>>> 68:     static int native2Java(int b) {
>>> 69:         try {
>>> 70:             Thread.sleep(500); // try yield, will pin, javaagent+tracePinnedThreads will crash here (because of the class `PinnedThreadPrinter`)
>> 
>> As noted in the JBS issue, -Djdk.tracePinnedThreads has been very problematic and has been removed in the loom repo as part of the object monitor changes.
>
> I have read the code in loom and this issue can be resolved by using JFR event instead. But I hope this could be fixed since using javaagent is very common in java application. The root cause is that no new class should be use after the vthread is pinned, since a agent can change the class bytecode and need to use `JvmtiVTMSTransitionDisabler` when transforming class. However, this vthread is in VTMS, it cannot jump out the loop.
> 
> Using `-Djdk.tracePinnedThreads=full` will use the class `PinnedThreadPrinter` so we end in a deadlock.

I have no objection to fixing JVMTI, I'm just pointing out that -Djdk.tracePinnedThreads has been very problematic and many other reasons so it will be proposed to be removed when we bring the changes to main line.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/20373#discussion_r1695136030