RFR: 8281615: Deadlock caused by jdwp agent [v4]

[Zhengyu Gu]

On Thu, 17 Feb 2022 02:25:15 GMT, David Holmes <dholmes at openjdk.org> wrote:

> The correctness here depends on exactly how these different functions can be called. I would have expected the allowed sequence to be: activate -> 1or more processUnloads -> reset
> 
> but given processUnloads can somehow occur concurrently with activate, there is obviously more at play here. The main question for me is whether reset can be called more than once and whether it too can be called when processUnloads is still executing? If yes then there are still problems.

`classTrack_activate()` is called inside `debugLoop_run()`, so it looks like that jvmti callback is enabled at the time.

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libjdwp.so+0xf0ea]  classTrack_activate+0x3a
C  [libjdwp.so+0x166dd]  installHandler+0xdd
C  [libjdwp.so+0x618c]  setCommand+0x17c
C  [libjdwp.so+0x13cd9]  debugLoop_run+0x299
C  [libjdwp.so+0x275e4]  attachThread+0x54
V  [libjvm.so+0x12be611]  JvmtiAgentThread::call_start_function()+0x181
V  [libjvm.so+0x1a9e496]  JavaThread::thread_main_inner()+0x5c6
V  [libjvm.so+0x1aa7100]  Thread::call_run()+0x100
V  [libjvm.so+0x1666814]  thread_native_entry(Thread*)+0x104

-------------

PR: https://git.openjdk.java.net/jdk/pull/7461