RFR: 8281615: Deadlock caused by jdwp agent [v4]

[Zhengyu Gu]

On Thu, 17 Feb 2022 17:34:41 GMT, Chris Plummer <cjplummer at openjdk.org> wrote:

>>> The correctness here depends on exactly how these different functions can be called. I would have expected the allowed sequence to be: activate -> 1or more processUnloads -> reset
>>> 
>>> but given processUnloads can somehow occur concurrently with activate, there is obviously more at play here. The main question for me is whether reset can be called more than once and whether it too can be called when processUnloads is still executing? If yes then there are still problems.
>> 
>> `classTrack_activate()` is called inside `debugLoop_run()`, so it looks like that jvmti callback is enabled at the time.
>> 
>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
>> C  [libjdwp.so+0xf0ea]  classTrack_activate+0x3a
>> C  [libjdwp.so+0x166dd]  installHandler+0xdd
>> C  [libjdwp.so+0x618c]  setCommand+0x17c
>> C  [libjdwp.so+0x13cd9]  debugLoop_run+0x299
>> C  [libjdwp.so+0x275e4]  attachThread+0x54
>> V  [libjvm.so+0x12be611]  JvmtiAgentThread::call_start_function()+0x181
>> V  [libjvm.so+0x1a9e496]  JavaThread::thread_main_inner()+0x5c6
>> V  [libjvm.so+0x1aa7100]  Thread::call_run()+0x100
>> V  [libjvm.so+0x1666814]  thread_native_entry(Thread*)+0x104
>
>> `classTrack_activate()` is called inside `debugLoop_run()`, so it looks like that jvmti callback is enabled at the time.
> 
> Yes. This is due to the debugger sending a JDWP EventRequest command for CLASS_UNLOAD events. At that point class tracking is activated and the debug agent enables OBJECT_FREE events.

@plummercj Thanks for the review.

@dholmes-ora Are you okay with current version? Thanks.

-------------

PR: https://git.openjdk.java.net/jdk/pull/7461