RFR: 8281615: Deadlock caused by jdwp agent [v2]

[Zhengyu Gu]

On Tue, 15 Feb 2022 06:31:46 GMT, Chris Plummer <cjplummer at openjdk.org> wrote:

>> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   David and Chris' comments
>
> src/jdk.jdwp.agent/share/native/libjdwp/classTrack.c line 65:
> 
>> 63:  * handler may acquire the same monitor(s), e.g. classTrackLock in cbTrackingObjectFree(),
>> 64:  * which can lead to deadlock.
>> 65:  */
> 
> The debug agent does a lot alloc/dealloc calls while processing JVMTI events and holding locks. So the question is why this is not problematic other than this classTrackLock issue. I believe the answer is that cbTrackingObjectFree() is unique in that it is handled outside of the debug agent's normal event handling code, which is serialized. cbTrackingObjectFree() is called so the debug agent can maintain a list of loaded classes, and is not an event that gets passed on to the debugger like most JVMTI events are. So we have a case here where classTrackLock can be grabbed by both "typical" JVMTI event handling code via the classTrack_processUnloads() call, and then this special cbTrackingObjectFree() event handling.
> 
> I think having this comment here doesn't help the reader of the code below unless they somehow read the comment first and then recognized it's application in the code below. At the very least the code below should tersely state while the lock is being released, and then refer to this comment for details.

I moved comments inside `classTrack_processUnloads()`, let me know if you prefer other places.

-------------

PR: https://git.openjdk.java.net/jdk/pull/7461