RFR: 8281615: Deadlock caused by jdwp agent [v2]

[Zhengyu Gu]

On Wed, 16 Feb 2022 00:53:27 GMT, Chris Plummer <cjplummer at openjdk.org> wrote:

>> I am not sure if it is possible, but checking `bagSize(deletedSignatures) == 0` seems to race against `classTrack_reset()` where it does not take `handlerLock` lock.
>
>> I am not sure if it is possible, but checking bagSize(deletedSignatures) == 0 seems to race against classTrack_reset() where it does not take handlerLock lock.
> 
> I had thought of that too, but I think the way `classTrack_reset()` is called, it is likely not possible for there to be a `classTrack_processUnloads()` also coming in because everything is shut down:
> 
> ``` 
>     threadControl_onDisconnect();
>     standardHandlers_onDisconnect();
> 
>     /*
>      * Cut off the transport immediately. This has the effect of
>      * cutting off any events that the eventHelper thread might
>      * be trying to send.
>      */
>     transport_close();
>     debugMonitorDestroy(cmdQueueLock);
> 
>     /* Reset for a new connection to this VM if it's still alive */
>     if ( ! gdata->vmDead ) {
>         debugInit_reset(getEnv());    <--- calls classTrack_reset()
>     }

But shutting down debug loop does not seem to have effect on ongoing jvmti callback, e.g. thread 4 in the bug report.

-------------

PR: https://git.openjdk.java.net/jdk/pull/7461