RFR: 8274687: JDWP can deadlock VM if Java thread waits in blockOnDebuggerSuspend

[Richard Reingruber]

On Tue, 5 Oct 2021 20:56:43 GMT, Chris Plummer <cjplummer at openjdk.org> wrote:

>> Regarding threadControl_resumeThread() it does appear that it would block, as would threadControl_resumeAll(), which seems problematic in that blockOnDebuggerSuspend() won't exit until the suspendCount == 0. So it's unclear to me how this is suppose to work if a resume() can't be done.
>
> It seems if we had a test case where the debugger did a ThreadReference.suspend(), then the debuggee did a Thread.resume() on the same thread, and then debugger did a ThreadReference.resume(), it would block, and with no way to unblock it.

I'll see if I can implement such a test. Also I have I prototype that defers the actions of handleAppResumeBreakpoint() until doPendingTasks() is running. This seems to work in the various jdi/jdwp tests which likely do not cover the functionality very well though.

-------------

PR: https://git.openjdk.java.net/jdk/pull/5805