RFR: 8274687: JDWP deadlocks if some Java thread reaches wait in blockOnDebuggerSuspend

[Chris Plummer]

On Thu, 7 Oct 2021 13:50:49 GMT, Richard Reingruber <rrich at openjdk.org> wrote:

> This change fixes deadlocks described in the JBS-bug by:
> 
> * Releasing `handlerLock` before waiting on `threadLock` in `blockOnDebuggerSuspend()`
> 
> * Notifying on `threadLock` in `threadControl_reset()`
> 
> Also the actions in handleAppResumeBreakpoint() are moved/deferred until
> doPendingTasks() runs. This is necessary because an event handler must not
> release handlerLock first and foremost because handlers are called while
> iterating the handler chain for an event type which is protected by handlerLock
> (see https://github.com/openjdk/jdk/pull/5805)
> 
> The first commit delays the cleanup actions after leaving the loop in
> `debugLoop_run()`. It allows to reproduce the deadlock running the dispose003
> test with the command
> 
> 
> make run-test TEST=test/hotspot/jtreg/vmTestbase/nsk/jdi/VirtualMachine/dispose/dispose003
> 
> 
> The second commit adds a new test that reproduces the deadlock when calling
> threadControl_resumeThread() while a thread is waiting in
> blockOnDebuggerSuspend().
> 
> The third commit contains the fix described above. With it the deadlocks
> cannot be reproduced anymore.
> 
> The forth commit removes the diagnostic code introduced with the first commit again.
> 
> The fix passed
> 
> test/hotspot/jtreg/serviceability/jdwp
> test/jdk/com/sun/jdi
> test/hotspot/jtreg/vmTestbase/nsk/jdwp
> test/hotspot/jtreg/vmTestbase/nsk/jdi

test/jdk/com/sun/jdi/ResumeAfterThreadResumeCallTest.java line 31:

> 29:  *          the JDWP agent (in blockOnDebuggerSuspend()) because it called
> 30:  *          j.l.Thread.resume() on a thread R that was suspended by the
> 31:  *          debugger.

This is hard to follow. Maybe instead document the steps the test takes. For example maybe something like: Suspend Thread R via breakpoint. Thread T calls j.l.Thread.resume() on Thread R, resulting in Thread T blocking in blockOnDebuggerSuspend. Resume Thread R using ThreadReference.resume(). Verify that Thread T is no longer  blocked in blockOnDebuggerSuspend.

Also, it would be nice if thread names in the description matched the names used in the implementation.

-------------

PR: https://git.openjdk.java.net/jdk/pull/5849