RFR: 8328866: Add raw monitor rank support to the debug agent. [v7]

[Chris Plummer]

On Thu, 16 May 2024 20:02:22 GMT, Chris Plummer <cjplummer at openjdk.org> wrote:

> Ironically right now I'm looking at a very rare deadlock that involves this code. It doesn't seem to happen when I disabled ranked locking. It might be instigated by the dbgRawMonitor that ranked locking uses.

@kevinjwalls I tracked down this deadlock and filed [JDK-8332738](https://bugs.openjdk.org/browse/JDK-8332738). It's really a pre-existing issue, but we get lucky with the current implementation because RawMonitorExit does not self suspend until after releasing the monitor, thus avoiding the deadlock. The ranked monitors implementation adds a self suspend opportunity when we release a raw monitor, which is it hits this bug. This is a very ugly issue that involves two threads getting events at the same time, and the debugger doing a StackFrame.PopFrames. Fortunately the "pop frames" locks are not involved.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19044#discussion_r1610404920