RFR: 8301639: JDI and JDWP specs should clarify potential deadlock issues with method invocation

[David Holmes]

On Wed, 13 Sep 2023 01:03:49 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> This PR includes a couple of clarifications of the JDWP and JDI invoke method support. The first is that deadlocks can occur due to a needed "resource" being held. The spec previously just mentioned monitors being held, but this is too specific. The second is to clarify that an invoke on a virtual thread can be more prone to deadlocks than a platform thread may be.
>> 
>> I did a doc build you can look at to view the changes. Links to the relevant sections are below:
>> 
>> https://cr.openjdk.org/~cjplummer/8301639/docs.00/specs/jdwp/jdwp-protocol.html#JDWP_ClassType_InvokeMethod
>> 
>> https://cr.openjdk.org/~cjplummer/8301639/docs.00/specs/jdwp/jdwp-protocol.html#JDWP_ClassType_NewInstance
>> 
>> https://cr.openjdk.org/~cjplummer/8301639/docs.00/specs/jdwp/jdwp-protocol.html#JDWP_InterfaceType_InvokeMethod
>> 
>> https://cr.openjdk.org/~cjplummer/8301639/docs.00/specs/jdwp/jdwp-protocol.html#JDWP_ObjectReference_InvokeMethod
>> 
>> https://cr.openjdk.org/~cjplummer/8301639/docs.00/api/jdk.jdi/com/sun/jdi/ClassType.html#invokeMethod(com.sun.jdi.ThreadReference,com.sun.jdi.Method,java.util.List,int)
>> 
>> https://cr.openjdk.org/~cjplummer/8301639/docs.00/api/jdk.jdi/com/sun/jdi/ClassType.html#newInstance(com.sun.jdi.ThreadReference,com.sun.jdi.Method,java.util.List,int)
>> 
>> https://cr.openjdk.org/~cjplummer/8301639/docs.00/api/jdk.jdi/com/sun/jdi/InterfaceType.html#invokeMethod(com.sun.jdi.ThreadReference,com.sun.jdi.Method,java.util.List,int)
>> 
>> https://cr.openjdk.org/~cjplummer/8301639/docs.00/api/jdk.jdi/com/sun/jdi/ObjectReference.html#invokeMethod(com.sun.jdi.ThreadReference,com.sun.jdi.Method,java.util.List,int)
>
> src/java.se/share/data/jdwp/jdwp.spec line 1178:
> 
>> 1176:         "support the timer mechanism for virtual threads, and thus methods such as "
>> 1177:         "<a href=../../api/java.base/java/lang/Thread.html#sleep(long)>Thread.sleep(long)</a> "
>> 1178:         "may deadlock."
> 
> I think this is an implementation detail that you don't really want to document in these unrelated specifications. I'm also unclear why virtual threads are more of a problem here than regular threads, if you have suspended them all - a regular thread won't return from sleep until it is unsuspended. In fact the more I think about this, the less I see why virtual threads need special mention here at all.

Of course as soon as I posted that I now realize the issue. There are more things that could "deadlock" virtual threads when all other threads are suspended - such as calling sleep, because it requires a "system" thread to be active. This suggest to me that we need modifications to JDWP/JDI to support effective debugging of virtual threads, by allowing for the notion of "system threads" that by default are not suspended (or else always resumed for an invoke).

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15695#discussion_r1323807591