RFR 8193879: Java debugger hangs on method invocation

[Gary Adams]

In TTY.java do you need to force a simple prompt for the
breakpoint event output? What prevents currentThread from
being set at the time you are printing the prompt?


  103         // Print the prompt if suspend policy is 
SUSPEND_EVENT_THREAD. In case of
  104         // SUSPEND_ALL policy this is handled by vmInterrupted() 
method.
  105         if (be.request().suspendPolicy() == 
EventRequest.SUSPEND_EVENT_THREAD) {
  106             MessageOutput.println();
  107             MessageOutput.printPrompt();


In Jdb.java you allow the waiting for the simple prompt.
I don't see waitForSimplePrompt used in any existing tests.
Since it is only looking for a single character it could
produce false positives if the '>' was produced in the
output stream. Is this wait paired to the added prompt for the
break point event?

  236         return waitForSimplePrompt ? waitForSimplePrompt(1, 
cmd.allowExit) : waitForPrompt(1, cmd.allowExit);

It might be a good idea to include a test with multiple
threads each with a breakpoint that will trigger SUSPEND_EVENT_THREAD
behavior.

On 10/4/18, 12:29 AM, Daniil Titov wrote:
> Please review the changes that fix the deadlock in the debugger when the debugger is running with the tracing option on.
>
> The problem here is that when the tracing is on the "JDI Target VM Interface" thread (the thread that reads all replies and then notifies the thread that sent the request that the reply has been
> received) is waiting for a lock which is already acquired by the thread that sent the request and is waiting for reply.
>
> The fix itself is in src/jdk.jdi/share/classes/com/sun/tools/jdi/VMState.java.
>
> The patch also reverts the changes done in 8129348 "Debugger hangs in trace mode with TRACE_SENDS" in src/jdk.jdi/share/classes/com/sun/tools/jdi/InvokableTypeImpl.java since they address only a specific case (VM is suspended and static method is invoked) while the proposed fix also solves issue 8129348 as well as issue 8193801 "Debugger hangs in trace mode for non-static methods".
>
> The changes include new tests for issues 8193879, 8193801 and  8129348.
>
> The changes in src/jdk.jdi/share/classes/com/sun/tools/example/debug/tty/TTY.java solve the problem that the prompt is not printed in the debugger output when the breakpoint is hit and the suspend policy is SUSPEND_EVENT_THREAD. This is required for new tests to detect that command "stop thread at ..." is completed.
>
> Mach5 build and jdk_jdi tests succeeded.
>
> Webrev: http://cr.openjdk.java.net/~dtitov/8193879/webrev.01/
> Issue: https://bugs.openjdk.java.net/browse/JDK-8193879
>
> Thanks,
> --Daniil
>
>
>