RFR: 8306705: com/sun/jdi/PopAndInvokeTest.java fails with NativeMethodException

[Leonid Mesnik]

On Tue, 25 Apr 2023 21:07:34 GMT, Chris Plummer <cjplummer at openjdk.org> wrote:

> Recently [JDK-8305511](https://bugs.openjdk.org/browse/JDK-8305511) removed the "@ignore 6951287 ", allowing this test to run, which is why this failure mode is now turning up.
> 
> There is a race condition, leading to the `popFrames()` call being done while a native method is in the set of frames to be popped. This results in a `NativeMethodException` instead of the frames being popped. The debuggee has:
> 
> 
>     public static void waiter() {
>         if (waiting) {
>             return;
>         }
>         waiting = true;
>         System.out.println(" debuggee: in waiter");
>         while (true) {
>         }
>     }
> 
> 
> And the debugger waits for `waiting == true` (checked via JDI calls) before suspending and doing the `popFrames()`. The problem is the println() after setting `waiting = true`. The debugger side can detect that `waiting == true` before the println() is complete, and the println() involves native code. Once `waiting` is set true, we need to make sure no other method calls are made so we can be sure that only the `waiter()` method is on the stack.
> 
> Note there is a lot of interesting history to this CR, including [JDK-6417053](https://bugs.openjdk.org/browse/JDK-6417053), which actually reproduced this issue long ago (but got closed as CNR), although it failed with a different error message (even though it was the same issue), and the different error message was itself the result of another bug that was inadvertently fixed when virtual threads support was added to JDI. See the CR for details if you are interested.

Marked as reviewed by lmesnik (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/13657#pullrequestreview-1400777256