RFR: 8369505: jhsdb jstack --mixed cannot handle continuation stub on Linux AMD64

[Patricio Chilano Mateo]

On Thu, 9 Oct 2025 14:11:39 GMT, Yasumasa Suenaga <ysuenaga at openjdk.org> wrote:

> I tried to get mixed thread dump of the application which runs virtual threads (see [Test.java on JBS](https://bugs.openjdk.org/secure/attachment/116453/Test.java)) via `jhsdb jstack --mixed`, then I got following message:
> 
> 
> sun.jvm.hotspot.utilities.AssertionFailure: must have non-zero frame size
>         at jdk.hotspot.agent/sun.jvm.hotspot.utilities.Assert.that(Assert.java:32)
>         at jdk.hotspot.agent/sun.jvm.hotspot.runtime.x86.X86Frame.senderForCompiledFrame(X86Frame.java:374)
>         at jdk.hotspot.agent/sun.jvm.hotspot.runtime.x86.X86Frame.sender(X86Frame.java:273)
>         at jdk.hotspot.agent/sun.jvm.hotspot.runtime.Frame.sender(Frame.java:225)
>         at jdk.hotspot.agent/sun.jvm.hotspot.runtime.Frame.realSender(Frame.java:230)
>         at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VFrame.sender(VFrame.java:120)
>         at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VFrame.javaSender(VFrame.java:150)
>         at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.initJFrameCache(PStack.java:224)
>         at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:73)
>         at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:65)
>         at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:60)
>         at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.run(JStack.java:67)
>         at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:278)
>         at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:241)
>         at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:134)
>         at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.runWithArgs(JStack.java:90)
>         at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJSTACK(SALauncher.java:306)
>         at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:507)
> 
> 
> And also I got following (strange) stacks which causes `AssersionFailure` in above:
> 
> 
> ----------------- 70094 -----------------
> "ForkJoinPool-1-worker-4" #32 daemon prio=5 tid=0x00007f8f5c371660 nid=70094 runnable [0x00007f8f406d9000]
>    java.lang.Thread.State: RUNNABLE
>    JavaThread state: _thread_in_native
> 0x00007f8f64658462 __syscall_cancel_arch + 0x32
> 0x00007f8f6464c75c __internal_syscall_cancel + 0x5c
> 0x00007f8f646a8c37 __GI___nanosleep + 0x17
> 0x00007f8f646bb14e __sleep + 0x3e
> 0x00007f8f4b3a8e1e <nep_invoker_blob>
> 0x00007f8f4b33fe48 * java.lang.invoke.LambdaForm$MH+0x000000000c047000.invoke(java.lang.Object, long, int) bci:10 (Interpreted frame)
> 0x00007f8f4b33fe48...

I can reproduce it with the attached test too. The issue happens when the return pc of the sender frame is the return barrier, which will be the case if the virtual thread was unmounted at some earlier point and there are still frames freezed in the stackChunk (see `frame::sender_for_interpreter_frame` and `frame::sender_for_compiled_frame` for handling this case in the VM). In your test, unmounting can happen at the `System.out.println` call.
So this issue will be present in the other platforms too. I tested the fix for x64 and looks good.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/27728#issuecomment-3390733064