RFR: 8369505: jhsdb jstack cannot handle continuation stub [v9]
Yasumasa Suenaga
ysuenaga at openjdk.org
Wed Oct 15 00:53:17 UTC 2025
On Tue, 14 Oct 2025 05:27:46 GMT, Yasumasa Suenaga <ysuenaga at openjdk.org> wrote:
>> I tried to get mixed thread dump of the application which runs virtual threads (see [Test.java on JBS](https://bugs.openjdk.org/secure/attachment/116453/Test.java)) via `jhsdb jstack --mixed`, then I got following message:
>>
>>
>> sun.jvm.hotspot.utilities.AssertionFailure: must have non-zero frame size
>> at jdk.hotspot.agent/sun.jvm.hotspot.utilities.Assert.that(Assert.java:32)
>> at jdk.hotspot.agent/sun.jvm.hotspot.runtime.x86.X86Frame.senderForCompiledFrame(X86Frame.java:374)
>> at jdk.hotspot.agent/sun.jvm.hotspot.runtime.x86.X86Frame.sender(X86Frame.java:273)
>> at jdk.hotspot.agent/sun.jvm.hotspot.runtime.Frame.sender(Frame.java:225)
>> at jdk.hotspot.agent/sun.jvm.hotspot.runtime.Frame.realSender(Frame.java:230)
>> at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VFrame.sender(VFrame.java:120)
>> at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VFrame.javaSender(VFrame.java:150)
>> at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.initJFrameCache(PStack.java:224)
>> at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:73)
>> at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:65)
>> at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:60)
>> at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.run(JStack.java:67)
>> at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:278)
>> at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:241)
>> at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:134)
>> at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.runWithArgs(JStack.java:90)
>> at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJSTACK(SALauncher.java:306)
>> at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:507)
>>
>>
>> And also I got following (strange) stacks which causes `AssersionFailure` in above:
>>
>>
>> ----------------- 70094 -----------------
>> "ForkJoinPool-1-worker-4" #32 daemon prio=5 tid=0x00007f8f5c371660 nid=70094 runnable [0x00007f8f406d9000]
>> java.lang.Thread.State: RUNNABLE
>> JavaThread state: _thread_in_native
>> 0x00007f8f64658462 __syscall_cancel_arch + 0x32
>> 0x00007f8f6464c75c __internal_syscall_cancel + 0x5c
>> 0x00007f8f646a8c37 __GI___nanosleep + 0x17
>> 0x00007f8f646bb14e __sleep + 0x3e
>> 0x00007f8f4b3a8e1e <nep_invoker_blob>
>> 0x00007f8f4b33fe48 * java.lang.invoke.LambdaForm$MH+0x000000000c047000.invoke(...
>
> Yasumasa Suenaga has updated the pull request incrementally with one additional commit since the last revision:
>
> Update TestJhsdbJstackWithVirtualThread.java
I guess `jhsdb jstack --mixed` might have potential bug(s) in `-Xcomp` regardless of continuation stub handling.
I ran `jhsdb jstack --mixed` with this PR to attach to the process running with `-Xcomp` on Linux x86_64, I got similar strange stack traces (without error) as following. It seems not to unwind completely in `-Xcomp`. LWP 1332 was failed to find sender frame from 0x7f9debcd96c0, LWP 1331 and 1330 are not compiled frame, but they couldn't find sender frame (maybe stack unwinder in jhsdb decide to stop unwinding because sender frame is NULL).
----------------- 1332 -----------------
"ForkJoinPool-1-worker-2" #29 daemon prio=5 tid=0x00007f9dfc2ad7b0 nid=1332 runnable [0x00007f9dd19fd000]
java.lang.Thread.State: RUNNABLE
JavaThread state: _thread_in_native
0x00007f9e0337e462 __syscall_cancel_arch + 0x32
0x00007f9e0337275c __internal_syscall_cancel + 0x5c
0x00007f9e033cec37 __GI___nanosleep + 0x17
0x00007f9e033e114e __sleep + 0x3e
0x00007f9deb4b361e <nep_invoker_blob>
0x00007f9debcd96c0 * java.lang.invoke.LambdaForm$MH+0x000000000f047000.invoke(java.lang.Object, long, int) bci:10 (Compiled frame)
0x8b486428ec834853 ????????
----------------- 1331 -----------------
"ForkJoinPool-1-worker-1" #27 daemon prio=5 tid=0x00007f9dfc2ac5b0 nid=1331 runnable [0x00007f9dd1afd000]
java.lang.Thread.State: RUNNABLE
JavaThread state: _thread_in_native
0x00007f9e0337e462 __syscall_cancel_arch + 0x32
0x00007f9e0337275c __internal_syscall_cancel + 0x5c
0x00007f9e033cec37 __GI___nanosleep + 0x17
0x00007f9e033e114e __sleep + 0x3e
0x00007f9deb4b361e <nep_invoker_blob>
0x00007f9de45d5c2c * java.lang.invoke.LambdaForm$MH+0x000000000f047000.invoke(java.lang.Object, long, int) bci:10 (Compiled frame)
* java.lang.invoke.LambdaForm$MH+0x000000000f04f000.invokeExact_MT(java.lang.Object, long, int, java.lang.Object) bci:21 (Compiled frame)
* jdk.internal.foreign.abi.DowncallStub+0x000000000f048000.invoke(java.lang.foreign.SegmentAllocator, java.lang.foreign.MemorySegment, int) bci:44 (Interpreted frame)
----------------- 1330 -----------------
"VirtualThread-unblocker" #25 daemon prio=5 tid=0x00007f9dfc297b50 nid=1330 runnable [0x00007f9dd1bfd000]
java.lang.Thread.State: RUNNABLE
JavaThread state: _thread_blocked
0x00007f9e0337e462 __syscall_cancel_arch + 0x32
0x00007f9e0337275c __internal_syscall_cancel + 0x5c
0x00007f9e0337549e __GI___pthread_cond_wait + 0x14e
0x00007f9e01f0d834 PlatformEvent::park() + 0xe4
0x00007f9e019889ca JVM_TakeVirtualThreadListToUnblock + 0x1ba
0x00007f9debc9cc16 java.lang.VirtualThread.takeVirtualThreadListToUnblock() + 0x96 (Native method)
In addition, I saw following error when I added both `-Xcomp` and `-XX:+PreserveFramePointer` in Linux x86_64. Note that I haven't seen this with `-XX:+PreserveFramePointer` only.
java.lang.RuntimeException: newSP(0x00007f345cb14690) is not above oldSP(0x00007f345cb14690)
at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VFrame.javaSender(VFrame.java:156)
at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.initJFrameCache(PStack.java:224)
at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:73)
at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:65)
at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:60)
at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.run(JStack.java:67)
at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:278)
at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:241)
at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:134)
at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.runWithArgs(JStack.java:90)
at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJSTACK(SALauncher.java:306)
at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:507)
----------------- 1425 -----------------
"ForkJoinPool-1-worker-4" #33 daemon prio=5 tid=0x00007f34782bac50 nid=1425 runnable [0x00007f345ca14000]
java.lang.Thread.State: RUNNABLE
JavaThread state: _thread_in_native
0x00007f3482592462 __syscall_cancel_arch + 0x32
0x00007f348258675c __internal_syscall_cancel + 0x5c
0x00007f34825e2c37 __GI___nanosleep + 0x17
0x00007f34825f514e __sleep + 0x3e
0x00007f34673a899e <nep_invoker_blob>
0x00007f3467ce4d44 * java.lang.invoke.LambdaForm$MH+0x0000000034047000.invoke(java.lang.Object, long, int) bci:10 (Compiled frame)
0x00007f3467ce5094 * java.lang.invoke.LambdaForm$MH+0x000000003404e800.invokeExact_MT(java.lang.Object, long, int, java.lang.Object) bci:21 (Compiled frame)
0x00007f346733fe48 * jdk.internal.foreign.abi.DowncallStub+0x0000000034048000.invoke(java.lang.foreign.SegmentAllocator, java.lang.foreign.MemorySegment, int) bci:44 (Interpreted frame)
0x00007f3460674744 * java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(java.lang.Object, java.lang.Object, java.lang.Object, int) bci:14 (Compiled frame)
0x00007f3460739944 * java.lang.invoke.LambdaForm$MH+0x000000003404c800.invoke(java.lang.Object, int) bci:44 (Compiled frame)
0x00007f346072f3ac * java.lang.invoke.LambdaForm$MH+0x000000003404b800.invoke_MT(java.lang.Object, int, java.lang.Object) bci:18 (Compiled frame)
0x00007f346733fd00 * Test.run() bci:9 line:40 (Interpreted frame)
0x00007f346733e098 <StubRoutines (continuation stubs)>
0x00007f3467cbeb50 * jdk.internal.vm.Continuation.run() bci:152 line:251 (Compiled frame)
0x00007f3467cd0eb8 * java.lang.VirtualThread.runContinuation() bci:100 line:293 (Compiled frame)
0x00007f3467cbbf40 * java.lang.VirtualThread$$Lambda+0x000000003402a848.run() bci:4 (Compiled frame)
0x00007f3467cd13e8 * java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec() bci:4 line:1596 (Compiled frame)
0x00007f3467cd01e0 * java.util.concurrent.ForkJoinTask.doExec() bci:10 line:511 (Compiled frame)
* java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(java.util.concurrent.ForkJoinTask, int) bci:5 line:1450 (Compiled frame)
0x00007f346733fd00 * java.util.concurrent.ForkJoinPool.runWorker(java.util.concurrent.ForkJoinPool$WorkQueue) bci:364 line:2019 (Interpreted frame)
0x00007f3467cb6b04 * java.util.concurrent.ForkJoinWorkerThread.run() bci:31 line:187 (Compiled frame)
0x00007f34673386fd <StubRoutines (initial stubs)>
0x00007f34809c0a7e JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*) + 0x4ce
0x00007f34809c11b3 JavaCalls::call_virtual(JavaValue*, Klass*, Symbol*, Symbol*, JavaCallArguments*, JavaThread*) + 0x2d3
0x00007f34809c17bb JavaCalls::call_virtual(JavaValue*, Handle, Klass*, Symbol*, Symbol*, JavaThread*) + 0xab
0x00007f3480b83a90 thread_entry(JavaThread*, JavaThread*) + 0xd0
0x00007f3480a02806 JavaThread::thread_main_inner() + 0x256
0x00007f3481665b57 Thread::call_run() + 0xb7
0x00007f34810fc588 thread_native_entry(Thread*) + 0x128
0x00007f3482589f54 start_thread + 0x2e4
----------------- 1424 -----------------
"ForkJoinPool-1-worker-3" #31 daemon prio=5 tid=0x00007f34782b6d00 nid=1424 runnable [0x00007f345cb14000]
java.lang.Thread.State: RUNNABLE
JavaThread state: _thread_in_native
0x00007f3482592462 __syscall_cancel_arch + 0x32
0x00007f348258675c __internal_syscall_cancel + 0x5c
0x00007f34825e2c37 __GI___nanosleep + 0x17
0x00007f34825f514e __sleep + 0x3e
0x00007f34673a899e <nep_invoker_blob>
0x00007f3467ce4d44 * java.lang.invoke.LambdaForm$MH+0x0000000034047000.invoke(java.lang.Object, long, int) bci:10 (Compiled frame)
0x00007f346074ed84 * java.lang.invoke.LambdaForm$MH+0x000000003404e800.invokeExact_MT(java.lang.Object, long, int, java.lang.Object) bci:21 (Compiled frame)
0x00007f346733fe48 * jdk.internal.foreign.abi.DowncallStub+0x0000000034048000.invoke(java.lang.foreign.SegmentAllocator, java.lang.foreign.MemorySegment, int) bci:44 (Interpreted frame)
0x00007f3467cda978 * java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(java.lang.Object, java.lang.Object, java.lang.Object, int) bci:14 (Compiled frame)
0x00007f3467cd9118 * java.lang.invoke.LambdaForm$MH+0x000000003404d000.invoke(java.lang.Object, int) bci:44 (Compiled frame)
0x00007f3467cd5c94 * java.lang.invoke.LambdaForm$MH+0x000000003404b800.invoke_MT(java.lang.Object, int, java.lang.Object) bci:18 (Compiled frame)
0x00007f34607129fc * Test.run() bci:9 line:40 (Compiled frame)
0x00007f346733e098 <StubRoutines (continuation stubs)>
0x00007f3467cbeb50 <Unknown compiled code>
0x00007f3467cd0eb8 <Unknown compiled code>
0x00007f3467cbbf40 <Unknown compiled code>
0x00007f3460726924 <Unknown compiled code>
0x00007f3460723b24 <Unknown compiled code>
0x00007f346733fd00 <interpreter> return entry points
0x00007f3467cb6b04 <Unknown compiled code>
0x00007f34673386fd <StubRoutines (initial stubs)>
0x00007f34809c0a7e JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*) + 0x4ce
0x00007f34809c11b3 JavaCalls::call_virtual(JavaValue*, Klass*, Symbol*, Symbol*, JavaCallArguments*, JavaThread*) + 0x2d3
0x00007f34809c17bb JavaCalls::call_virtual(JavaValue*, Handle, Klass*, Symbol*, Symbol*, JavaThread*) + 0xab
0x00007f3480b83a90 thread_entry(JavaThread*, JavaThread*) + 0xd0
0x00007f3480a02806 JavaThread::thread_main_inner() + 0x256
0x00007f3481665b57 Thread::call_run() + 0xb7
0x00007f34810fc588 thread_native_entry(Thread*) + 0x128
0x00007f3482589f54 start_thread + 0x2e4
This is a problem in restoring Java frame, not native frame - native frame unwinding depends on DWARF in Linux x86_64, on FP ( `X29` ) in AArch64. Thus we might need to fix unwinder for Java frames. But it is another issue.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/27728#issuecomment-3404126783
More information about the hotspot-runtime-dev
mailing list