RFR: 8374482: SA does not handle signal handler frame in mixed jstack [v4]

[Yasumasa Suenaga]

On Thu, 22 Jan 2026 11:30:34 GMT, Kevin Walls <kevinw at openjdk.org> wrote:

>> Yasumasa Suenaga has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
>> 
>>  - Merge remote-tracking branch 'origin/master' into jhsdb-jstack-sighandler
>>  - Merge remote-tracking branch 'origin/master' into jhsdb-jstack-sighandler
>>  - Merge remote-tracking branch 'origin/master' into jhsdb-jstack-sighandler
>>  - Fix
>>  - 8374482: SA does not handle signal handler frame in mixed jstack
>
> (sorry about delay)
> Just to discuss an alternative:
> 
> When LinuxAMD64CFrame.sender() finds nextPC, it could recognise the signal handler trampoline, 
> by looking at the return address nextPC and checking if it points to the bytes of the trampoline:
> 0x7f5e5e094610 <__restore_rt>:  0x48    0xc7    0xc0    0x0f    0x00    0x00    0x00    0x0f
> (which look as stable as everything else we are relying on here)
> 
> Then the new frame is created with boolean isSigTrampoline set as a member in LinuxAMD64CFrame.
> 
> That avoids needing the symbol lookup to decide if it's a trampoline.
> Then in LinuxAMD64CFrame:
> closestSymbolToPC() would return the special string if isSigTrampoline without a lookup.
> isValidFrame would use the isSigTrampoline member, no new parameter needed.
> getNextCFA would not need to check if it's a trampoline and pass a value to isValidFrame.
> sender() would not need the symbol lookup, it has the member boolean.
> 
> Does that make it simpler overall?
> 
> It would avoid using the symbol name to know if this is a signal handler trampoline, and having special meaning of "<signal handler called>" in both places.

@kevinjwalls Thanks for your comment! It sounds good. I updated PR.

> When LinuxAMD64CFrame.sender() finds nextPC, it could recognise the signal handler trampoline,
by looking at the return address nextPC and checking if it points to the bytes of the trampoline:
0x7f5e5e094610 <__restore_rt>: 0x48 0xc7 0xc0 0x0f 0x00 0x00 0x00 0x0f
(which look as stable as everything else we are relying on here)

I guess bytes of `__restore_rt`  what you show is the contents of RIP like `x/8xb $rip` in GDB. If so, it is for AMD64 only at least, and it might be different depending on compiler/compiler options for glibc.
Thus I added list of signal trampoline in `LinuxDebugger` (for AMD64 only for now). We can add if need for other architectures (AArch64 is different (`__kernel_rt_sigreturn`) at least AFAIK). It is used to detect signal trampoline in `LinuxDebuggerLocal::isSignalTrampoline`.
`closestSymbolToPC()` returns `<signal handler called>` if the PC is in signal trampoline.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/29023#issuecomment-3787822614