RFR: 8339307: jhsdb jstack could not trace FFM upcall frame [v4]

[Jorn Vernee]

On Mon, 2 Sep 2024 10:29:40 GMT, Yasumasa Suenaga <ysuenaga at openjdk.org> wrote:

>> I attempted to check stack trace in the core generated by [SEGV example in upcall](https://github.com/YaSuenag/garakuta/blob/841452d9176dab1ddbb552009c180530eb81190b/NativeSEGV/ffm/upcall/src/main/java/com/yasuenag/garakuta/nativesegv/upcall/Main.java) with `jhsdb jstack`, however it failed with following exception.
>> 
>> 
>> Error occurred during stack walking:
>> java.lang.RuntimeException: Couldn't deduce type of CodeBlob @0x00007fa04c265990 for PC=0x00007fa04c265aa6
>>         at jdk.hotspot.agent/sun.jvm.hotspot.code.CodeCache.findBlobUnsafe(CodeCache.java:124)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.code.CodeCache.findBlob(CodeCache.java:83)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.runtime.Frame.cb(Frame.java:119)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.runtime.x86.X86Frame.adjustUnextendedSP(X86Frame.java:334)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.runtime.x86.X86Frame.initFrame(X86Frame.java:137)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.runtime.x86.X86Frame.<init>(X86Frame.java:163)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.runtime.x86.X86Frame.senderForInterpreterFrame(X86Frame.java:361)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.runtime.x86.X86Frame.sender(X86Frame.java:281)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.runtime.Frame.sender(Frame.java:207)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.runtime.Frame.realSender(Frame.java:212)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VFrame.sender(VFrame.java:120)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VFrame.javaSender(VFrame.java:144)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:81)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.run(JStack.java:67)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:278)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:241)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:134)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.runWithArgs(JStack.java:90)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJSTACK(SALauncher.java:302)
>>         at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:500)
>> Caused by: sun.jvm.hotspot.types.WrongTypeException: No suitable match for type of address 0x00007fa04c265990 (nearest symbol is _ZTV10Upcall...
>
> Yasumasa Suenaga has updated the pull request incrementally with three additional commits since the last revision:
> 
>  - Update test/hotspot/jtreg/serviceability/sa/LingeredAppWithFFMUpcall.java
>    
>    Co-authored-by: Andrey Turbanov <turbanoff at gmail.com>
>  - Update test/hotspot/jtreg/serviceability/sa/LingeredAppWithFFMUpcall.java
>    
>    Co-authored-by: Andrey Turbanov <turbanoff at gmail.com>
>  - Update test/hotspot/jtreg/serviceability/sa/LingeredAppWithFFMUpcall.java
>    
>    Co-authored-by: Andrey Turbanov <turbanoff at gmail.com>

I had a look at the SA agent code in `src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/x86/X86Frame.java` and it looks like this has it's own separate stack walking implementation.

I understand that adding the `UpcallStub` type to the SA agent code makes the `WrongTypeException` go away, and then we run into an assertion failure because the frame size is zero?


Error occurred during stack walking:
sun.jvm.hotspot.utilities.AssertionFailure: must have non-zero frame size
        at jdk.hotspot.agent/sun.jvm.hotspot.utilities.Assert.that(Assert.java:32)
        at jdk.hotspot.agent/sun.jvm.hotspot.runtime.x86.X86Frame.senderForCompiledFrame(X86Frame.java:383)
        at jdk.hotspot.agent/sun.jvm.hotspot.runtime.x86.X86Frame.sender(X86Frame.java:292)
        at jdk.hotspot.agent/sun.jvm.hotspot.runtime.Frame.sender(Frame.java:207)
        at jdk.hotspot.agent/sun.jvm.hotspot.runtime.Frame.realSender(Frame.java:212)
        at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VFrame.sender(VFrame.java:120)
        at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VFrame.javaSender(VFrame.java:150)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:81)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.run(JStack.java:67)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:278)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:241)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:134)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.runWithArgs(JStack.java:90)
        at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJSTACK(SALauncher.java:302)
        at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:500)


The issue here appears to be that the stack walking code in the SA agent doesn't handle upcall stub frames at all at the moment, so the code falls back to handling for compiled frames. However, that code looks wrong for upcall stub frames, since we need to look at the frame anchor to get to the caller (see how this is implemented in e.g. `src/hotspot/cpu/x86/frame_x86.cpp`). Note how there is also special handling for (JNI) entry frames in the SA.

>From the output, this still seems to work out, I'm guessing because we end up walking the native frames until we get back to Java, and the native frames are simply ignored. I'm not sure if that will always work for arbitrary native code though.

I think the right fix here is to implement handling for upcall stub frames in the SA agent, since that's also how entry frames are handled. I don't think setting the frame size in hotspot is actually needed if we do that.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20789#issuecomment-2324580579