Segmentation fault with new Linker.Option.CaptureCallState feature

[Maurizio Cimadamore]

Hi Mark,
many thanks for the bug report (and the kind words :-) ).

This looks like a bug - in the sense that I'm not aware of any 
limitations in this area. The fact that the function returns a value 
must be tripping something up in the assembly trampoline we generate.

For now, I've filed this:

https://bugs.openjdk.org/browse/JDK-8296973

Thanks!
Maurizio

On 14/11/2022 21:56, Mark Schroeder wrote:
> Hi everyone,
>
> first of all, I wanted to thank you for the amazing work you're doing 
> in this project. I really enjoy working with the API.
> While experimenting with some system calls under Linux, I encountered 
> an issue with the new Linker.Option.CaptureCallState feature delivered 
> here: https://github.com/openjdk/panama-foreign/pull/742. When calling 
> a function that returns a value (such as a system call returning -1 in 
> case of an error), the JVM exits due to a segmentation fault. As a 
> minimal example, I adapted the test code from the pull request (the 
> issue is not limited to system calls).
> This is the C code:
>
> #include <errno.h>
> #include <stdint.h>
>
> void set_errno(int32_t value) {
>     errno = value;
> }
>
> int32_t set_errno2(int32_t value) {
>     errno = (int) value;
>     return value * 2;
> }
>
> I compiled the code on Linux (Fedora 36) with the following command:
> gcc -o libCaptureCallState.so -shared -Wall libCaptureCallState.c
> This is the Java code causing the issue:
>
> import java.lang.foreign.FunctionDescriptor;
> import java.lang.foreign.Linker;
> import java.lang.foreign.MemoryLayout;
> import java.lang.foreign.MemorySegment;
> import java.lang.foreign.MemorySession;
> import java.lang.foreign.SymbolLookup;
> import java.lang.foreign.ValueLayout;
> import java.lang.invoke.MethodHandle;
> import java.lang.invoke.VarHandle;
> import java.nio.file.Path;
>
> public class TestCaptureCallState {
>
>     public static void main(String[] args) throws Throwable {
> System.load(Path.of("libCaptureCallState.so").toAbsolutePath().toString());
>         Linker linker = Linker.nativeLinker();
>         SymbolLookup lookup = SymbolLookup.loaderLookup();
>         Linker.Option.CaptureCallState state = 
> Linker.Option.captureCallState("errno");
>         VarHandle errno = 
> state.layout().varHandle(MemoryLayout.PathElement.groupElement("errno"));
>         MethodHandle setErrno = linker.downcallHandle(
>             lookup.find("set_errno").orElseThrow(),
>             FunctionDescriptor.ofVoid(ValueLayout.JAVA_INT),
>             state
>         );
>         MethodHandle setErrno2 = linker.downcallHandle(
>             lookup.find("set_errno2").orElseThrow(),
>             FunctionDescriptor.of(ValueLayout.JAVA_INT, 
> ValueLayout.JAVA_INT),
>             state
>         );
>
>         try (MemorySession session = MemorySession.openConfined()) {
>             MemorySegment saveSeg = session.allocate(state.layout());
>             System.out.println("Testing set_errno ...");
>             setErrno.invoke(saveSeg, 42);
>             int savedErrno = (int) errno.get(saveSeg);
>             System.out.printf("errno: %d\n", savedErrno);
>         }
>         try (MemorySession session = MemorySession.openConfined()) {
>             System.out.println("Testing set_errno2 ...");
>             MemorySegment saveSeg = session.allocate(state.layout());
>             int result = (int) setErrno2.invoke(saveSeg, 42);
>             int savedErrno = (int) errno.get(saveSeg);
>             System.out.printf("errno: %d, result: %d\n", savedErrno, 
> result);
>         }
>     }
> }
>
> I executed the code with the following commands:
> javac --enable-preview --release 20 TestCaptureCallState.java
> java --enable-preview --enable-native-access=ALL-UNNAMED 
> TestCaptureCallState
> The code fails when calling the function with the return value:
>
> Testing set_errno ...
> errno: 42
> Testing set_errno2 ...
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x00007fea8ed0dce8, pid=47154, tid=47155
> #
> # JRE version: OpenJDK Runtime Environment (20.0) (build 
> 20-internal-adhoc.maschroeder.openjdk--panama-foreign)
> # Java VM: OpenJDK 64-Bit Server VM 
> (20-internal-adhoc.maschroeder.openjdk--panama-foreign, mixed mode, 
> sharing, tiered, compressed oops, compressed class ptrs, g1 gc, 
> linux-amd64)
> # Problematic frame:
> # V  [libjvm.so+0x70dce8]  DowncallLinker::capture_state(int*, int)+0x18
> ...
>
> Is there something that I overlooked? Is this a known limitation of 
> the new feature?
>
> Mark