[foreign] programmatic access to native libraries w/o binder

[Maurizio Cimadamore]

Hi,
I took the patch at [1] for a spin with jshell. I wanted to see how easy 
it was to call native functions using the SystemABI interface - e.g. w/o 
binder and annotated interface. I think I'm pleasantly impressed! I did 
some changes to the proposed patch as I wanted to put everything 
SystemABI-specific in a public package (java.foreign) and I also added a 
method on UpcallHandle to get a pointer. But overall these were minor 
modification, mostly in order to be able to use this setup on jshell. 
But let's dive on some code.

First, some setup:


$ jshell
|  Welcome to JShell -- Version 12-internal
|  For an introduction type: /help intro

jshell> import java.foreign.*

jshell> import java.foreign.memory.*

jshell> import java.lang.invoke.*

jshell> var abi = SystemABI.getInstance()
abi ==> jdk.internal.foreign.abi.sysv.x64.SysVx64ABI at 41cf53f9

Now that we have an ABI, let's try getpid:


jshell> var getpid = 
abi.downcallHandle(Libraries.getDefaultLibrary().lookup("getpid"), new 
NativeMethodType(NativeTypes.INT32))
getpid ==> MethodHandle()int

jshell> (int)getpid.invokeExact()
$6 ==> 20308


That was easy; let's try a function which actually takes some args:


jshell> var pow = 
abi.downcallHandle(Libraries.getDefaultLibrary().lookup("pow"), new 
NativeMethodType(NativeTypes.DOUBLE, NativeTypes.DOUBLE, 
NativeTypes.DOUBLE))
pow ==> MethodHandle(double,double)double

jshell> (double)pow.invokeExact(2d, 3d)
$12 ==> 8.0


That works too. Finally, let's see if we can make qsort work - first 
let's declare the comparison logic:


jshell> class Comp {
    ...>       int comp(Pointer<Integer> p1, Pointer<Integer> p2) {
    ...>          return p1.get() - p2.get();
    ...>       }
    ...>    }
|  created class Comp


And allocate the stub:



jshell> var up = abi.upcallStub(abi.defaultCallingConvention(), new Comp(),
    ...> MethodHandles.lookup().findVirtual(Comp.class, "comp", 
MethodType.methodType(int.class, Pointer.class, Pointer.class)),
    ...>                         new NativeMethodType(NativeTypes.INT32, 
NativeTypes.INT.pointer(), NativeTypes.INT.pointer()));
up ==> jdk.internal.foreign.invokers.DirectUpcallHandle at dcf3e99


And finally, let's call qsort and print the results.


jshell> var qsort = 
abi.downcallHandle(Libraries.getDefaultLibrary().lookup("qsort"),
    ...>                         new NativeMethodType(NativeTypes.VOID, 
NativeTypes.INT.pointer(), NativeTypes.INT, NativeTypes.INT, 
NativeTypes.VOID.pointer()));
qsort ==> MethodHandle(Pointer,int,int,Pointer)void

jshell> Scope sc = Scope.newNativeScope()
sc ==> jdk.internal.foreign.ScopeImpl$NativeScope at 5fdef03a

jshell> var arr = sc.allocateArray(NativeTypes.INT32, new int[] { 2, 6, 
4, 8, 1, 10 });
arr ==> jdk.internal.foreign.memory.BoundedArray at 311d617d

jshell> qsort.invoke(arr.elementPointer(), 6, 4, up.entryPoint());
$13 ==> null

jshell> System.err.println(Arrays.toString(arr.toArray(int[]::new)));
[1, 2, 4, 6, 8, 10]

Cool! We have been able to do all this without using a single annotated 
interface, which suggests that SystemABI might indeed be general enough 
to build on top of it.

There are, I think, some hiccups in the above snippets which I found 
annoying:

* having to create a NativeMethodType when I just want to pass the list 
of layout types leads to more verbose code

* we might need NativeTypes.POINTER - doing NativeTypes.VOID.pointer() 
is an explicit workaround, but a tad verbose

* as we already know, the upcallStub API point could use some 
simplification: omit the receiver argument and directly return the 
pointer; that will also simplify things

Maurizio

[1] - 
https://cr.openjdk.java.net/~henryjen/panama/SystemABI/webrev.01/webrev/