[foreign] programmatic access to native libraries w/o binder
Jorn Vernee
jbvernee at xs4all.nl
Wed Nov 28 16:10:18 UTC 2018
I don't imagine framework code to look like this;
var getpid =
abi.downcallHandle(Libraries.getDefaultLibrary().lookup("getpid"), new
NativeMethodType(NativeTypes.INT32))
But more like this:
public MetodHandle bind(BindTarget target) { // some framework type
Library.Symbol sym = Libraries.loadLibrary(lookup(),
target.libName()).lookup(target.funcName());
NativeMethodType nmt = deriveNativeMethodType(target);
return abi.downcallHandle(sym, nmt);
}
In which case NativeMethodType is a nice abstraction to have, otherwise
a framework writer will have to write it themselves.
The only cases where I expect the first form to appear is when you
already know the exact target of the binding when writing the program.
Jorn
Jorn Vernee schreef op 2018-11-28 16:41:
>> Internally, we can have a NativeMethodType abstraction, since for the
>> binder it seems to make sense - I'm wondering how much that should be
>> public API, given that low level users will probably see that as a
>> somewhat redundant intermediate step.
>
> Well, if it makes sense for the binder, wouldn't it make sense for
> framework writers as well? I'd imagine they'd want to have some kind
> of MethodType analog for native binding.
>
> Jorn
>
> Maurizio Cimadamore schreef op 2018-11-28 16:32:
>> On 28/11/2018 15:27, Jorn Vernee wrote:
>>> Looking good!
>>>
>>>> * having to create a NativeMethodType when I just want to pass the
>>>> list of layout types leads to more verbose code
>>>>
>>>> * we might need NativeTypes.POINTER - doing
>>>> NativeTypes.VOID.pointer()
>>>> is an explicit workaround, but a tad verbose
>>>
>>> Reading through the code, I prefer both of these actually, since it
>>> makes the code more visually explicit as to what's happening. The
>>> `new NativeType(...)` call also gives a visual grouping to the
>>> arguments. Though I admit, I like being very descriptive when writing
>>> code.
>>>
>>> Tbh, I mostly expect the NativeMethodType to be constructed
>>> automatically, e.g. by deriving the signature from a mangled symbol
>>> name, so I'm not sure how much verbosity will be a problem in
>>> practice.
>>>
>>> To save on characters I'd suggest using `import static
>>> NativeTypes.*`, and perhaps we could add a convenience overload of
>>> upcallStub which doesn't take the target MethodHandle, and instead
>>> tries to infer it from the passed receiver object?
>>>
>>> default UpcallHandle upcallStub(Object receiver, NativeMethodType
>>> nmt) {
>>> Method m = receiver.getClass().getDeclaredMethods()[0]; //
>>> you get the gist
>>> return upcallStub(cc, receiver, publicLookup().unreflect(m),
>>> nmt);
>>> }
>>
>> It's not about saving characters - I was trying to imagine what a
>> typical usage of this API would look like. I assume that people that
>> reach low level to SystemABI directly (bypassing the binder) basically
>> want to be more in control on how the mapping is done. This could be a
>> powerful options for framework writers. But those users, I believe,
>> will almost never use the API point that takes a j.l.Method and a
>> Function - they would prefer to specific a list of
>> NativeTypes/LayoutTypes directly.
>>
>> Internally, we can have a NativeMethodType abstraction, since for the
>> binder it seems to make sense - I'm wondering how much that should be
>> public API, given that low level users will probably see that as a
>> somewhat redundant intermediate step.
>>
>>>
>>> Btw, now that we're starting to have NativeTypes and NativeMethodType
>>> maybe we should consider renaming LayoutType to NativeType as well?
>>
>> Sure, I was thinking along the same lines
>>
>> Maurizio
>>
>>>
>>> Jorn
>>>
>>> Maurizio Cimadamore schreef op 2018-11-28 15:00:
>>>> Hi,
>>>> I took the patch at [1] for a spin with jshell. I wanted to see how
>>>> easy it was to call native functions using the SystemABI interface -
>>>> e.g. w/o binder and annotated interface. I think I'm pleasantly
>>>> impressed! I did some changes to the proposed patch as I wanted to
>>>> put
>>>> everything SystemABI-specific in a public package (java.foreign) and
>>>> I
>>>> also added a method on UpcallHandle to get a pointer. But overall
>>>> these were minor modification, mostly in order to be able to use
>>>> this
>>>> setup on jshell. But let's dive on some code.
>>>>
>>>> First, some setup:
>>>>
>>>>
>>>> $ jshell
>>>> | Welcome to JShell -- Version 12-internal
>>>> | For an introduction type: /help intro
>>>>
>>>> jshell> import java.foreign.*
>>>>
>>>> jshell> import java.foreign.memory.*
>>>>
>>>> jshell> import java.lang.invoke.*
>>>>
>>>> jshell> var abi = SystemABI.getInstance()
>>>> abi ==> jdk.internal.foreign.abi.sysv.x64.SysVx64ABI at 41cf53f9
>>>>
>>>> Now that we have an ABI, let's try getpid:
>>>>
>>>>
>>>> jshell> var getpid =
>>>> abi.downcallHandle(Libraries.getDefaultLibrary().lookup("getpid"),
>>>> new
>>>> NativeMethodType(NativeTypes.INT32))
>>>> getpid ==> MethodHandle()int
>>>>
>>>> jshell> (int)getpid.invokeExact()
>>>> $6 ==> 20308
>>>>
>>>>
>>>> That was easy; let's try a function which actually takes some args:
>>>>
>>>>
>>>> jshell> var pow =
>>>> abi.downcallHandle(Libraries.getDefaultLibrary().lookup("pow"), new
>>>> NativeMethodType(NativeTypes.DOUBLE, NativeTypes.DOUBLE,
>>>> NativeTypes.DOUBLE))
>>>> pow ==> MethodHandle(double,double)double
>>>>
>>>> jshell> (double)pow.invokeExact(2d, 3d)
>>>> $12 ==> 8.0
>>>>
>>>>
>>>> That works too. Finally, let's see if we can make qsort work - first
>>>> let's declare the comparison logic:
>>>>
>>>>
>>>> jshell> class Comp {
>>>> ...> int comp(Pointer<Integer> p1, Pointer<Integer> p2) {
>>>> ...> return p1.get() - p2.get();
>>>> ...> }
>>>> ...> }
>>>> | created class Comp
>>>>
>>>>
>>>> And allocate the stub:
>>>>
>>>>
>>>>
>>>> jshell> var up = abi.upcallStub(abi.defaultCallingConvention(), new
>>>> Comp(),
>>>> ...> MethodHandles.lookup().findVirtual(Comp.class, "comp",
>>>> MethodType.methodType(int.class, Pointer.class, Pointer.class)),
>>>> ...> new
>>>> NativeMethodType(NativeTypes.INT32, NativeTypes.INT.pointer(),
>>>> NativeTypes.INT.pointer()));
>>>> up ==> jdk.internal.foreign.invokers.DirectUpcallHandle at dcf3e99
>>>>
>>>>
>>>> And finally, let's call qsort and print the results.
>>>>
>>>>
>>>> jshell> var qsort =
>>>> abi.downcallHandle(Libraries.getDefaultLibrary().lookup("qsort"),
>>>> ...> new
>>>> NativeMethodType(NativeTypes.VOID,
>>>> NativeTypes.INT.pointer(), NativeTypes.INT, NativeTypes.INT,
>>>> NativeTypes.VOID.pointer()));
>>>> qsort ==> MethodHandle(Pointer,int,int,Pointer)void
>>>>
>>>> jshell> Scope sc = Scope.newNativeScope()
>>>> sc ==> jdk.internal.foreign.ScopeImpl$NativeScope at 5fdef03a
>>>>
>>>> jshell> var arr = sc.allocateArray(NativeTypes.INT32, new int[] { 2,
>>>> 6, 4, 8, 1, 10 });
>>>> arr ==> jdk.internal.foreign.memory.BoundedArray at 311d617d
>>>>
>>>> jshell> qsort.invoke(arr.elementPointer(), 6, 4, up.entryPoint());
>>>> $13 ==> null
>>>>
>>>> jshell>
>>>> System.err.println(Arrays.toString(arr.toArray(int[]::new)));
>>>> [1, 2, 4, 6, 8, 10]
>>>>
>>>> Cool! We have been able to do all this without using a single
>>>> annotated interface, which suggests that SystemABI might indeed be
>>>> general enough to build on top of it.
>>>>
>>>> There are, I think, some hiccups in the above snippets which I found
>>>> annoying:
>>>>
>>>> * having to create a NativeMethodType when I just want to pass the
>>>> list of layout types leads to more verbose code
>>>>
>>>> * we might need NativeTypes.POINTER - doing
>>>> NativeTypes.VOID.pointer()
>>>> is an explicit workaround, but a tad verbose
>>>>
>>>> * as we already know, the upcallStub API point could use some
>>>> simplification: omit the receiver argument and directly return the
>>>> pointer; that will also simplify things
>>>>
>>>> Maurizio
>>>>
>>>> [1] -
>>>> https://cr.openjdk.java.net/~henryjen/panama/SystemABI/webrev.01/webrev/
More information about the panama-dev
mailing list