Native call framework prototype

[Vladimir Ivanov]

FYI I've just pushed recent enhancements to the Lookup.findNative prototype:
   http://hg.openjdk.java.net/panama/panama/jdk/rev/e2648e33b231
   http://hg.openjdk.java.net/panama/panama/hotspot/rev/46867db7b6f4

What's new:
   * argument shuffling (Java->C calling convention) in linkToNative 
adapater;
   * better interpreter support (now uses compiled version through i2c);
   * JIT-compiler support for direct native calls in generated code (C2 
only);
   * additional test cases (see NativeCallTest & NativeAdapterTest [1]);

What's still missing:
   * no safepoint checks and thread state transitions before/after 
native call;
   * the framework is still inherently unsafe and very low-level (no 
Java<->C type conversions);
   * API for DLL management;

In addition to getpid example, I added gettimeofday, getlogin & setlogin 
(see NativeCallTest [2]).

As you can see, it requires some boilerplate code and manual bookkeeping 
of native memory to make it work as expected.

Though new data layouts in JVM heap are advertized on Project Panama 
page [4], I haven't experimented with custom Java object layout so 
native code could access it directly. Instead, a piece of native memory 
is allocated. Native code uses it directly, but Java code accesses it 
through a wrapper interface (see TimeVal/TimeZone intfs in 
NativeCallTest).

It would be interesting to try to wire it up with Groveller/LDL to avoid 
writing native API by hand.

Also, there's a first attempt to access a global variable (errno). I 
used Unsafe.findNativeAddress & Unsafe.getInt for now, but VarHandles 
should be the right tool.

The next step are:
   * support C->Java upcalls through function pointers (like qsort call 
with comparator in Java, see [3]);
   * do argument wrapping/result unwrapping transparently to the user 
(MT(TimeValIntf, TimeZoneIntf)int instead of MT(long,long)int); it 
requires tight integration with how native API is exposed (e.g. the 
framework should know about Struct interface to be able);

Stay tuned! Thanks!

Best regards,
Vladimir Ivanov

[1] http://hg.openjdk.java.net/panama/panama/jdk/rev/e2648e33b231
[2] http://hg.openjdk.java.net/panama/panama/jdk/rev/e2648e33b231#l3.111
[3] 
https://github.com/jnr/jnr-ffi-examples/blob/master/qsort/src/main/java/qsort/Qsort.java
[4] http://openjdk.java.net/projects/panama/

On 2/19/15 9:23 PM, Vladimir Ivanov wrote:
> Hi,
>
> As you, probably, already noticed, I've just pushed raw prototype of
> java.lang.invoke-based native function invocation framework, inspired by
> [1].
>
> The first version is a raw prototype which is able to call getpid (see
> jdk/test/java/lang/invoke/NativeCallTest.java):
>
>    MethodHandle mh =
>      LOOKUP.findNative(null, "getpid", MethodType.methodType(int.class));
>    int pid = (int) mh.invokeExact();
>
> NativeMethodHandle and NativeEntryPoint (MemberName counterpart) are
> used to represent native function.
>
> The implementation relies on 2 VM primitives:
>    MethodHandle.linkToNative() & Unsafe.findNativeAddress()
>
> Native part is mostly a skeleton for future work:
>    * linkToNative adapter generation is present only on x64;
>    * supported adapter signature is trivial, just enough to call getpid()
>    * generated adapter doesn't do thread state transition before calling
> native function (it corresponds to privileged "raw" invocation mode in
> [2]);
>
> On the other hand, Java part is in much better shape. No access &
> security checks yet, but once VM can generate more adapters, everything
> should "just work" :-)
>
> The next steps are:
>
>    - extend linkToNative adapter to work on all Java signatures in raw
> invocation mode
>
>    - introduce "state transitioning" calling mode (like JNI does) and
> different calling conventions support; the idea is to encapsulate such
> info in NativeEntryPoint
>
>    - JIT-compiler support for "raw" mode: issue direct native call
> instead of using linkToNative adapter when possible
>
>    - try some simple Java<->C type conversions (like String <-> char*);
> they can be done on LambdaForm level
>
>    - try commoning thread state transitions in JIT-compiled code in
> order to reduce native call overhead
>
> Best regards,
> Vladimir Ivanov
>
> [1] http://cr.openjdk.java.net/~jrose/panama/native-call-primitive.html
> [2] https://blogs.oracle.com/jrose/entry/the_isthmus_in_the_vm