panama/foreign status update
maurizio.cimadamore at oracle.com
Fri Dec 3 18:21:48 UTC 2021
as you might have noticed, we have integrated the latest set of API
(JEP-419) changes into Java 18, so I think it is now a good time to take
a look at where we are, and where we are headed. When we started this
journey (back in Java 14), we had three main goals:
1. Provide an alternative to the byte buffer API with similar
performance/safety characteristics, but without some of the downsides
(limited addressing space, lack of deterministic deallocation);
2. Make native libraries more accessible, by replacing JNI with a more
3. Provide basic building blocks which would enable other frameworks to
define higher-level native interop.
After few years of having the API available as an incubating API in the
JDK, it is natural to ask whether the API we have delivered is
successful in addressing the above goals.
On (1), the fact that big frameworks such as Netty  and Lucene  (I
hope Sketches will join the party soon) have come up with experimental
branches written on top of the new API is a good sign. A couple of years
ago there were some issues, mainly due to the ownership model being too
restrictive, but those issues have been sorted out (since Java 16).
On (2), over the last few months we had some fairly significant examples
of developers using jextract + foreign API to tackle pretty sizeable
native APIs, from OpenGL , to OpenSSL , to UCX , to FUSE .
(Apologies in advance for the ones that I might have forgotten about!).
On (3) we don't have much data yet - for Java - as some of the biggest
frameworks (JNA, JNR and JavaCPP) did not have a chance to play with the
API as of yet. But what I find interesting/encouraging is that other VM
languages have picked up the challenge and came up with creative ways to
use the foreign API to implement a more direct native interop layer, in
Scala  and in Clojure .
I think all this makes us confident that the API we have is solid enough
to write real world software.
Over then next few months we're going to transition the API from an
incubating API (module jdk.incubator.foreign) to a _preview_ API (in
java.base, package name TBD). We will soon open a new branch and start
moving the code across modules. In parallel, we will keep making changes
to the foreign linker runtime (see ), to simplify the implementation,
reduce the number of code paths and, ultimately, make it easier for
other JVM port maintainers to support this API in full.
On the jextract front, as mentioned in the past  we would like to
transition to a model where jextract is delivered as a separate tool.
This means that jextract will live in its own github repo, which will
hopefully make it easier for developers to contribute. Oracle will still
provide some EA ready-made jextract binaries (location TBD). In the past
few weeks we've been making slow but steady progress in order to make
that happen - I still cannot say definitively _when_ that will happen,
but it looks certainly closer now than it was few months ago.
I'd like to, once again, thank the team for working so hard on this
project - and I would like to thank all of you for trying out the
various bits, reading documents and providing feedback. A projects like
this cannot exist in isolation - and we would not be in the position we
are in now without your help.
 - https://github.com/netty/netty-incubator-buffer-api
 - https://github.com/apache/lucene/pull/173
 - https://github.com/jzy3d/panama-gl
 - https://github.com/rmaucher/openssl-panama-foreign
 - https://github.com/hhu-bsinfo/infinileap
 - https://github.com/skymatic/fuse-panama
 - https://gitlab.com/mhammons/slinc
 - https://github.com/IGJoshua/coffi
More information about the panama-dev