Some scattered feedback on the Foreign Linker API

[Chris Hegarty]

Hi,

As part of a recent prototyping effort, we evaluated replacing the
usage of JNA in the core of the Elasticsearch server with the incubating
Foreign Linker API.

TL;DR  things just worked,  perf improved,  and we really like the
restricted native access.

Our usage (at least in this particular case) is not really performance
sensitive, in that we reach into native to setup syscall filtering and
check resource limits - but hey, faster is always better! We measured
approximate perf improvements between 8 and 20 times faster for simple
downcalls. The reason the numbers vary so much is more to do with JNA
rather than Panama, since the Panama times from run to run were
consistent, while the times measured with JNA varied quite a bit - which
we observed on several different platforms but did not investigate
further.

While the aforementioned effort did not focus on a particularly
performance sensitive area of code, there are other areas of the server
that do reach into native, and are more performance sensitive. Our
intuition is that they would benefit from using the Linker API too - an
experiment yet to be done, but given the potential perf improvement we
observed above, an experiment we are excited and motivated to do.

I’m not particularly familiar with JNA, so this effort was also a
earning exercise for me. What struck me was that with JNA things work
with minimal fuss. That is to say that JNA would appear to be a
higher-level API that hides much of the detail, detail which you need to
concern yourself with when using Panama’s Linker API. I found this ok,
and the eventual end result in terms of the consuming code was
reasonably similar to that of JNA, but with Panama much more “glue” code
was needed - which we generated with jextract. This is not a problem
per se, but an observation that was glaring when comparing the the two
implementations. JNA “hides” the much of the detail from the developer.

We love restricted native access. We’re currently modularizing the core
of the Elasticsearch server, and the ability to restrict native access
to just a couple of these modules is a big positive. This allows to
improve security and integrity of the system, e.g. custom server plugins
cannot just use JNA from the classpath. All without resorting to playing
tricks with classloaders in an attempt to "hide" JNA. We can restrict
native access to just the ES modules that require it - all other parts
of the system will be disallowed (irrespective of security manager).

We believe that the model of protecting sensitive operations behind
module boundaries with caller-sensitive access checks upon invocation is
the right model. The mechanics of this could be improved or built upon
through additional module support, like say the ability for module
descriptors to carry information about which other modules require, or
are allowed to perform, restricted access operations. And additional
consistency checks akin to the tying of a set of modules with hashes, to
offer further protection beyond the module name itself.

-Chris.