Fwd: JMOD, native libraries and the packaging of JavaFX

[Mike Hearn]

I did a bit of experimentation to learn how different operating systems
support loading shared libraries in-memory. I also did a bit of thinking on
the topic of "native classloaders". Here's a braindump, which may lead
nowhere but at least it'll be written down.

Linux: This OS has the best support. The memfd_create syscall was added in
Linux 3.17 (released in 2014). It's not exposed by glibc but is easy to
invoke by hand. It creates a file descriptor that supports all normal
operations from an in-memory region. After creation it can be fed to the
rtld using dlopen(/proc/self/fd/num). I've tried this and it works fine.

Windows: Runner up support. An in-memory file can be created using
FILE_ATTRIBUTE_TEMPORARY
| FILE_FLAG_DELETE_ON_CLOSE passed to CreateFile. However, it still
occupies a location in the namespace, probably permission checks still
apply. Additionally the file is not truly memory only. Under memory
pressure the VMM may flush it to disk to free up resources. I haven't tried
this API myself.

macOS: Worst support. There is a deprecated NSCreateObjectFromMemoryFile
API but internally all it does is save the contents to a file and load it
again. The core rtld cannot load from anything except a file and macOS does
not appear to have any way to create in-memory fds. shm_open doesn't work,
the SHM implementation is too broken; you can't write to such an fd, you
can mmap it but then trying to open /dev/fd/x doesn't work on the resulting
fd.

Obviously with any such approach you face the problem of dependencies. Even
if the DLL/DSO itself is in memory, the rtld will still try to load any
dependent libraries from disk.

Playing around with this led me to start pondering something more
ambitious, namely, making native code loading customisable by Java code,
via some sort of NativeLoader API that's conceptually similar to
ClassLoader. Methods on it would be called to resolve a library given a
name and to look up symbols in the returned code module (returning a native
pointer to an entry point that uses a calling convention passed into the
lookup). System.load() would be redirected to call into this class and the
JNI linker would upcall into it. NativeLoaders could be exposed via a SPI.

This might sound extreme, but we can see some advantages:

   - The default NativeLoader would just use the platform APIs as today,
   meaning little behaviour or code change on the JDK side.
   - Samuel could write a NativeLoader for his unpack+cache implementation
   that standardises this behaviour in an ordinary library anyone can use.
   - A Linux implementation could be written that gives faster performance
   and more robustness using memfd_create, again, it could be done by the
   community instead of HotSpot implementors.
   - It opens the possibility of the community developing a new,
   platform-independent native code format by itself. Why might this be
   interesting?
   - ELF, PE and Mach-O do not vary significantly in feature set, and
      differ only due to the underlying platforms evolving separately. From the
      perspective of a Java developer who just wants to package some
native code
      for distribution the distinction is annoying and unnecessary. It means
      building the same code three times, on three different platforms
with three
      different toolchains, each time the native code changes, even if
the change
      itself has no platform specific parts.
      - A new format could potentially simplify (e.g. do we still need
      support for relocation given the relatively poor success of ASLR and the
      big success of 64 bit platforms?), but it could also have features Java
      developers might want, like:
      - The ability to have OS specific symbols. If your library differs
         between platforms only in a few places, do you really need to
ship three
         artifacts for that or would one artifact that contained 3
versions of the
         underlying function be sufficient?
         - The ability to smoothly up-call to Java by linking dependent
         symbols back to Java via the callback capabilities in Panama.
         - The ability to internally link symbols based on information
         discovered by the JVM, e.g. if the JVM is able and willing to
use AVX512
         the new format could simply use that information instead of
requiring the
         developer to write native code to detect CPU capabilities.
         - Fat binaries that support different CPU architectures inside a
         single file, and/or LLVM bitcode. If you have LLVM as a supported
         "architecture" then the NativeLoader could perhaps bridge to
Sulong, so
         Java code that wasn't written with the Graal/Truffle API in
mind can still
         benefit from calling into JIT-compiled bitcode. This probably
would require
         NativeLoader to return a MethodHandle of some sort rather
than a 'long'
         pointer.
         - Alternatively write a cross platform ELF loader. The Java IO
      APIs provide everything needed already, as far as I know.
      - The Arabica project explored sandboxing JNI libraries using Google
      NativeClient. It appeared to work and calls from the sandboxed code into
      the OS were transparently redirected back into the JVM where normal
      permission checks occurred. Sandboxed, cross platform native
code would be
      a powerful capability. http://www.cse.psu.edu/~gxt29/paper/arabica.pdf
      - It opens the possibility of "assembly spinning" as an analogue to
   "bytecode spinning", as there would be no requirement that the NativeLoader
   return a pointer to code that it loaded off disk. Java-level JIT compilers
   like Graal could potentially be used to spin little snippets of code at
   runtime, or when only small amounts are needed (e.g. to invoke a syscall
   that isn't exposed via glibc, like, say, memfd_create) the bytes can simply
   be prepared ahead of time and read from the constant pool.

Getting back to JavaFX for a moment, it sounds like it's too late for
anything to go into JDK11 at this point, which is a pity. It will be up to
the community to find a solution like Samuel's cache for now.

thanks,
-mike