Notes on packaging of native code in libraries

[Mike Hearn]

Hello,

Recently there was a discussion of hermetic Java and how native libraries
are packaged in Google's solution. The approach is Linux specific but once
you leave that OS new issues emerge. We've just published a blog post that
explores the problems that arise when distributing Java apps to platforms
that require code signing:

https://hydraulic.software/blog/11-in-jar-signing.html

Conveyor is a build tool that layers on top of jlink and it does extensive
work to turn JARs into something that can be shipped to Mac/Windows users
successfully. The blog post discusses some of these challenges, which any
approach to static Java will also face (assuming it's supported on more
than just Linux of course).

The core issues are related to code signing, and the practice of extracting
libraries on the fly from JARs to caches or temp directories. Conveyor has
two modes, one which does this extraction ahead of time (but many libraries
don't expect this and must be explicitly configured via system properties,
or they just break), and the other which code signs libraries inside the
JARs. The latter is less likely to break apps but yields other problems.
Neither approach is perfect.

Especially once Panama ships the Java ecosystem could strongly benefit from
a standardization of how native components are bundled into and loaded from
libraries, as current build systems and the JVM tooling don't have much to
say on the topic. The result is a lot of wheel reinvention across the
ecosystem in the form of NativeLibraryLoader.java classes, always unique
per project, and a bunch of bugs / developer friction that doesn't need to
happen.

The good news is that all this would be very cheap to improve. All that's
needed is:

   - A defined layout for JARs (or JMODs) that standardizes where to place
   native libraries given an OS and CPU architecture.

   - Tooling that extracts native code to a user-specified directory that's
   then appended to the java.library.path at runtime (e.g. a flag to the java
   launcher?), so that once build systems learn to pass this flag or do the
   extraction themselves library authors can just deprecate and eventually
   remove all their custom loader code (which is large, complex, copy/pasted
   between projects and inconsistent).

   - Support for that mechanism in jlink.

The current JMOD mechanism improved things over the prior situation, but
unfortunately doesn't quite supply everything needed. A solution that works
with JARs and supports multiple libraries in one JAR would be easily
adopted.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20230224/dec1627d/attachment.htm>