JMOD, native libraries and the packaging of JavaFX

[Mike Hearn]

Hello,

The JMOD format is not documented directly, but is essentially a JAR-like
format which can also contain native libraries, license texts, man pages
and config files. However, JMODs are not just JARs with extra features.
They are incompatible, because class files go under a classes/ directory.

JEP 261 says:

JMOD files can be used at compile time and link time, but not at run time.
To support them at run time would require, in general, that we be prepared
to extract and link native-code libraries on-the-fly. This is feasible on
most platforms, though it can be very tricky, and we have not seen many use
cases that require this capability, so for simplicity we have chosen to
limit the utility of JMOD files in this release.


I was a bit surprised when I first read this because JARs that contain
native libraries, along with hacky custom code to extract and load them,
are actually quite common. One example is Conscrypt but there are others.
An extended JAR format in which the JVM took care of loading the right
native library would be a very helpful thing to have. Whilst the task can
be tricky if you want to do it as efficiently as possible e.g. not save the
DLL / DSO to disk, a "good enough" approach can't be that hard because the
community has implemented it many times.

I'm thinking about this issue now because I quite like JavaFX and its
future is clearly as a regular Java library, albeit a big one, distributed
either via (not ideal) an SDK or (better) a set of regular libraries
published to a Maven repository.

Publishing JavaFX as a set of modules that developers can depend on with
Maven or Gradle will require either that JavaFX include the sort of hacky
extract-to-a-temp-dir-and-load code that is standard today, or that it's
not published as ordinary JARs at all, or that the JPMS is extended in time
for Java 11 to provide a uniform solution.

As far as I can see the JMOD format is probably not going to gain adoption.
Whilst it can be used by developers in theory:

   - There's no support in Maven or Gradle.
   - The format isn't documented.
   - Using it requires jlinking, which isn't a part of the regular
   developer workflow.
   - jlink doesn't do much for the vast majority of apps that can't be
   fully modularised yet.
   - It's not clear why it's better than an extended JAR format.
   - It behaves in puzzling ways, for example the "jar" tool can print the
   file listing of a jmod but not extract it.

The bulk of the JMOD feature set could be delivered with two small
extensions to the JAR format:

   1. A common directory structure for storing native libraries. That would
   allow native library extraction and loading to be provided via a small
   library, if not provided by the JVM itself.
   2. A common directory structure for including license files (this just
   has to be announced, as nothing needs to load them).
   3. Metadata linking command line program names to main class names and
   JVM parameters.

For example the jdk.javadoc.jmod contains a program bin/javadoc which is a
native binary specific to the host platform of the JDK. But all it does is
run the JavaDoc program itself, which is written in Java. This sort of
startup program could be easily generated on the fly given startup
parameters in the same way the javapackager tool does it.

If the JAR format was extended in this way, it would become possible to
write a tool that given a Maven coordinate would resolve and then install
into a bin/ directory on the path a starter program with the suggested
name. This would be a nice feature to have (I am writing such a tool
already, but it'd be good to standardise the way to express a desired
'short name').

I realise there isn't much time left until the Java 11 ship sails. But for
JavaFX users at least, it'd be convenient to have this small extension.
Even if the JVM isn't extended to support it in 11, a common format would
allow it to take over responsibility later.

thanks,
-mike