Loading native libraries from the classpath

[Mike Hearn]

We can imagine a slightly tweaked Jigsaw that would approach things
from a similar but different angle:

- JMODs work on the module and classpath (perhaps being moved onto the
module path automatically, if found on the classpath)

- JMOD format is extended to be more portable, by allowing binaries
for different OS/CPU combinations in a single archive

- jlink then takes the right files for the target out of the JMOD and
discards the rest during link-time optimization

- The JVM can do the same extraction from jmods on the
module/classpath at runtime, to /tmp, but this is meant only for
development

This would fit neatly with packaging tools like jpackage or Conveyor
because they take jlink output as a starting point.

It might also require bringing jmods and jars closer together (e.g.
allowing a jmod to use the .jar file extension), to avoid rough edges
with existing tools.

Getting rid of on-the-fly extraction is really hard though, because
the model of a library being a collection of JARs is so deeply
embedded in the ecosystem. It's a really fundamental part of what Java
is. It's also just really convenient compared to what other platforms
do. It's easy for things to go very badly wrong here, see for example
the native library dependencies story for Python or C++, which is much
hackier and harder for devs than what Java does.

It's great to try and improve on what the wider community has
developed, but the existing ways have the significant advantage of
having proven acceptance already. It's survived the natural selection
process and been proven at least a somewhat fit animal. New
clean-slate designs don't always survive the same test, as it's easy
to create things that suffer from some new unforeseen issue whilst
fixing others.

If the goal is to avoid workarounds it's very hard because the things
being worked around are build systems, IDEs and operating systems.
Changing those ranges from hard to impossible. It's easier to work
within their constraints than design a system assuming they'll change
that then never gets adopted.