jmods-less jlinking prototype

[Severin Gehwolf]

The Problem:
------------
Why do we need the 'jmods' directory when running jlink when 'java --
list-modules' lists all the modules and all the files from the JDK
build are in <install/location/of/jdk> (only 'jmods' directory is
missing)[1]?

Example:

$ rm -rf ./images/jdk/jmods/
$ ./images/jdk/bin/jlink --add-modules java.base --output build/test-jlink-jdk 
Error: --module-path is not specified and this runtime image does not contain jmods directory.
Usage: jlink <options> --module-path <modulepath> --add-modules <module>[,<module>...]
Use --help for a list of possible options

The Proposed Solution:
----------------------
Instead of using the <module-name>.jmod archives when jlinking an
application image, use the JDK installation on the file system together
with the module contents from the jimage to produce the desired result,
provided we initially start with a full JDK (including 'jmods'
directory). Only use this mode *if and only if* the 'jmods' directory
is missing in the JDK image. Use a jlink plugin that records resource
files - other than classes and resources which are in the jimage file -
in a new resource file 'jmod_resources' and add it to the jimage.

In later jlink passes use the 'jmod_resources' resource file of the
jimage in order to track other files part of the application image. The
union of 'jmod_resources' (if any[2]) and the classes and resources
files from the jimage form the equivalent of what '<module-name>.jmod'
archives track today.

The size of those extra 'jmod_resources' files are fairly small: about
~170 bytes on average over all current JDK modules. The one for
'java.base' is "largest" with 1308 bytes on my system. This seems a
good trade-off.

In the proposed prototype the supporting jlink plugin is enabled by
default but could instead only enable given a specific option.

It seems conceivable that such recursive jlink passes would become more
common in context of Leyden, provided jlink serves as the tool for
driving condensation passes. Supporting jmods-less jlinking would
further strengthen that use-case.

An additional advantage of such an approach would be that distributors
could ship a lighter weight JDK distribution (without jmods), yet still
support jlinking specific application images.

The code is here:
https://github.com/jerboaa/jdk/compare/jlink-jmods-base...jerboaa:jdk:jlink-jmods-less?expand=1
Or the branch: 
https://github.com/jerboaa/jdk/tree/jlink-jmods-less

Current Limitations:
--------------------
The prototype only works on the actual target runtime platform. Cross-
jlinking isn't supported. However, in context of Leyden this might be
OK as some AOT compilations as part of some condenser runs might have
similar restrictions.

Also, only modules already present in the base jimage can be included
in derivatives. Consider a base JDK image including JDK modules
'java.base', 'app.module1' and 'app.module2'. Further jlink passes
would only be able to create images including a *subset* of those
modules.

Alternatives:
-------------
There is a '--keep-packaged-modules <output/to/jmods/packages>' option,
but that seems unsatisfying since it copies large amounts of data for
no good reason. Especially when the java.base.jmod includes debuginfo
in native libraries itself (in particular, libjvm.so).

Example Usage:
--------------
$ ./images/jdk/bin/jlink --add-modules ALL-MODULE-PATH --output build/jlinked-image
WARNING: Using incubator modules: jdk.incubator.concurrent, jdk.incubator.vector
$ ls ./build/jlinked-image/jmods
ls: cannot access './build/jlinked-image/jmods': No such file or directory
$ ./build/jlinked-image/bin/jlink --verbose --add-modules java.base \
                                  --output ./build/java.base-image
java.base jrt:/java.base (jmod-less)

Providers:
  java.base provides java.nio.file.spi.FileSystemProvider used by java.base
  java.base provides java.util.random.RandomGenerator used by java.base
$ ./build/java.base-image/bin/java --version
openjdk 21-internal 2023-09-19
OpenJDK Runtime Environment (build 21-internal-adhoc.sgehwolf.jdk-jdk)
OpenJDK 64-Bit Server VM (build 21-internal-adhoc.sgehwolf.jdk-jdk, mixed mode)

Thoughts? Opinions?

Note that depending on feedback, we could consider getting something
like this into mainline JDK as it seems useful outside of Leyden as
well.

Thanks,
Severin


[1] In the Leyden context there is also this issue:
    Let 'jlink' be the tool to "drive" condensor passes. Would I be
    able to use it recursively, minus after the last step? Not in it's
    current form without using '--keep-packaged-modules':
    $ ./images/jdk/bin/jlink --add-modules jdk.jlink --output build/test-jlink-image
    $ ./build/test-jlink-image/bin/jlink --add-modules java.base --output build/test-java.base-image
    Error: --module-path is not specified and this runtime image does not contain jmods directory.
    Usage: jlink <options> --module-path <modulepath> --add-modules <module>[,<module>...]
    Use --help for a list of possible options

[2] Not all modules contain non-class-resource data, like native
    libraries or binaries, but only contain Java class files and
    resources. If so, all required files are already in the jimage
    file.