jmods-less jlinking prototype

[Glavo]

As a reference, I implemented a tool in the past:

    https://github.com/Glavo/fallback-jmod

This tool creates an incomplete jmod file, which contains a list of missing
files.
This tool also includes a jlink plugin. When performing a jlink, it will
use the classes and binaries in the specified JRE path to complete the
contents of this jmod according to this file list.

I'm glad to see that other people are paying attention to this issue. The
huge size of JDK 9+makes me feel irritated. I really hope to remove the
duplicate files in the jmod file in some way.

On Tue, Mar 14, 2023 at 6:36 PM Severin Gehwolf <sgehwolf at redhat.com> wrote:

> The Problem:
> ------------
> Why do we need the 'jmods' directory when running jlink when 'java --
> list-modules' lists all the modules and all the files from the JDK
> build are in <install/location/of/jdk> (only 'jmods' directory is
> missing)[1]?
>
> Example:
>
> $ rm -rf ./images/jdk/jmods/
> $ ./images/jdk/bin/jlink --add-modules java.base --output
> build/test-jlink-jdk
> Error: --module-path is not specified and this runtime image does not
> contain jmods directory.
> Usage: jlink <options> --module-path <modulepath> --add-modules
> <module>[,<module>...]
> Use --help for a list of possible options
>
> The Proposed Solution:
> ----------------------
> Instead of using the <module-name>.jmod archives when jlinking an
> application image, use the JDK installation on the file system together
> with the module contents from the jimage to produce the desired result,
> provided we initially start with a full JDK (including 'jmods'
> directory). Only use this mode *if and only if* the 'jmods' directory
> is missing in the JDK image. Use a jlink plugin that records resource
> files - other than classes and resources which are in the jimage file -
> in a new resource file 'jmod_resources' and add it to the jimage.
>
> In later jlink passes use the 'jmod_resources' resource file of the
> jimage in order to track other files part of the application image. The
> union of 'jmod_resources' (if any[2]) and the classes and resources
> files from the jimage form the equivalent of what '<module-name>.jmod'
> archives track today.
>
> The size of those extra 'jmod_resources' files are fairly small: about
> ~170 bytes on average over all current JDK modules. The one for
> 'java.base' is "largest" with 1308 bytes on my system. This seems a
> good trade-off.
>
> In the proposed prototype the supporting jlink plugin is enabled by
> default but could instead only enable given a specific option.
>
> It seems conceivable that such recursive jlink passes would become more
> common in context of Leyden, provided jlink serves as the tool for
> driving condensation passes. Supporting jmods-less jlinking would
> further strengthen that use-case.
>
> An additional advantage of such an approach would be that distributors
> could ship a lighter weight JDK distribution (without jmods), yet still
> support jlinking specific application images.
>
> The code is here:
>
> https://github.com/jerboaa/jdk/compare/jlink-jmods-base...jerboaa:jdk:jlink-jmods-less?expand=1
> Or the branch:
> https://github.com/jerboaa/jdk/tree/jlink-jmods-less
>
> Current Limitations:
> --------------------
> The prototype only works on the actual target runtime platform. Cross-
> jlinking isn't supported. However, in context of Leyden this might be
> OK as some AOT compilations as part of some condenser runs might have
> similar restrictions.
>
> Also, only modules already present in the base jimage can be included
> in derivatives. Consider a base JDK image including JDK modules
> 'java.base', 'app.module1' and 'app.module2'. Further jlink passes
> would only be able to create images including a *subset* of those
> modules.
>
> Alternatives:
> -------------
> There is a '--keep-packaged-modules <output/to/jmods/packages>' option,
> but that seems unsatisfying since it copies large amounts of data for
> no good reason. Especially when the java.base.jmod includes debuginfo
> in native libraries itself (in particular, libjvm.so).
>
> Example Usage:
> --------------
> $ ./images/jdk/bin/jlink --add-modules ALL-MODULE-PATH --output
> build/jlinked-image
> WARNING: Using incubator modules: jdk.incubator.concurrent,
> jdk.incubator.vector
> $ ls ./build/jlinked-image/jmods
> ls: cannot access './build/jlinked-image/jmods': No such file or directory
> $ ./build/jlinked-image/bin/jlink --verbose --add-modules java.base \
>                                   --output ./build/java.base-image
> java.base jrt:/java.base (jmod-less)
>
> Providers:
>   java.base provides java.nio.file.spi.FileSystemProvider used by java.base
>   java.base provides java.util.random.RandomGenerator used by java.base
> $ ./build/java.base-image/bin/java --version
> openjdk 21-internal 2023-09-19
> OpenJDK Runtime Environment (build 21-internal-adhoc.sgehwolf.jdk-jdk)
> OpenJDK 64-Bit Server VM (build 21-internal-adhoc.sgehwolf.jdk-jdk, mixed
> mode)
>
> Thoughts? Opinions?
>
> Note that depending on feedback, we could consider getting something
> like this into mainline JDK as it seems useful outside of Leyden as
> well.
>
> Thanks,
> Severin
>
>
> [1] In the Leyden context there is also this issue:
>     Let 'jlink' be the tool to "drive" condensor passes. Would I be
>     able to use it recursively, minus after the last step? Not in it's
>     current form without using '--keep-packaged-modules':
>     $ ./images/jdk/bin/jlink --add-modules jdk.jlink --output
> build/test-jlink-image
>     $ ./build/test-jlink-image/bin/jlink --add-modules java.base --output
> build/test-java.base-image
>     Error: --module-path is not specified and this runtime image does not
> contain jmods directory.
>     Usage: jlink <options> --module-path <modulepath> --add-modules
> <module>[,<module>...]
>     Use --help for a list of possible options
>
> [2] Not all modules contain non-class-resource data, like native
>     libraries or binaries, but only contain Java class files and
>     resources. If so, all required files are already in the jimage
>     file.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20230314/7783e39d/attachment.htm>