<div dir="ltr">As a reference, I implemented a tool in the past:<br><div><br></div><div> <a href="https://github.com/Glavo/fallback-jmod">https://github.com/Glavo/fallback-jmod</a></div><div><br></div><div>This tool creates an incomplete jmod file, which contains a list of missing files.<br></div><div>This tool also includes a jlink plugin. When performing a jlink, it will use the classes and binaries in the specified JRE path to complete the contents of this jmod according to this file list.<br></div><div><br></div><div>I'm glad to see that other people are paying attention to this issue. The huge size of JDK 9+makes me feel irritated. I really hope to remove the duplicate files in the jmod file in some way.<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Mar 14, 2023 at 6:36 PM Severin Gehwolf <<a href="mailto:sgehwolf@redhat.com">sgehwolf@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">The Problem:<br>
------------<br>
Why do we need the 'jmods' directory when running jlink when 'java --<br>
list-modules' lists all the modules and all the files from the JDK<br>
build are in <install/location/of/jdk> (only 'jmods' directory is<br>
missing)[1]?<br>
<br>
Example:<br>
<br>
$ rm -rf ./images/jdk/jmods/<br>
$ ./images/jdk/bin/jlink --add-modules java.base --output build/test-jlink-jdk <br>
Error: --module-path is not specified and this runtime image does not contain jmods directory.<br>
Usage: jlink <options> --module-path <modulepath> --add-modules <module>[,<module>...]<br>
Use --help for a list of possible options<br>
<br>
The Proposed Solution:<br>
----------------------<br>
Instead of using the <module-name>.jmod archives when jlinking an<br>
application image, use the JDK installation on the file system together<br>
with the module contents from the jimage to produce the desired result,<br>
provided we initially start with a full JDK (including 'jmods'<br>
directory). Only use this mode *if and only if* the 'jmods' directory<br>
is missing in the JDK image. Use a jlink plugin that records resource<br>
files - other than classes and resources which are in the jimage file -<br>
in a new resource file 'jmod_resources' and add it to the jimage.<br>
<br>
In later jlink passes use the 'jmod_resources' resource file of the<br>
jimage in order to track other files part of the application image. The<br>
union of 'jmod_resources' (if any[2]) and the classes and resources<br>
files from the jimage form the equivalent of what '<module-name>.jmod'<br>
archives track today.<br>
<br>
The size of those extra 'jmod_resources' files are fairly small: about<br>
~170 bytes on average over all current JDK modules. The one for<br>
'java.base' is "largest" with 1308 bytes on my system. This seems a<br>
good trade-off.<br>
<br>
In the proposed prototype the supporting jlink plugin is enabled by<br>
default but could instead only enable given a specific option.<br>
<br>
It seems conceivable that such recursive jlink passes would become more<br>
common in context of Leyden, provided jlink serves as the tool for<br>
driving condensation passes. Supporting jmods-less jlinking would<br>
further strengthen that use-case.<br>
<br>
An additional advantage of such an approach would be that distributors<br>
could ship a lighter weight JDK distribution (without jmods), yet still<br>
support jlinking specific application images.<br>
<br>
The code is here:<br>
<a href="https://github.com/jerboaa/jdk/compare/jlink-jmods-base...jerboaa:jdk:jlink-jmods-less?expand=1" rel="noreferrer" target="_blank">https://github.com/jerboaa/jdk/compare/jlink-jmods-base...jerboaa:jdk:jlink-jmods-less?expand=1</a><br>
Or the branch: <br>
<a href="https://github.com/jerboaa/jdk/tree/jlink-jmods-less" rel="noreferrer" target="_blank">https://github.com/jerboaa/jdk/tree/jlink-jmods-less</a><br>
<br>
Current Limitations:<br>
--------------------<br>
The prototype only works on the actual target runtime platform. Cross-<br>
jlinking isn't supported. However, in context of Leyden this might be<br>
OK as some AOT compilations as part of some condenser runs might have<br>
similar restrictions.<br>
<br>
Also, only modules already present in the base jimage can be included<br>
in derivatives. Consider a base JDK image including JDK modules<br>
'java.base', 'app.module1' and 'app.module2'. Further jlink passes<br>
would only be able to create images including a *subset* of those<br>
modules.<br>
<br>
Alternatives:<br>
-------------<br>
There is a '--keep-packaged-modules <output/to/jmods/packages>' option,<br>
but that seems unsatisfying since it copies large amounts of data for<br>
no good reason. Especially when the java.base.jmod includes debuginfo<br>
in native libraries itself (in particular, libjvm.so).<br>
<br>
Example Usage:<br>
--------------<br>
$ ./images/jdk/bin/jlink --add-modules ALL-MODULE-PATH --output build/jlinked-image<br>
WARNING: Using incubator modules: jdk.incubator.concurrent, jdk.incubator.vector<br>
$ ls ./build/jlinked-image/jmods<br>
ls: cannot access './build/jlinked-image/jmods': No such file or directory<br>
$ ./build/jlinked-image/bin/jlink --verbose --add-modules java.base \<br>
--output ./build/java.base-image<br>
java.base jrt:/java.base (jmod-less)<br>
<br>
Providers:<br>
java.base provides java.nio.file.spi.FileSystemProvider used by java.base<br>
java.base provides java.util.random.RandomGenerator used by java.base<br>
$ ./build/java.base-image/bin/java --version<br>
openjdk 21-internal 2023-09-19<br>
OpenJDK Runtime Environment (build 21-internal-adhoc.sgehwolf.jdk-jdk)<br>
OpenJDK 64-Bit Server VM (build 21-internal-adhoc.sgehwolf.jdk-jdk, mixed mode)<br>
<br>
Thoughts? Opinions?<br>
<br>
Note that depending on feedback, we could consider getting something<br>
like this into mainline JDK as it seems useful outside of Leyden as<br>
well.<br>
<br>
Thanks,<br>
Severin<br>
<br>
<br>
[1] In the Leyden context there is also this issue:<br>
Let 'jlink' be the tool to "drive" condensor passes. Would I be<br>
able to use it recursively, minus after the last step? Not in it's<br>
current form without using '--keep-packaged-modules':<br>
$ ./images/jdk/bin/jlink --add-modules jdk.jlink --output build/test-jlink-image<br>
$ ./build/test-jlink-image/bin/jlink --add-modules java.base --output build/test-java.base-image<br>
Error: --module-path is not specified and this runtime image does not contain jmods directory.<br>
Usage: jlink <options> --module-path <modulepath> --add-modules <module>[,<module>...]<br>
Use --help for a list of possible options<br>
<br>
[2] Not all modules contain non-class-resource data, like native<br>
libraries or binaries, but only contain Java class files and<br>
resources. If so, all required files are already in the jimage<br>
file.<br>
<br>
</blockquote></div>