RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v3]

[Alan Bateman]

On Thu, 10 Aug 2023 17:21:31 GMT, Severin Gehwolf <sgehwolf at openjdk.org> wrote:

>> Please review this patch which adds a "jmodless" jlink mode to the JDK. Fundamentally this patch adds an option to use `jlink` even though your JDK install might not come with the packaged modules (directory `jmods`). This is particularly useful to further reduce the size of a jlinked runtime. After the removal of the concept of a JRE, a common distribution mechanism is still the full JDK with all modules and packaged modules. However, packaged modules can incur an additional size tax. For example in a container scenario it could be useful to have a base JDK container including all modules, but without also delivering the packaged modules. This comes at a size advantage of `~25%`. Such a base JDK container could then be used to `jlink` application specific runtimes, further reducing the size of the application runtime image (App + JDK runtime; as a single image *or* separate bundles, depending on the app being modularized).
>> 
>> The basic design of this approach is to add a jlink plugin for tracking non-class and non-resource files of a JDK install. I.e. files which aren't present in the jimage (`lib/modules`). This enables producing a `JmodLessArchive` class which has all the info of what constitutes the final jlinked runtime.
>> 
>> Basic usage example:
>> 
>> $ diff -u <(./bin/java --list-modules --limit-modules java.se) <(../linux-x86_64-server-release/images/jdk/bin/java --list-modules --limit-modules java.se)
>> $ diff -u <(./bin/java --list-modules --limit-modules java.se) <(../linux-x86_64-server-release/images/jdk/bin/java --list-modules --limit-modules jdk.jlink)
>> $ ls ../linux-x86_64-server-release/images/jdk/jmods
>> java.base.jmod            java.net.http.jmod       java.sql.rowset.jmod      jdk.crypto.ec.jmod         jdk.internal.opt.jmod                     jdk.jdi.jmod         jdk.management.agent.jmod  jdk.security.auth.jmod
>> java.compiler.jmod        java.prefs.jmod          java.transaction.xa.jmod  jdk.dynalink.jmod          jdk.internal.vm.ci.jmod                   jdk.jdwp.agent.jmod  jdk.management.jfr.jmod    jdk.security.jgss.jmod
>> java.datatransfer.jmod    java.rmi.jmod            java.xml.crypto.jmod      jdk.editpad.jmod           jdk.internal.vm.compiler.jmod             jdk.jfr.jmod         jdk.management.jmod        jdk.unsupported.desktop.jmod
>> java.desktop.jmod         java.scripting.jmod      java.xml.jmod             jdk.hotspot.agent.jmod     jdk.internal.vm.compiler.management.jmod  jdk.jlink.jmod       jdk.naming.dns.j...
>
> Severin Gehwolf has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Implementation for run-image link and single-hop
>    
>    This uses a stamp file in 'lib/modules' jimage in order to recognize
>    multi-hop links. However, this has the caveat of no longer producing
>    reproducible builds as compared to a packaged module-based link.
>    
>    Add an option to allow for multi-hop, which disables the stamp file
>    addition and makes it reproducible again.
>  - Exit the jlink on modified files by default
>    
>    This is configurable so add tests for both scenarios.

I did a pass over this to see where this proposal is currently at. At a high-level I think good progress since the discussion on leyden-dev some time ago. A few comments this from this pass:

If I read it correctly, the default behavior is now to fail if jlink is executed from a run-time image that has been modified from the original and the packaged modules are not observable on the module path (this includes the default module path effectively appended by jlink). That seems the right policy as it avoids modifications by plugins or conf changes silently propagating.

If I read the code correctly, the error when a hash doesn't match is a very terse "Run image links only allow single-hop" so that probably needs to be looked to make sure the message says that the run-time image has been modified and maybe include one of the files/resources that has changed so there is something real to go with the error.

The command line options to override and change the error to a warning or silently ignore seems to be "-run-time-ignore-single-hop" and "--run-image-only-warnings". Maybe it should be reduced to just one option and maybe it should contain the word "clone" to something that makes it more obvious that it's just copying or cloning the contents of the modules in the run-time image (I suspect "single hop" won't be understood).

Adding a new jdk.tools.jlink.internal.Archive implementation where the resources come from the run-time image seems a sensible approach. I don't particularly like the name "JmodLessArchive" because the original module may not have been packaged as a JMOD file, it may have been a modular JAR. Same comment on the BOM file "jmod_resources" added to the top-level directory of each module. I think it's clever to add this to each module in the container image but part of me wonders if it should be hidden in some way to avoid tools depending on it. I don't have a concrete suggestion except to wonder if jrtfs should filter it out.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/14787#issuecomment-1726205365