RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present

Fri Jul 7 08:40:47 UTC 2023

Please review this patch which adds a "jmodless" jlink mode to the JDK. Fundamentally this patch adds an option to use `jlink` even though your JDK install might not come with the packaged modules (directory `jmods`). This is particularly useful to further reduce the size of a jlinked runtime. After the removal of the concept of a JRE, a common distribution mechanism is still the full JDK with all modules and packaged modules. However, packaged modules can incur an additional size tax. For example in a container scenario it could be useful to have a base JDK container including all modules, but without also delivering the packaged modules. This comes at a size advantage of `~25%`. Such a base JDK container could then be used to `jlink` application specific runtimes, further reducing the size of the application runtime image (App + JDK runtime; as a single image *or* separate bundles, depending on the app being modularized).

The basic design of this approach is to add a jlink plugin for tracking non-class and non-resource files of a JDK install. I.e. files which aren't present in the jimage (`lib/modules`). This enables producing a `JmodLessArchive` class which has all the info of what constitutes the final jlinked runtime.

Basic usage example:

$ diff -u <(./bin/java --list-modules --limit-modules java.se) <(../linux-x86_64-server-release/images/jdk/bin/java --list-modules --limit-modules java.se)
$ diff -u <(./bin/java --list-modules --limit-modules java.se) <(../linux-x86_64-server-release/images/jdk/bin/java --list-modules --limit-modules jdk.jlink)
$ ls ../linux-x86_64-server-release/images/jdk/jmods
java.base.jmod            java.net.http.jmod       java.sql.rowset.jmod      jdk.crypto.ec.jmod         jdk.internal.opt.jmod                     jdk.jdi.jmod         jdk.management.agent.jmod  jdk.security.auth.jmod
java.compiler.jmod        java.prefs.jmod          java.transaction.xa.jmod  jdk.dynalink.jmod          jdk.internal.vm.ci.jmod                   jdk.jdwp.agent.jmod  jdk.management.jfr.jmod    jdk.security.jgss.jmod
java.datatransfer.jmod    java.rmi.jmod            java.xml.crypto.jmod      jdk.editpad.jmod           jdk.internal.vm.compiler.jmod             jdk.jfr.jmod         jdk.management.jmod        jdk.unsupported.desktop.jmod
java.desktop.jmod         java.scripting.jmod      java.xml.jmod             jdk.hotspot.agent.jmod     jdk.internal.vm.compiler.management.jmod  jdk.jlink.jmod       jdk.naming.dns.jmod        jdk.unsupported.jmod
java.instrument.jmod      java.security.jgss.jmod  jdk.accessibility.jmod    jdk.httpserver.jmod        jdk.jartool.jmod                          jdk.jpackage.jmod    jdk.naming.rmi.jmod        jdk.xml.dom.jmod
java.logging.jmod         java.security.sasl.jmod  jdk.attach.jmod           jdk.incubator.vector.jmod  jdk.javadoc.jmod                          jdk.jshell.jmod      jdk.net.jmod               jdk.zipfs.jmod
java.management.jmod      java.se.jmod             jdk.charsets.jmod         jdk.internal.ed.jmod       jdk.jcmd.jmod                             jdk.jsobject.jmod    jdk.nio.mapmode.jmod
java.management.rmi.jmod  java.smartcardio.jmod    jdk.compiler.jmod         jdk.internal.jvmstat.jmod  jdk.jconsole.jmod                         jdk.jstatd.jmod      jdk.random.jmod
java.naming.jmod          java.sql.jmod            jdk.crypto.cryptoki.jmod  jdk.internal.le.jmod       jdk.jdeps.jmod                            jdk.localedata.jmod  jdk.sctp.jmod
$ ls jmods
ls: cannot access 'jmods': No such file or directory
$ ./bin/jlink --version
22-internal
$ ./bin/jlink --add-modules java.se --output ../build/java.se-runtime --verbose
java.base jrt:/java.base (jmod-less)
java.compiler jrt:/java.compiler (jmod-less)
java.datatransfer jrt:/java.datatransfer (jmod-less)
java.desktop jrt:/java.desktop (jmod-less)
java.instrument jrt:/java.instrument (jmod-less)
java.logging jrt:/java.logging (jmod-less)
java.management jrt:/java.management (jmod-less)
java.management.rmi jrt:/java.management.rmi (jmod-less)
java.naming jrt:/java.naming (jmod-less)
java.net.http jrt:/java.net.http (jmod-less)
java.prefs jrt:/java.prefs (jmod-less)
java.rmi jrt:/java.rmi (jmod-less)
java.scripting jrt:/java.scripting (jmod-less)
java.se jrt:/java.se (jmod-less)
java.security.jgss jrt:/java.security.jgss (jmod-less)
java.security.sasl jrt:/java.security.sasl (jmod-less)
java.sql jrt:/java.sql (jmod-less)
java.sql.rowset jrt:/java.sql.rowset (jmod-less)
java.transaction.xa jrt:/java.transaction.xa (jmod-less)
java.xml jrt:/java.xml (jmod-less)
java.xml.crypto jrt:/java.xml.crypto (jmod-less)

Providers:
  java.desktop provides java.net.ContentHandlerFactory used by java.base
  java.base provides java.nio.file.spi.FileSystemProvider used by java.base
  java.naming provides java.security.Provider used by java.base
  java.security.jgss provides java.security.Provider used by java.base
  java.security.sasl provides java.security.Provider used by java.base
  java.xml.crypto provides java.security.Provider used by java.base
  java.base provides java.util.random.RandomGenerator used by java.base
  java.management.rmi provides javax.management.remote.JMXConnectorProvider used by java.management
  java.management.rmi provides javax.management.remote.JMXConnectorServerProvider used by java.management
  java.desktop provides javax.print.PrintServiceLookup used by java.desktop
  java.desktop provides javax.print.StreamPrintServiceFactory used by java.desktop
  java.management provides javax.security.auth.spi.LoginModule used by java.base
  java.desktop provides javax.sound.midi.spi.MidiDeviceProvider used by java.desktop
  java.desktop provides javax.sound.midi.spi.MidiFileReader used by java.desktop
  java.desktop provides javax.sound.midi.spi.MidiFileWriter used by java.desktop
  java.desktop provides javax.sound.midi.spi.SoundbankReader used by java.desktop
  java.desktop provides javax.sound.sampled.spi.AudioFileReader used by java.desktop
  java.desktop provides javax.sound.sampled.spi.AudioFileWriter used by java.desktop
  java.desktop provides javax.sound.sampled.spi.FormatConversionProvider used by java.desktop
  java.desktop provides javax.sound.sampled.spi.MixerProvider used by java.desktop
  java.logging provides jdk.internal.logger.DefaultLoggerFinder used by java.base
  java.desktop provides sun.datatransfer.DesktopDatatransferService used by java.datatransfer

One nice property of this patch is that it can produce an identical (as in binary identical) `java.se` JDK image in "jmod-less" mode as compared to a regular jlink with packaged modules present. This has been asserted in newly added tests. In order to prevent accidental copy of modified files in the base JDK a checksum mechanism is in place to warn about this when jlinking. This is also asserted in tests.

One limitation of this mode is that there is no way to use it for cross-linking (at least currently). That is, a jmod-less `jlink` needs to happen on the same runtime platform the resulting image is intended to get deployed on.

Testing:
- [x] GHA (including `jdk/tools/jlink` tests). See for example [the latest run](https://github.com/jerboaa/jdk/actions/runs/5477743443) and [here](https://github.com/jerboaa/jdk/actions/runs/5475484765).
- [x] Some tests on our internal infra (JDK 17-based). It didn't show any regressions.
- [x] Added tests as part of the patch (8 of them). All pass on the major platforms.

Thoughts?

-------------

Commit messages:
 - 8311302: Allow for jlinking a custom runtime without packaged modules being present

Changes: https://git.openjdk.org/jdk/pull/14787/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14787&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8311302
  Stats: 2083 lines in 23 files changed: 2016 ins; 33 del; 34 mod
  Patch: https://git.openjdk.org/jdk/pull/14787.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/14787/head:pull/14787

PR: https://git.openjdk.org/jdk/pull/14787