RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v12]

[Mandy Chung]

On Fri, 8 Mar 2024 17:19:41 GMT, Severin Gehwolf <sgehwolf at openjdk.org> wrote:

>> I tried out the latest commit (a797ea69).
>> 
>> The output "The default module path, '$java.home/jmods' not present. Use --verbose to show the full list of plugin options applied" is bit confusing as it looks like jlink failed but it actually succeeded. Blowing away the generated image and retrying with --verbose tripped this assert
>> 
>> java.lang.AssertionError: handling of scratch options failed
>> 	at jdk.jlink/jdk.tools.jlink.internal.JlinkTask.logPackagedModuleEquivalent(JlinkTask.java:675)
>> 	at jdk.jlink/jdk.tools.jlink.internal.JlinkTask.createImageProvider(JlinkTask.java:581)
>> 	at jdk.jlink/jdk.tools.jlink.internal.JlinkTask.createImage(JlinkTask.java:430)
>> 	at jdk.jlink/jdk.tools.jlink.internal.JlinkTask.run(JlinkTask.java:302)
>> 	at jdk.jlink/jdk.tools.jlink.internal.Main.run(Main.java:56)
>> 	at jdk.jlink/jdk.tools.jlink.internal.Main.main(Main.java:34)
>> Caused by: jdk.tools.jlink.internal.TaskHelper$BadArgs: (...my-original-jdk-directory..)/build/linux-x64/images/jdk/jmods already exists
>> 	at jdk.jlink/jdk.tools.jlink.internal.TaskHelper.newBadArgs(TaskHelper.java:730)
>> 	at jdk.jlink/jdk.tools.jlink.internal.JlinkTask.lambda$static$12(JlinkTask.java:183)
>> 	at jdk.jlink/jdk.tools.jlink.internal.TaskHelper$Option.process(TaskHelper.java:177)
>> 	at jdk.jlink/jdk.tools.jlink.internal.TaskHelper$OptionsHelper.handleOptions(TaskHelper.java:600)
>> 	at jdk.jlink/jdk.tools.jlink.internal.JlinkTask.logPackagedModuleEquivalent(JlinkTask.java:672)
>> 	... 5 more
>> 
>> I haven't dug into this yet but I'm puzzled that the file path to where the original build was created is in the exception messages, is that recorded?
>
>> > @AlanBateman @mlchung I've now pushed an update of this patch which now uses a build-time approach as discussed elsewhere. In order to produce a linkable runtime JDK image, one needs to set --enable-runtime-link-image at configure time.
>> 
>> What is the rationale for introducing a special configure flag for this functionality? I've tried to look though all comments in this PR, and read the JBS issue and CSR, but I have not found any motivation for this. I'm sorry if it's there and I missed it, but this is a huge PR with a lot of discussion.
> 
> Sorry, yes this was part of a meeting discussion we had outside this PR. My understanding is that by default the produced jimage isn't runtime-link enabled. We (Red Hat) would turn it on in our builds, though. @AlanBateman or @mlchung might have more details. I think it was a requirement to get this patch in. At least for the initial contribution.
> 
>> In general, it is better not to introduce variants of the build like this. The testing matrix just explodes a bit more. And my understanding from the discussion is that this functionality is likely to be turned on anyway, otherwise you'll build a crippled jlink without full functionality.
> 
> I would be happy if this could be on by default. For now, I think though we need to work on the premise that whether or not the resulting JDK image is suitable for runtime linking (without jmods) is a build-time config decision. Therefore we need the configure option.

@jerboaa thanks for the update.  First to recap the revised proposal (based on Alan's notes shared with me earlier):

The JDK build is capable of producing a JDK run-time image that does not include packaged modules and the  new JDK image is capable to create custom run-time images (call it "linkable" JDK image for now).   To reconstitute to the original module content, the new JDK image conceptually needs to contain the "diffs" from the original packaged packaged.   This makes it possible for the jlink plugins to run "as if" the resources for each module were coming from the original packaged module.  The new image also has the checksums of at least the user-editable configuration files so that edits can be detected. 

The revised proposal has a few limitations:

1. The "linkable" JDK image can only be created by the JDK build.
2. The "linkable" JDK image is created from the JDK image produced by the JDK build today.  It contains the same set of modules, there is no possibility to combine its generation with other jlink options or code transformations.
3. The "linkable" JDK image cannot create a run-time image that contains the "jdk.jlink" module.
4. The "linkable" JDK image only reconstitutes classes/resources to the original module bits.   Other plugins such as man pages and dedup legal files are not reconstituted.

These limitations are trade-off to make for a much simpler approach. It removes the issues of creating a linkable JDK, or creating another image using a linkable JDK, while at the same time executing a pipeline of plugins that do other transformations.   

Here is my feedback to the prototype:

- I think the reader and writer code for generating the diffs should be in `jdk.jlink` so that they are always kept in sync.
- The diffs generation can be done as a tool or `jlink` option.  If it's a tool, it should be run on the interim image (like the tool generating classlist - see `GenerateLinkOptData.gmk`).

I have no strong opinion in which approach.   It seems worth exploring the tool approach residing in `jdk.jlink` to generate the diffs as well as create the linkable image could remove the need of a jlink internal `--create-linkable-runtime` option.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/14787#issuecomment-1986325303