jmods-less jlinking prototype

[Alan Bateman]

On 16/06/2023 13:48, Severin Gehwolf wrote:
> :
> OK. Looking at java.se I get this (the jimage ends up different, but
> that's it):
>
> $ ./build/linux-x86_64-server-release/images/jdk/bin/jlink --add-modules java.se --output build/jmod-full-jlink-java.se --verbose
> $ ./build/linux-x86_64-server-release/images/jdk/bin/jlink --add-modules java.se,jdk.jlink --output build/jmod-less-jlink1 --verbose
> $ ./build/jmod-less-jlink1/bin/jlink --add-modules java.se --output build/jmod-less-jlink-java.se --verbose
> $ diff -r -u ./build/jmod-less-jlink-java.se/  jmod-full-jlink-java.se/
> Binary files ./build/jmod-less-jlink-java.se/lib/modules and build/jmod-full-jlink-java.se/lib/modules differ
> $ ./build/linux-x86_64-server-release/images/jdk/bin/jimage extract --dir build/jimage-java.se-jmodfull ./build/jmod-full-jlink-java.se/lib/modules
> $ ./build/linux-x86_64-server-release/images/jdk/bin/jimage extract --dir build/jimage-java.se-jmodless ./build/jmod-less-jlink-java.se/lib/modules
> $ diff -r -u build/jimage-java.se-jmodfull build/jimage-java.se-jmodless
> <nothing>
>
> $ diff -u <(./build/linux-x86_64-server-release/images/jdk//bin/jimage info build/jmod-less-jlink-java.se/lib/modules) <(./build/linux-x86_64-server-release/images/jdk//bin/jimage info build/jmod-full-jlink-java.se/lib/modules)
> --- /dev/fd/63	2023-06-16 14:45:48.800145724 +0200
> +++ /dev/fd/62	2023-06-16 14:45:48.801145733 +0200
> @@ -5,6 +5,6 @@
>    Table Length:   19474
>    Offsets Size:   77896
>    Redirects Size: 77896
> - Locations Size: 391737
> + Locations Size: 391733
>    Strings Size:   446073
> - Index Size:     993630
> + Index Size:     993626
>
>
> So this might be a side-effect of how the jimage is serialized to disk.
> I'll keep looking but it seems the patch is already fairly close?
Reproducible builds have been a bit of whack-a-mole that has included 
the image writer. I'm not aware of any remaining issues so what you are 
seeing may be the ordering that strings are hashed or it may be be 
contents (use `jimage extract --dir <dir> lib/modules` and compare).


> :
> I'm not sure what you mean by reconstitute the original resources?
> Could you clarify, please?
>
> Given an unmodified JDK image, a jmod-less jlink will preserve the
> resources (doesn't change them).
>
> Is the idea that if somebody, say changed 'conf/net.properties' a
> subsequent jmod-less jlink should fail? Something like:
>
> T_1: jlink --add-modules ALL-MODULE-PATH --output build/jmod-less-jlink-all-mods
> T_2: Edit build/jmod-less-jlink-all-mods/conf/net.properties
> T_3: ./build/jmod-less-all-mods/bin/jlink --add-modules java.se --output build/jmod-less-modified-net
>
> Make the jlink at time T_3 fail? Is that the idea? Something else?
The SystemModulesPlugin is one of complicated plugins as it replaces 
SystemModulesMap and generates classes that are specific to the set of 
modules that go into the target runtime. If the linking is "as if" the 
packaged modules are present then the resources that go through the 
pipeline shouldn't include the generated/modified classes. If T_3 is 
changed to generate a run-image with a subset of the modules then you'll 
see what I mean.

In T_1, the target run-time image will include the jdk.jlink module. I 
think that is your signal to identify the resources that have been 
added, removed or modified in the plugin pipeline. There are many ways 
that the diffs can be expressed and saved so that the resources in the 
modules that go through the pipeline in T_3 are "as if" they came from 
the packaged modules used in T_1.

A motivating example for user editable configuration is where 
conf/security/java.security at T_2 is changed to allow weak crypto. 
That's not something to silently propagate. It could be that T_3 fails 
because the hashes don't match or it has saved the original conf resource.

For the native libs then I think the ModuleFinder can detect that the 
hashes have changed. There's a choice here on whether it just warns or 
fails. As you know, the only plugin that modifies native libs right now 
is the debug symbol stripping. There is also the VM selection plugin but 
that just works as a filter.

-Alan