jmods-less jlinking prototype

[Severin Gehwolf]

On Sat, 2023-06-17 at 17:05 +0100, Alan Bateman wrote:
> > On 16/06/2023 13:48, Severin Gehwolf wrote:
> > > > :
> > > > OK. Looking at java.se I get this (the jimage ends up different, but
> > > > that's it):
> > > > 
> > > > $ ./build/linux-x86_64-server-release/images/jdk/bin/jlink --add-modules java.se --output build/jmod-full-jlink-java.se --verbose
> > > > $ ./build/linux-x86_64-server-release/images/jdk/bin/jlink --add-modules java.se,jdk.jlink --output build/jmod-less-jlink1 --verbose
> > > > $ ./build/jmod-less-jlink1/bin/jlink --add-modules java.se --output build/jmod-less-jlink-java.se --verbose
> > > > $ diff -r -u ./build/jmod-less-jlink-java.se/  jmod-full-jlink-java.se/
> > > > Binary files ./build/jmod-less-jlink-java.se/lib/modules and build/jmod-full-jlink-java.se/lib/modules differ
> > > > $ ./build/linux-x86_64-server-release/images/jdk/bin/jimage extract --dir build/jimage-java.se-jmodfull ./build/jmod-full-jlink-java.se/lib/modules
> > > > $ ./build/linux-x86_64-server-release/images/jdk/bin/jimage extract --dir build/jimage-java.se-jmodless ./build/jmod-less-jlink-java.se/lib/modules
> > > > $ diff -r -u build/jimage-java.se-jmodfull build/jimage-java.se-jmodless
> > > > <nothing>
> > > > 
> > > > $ diff -u <(./build/linux-x86_64-server-release/images/jdk//bin/jimage info build/jmod-less-jlink-java.se/lib/modules) <(./build/linux-x86_64-server-release/images/jdk//bin/jimage info build/jmod-full-jlink-java.se/lib/modules)
> > > > --- /dev/fd/63  2023-06-16 14:45:48.800145724 +0200
> > > > +++ /dev/fd/62  2023-06-16 14:45:48.801145733 +0200
> > > > @@ -5,6 +5,6 @@
> > > >    Table Length:   19474
> > > >    Offsets Size:   77896
> > > >    Redirects Size: 77896
> > > > - Locations Size: 391737
> > > > + Locations Size: 391733
> > > >    Strings Size:   446073
> > > > - Index Size:     993630
> > > > + Index Size:     993626
> > > > 
> > > > 
> > > > So this might be a side-effect of how the jimage is serialized to disk.
> > > > I'll keep looking but it seems the patch is already fairly close?
> > Reproducible builds have been a bit of whack-a-mole that has included
> > the image writer. I'm not aware of any remaining issues so what you are 
> > seeing may be the ordering that strings are hashed or it may be be 
> > contents (use `jimage extract --dir <dir> lib/modules` and compare).
> > 

I've done the 'jimage extract' compare already (see above commands) and
they're identical. So it's probably strings ordering.

> > 
> > > > :
> > > > I'm not sure what you mean by reconstitute the original resources?
> > > > Could you clarify, please?
> > > > 
> > > > Given an unmodified JDK image, a jmod-less jlink will preserve the
> > > > resources (doesn't change them).
> > > > 
> > > > Is the idea that if somebody, say changed 'conf/net.properties' a
> > > > subsequent jmod-less jlink should fail? Something like:
> > > > 
> > > > T_1: jlink --add-modules ALL-MODULE-PATH --output build/jmod-less-jlink-all-mods
> > > > T_2: Edit build/jmod-less-jlink-all-mods/conf/net.properties
> > > > T_3: ./build/jmod-less-all-mods/bin/jlink --add-modules java.se --output build/jmod-less-modified-net
> > > > 
> > > > Make the jlink at time T_3 fail? Is that the idea? Something else?
> > The SystemModulesPlugin is one of complicated plugins as it replaces 
> > SystemModulesMap and generates classes that are specific to the set of 
> > modules that go into the target runtime. If the linking is "as if" the 
> > packaged modules are present then the resources that go through the 
> > pipeline shouldn't include the generated/modified classes. If T_3 is 
> > changed to generate a run-image with a subset of the modules then you'll 
> > see what I mean.

OK, thanks. Noted.

> > In T_1, the target run-time image will include the jdk.jlink module. I 
> > think that is your signal to identify the resources that have been 
> > added, removed or modified in the plugin pipeline. There are many ways 
> > that the diffs can be expressed and saved so that the resources in the 
> > modules that go through the pipeline in T_3 are "as if" they came from 
> > the packaged modules used in T_1.
> > 
> > A motivating example for user editable configuration is where 
> > conf/security/java.security at T_2 is changed to allow weak crypto. 
> > That's not something to silently propagate. It could be that T_3 fails 
> > because the hashes don't match or it has saved the original conf resource.

OK, got it.

> > For the native libs then I think the ModuleFinder can detect that the
> > hashes have changed. There's a choice here on whether it just warns or 
> > fails. As you know, the only plugin that modifies native libs right now 
> > is the debug symbol stripping. There is also the VM selection plugin but 
> > that just works as a filter.

Sounds good. I'll go for producing warnings first.

Thanks,
Severin