jmods-less jlinking prototype

[Severin Gehwolf]

On Thu, 2023-06-15 at 18:03 +0100, Alan Bateman wrote:
> On 15/06/2023 09:50, Severin Gehwolf wrote:
> > :
> > I'd like to restart this conversation in order to ultimately arrive at
> > a go/no-go decision about this patch. Hope that's OK.
> > 
> > We do agree it's interesting to explore further. Since last time we
> > spoke the patch has been improved further and I think it's viable. It
> > supports the three main platforms and passes the basic jtreg testing
> > (including the added test; see below for the GHA runs). So if we were
> > to contribute this into a shared repo, which one should this go? a)
> > openjdk/jdk (a.k.a JDK mainline) or b) leyden/jdk c) something else?
> > Would you have any thoughts as to which repo would be most suitable at
> > this point to discuss a PR?
> > 
> > We are reasonably confident that it works well for a majority of use-
> > cases (also outside of leyden) so it might be useful to have in
> > mainline JDK. But maybe it makes sense to first add it to leyden? I'm
> > not sure. In the leyden context it would be a great first building
> > block to further explore composability with a simple enough patch[1].
> > Consider:
> > 
> > j_A -> j_B -> j_C -> j_D
> > 
> > where j_A is a full jlink from jmods (produced during the OpenJDK
> > build), and then j_B..j_D would be further reductions (condensor runs
> > if you will).
> > 
> > Happy to document the caveats of this new jmod-less mode in a CSR if
> > that helps. IMHO, there aren't that many in practice.
> 
> I think a nice property to have would be for a jlink command without 
> packaged modules generate an identical run-time image as it would with 
> the same command when using the packaged modules.

OK. Looking at java.se I get this (the jimage ends up different, but
that's it):

$ ./build/linux-x86_64-server-release/images/jdk/bin/jlink --add-modules java.se --output build/jmod-full-jlink-java.se --verbose
$ ./build/linux-x86_64-server-release/images/jdk/bin/jlink --add-modules java.se,jdk.jlink --output build/jmod-less-jlink1 --verbose
$ ./build/jmod-less-jlink1/bin/jlink --add-modules java.se --output build/jmod-less-jlink-java.se --verbose
$ diff -r -u ./build/jmod-less-jlink-java.se/  jmod-full-jlink-java.se/
Binary files ./build/jmod-less-jlink-java.se/lib/modules and build/jmod-full-jlink-java.se/lib/modules differ
$ ./build/linux-x86_64-server-release/images/jdk/bin/jimage extract --dir build/jimage-java.se-jmodfull ./build/jmod-full-jlink-java.se/lib/modules 
$ ./build/linux-x86_64-server-release/images/jdk/bin/jimage extract --dir build/jimage-java.se-jmodless ./build/jmod-less-jlink-java.se/lib/modules
$ diff -r -u build/jimage-java.se-jmodfull build/jimage-java.se-jmodless
<nothing>

$ diff -u <(./build/linux-x86_64-server-release/images/jdk//bin/jimage info build/jmod-less-jlink-java.se/lib/modules) <(./build/linux-x86_64-server-release/images/jdk//bin/jimage info build/jmod-full-jlink-java.se/lib/modules)
--- /dev/fd/63	2023-06-16 14:45:48.800145724 +0200
+++ /dev/fd/62	2023-06-16 14:45:48.801145733 +0200
@@ -5,6 +5,6 @@
  Table Length:   19474
  Offsets Size:   77896
  Redirects Size: 77896
- Locations Size: 391737
+ Locations Size: 391733
  Strings Size:   446073
- Index Size:     993630
+ Index Size:     993626


So this might be a side-effect of how the jimage is serialized to disk.
I'll keep looking but it seems the patch is already fairly close?

> In my view at least, 
> if it can't guarantee to generate the same run-time image then it should 
> fail. I can't tell how close you are to this but it means being able to 
> reconstitute the original resources or detect that they have changed.
> 

I'm not sure what you mean by reconstitute the original resources?
Could you clarify, please?

Given an unmodified JDK image, a jmod-less jlink will preserve the
resources (doesn't change them).

Is the idea that if somebody, say changed 'conf/net.properties' a
subsequent jmod-less jlink should fail? Something like:

T_1: jlink --add-modules ALL-MODULE-PATH --output build/jmod-less-jlink-all-mods
T_2: Edit build/jmod-less-jlink-all-mods/conf/net.properties
T_3: ./build/jmod-less-all-mods/bin/jlink --add-modules java.se --output build/jmod-less-modified-net

Make the jlink at time T_3 fail? Is that the idea? Something else?

> So I think it means plugins to un-do transformations where feasible, maybe 
> hashes to detect changes to resources. If we had that they I think it
> would be a useful change to have in the main line.

Glad to hear!

Detecting modifications to files not in the jimage seems doable.
Although, I'm less sure what transformation you think of to un-do?!

> Clearly if there is 
> stripping of native debug symbols, and some other transformations, then 
> you aren't going to be able to reconstitute the original natives with
> debug info. So if someone were to create a run-image with stripping and 
> including jdk.jlink module then the resulting run-image could not be 
> used to stamp down further run-images without specifying the location of 
> the packaged modules.

Sounds good.

> I'm less sure about connection to Leyden and condensers.

Fair enough. Since leyden inherits from mainline that's a non-issue.

Cheers,
Severin