jmods-less jlinking prototype

[Severin Gehwolf]

On Wed, 2023-03-15 at 14:51 +0000, Alan Bateman wrote:
> On 14/03/2023 10:36, Severin Gehwolf wrote:
> > The Problem:
> > ------------
> > Why do we need the 'jmods' directory when running jlink when 'java --
> > list-modules' lists all the modules and all the files from the JDK
> > build are in <install/location/of/jdk> (only 'jmods' directory is
> > missing)[1]?
> 
> Just some history around this. During JDK 9, there was a suggestion that 
> the packaged modules should be a separate download but that had its own 
> set of issues.  There were also suggestions that jlink should use the
> classes/resources from the current run-time image to avoid needing to
> include the complete contents of all packaged modules, it just wasn't
> interesting to pursue at the time. That was 6+ years ago and there is a 
> lot of experience with jlink since then, plus we've now on a road where 
> jlink may be working with condensers in the future. So it could be 
> interesting to explore to see how it might fit in.

Thanks for the history lesson!

> More background is that generating a run-time image may involve a number 
> of transformations, e.g jlink --strip-debug may remove debugging symbols 
> and strip debug related class file attributes, other plugins, like 
> generate-jli-classes and system-modules, generate code and/or modify a 
> number of classes.  Post generation, there may also be changes to 
> configuration files in conf/**. I think the point is that all types of 
> resources (native libs, executables, classes/resources, conf) can be 
> filtered out, stripped, or irreversibly transformed.

Agreed to the above. Incidentally, the generate-jli-classes plugin
isn't designed for recursive jlinking so I had to work around it[1]. In
the Leyden context this does not seem so bad a trade-off, though. You
can only sub-set the JDK image you start out with with this mode. So if
the libraries had debug info stripped away the result would have that
too.

I'd argue that the case where users actually want things back (add
back, rather than remove) from any base JDK image is rather small.
Though, I have no numbers about this so this is only anecdotal. Have
you seen use-cases doing that (add-back, taking from jmods)?

> Even if the run-time image has the jdk.jlink module, it might not have many of the 
> modules that were in the original JDK builds so it's not really a good 
> starting point for stamping out further images.

While it might not have many modules in the JDK, the prime use case of
this prototype is focused on further reduction, so it might in fact be
a good starting point for many cases. In particular, the base case of 
ALL-MODULE-PATH -> jlink again to get a small application image.

> On container environments, we see a lot of examples where jlink is run 
> to produce a run-time image with a subset of the modules and with some 
> stripping and compression to get a smaller run-time image. Smaller 
> container means faster to transport across network and deploy, great!
> Most of the recipes that I've seen published don't allow for further 
> jlink-ing, either because they only have a small subset of the modules 
> or they don't have the jdk.jlink module so they don't have the tool. 
> Maybe this effort might change that.

+1

> So I think it could be interesting to explore to at least see how far it 
> can get. I read your proposal/prototype as generating a patch/diff of
> the bits in the run-time image so you can essentially go back to the 
> contents of the original packages modules. If I read your mail 
> correctly, if debug symbols are stripped then it doesn't currently 
> recover the original libjvm.so or other libraries.

Correct. Recovering something that wasn't in the the JDK image to begin
with is not a goal. That's where the original jmods-full usecase would
come in (although, if the JDK build had debug symbols already stripped,
there is also no way to go back and recover those in the jmods-full
case).

> Also I couldn't see 
> if the original conf/templates are saved for future images or whether
> configuration edits get copied.

Original conf/templates aren't saved. They're taken as is from the base
image. I.e. modifications propagate throughout.

I'd like to stress that this proposal isn't supposed to replace the
jmods-full jlink. At least the first stage need to be jmods-full (e.g.
initial JDK build). Or at least it wasn't designed for that.

[1] https://github.com/jerboaa/jdk/commit/41339bf7938c2070b68b34a64cf86241282e1c66#diff-fe9ad54f129e10e65432e62154d18797929677180122263f50ab454b59658cd8R65