jlink SystemModulesPlugin use of hashCode breaks reproducibility

[Craig Raw]

Sure - I'm using the Gradle badass-jlink-plugin. The command it is
executing is:

/Users/craigraw/.sdkman/candidates/java/16.0.1.hs-adpt/bin/jlink -v
--strip-debug --compress 2 --no-header-files --no-man-pages
--ignore-signing-information --exclude-files **.png --exclude-resources
glob:/com.sparrowwallet.merged.module/META-INF/* --module-path
/Users/craigraw/.sdkman/candidates/java/16.0.1.hs-adpt/jmods/:/Users/craigraw/git/sparrow/build/jlinkbase/jlinkjars
--add-modules com.sparrowwallet.sparrow --output
/Users/craigraw/git/sparrow/build/image

I've had no experience with JDK 17 yet - I'm specifying that users build
with a constant version (JDK 16 at present) to maintain
reproducibility with jpackage.

Craig

On Mon, Sep 20, 2021 at 11:54 AM Alan Bateman <Alan.Bateman at oracle.com>
wrote:

> On 20/09/2021 07:52, Craig Raw wrote:
> > Hi,
> >
> > While I've noticed significant improvements in making reproducible builds
> > with JDK 16, there is a remaining issue I'm stumped by. The issue is
> this:
> >
> > Using jlink invokes the SystemModulesPlugin, which creates .class files
> > containing module descriptors for all the modules of the application.
> > Specifically, these .class files are SystemModules$all.class and
> > SystemModules$default.class. The class files create instances of
> > jdk.internal.module.Builder to create the ModuleDescriptor objects. The
> > ModuleDescriptors are created by use of the Builder.build(int hashCode)
> > method. The problem is this: the value of the hashCode parameter can
> change
> > when building across different systems, changing the bytecode of the
> > SystemModules$*.class files, and therefore the modules file created by
> > jlink.
> >
> > It is unclear to me why the hashCode should differ. The
> SystemModulesPlugin
> > invokes ModuleDescriptor.hashCode() on line 1116, and that method should
> be
> > deterministic. However, I do note that the value is cached inside
> > ModuleDescriptor, and in fact a ModuleDescriptor can be constructed with
> a
> > provided hashCode value. Perhaps an outdated value is cached? My other
> > consideration is that integer overflow is behaving differently on some
> > systems, but I am targeting 64 bit systems only, and this issue is
> > occurring across similar Debian-based platforms running identical JDK
> > versions (adoptopenjdk-16-hotspot=16.0.1+9).
> >
> > The issue is illustrated with an example here:
> > https://github.com/sparrowwallet/sparrow/issues/197
> >
> > Note that this issue is intermittent - many users are able to create
> > reproducible builds of this project, but some are not. For reference, the
> > steps to creating a reproducible build are documented here (note only the
> > 1.5.0-beta1 release is reproducible):
> >
> https://github.com/sparrowwallet/sparrow/blob/master/docs/reproducible.md
> >
> > Any assistance in resolving this is appreciated.
> We fixed a lots of issues with reproducible builds in recent releases,
> including several issues related to iteration order that were
> problematic for the jlink plugins. Off hand I can't think of any issues
> with the ModuleDescriptor hashCode. It is computed at link time and
> should be deterministic. If I were to guess then then this may be
> something to do with the module version recorded at compile-time at that
> is one of the components that the hash is based on. That's just a guess,
> I think I would need to see the module path specified to jlink to have a
> better idea.
>
> BTW: Have you had any reports with JDK 17 yet?
>
> -Alan
>
>
>
>
>