jlink SystemModulesPlugin use of hashCode breaks reproducibility

[Alan Bateman]

On 20/09/2021 07:52, Craig Raw wrote:
> Hi,
>
> While I've noticed significant improvements in making reproducible builds
> with JDK 16, there is a remaining issue I'm stumped by. The issue is this:
>
> Using jlink invokes the SystemModulesPlugin, which creates .class files
> containing module descriptors for all the modules of the application.
> Specifically, these .class files are SystemModules$all.class and
> SystemModules$default.class. The class files create instances of
> jdk.internal.module.Builder to create the ModuleDescriptor objects. The
> ModuleDescriptors are created by use of the Builder.build(int hashCode)
> method. The problem is this: the value of the hashCode parameter can change
> when building across different systems, changing the bytecode of the
> SystemModules$*.class files, and therefore the modules file created by
> jlink.
>
> It is unclear to me why the hashCode should differ. The SystemModulesPlugin
> invokes ModuleDescriptor.hashCode() on line 1116, and that method should be
> deterministic. However, I do note that the value is cached inside
> ModuleDescriptor, and in fact a ModuleDescriptor can be constructed with a
> provided hashCode value. Perhaps an outdated value is cached? My other
> consideration is that integer overflow is behaving differently on some
> systems, but I am targeting 64 bit systems only, and this issue is
> occurring across similar Debian-based platforms running identical JDK
> versions (adoptopenjdk-16-hotspot=16.0.1+9).
>
> The issue is illustrated with an example here:
> https://github.com/sparrowwallet/sparrow/issues/197
>
> Note that this issue is intermittent - many users are able to create
> reproducible builds of this project, but some are not. For reference, the
> steps to creating a reproducible build are documented here (note only the
> 1.5.0-beta1 release is reproducible):
> https://github.com/sparrowwallet/sparrow/blob/master/docs/reproducible.md
>
> Any assistance in resolving this is appreciated.
We fixed a lots of issues with reproducible builds in recent releases, 
including several issues related to iteration order that were 
problematic for the jlink plugins. Off hand I can't think of any issues 
with the ModuleDescriptor hashCode. It is computed at link time and 
should be deterministic. If I were to guess then then this may be 
something to do with the module version recorded at compile-time at that 
is one of the components that the hash is based on. That's just a guess, 
I think I would need to see the module path specified to jlink to have a 
better idea.

BTW: Have you had any reports with JDK 17 yet?

-Alan