RFR(S): 8240734: ModuleHashes attribute not reproducible between builds

[Alan Bateman]

On 12/03/2020 07:32, Yangfei (Felix) wrote:
> :
> I looked into the JLinkReproducibleTest.
> There are only two modules contained in lib/modules of the image: main and java.base.
> I changed the test specifying --keep-packaged-modules:
> diff -r 8c5697ed51b2 test/jdk/tools/jlink/JLinkReproducibleTest.java
> --- a/test/jdk/tools/jlink/JLinkReproducibleTest.java   Wed Mar 11 08:34:14 2020 +0100
> +++ b/test/jdk/tools/jlink/JLinkReproducibleTest.java   Thu Mar 12 15:30:45 2020 +0800
> @@ -49,6 +49,8 @@
>           cmd.addAll(List.of(
>               "--module-path", JMODS_DIR.toString() + File.pathSeparator + CLASS_DIR.toString(),
>               "--add-modules", "main",
> +            "--keep-packaged-modules",
> +            image.resolve("jmods").toString(),
>               "--compress=2",
>               "--output", image.toString()
>           ));
>
> One extra java.base.jmod file will be emitted for each image created:
>    image-second/jmods/java.base.jmod
>    image-fourth/jmods/java.base.jmod
>    image-third/jmods/java.base.jmod
>    image-first/jmods/java.base.jmod
>
> These jmod files are the same with the one located in the JDK install directory.
> The ModuleHashes attribute is written in java.base.jmod file when building JDK.
> So the java.base.jmod will always be the same for each run of the JLinkReproducibleTest.
> Maybe I missed something? Could you please say something more on how extending the test?
>
The `jmod` and `jar` tools can be used to compute the hashes of a set of 
intimately connected modules. If modules A, B and C are tightly 
connected, and A requires B and C, then you can use the tooling to 
create a packaged form of A that includes the hashes of B and C. The 
hashes are checked at link-time (and in the case of modular JARs on the 
modular path, at run-time too). So I think you'll need a new test. It 
might be that running `jmod --hash-modules` with just two dependences is 
enough to have the entries written to the ModuleHashes attribute in 
random order. A buddy test to JLinkReproducibleTest that creates a 
run-time image containing "user modules" that are tightly connected 
would be a good test.

I'm pretty sure that this isn't the last issue related to reproducible 
builds. Amy is chasing another one that arises with compressed images 
(jlink --compress=2) that may need work too.

-Alan