JDK-8275509: (jlink) SystemModulesPlugin generates a jdk.internal.module.SystemModules$all.class which isn't reproducible

[Jaikiran Pai]

Hello Alan,

On 19/10/21 7:40 pm, Alan Bateman wrote:
> On 19/10/2021 14:49, Jaikiran Pai wrote:
>>
>> Ah! So this exact same investigation had already happened a few weeks 
>> back then. I haven't subscribed to that list, so missed it. I see in 
>> one of those messages this part:
>>
>> "Off hand I can't think of any issues with the ModuleDescriptor 
>> hashCode. It is computed at link time and should be deterministic. If 
>> I were to guess then then this may be something to do with the module 
>> version recorded at compile-time at that is one of the components 
>> that the hash is based on."
>>
>> To be clear, is the ModuleDescriptor#hashCode() expected to return 
>> reproducible (same) hashCode across multiple runs? What currently 
>> changes the hashCode() across multiple runs is various components 
>> within ModuleDescriptor's hashCode() implementation using the 
>> hashCode() of the enums (specifically the various Modifier enums).
>
> The discussion on jigsaw-dev didn't get to the bottom of the issue at 
> the time, mostly because it wasn't easy to reproduce.
>
> Now that the issue is clearer then we should fix it. Aside from 
> reproducible builds then I expect it is possible to use a 
> ModuleDescriptor.Builder to create a ModuleDescriptor that is equal to 
> a ModuleDescriptor in boot layer configuration but with a different 
> hashCode.
>
Based on this input, one of the tests I have included for verifying this 
proposed hashCode fix, involves dealing with a boot layer 
ModuleDescriptor and then verifying it's hashCode against a 
ModuleDescriptor that is built for the same module using the 
ModuleDescriptor.Builder. It does reproduce the hashCode issue. However, 
that test seems to have exposed a different bug with CDS and equality 
checks against enums (which impact ModuleDescriptor#equals()).

More precisely, consider this trivial Java code:

import java.lang.module.*;
import java.io.*;
import java.util.*;

public class EnumEquality {

     public static void main(final String[] args) throws Exception {
         String moduleName = "java.sql";
         // load the "java.sql" module from boot layer
         Optional<Module> bootModule = 
ModuleLayer.boot().findModule(moduleName);
         if (bootModule.isEmpty()) {
             throw new RuntimeException(moduleName + " module is missing 
in boot layer");
         }
         ModuleDescriptor m1 = bootModule.get().getDescriptor();
         // now recreate the same module using the ModuleDescriptor.read 
on the module's module-info.class
         ModuleDescriptor m2;
         try (InputStream moduleInfo = 
bootModule.get().getResourceAsStream("module-info.class")) {
             if (moduleInfo == null) {
                 throw new RuntimeException("Could not locate 
module-info.class in " + moduleName + " module");
             }
             // this will internally use a ModuleDescriptor.Builder to 
construct the ModuleDescriptor
             m2 = ModuleDescriptor.read(moduleInfo);
         }
         if (!m1.equals(m2)) {
             // root cause - the enums aren't "equal"
             for (ModuleDescriptor.Requires r1 : m1.requires()) {
                 if (r1.name().equals("java.base")) {
                     for (ModuleDescriptor.Requires r2 : m2.requires()) {
                         if (r2.name().equals("java.base")) {
                             System.out.println("Modifiers r1 " + 
r1.modifiers() + " r2 " + r2.modifiers()
                                 + " --> equals? " + 
r1.modifiers().equals(r2.modifiers()));
                         }
                     }
                 }
             }

             throw new RuntimeException("ModuleDescriptor(s) aren't 
equal: \n" + m1 + "\n" + m2);
         }
         System.out.println("Success");
     }
}

This program uses "java.sql" as the module under test. This "java.sql" 
is a boot layer module and among other things has:

     requires transitive java.logging;
     requires transitive java.transaction.xa;
     requires transitive java.xml;

in its module definition[1]. The program first loads this module's 
ModuleDescriptor into an instance m1, using the boot() module layer. It 
then "reconstructs" this same module by reading the module-info.class of 
this module, using the ModuleDescriptor.read() API (which internally 
calls ModuleDescriptor.Builder). This is stored into m2. m1 and m2 are 
then checked for equality (using a call to equals() method). This 
equality check keeps failing consistently.

Digging into it, it appears that since the ModuleDescriptor#equals() 
calls equals() on enum types (in this specific case on 
ModuleDescriptor.Requires.Modifier) and since enum type equality is 
implemented as identity checks, those identity checks are surprisingly 
failing. More specifically ModuleDescriptor.Requires.Modifier.MANDATED 
== ModuleDescriptor.Requires.Modifier.MANDATED is equating to false 
because at runtime I see that two different instances of 
ModuleDescriptor.Requires.Modifier.MANDATED have been loaded (by the 
same boot module classloader). Although I use 
ModuleDescriptor.Requires.Modifier.MANDATED as an example, the same is 
reproducible with other enum values like 
ModuleDescriptor.Requires.Modifier.TRANSITIVE.

This appears to be specific to CDS since running the above program with:

java -Xshare:off EnumEquality

succeeds and the ModuleDescriptor equality check passes.

In short, it looks like there is some general issue with CDS and 
equality checks with enums and perhaps deserves a separate JBS issue?

[1] 
https://github.com/openjdk/jdk/blob/master/src/java.sql/share/classes/module-info.java#L34

-Jaikiran