RFR: 8377712: ConstantPool of WeakReferenceKey is not deterministic in CDS archive
We are seeing intermittent failures (12 times in Jan 2026) where the CDS archives generated in the JDK build is not deterministic. The symptom is that (perhaps due to the peculiarity of hash codes and memory addressing??) the constant pool entry for `ReferenceKey<?>` in the `WeakReferenceKey` class is not resolved when the failure happens. The fix is for the JVM to call a method in `WeakReferenceKey` to force this entry to be resolved. ------------- Commit messages: - 8370855: ConstantPool of WeakReferenceKey is not deterministic in CDS archive Changes: https://git.openjdk.org/jdk/pull/29678/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=29678&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8377712 Stats: 41 lines in 4 files changed: 39 ins; 1 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/29678.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/29678/head:pull/29678 PR: https://git.openjdk.org/jdk/pull/29678
On Wed, 11 Feb 2026 20:28:15 GMT, Ioi Lam <iklam@openjdk.org> wrote:
We are seeing intermittent failures (12 times in Jan 2026) where the CDS archives generated in the JDK build is not deterministic.
The symptom is that (perhaps due to the peculiarity of hash codes and memory addressing??) the constant pool entry for `ReferenceKey<?>` in the `WeakReferenceKey` class is not resolved when the failure happens.
The fix is for the JVM to call a method in `WeakReferenceKey` to force this entry to be resolved.
Is there a reason why this only happens for WeakReferenceKey but not SoftReferenceKey? ------------- PR Comment: https://git.openjdk.org/jdk/pull/29678#issuecomment-3887762733
On Wed, 11 Feb 2026 23:14:25 GMT, Chen Liang <liach@openjdk.org> wrote:
Is there a reason why this only happens for WeakReferenceKey but not SoftReferenceKey?
WeakReferenceKey is used in this call stack when creating a CDS archive during the JDK build process: at java.base/jdk.internal.util.WeakReferenceKey.equals(WeakReferenceKey.java:73) at java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1054) at java.base/java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1558) at java.base/jdk.internal.util.ReferencedKeyMap.internKey(ReferencedKeyMap.java:460) at java.base/jdk.internal.util.ReferencedKeyMap.intern(ReferencedKeyMap.java:396) at java.base/jdk.internal.util.ReferencedKeySet.intern(ReferencedKeySet.java:178) at java.base/java.lang.invoke.MethodType.makeImpl(MethodType.java:410) at java.base/java.lang.invoke.MethodType.methodType(MethodType.java:312) at java.base/java.lang.invoke.MemberName.getMethodOrFieldType(MemberName.java:110) at java.base/java.lang.invoke.DirectMethodHandle.make(DirectMethodHandle.java:81) at java.base/java.lang.invoke.DirectMethodHandle.make(DirectMethodHandle.java:130) at java.base/java.lang.invoke.MethodHandles$Lookup.getDirectFieldCommon(MethodHandles.java:3917) at java.base/java.lang.invoke.MethodHandles$Lookup.getDirectField(MethodHandles.java:3912) at java.base/java.lang.invoke.MethodHandles$Lookup.findGetter(MethodHandles.java:2952) at java.base/java.lang.invoke.ClassSpecializer$Factory.findGetter(ClassSpecializer.java:861) at java.base/java.lang.invoke.ClassSpecializer$Factory.findGetters(ClassSpecializer.java:870) at java.base/java.lang.invoke.ClassSpecializer$Factory.linkSpeciesDataToCode(ClassSpecializer.java:898) at java.base/java.lang.invoke.ClassSpecializer$Factory.loadSpecies(ClassSpecializer.java:505) at java.base/java.lang.invoke.ClassSpecializer.findSpecies(ClassSpecializer.java:212) at java.base/java.lang.invoke.BoundMethodHandle$SpeciesData.extendWith(BoundMethodHandle.java:406) at java.base/java.lang.invoke.LambdaForm.createConstantForm(LambdaForm.java:1642) at java.base/java.lang.invoke.LambdaForm.constantForm(LambdaForm.java:1637) at java.base/java.lang.invoke.GenerateJLIClassesHelper.generateBasicFormsClassBytes(GenerateJLIClassesHelper.java:467) at java.base/java.lang.invoke.GenerateJLIClassesHelper$HolderClassBuilder.build(GenerateJLIClassesHelper.java:262) at java.base/java.lang.invoke.GenerateJLIClassesHelper.generateHolderClasses(GenerateJLIClassesHelper.java:444) at java.base/java.lang.invoke.MethodHandleImpl$1.generateHolderClasses(MethodHandleImpl.java:1560) at java.base/jdk.internal.misc.CDS.generateLambdaFormHolderClasses(CDS.java:238) SoftReferenceKeys are not used: $ java -Xshare:dump -Xlog:class+init | grep ReferenceKe Initializing 'jdk/internal/util/StrongReferenceKey'(no method) (0x000000005304c968) by thread "main" Initializing 'jdk/internal/util/WeakReferenceKey'(no method) (0x000000005304cfb8) by thread "main" ------------- PR Comment: https://git.openjdk.org/jdk/pull/29678#issuecomment-3887839876
On Wed, 11 Feb 2026 20:28:15 GMT, Ioi Lam <iklam@openjdk.org> wrote:
We are seeing intermittent failures (12 times in Jan 2026) where the CDS archives generated in the JDK build is not deterministic.
The symptom is that (perhaps due to the peculiarity of hash codes and memory addressing??) the constant pool entry for `ReferenceKey<?>` in the `WeakReferenceKey` class is not resolved when the failure happens.
The fix is for the JVM to call a method in `WeakReferenceKey` to force this entry to be resolved.
I think a better approach might be in ReferencedKeyMap.prepareForAOTCache, after for (ReferenceKey<K> key : map.keySet()) { Object referent = key.get(); inject a call `key.equals(new StrongReferenceKey(referent))`, so that we hit the instanceof. What do you think? This code should be less reliant to weird translation hacks and requires less VM hacks. ------------- PR Comment: https://git.openjdk.org/jdk/pull/29678#issuecomment-3887902436
On Wed, 11 Feb 2026 20:28:15 GMT, Ioi Lam <iklam@openjdk.org> wrote:
We are seeing intermittent failures (12 times in Jan 2026) where the CDS archives generated in the JDK build is not deterministic.
The symptom is that (perhaps due to the peculiarity of hash codes and memory addressing??) the constant pool entry for `ReferenceKey<?>` in the `WeakReferenceKey` class is not resolved when the failure happens.
The fix is for the JVM to call a method in `WeakReferenceKey` to force this entry to be resolved.
A better approach might be as simple as calling `MethodType.methodType(` to retrieve an existing method type, maybe `MethodType.methodType(void.class)` would be sufficient, in `MethodType::assemblySetup`. ------------- PR Comment: https://git.openjdk.org/jdk/pull/29678#issuecomment-3887917296
On Wed, 11 Feb 2026 20:28:15 GMT, Ioi Lam <iklam@openjdk.org> wrote:
We are seeing intermittent failures (12 times in Jan 2026) where the CDS archives generated in the JDK build is not deterministic.
The symptom is that (perhaps due to the peculiarity of hash codes and memory addressing??) the constant pool entry for `ReferenceKey<?>` in the `WeakReferenceKey` class is not resolved when the failure happens.
The fix is for the JVM to call a method in `WeakReferenceKey` to force this entry to be resolved.
I think a better approach might be in ReferencedKeyMap.prepareForAOTCache, after
``` for (ReferenceKey<K> key : map.keySet()) { Object referent = key.get(); ```
inject a call `key.equals(new StrongReferenceKey(referent))`, so that we hit the instanceof. What do you think? This code should be less reliant to weird translation hacks and requires less VM hacks.
A better approach might be as simple as calling `MethodType.methodType()` to retrieve an existing method type, maybe `MethodType.methodType(void.class)` would be sufficient, in `MethodType::assemblySetup`.
`ReferencedKeyMap.prepareForAOTCache()` is only called only when `AOTClassLinking` is true. However, when creating the default CDS archives, `AOTClassLinking` is false. Also, I am not sure if we should insert unnecessary code to try to achieve some unobvious side effect. ------------- PR Comment: https://git.openjdk.org/jdk/pull/29678#issuecomment-3888436219
On Wed, 11 Feb 2026 20:28:15 GMT, Ioi Lam <iklam@openjdk.org> wrote:
We are seeing intermittent failures (12 times in Jan 2026) where the CDS archives generated in the JDK build is not deterministic.
The symptom is that (perhaps due to the peculiarity of hash codes and memory addressing??) the constant pool entry for `ReferenceKey<?>` in the `WeakReferenceKey` class is not resolved when the failure happens.
The fix is for the JVM to call a method in `WeakReferenceKey` to force this entry to be resolved.
Then I think we should probably patch GenerateJLIClassesHelper to call `MethodType.methodType(void.class)` (or any other method type) twice somewhere. The 2nd call ensures there will be a method type lookup, which covers the missing code path that causes the problem. ------------- PR Comment: https://git.openjdk.org/jdk/pull/29678#issuecomment-3888726353
On Thu, 12 Feb 2026 05:13:03 GMT, Chen Liang <liach@openjdk.org> wrote:
Then I think we should probably patch GenerateJLIClassesHelper to call `MethodType.methodType(void.class)` (or any other method type) twice somewhere. The 2nd call ensures there will be a method type lookup, which covers the missing code path that causes the problem.
The actions in WeakReferenceKey.equals() is many levels away from `MethodType.methodType()`. Doing this is fragile if the implementation changes at any of those levels. It's also "unnecessary code that has an unobvious side effect". That's why I prefer to address the problem directly, as close to its source as possible. ------------- PR Comment: https://git.openjdk.org/jdk/pull/29678#issuecomment-3888819154
We are seeing intermittent failures (12 times in Jan 2026) where the CDS archives generated in the JDK build is not deterministic.
The symptom is that (perhaps due to the peculiarity of hash codes and memory addressing??) the constant pool entry for `ReferenceKey<?>` in the `WeakReferenceKey` class is not resolved when the failure happens.
The fix is for the JVM to call a method in `WeakReferenceKey` to force this entry to be resolved.
Ioi Lam has updated the pull request incrementally with one additional commit since the last revision: @liach comments ------------- Changes: - all: https://git.openjdk.org/jdk/pull/29678/files - new: https://git.openjdk.org/jdk/pull/29678/files/ecf3113f..184d575d Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=29678&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=29678&range=00-01 Stats: 15 lines in 2 files changed: 2 ins; 2 del; 11 mod Patch: https://git.openjdk.org/jdk/pull/29678.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/29678/head:pull/29678 PR: https://git.openjdk.org/jdk/pull/29678
On Sat, 14 Feb 2026 07:23:01 GMT, Ioi Lam <iklam@openjdk.org> wrote:
We are seeing intermittent failures (12 times in Jan 2026) where the CDS archives generated in the JDK build is not deterministic.
The symptom is that (perhaps due to the peculiarity of hash codes and memory addressing??) the constant pool entry for `ReferenceKey<?>` in the `WeakReferenceKey` class is not resolved when the failure happens.
The fix is for the JVM to call a method in `WeakReferenceKey` to force this entry to be resolved.
Ioi Lam has updated the pull request incrementally with one additional commit since the last revision:
@liach comments
Marked as reviewed by liach (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/29678#pullrequestreview-3801736292
On Sat, 14 Feb 2026 07:23:01 GMT, Ioi Lam <iklam@openjdk.org> wrote:
We are seeing intermittent failures (12 times in Jan 2026) where the CDS archives generated in the JDK build is not deterministic.
The symptom is that (perhaps due to the peculiarity of hash codes and memory addressing??) the constant pool entry for `ReferenceKey<?>` in the `WeakReferenceKey` class is not resolved when the failure happens. Probably it's due to a particular layout in the hashtable such that we have no collisions so the `WeakReferenceKey::equals()` method is never called.
The fix is for the JVM to call a method in `WeakReferenceKey` to force this entry to be resolved.
Ioi Lam has updated the pull request incrementally with one additional commit since the last revision:
@liach comments
Good. ------------- Marked as reviewed by kvn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/29678#pullrequestreview-3822602720
On Wed, 11 Feb 2026 20:28:15 GMT, Ioi Lam <iklam@openjdk.org> wrote:
We are seeing intermittent failures (12 times in Jan 2026) where the CDS archives generated in the JDK build is not deterministic.
The symptom is that (perhaps due to the peculiarity of hash codes and memory addressing??) the constant pool entry for `ReferenceKey<?>` in the `WeakReferenceKey` class is not resolved when the failure happens. Probably it's due to a particular layout in the hashtable such that we have no collisions so the `WeakReferenceKey::equals()` method is never called.
The fix is for the JVM to call a method in `WeakReferenceKey` to force this entry to be resolved.
This pull request has now been integrated. Changeset: 33c9f20b Author: Ioi Lam <iklam@openjdk.org> URL: https://git.openjdk.org/jdk/commit/33c9f20bef05239ee016d980dc69a3d583ce8293 Stats: 42 lines in 4 files changed: 39 ins; 1 del; 2 mod 8377712: ConstantPool of WeakReferenceKey is not deterministic in CDS archive Reviewed-by: liach, kvn ------------- PR: https://git.openjdk.org/jdk/pull/29678
participants (3)
-
Chen Liang
-
Ioi Lam
-
Vladimir Kozlov