RFR: 8357525: Default CDS archive becomes non-deterministic after JDK-8305895

[Aleksey Shipilev]

On Wed, 21 May 2025 22:01:40 GMT, Ioi Lam <iklam at openjdk.org> wrote:

> Since [JDK-8305895](https://bugs.openjdk.org/browse/JDK-8305895), interfaces and abstract classes are allocated outside of the class space. See [here for the triggering condition](https://github.com/openjdk/jdk/blame/4d7068923cd87fbfc2edee25406521b11580d153/src/hotspot/share/classfile/classFileParser.cpp#L5835-L5842)
> 
> Theoretically such InstanceKlasses can have arbitrary addresses, so the test of `(uintptr_t(existing) < uintptr_t(klass))` in [Klass::hash_insert()](https://github.com/openjdk/jdk/blob/4d7068923cd87fbfc2edee25406521b11580d153/src/hotspot/share/oops/klass.cpp#L458) no longer produces a deterministic result across two JVM runs. As a result, we have seen several occurrence of non-deterministic CDS archives in our test pipeline for a short period after  [JDK-8305895](https://bugs.openjdk.org/browse/JDK-8305895) for pushed.
> 
> Thereafter, for some unknown reason, the problem disappears. However, we could just be lucky due to allocation policy changes in metaspace. The underlying cause still exists.
> 
> I wrote a proof of concept that shows the problem. See https://github.com/openjdk/jdk/commit/33185705f85986e1ee1e529005898e834cc8a88f
> 
> 
> (Without fix)
> 
> $ java -Xshare:dump -XX:+UseNewCode -XX:SharedArchiveFile=f1.jsa
> $ java -Xshare:dump -XX:+UseNewCode2 -XX:SharedArchiveFile=f2.jsa
> $ cksum f1.jsa f2.jsa
>   2834014305 16105472 f1.jsa
>   4126337207 16105472 f2.jsa
> 
> (With the fix in this PR)
> 
> $ java -Xshare:dump -XX:+UseNewCode -XX:+UseNewCode3 -XX:SharedArchiveFile=f1.jsa
> $ java -Xshare:dump -XX:+UseNewCode2 -XX:+UseNewCode3 -XX:SharedArchiveFile=f2.jsa
> $ cksum f1.jsa f2.jsa
>   3637571299 16105472 f1.jsa
>   3637571299 16105472 f2.jsa
> 
> 
> With the fix, we re-hash the copy of the `secondary_supers` in the CDS archive. When InstanceKlasses are copied into the CDS archive, they are sorted by their names. This makes the `(uintptr_t(existing) < uintptr_t(klass))`  comparison produce deterministic results.
> 
> Unfortunately it's not possible to write a jtreg test for this problem.

This looks fine. 

Looks to me we mis-attributing the root cause a bit: it is not just about [JDK-8305895](https://bugs.openjdk.org/browse/JDK-8305895) that made some `IK`-s allocated at more arbitrary locations outside of Class Space. Even in Class Space there is a positional randomness, right? 

So I think the actual root cause is the the address-based tie-breaker introduced in [JDK-8180450](https://bugs.openjdk.org/browse/JDK-8180450). Let's link those issues up and change the synopsis accordingly?

src/hotspot/share/oops/klass.cpp line 819:

> 817:     // _secondary_supers, its elements will appear in a deterministic order.
> 818:     //
> 819:     // Note that the bitmap is guatanteed to be deterministic, regardless of the

Suggestion:

    // Note that the bitmap is guaranteed to be deterministic, regardless of the

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/25373#pullrequestreview-2860696682
PR Review Comment: https://git.openjdk.org/jdk/pull/25373#discussion_r2102220137