RFR: 8253495: CDS generates non-deterministic output [v2]

[David Holmes]

On Wed, 9 Mar 2022 05:10:44 GMT, Ioi Lam <iklam at openjdk.org> wrote:

>> This patch makes the result of "java -Xshare:dump" deterministic:
>> - Disabled new Java threads from launching. This is harmless. See comments in jvm.cpp
>> - Fixed a problem in hashtable ordering in heapShared.cpp
>> - BasicHashtableEntry has a gap on 64-bit platforms that may contain random bits. Added code to zero it.
>> - Enabled checking of $JAVA_HOME/lib/server/classes.jsa in make/scripts/compare.sh
>> 
>> Note:  $JAVA_HOME/lib/server/classes_ncoops.jsa is still non-deterministic. This will be fixed in [JDK-8282828](https://bugs.openjdk.java.net/browse/JDK-8282828).
>> 
>> Testing under way:
>> - tier1~tier5
>> - Run all *-cmp-baseline jobs 20 times each (linux-aarch64-cmp-baseline, windows-x86-cmp-baseline, .... etc).
>
> Ioi Lam has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fixed zero build

I have reservations about contorting things this way just to get "deterministic output".

The VM needs to fully initialize and then become quiescent before the dump occurs, and as I say below if you don't start other threads then you potentially remove part of the archive because classes won't be loaded by those threads.

I think if you care about the order of dumping classes then you should control that order, you don't try to force the order of loading. Can't you sort things before dumping? ie rehash/rebuild the hashtables etc so it has a canonical ordering? I see this was mentioned in the bug report and is considered a largish/complex fix, but it would be the proper fix IMO.

Thanks,
David

src/hotspot/share/prims/jvm.cpp line 2873:

> 2871:     // execute in parallel, symbols and classes may be loaded in
> 2872:     // random orders which will make the resulting CDS archive
> 2873:     // non-deterministic.

Yes but by not starting these threads you are potentially excluding a range of classes from the shared archive!

-------------

PR: https://git.openjdk.java.net/jdk/pull/7748