RFR: 8342283: CDS cannot handle a large number of classes

[Aleksey Shipilev]

On Thu, 31 Oct 2024 09:16:00 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> When lots of classes are loaded during `java -Xshare:dump`, the internal arrays used by some of the HashMaps and ArrayLists become too large to be archived by CDS (> 256KB).
>> 
>> At the very end of Java bytecode execution during `java -Xshare:dump`, we used to call `clear()` on these tables to free their elements (*) -- these tables are repopulated at run time when classes are loaded incrementally. However, the `clear()` call doesn't resize the internal arrays.
>> 
>> The fix is to re-ininitialize these tables to new, empty tables that have small internal arrays.
>> 
>> ===
>> (*) the call to `resetArchivedStates()` is made from [HeapShared::reset_archived_object_states()](https://github.com/openjdk/jdk/blob/688e92e7f5febddd2935cb7f500dd3f10fbd9401/src/hotspot/share/cds/metaspaceShared.cpp#L799)
>
> src/java.base/share/classes/java/lang/ClassLoader.java line 2737:
> 
>> 2735:      * @implNote This is done while the JVM is running in single-threaded mode,
>> 2736:      * and at the very end of Java bytecode execution. We know that no more classes
>> 2737:      * will be loaded and none of the fields modified by this method will be used again.
> 
> I wonder if moving all this to VM side, which is not bound to Java language rules, would be conceptually cleaner. It would be more work, but I think there any only three classes that implement `resetArchivedStates`, which limits the scope of the change. I am prototyping this locally...

Something like this: https://github.com/openjdk/jdk/compare/master...shipilev:jdk:JDK-8342283-cds-reset-archived-states. This passes `runtime/cds` at least.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21797#discussion_r1824260227