RFR: 8376125: Out of memory in the CDS archive error with lot of classes [v3]

[Ioi Lam]

On Mon, 2 Feb 2026 23:35:13 GMT, Xue-Lei Andrew Fan <xuelei at openjdk.org> wrote:

>> umm, I removed this change to `os.cpp` and ran `LargeArchive.java` test on x86-64 system and it passed. On which platform/OS did you see the failure?
>
> I was on MacOS.  Here is the failure without the os.cpp update:
> 
> % make test TEST="test/hotspot/jtreg/resourcehogs/runtime/aot/LargeArchive.java" JTREG="JAVA_OPTIONS=-Dtest.archive.large.all.workflows=true"
> ...
> [0.018s][info][aot] JVM_StartThread() ignored: java.lang.ref.Reference$ReferenceHandler
> [0.018s][info][aot] JVM_StartThread() ignored: java.lang.ref.Finalizer$FinalizerThread
> [0.039s][info][cds] Loading classes to share ...
> [0.039s][info][cds] Parsing LargeArchive.classlist
> [0.047s][info][aot] JVM_StartThread() ignored: jdk.internal.misc.InnocuousThread
> [136.124s][info][cds] Parsing /Users/xuelei.fan/workspace/openjdk/jdk-xf.git/build/macosx-aarch64-server-release/images/jdk/lib/classlist (lambda form invokers only)
> [136.126s][info][cds] Loading classes to share: done.
> [136.127s][info][aot] Rewriting and linking classes ...
> [136.245s][info][aot] Rewriting and linking classes: done
> [136.245s][info][aot] Regenerate MethodHandle Holder classes...
> [136.344s][info][aot] Regenerate MethodHandle Holder classes...done
> [136.351s][info][cds] Dumping shared data to file: LargeArchive.static.jsa
> [136.351s][info][cds] Gathering all archivable objects ... 
> [136.371s][info][cds] Skipping java/lang/invoke/BoundMethodHandle$Species_F: dynamically generated
> [136.371s][info][cds] Skipping java/lang/invoke/BoundMethodHandle$Species_J: dynamically generated
> [136.413s][info][cds] Skipping jdk/internal/misc/CDS$UnregisteredClassLoader$Source: used only when dumping CDS archive
> [136.413s][info][cds] Skipping jdk/internal/misc/CDS$UnregisteredClassLoader: used only when dumping CDS archive
> [136.417s][info][cds] Heap range = [0x00000003c0000000 - 0x00000004c0000000]
> [136.430s][info][aot] Archived 7975 interned strings
> [136.431s][info][cds] Gathering classes and symbols ... 
> [143.110s][info][cds] Sorting symbols ... 
> [143.113s][info][cds] Sorting classes ... 
> [149.873s][info][cds] Reserved output buffer space at 0x0000007000000000 [34359738368 bytes]
> [149.900s][info][cds] Allocating RW objects ... 
> [149.954s][info][cds] done (46510 objects)
> [149.954s][info][cds] Allocating RO objects ... 
> [151.033s][info][cds] done (142334 objects)
> [151.033s][info][cds] Relocating embedded pointers in core regions ... 
> [152.400s][info][cds] Relocating 150345176 pointers, 0 tagged, 17461 nulled
> [152.400s][info][aot] Make classes shareable
> [152.558s][info][cds] Number of classes 4485
> [152.558s][info][cds]     instance classes   =  4340, aot-linked =     0, inited =     0
> [152.558s][info][cds]  ...

According to https://gitlab.haskell.org/ghc/ghc/-/issues/17414 

> File reads/writes bigger than 2GB result in an "Invalid argument" exception on macOS. Files bigger than 2GB still work, but individual read/write operations bigger than 2GB fail.

I think it's better to move this fix into `os::pd_write()` (within `#ifdef __APPLE__`) to limit the writes to less than 2GB.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/29494#discussion_r2757096105