RFR: 8376125: Out of memory in the CDS archive error with lot of classes [v7]

Fri Feb 6 06:34:49 UTC 2026

On Wed, 4 Feb 2026 20:35:17 GMT, Xue-Lei Andrew Fan <xuelei at openjdk.org> wrote:

>> **Summary**
>> This change extends the CDS/AOT archive size limit from 2GB to 32GB by using scaled offset encoding.
>> 
>> **Problem**
>> Applications with a large number of classes (e.g., 300,000+) can exceed the current 2GB archive size limit, causing archive creation to fail with:
>> 
>> [error][aot] Out of memory in the CDS archive: Please reduce the number of shared classes.
>> 
>> 
>> **Solution**
>> Instead of storing raw byte offsets in u4 fields (limited to ~2GB), we now store scaled offset units where each unit represents 8 bytes (OFFSET_SHIFT = 3). This allows addressing up to 32GB (2^32 × 8 bytes) while maintaining backward compatibility with the existing u4 offset fields.
>> 
>> Current:   address = base + offset_bytes           (max ~2GB)
>> Proposed:  address = base + (offset_units << 3)    (max 32GB)
>> 
>> All archived objects are guaranteed to be 8-byte aligned. This means the lower 3 bits of any valid byte offset are always zero – we're wasting them!
>> 
>> Current byte offset (aligned to 8 bytes):
>>   0x00001000  =  0000 0000 0000 0000 0001 0000 0000 0|000
>>                                                       └── Always 000!
>> 
>> Scaled offset (shift=3):
>>   0x00000200  =  Same address, but stored in 29 bits instead of 32
>>                  Frees up 3 bits → 8x larger range!
>> Current byte offset (aligned to 8 bytes):  0x00001000  =  0000 0000 0000 0000 0001 0000 0000 0|000                                                      └── Always 000!Scaled offset (shift=3):  0x00000200  =  Same address, but stored in 29 bits instead of 32                 Frees up 3 bits → 8x larger range!
>> 
>> By storing `offset_bytes >> 3` instead of `offset_bytes`, we use all 32 bits of the u4 field to represent meaningful data, extending the addressable range from 2GB to 32GB.
>> 
>> **Test**
>> All tier1 and tier2 tests passed.  No visible performance impact.  Local benchmark shows significant performance improvement for CDS, Dynamic CDS and AOT Cache archive loading, with huge archive size (>2GB).
>> 
>> Archive:
>>   - 300000 simple classes
>>   - 2000 mega-classes
>>   - 5000 FieldObject classes
>>   - Total: 307000 classes
>> 
>> AOT Cache:
>>   Times (wall):      create=250020ms verify=2771ms baseline=15470ms perf_with_aot=2388ms
>>   Times (classload): verify=965ms baseline=14771ms perf_with_aot=969ms
>>   
>> Static CDS:
>>   Times (wall):      create=161859ms verify=2055ms baseline=15592ms perf_with_cds=1996ms
>>   Times (classload): verify=1027ms baseline=14852ms perf_with_cds=1...
>
> Xue-Lei Andrew Fan has updated the pull request incrementally with one additional commit since the last revision:
> 
>   remove LargeArchive test

> > > I think we should keep this PR simple -- only introduce the shift encoding and limit max archive size to 3.5GB.
> > > I was wondering if it is OK to support 4GB+ archive when UseCompactObjectHeaders is false.
> > 
> > 
> > I have strong reservations about that, at least until we have fixed CDS:
> 
> @tstuefe I totally agree with that. We prefer to use compact object headers as we see huge benefits in practice. The prototype was just to show that we are able to support big CDS even UseCompactObjectHeaders have to be false. But I don't think we need it as UseCompactObjectHeaders will be default.
> 
> > The problem is that CDS loads _all_ metadata into the Klass encoding range. That includes non-Klass metadata. That wastes a _huge_ amount of space.
> > The ratio of non-Klass data to Klass data is usually extreme - e.g. in Spring petclinic, it's 7:1. So we waste a large amount of encoding space for things that don't need it. Likely 70% .. > 80%.
> > That is why you can only load 300000 classes when the nKlass encoding range should give you space for 4 .. 6 million classes, depending on Lilliput and on the median object granularity.
> > I opened https://bugs.openjdk.org/browse/JDK-8377222 to track that problem.
> > We could fit far more Klass structures into class space. And that would also have another advantage: it might let you store huge CDS archives and still use compact object headers. Even a future proposed Lilliput2 with a 32-bit object header (reduces number of loadable classes to 512000) may still be in reach for huge CDS archives.
> > If CDS were to load only Klass structures into the encoding space, this would also open the door to certain performance enhancements I have in mind, since Klass structures are stored more densely and we wouldn't waste as many nKlass ID slots on non-Klass metadata.
> > So I think that would be the right way to go forward. If we have implemented that, and you still run out of Klass encoding range, we really should take a close look at what application needs more than 4..6 million classes.
> 
> I agreed. I tried to prototype the idea to support large CDS archives with UseCompactObjectHeaders: #29556. And it really works! There is no problem to get 10GB CDS archives (I did not test bigger archives yet, 7,500 mega classes + 1,000 small classes).
> 
> The prototype is based on this pull request to check out large archive, and you may look at [this commit](https://github.com/openjdk/jdk/pull/29556/changes/e7a12c372480f405d2a08a75bdabac91c7328346) only, for the idea to separate Klass objects from other metadata in the archive.
> 
> Again, it is just a prototype to show the idea. I will close JDK-8377137 as a duplicate of JDK-8377222.

Very nice. The re-calculating of the nKlass IDs is a beauty spot, though. Would be nice if we could avoid that.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/29494#issuecomment-3858292782