RFR: 8376125: Out of memory in the CDS archive error with lot of classes
Xue-Lei Andrew Fan
xuelei at openjdk.org
Tue Feb 3 15:49:54 UTC 2026
On Tue, 3 Feb 2026 00:38:31 GMT, Alexey Bakhtin <abakhtin at openjdk.org> wrote:
>>> There are two more apis that return "unchecked" offset: `ArchiveBuilder::buffer_to_offset()` and `ArchiveBuilder::any_to_offset()`. These apis are not returning the scaled offset. I think it is better to get rid of these apis and replace their usage with `_u4` version which has the offset range check. I noticed there are only 1-2 instances that use these "unchecked" apis.
>>
>> Thanks for the suggestion. I looked into this and found that buffer_to_offset() and any_to_offset() serve a different purpose than the _u4 versions. The _u4 versions use scaled encoding (with MetadataOffsetShift) and return a compact u4 for metadata pointer storage. The raw versions return unscaled byte offsets stored in larger types. These usages cannot switch to _u4 versions because they need raw byte offsets (not scaled) and store them in 64-bit types.
>>
>> However, the comments for the methods may be misleading after introducing the _u4 methods. What do you think to revise the comment as:
>>
>> // The address p points to an object inside the output buffer. When the archive is mapped
>> // at the requested address, what's the byte offset of this object from _requested_static_archive_bottom?
>> uintx buffer_to_offset(address p) const;
>>
>> // Same as buffer_to_offset, except that the address p points to either (a) an object
>> // inside the output buffer, or (b), an object in the currently mapped static archive.
>> uintx any_to_offset(address p) const;
>>
>> // The reverse of buffer_to_offset_u4() - converts scaled offset units back to buffered address.
>> address offset_to_buffered_address(u4 offset_units) const;
>>
>>
>> I am also OK to rename the method names to: `buffer_to_offset_bytes()` and `any_to_offset_bytes()`, if the new names are clearer.
>>
>> @ashu-mehra What do you think?
>
> Hi @XueleiFan,
>
> I've tried the suggested code with an archive size more than 4Gb, but it fails with an assertion:
>
> # Internal Error (aotMetaspace.cpp:1955), pid=96332, tid=4099
> # guarantee(archive_space_size < max_encoding_range_size - class_space_alignment) failed: Archive too large
>
> CDC archive was created successfully:
>
> [187.068s][info ][cds ] Shared file region (rw) 0: 822453584 bytes, addr 0x0000000800004000 file offset 0x00004000 crc 0x132b652e
> [189.176s][info ][cds ] Shared file region (ro) 1: 3576115584 bytes, addr 0x0000000831060000 file offset 0x31060000 crc 0x71b020a2
> [197.653s][info ][cds ] Shared file region (ac) 4: 0 bytes
> [198.870s][info ][cds ] Shared file region (bm) 2: 56555664 bytes, addr 0x0000000000000000 file offset 0x1062d4000 crc 0xbd87f804
> [199.504s][info ][cds ] Shared file region (hp) 3: 16091256 bytes, addr 0x00000000ff000000 file offset 0x1098c4000 crc 0x7834b7c3
> [199.684s][debug ][cds ] bm space: 56555664 [ 1.3% of total] out of 56555664 bytes [100.0% used]
> [199.684s][debug ][cds ] hp space: 16091256 [ 0.4% of total] out of 16091256 bytes [100.0% used] at 0x0000000c6d000000
> [199.684s][debug ][cds ] total : 4471216088 [100.0% of total] out of 4471228536 bytes [100.0% used]
@alexeybakhtin Thank you for testing of bigger archives (>4GB).
I was wondering if it is OK to support 4GB+ archive when UseCompactObjectHeaders is false. The following prototype works. However, we prefer UseCompactObjectHeaders in practice, and the biggest archive size (5.6M objects) is about 2.1G at this moment. Could we have 4GB archive size limit as a open issue, and address it separately?
diff --git a/src/hotspot/share/cds/aotMetaspace.cpp b/src/hotspot/share/cds/aotMetaspace.cpp
index 62d76957c0a..676d54cba33 100644
--- a/src/hotspot/share/cds/aotMetaspace.cpp
+++ b/src/hotspot/share/cds/aotMetaspace.cpp
@@ -1950,8 +1950,12 @@ char* AOTMetaspace::reserve_address_space_for_archives(FileMapInfo* static_mapin
const size_t ccs_begin_offset = align_up(archive_space_size, class_space_alignment);
const size_t gap_size = ccs_begin_offset - archive_space_size;
- // Reduce class space size if it would not fit into the Klass encoding range
- constexpr size_t max_encoding_range_size = 4 * G;
+ // Reduce class space size if it would not fit into the Klass encoding range.
+ // The max encoding range depends on narrow Klass pointer bits and max shift:
+ // - With UseCompactObjectHeaders: 22-bit + shift 10 = 4GB
+ // - Without UseCompactObjectHeaders (legacy): 32-bit + shift 3 = 32GB
+ const size_t max_encoding_range_size =
+ nth_bit(CompressedKlassPointers::narrow_klass_pointer_bits() + CompressedKlassPointers::max_shift());
guarantee(archive_space_size < max_encoding_range_size - class_space_alignment, "Archive too large");
if ((archive_space_size + gap_size + class_space_size) > max_encoding_range_size) {
class_space_size = align_down(max_encoding_range_size - archive_space_size - gap_size, class_space_alignment);
diff --git a/src/hotspot/share/cds/archiveBuilder.cpp b/src/hotspot/share/cds/archiveBuilder.cpp
index e1130e7befc..85f6922dd50 100644
--- a/src/hotspot/share/cds/archiveBuilder.cpp
+++ b/src/hotspot/share/cds/archiveBuilder.cpp
@@ -1122,16 +1122,16 @@ class RelocateBufferToRequested : public BitMapClosure {
#ifdef _LP64
int ArchiveBuilder::precomputed_narrow_klass_shift() {
- // Legacy Mode:
- // We use 32 bits for narrowKlass, which should cover the full 4G Klass range. Shift can be 0.
- // CompactObjectHeader Mode:
- // narrowKlass is much smaller, and we use the highest possible shift value to later get the maximum
- // Klass encoding range.
+ // We use the highest possible shift value to get the maximum Klass encoding range:
+ // - Legacy Mode (UseCompactObjectHeaders=false):
+ // 32-bit narrowKlass + shift 3 = 32GB encoding range
+ // - CompactObjectHeader Mode (UseCompactObjectHeaders=true):
+ // 22-bit narrowKlass + shift 10 = 4GB encoding range
//
// Note that all of this may change in the future, if we decide to correct the pre-calculated
// narrow Klass IDs at archive load time.
assert(UseCompressedClassPointers, "Only needed for compressed class pointers");
- return UseCompactObjectHeaders ? CompressedKlassPointers::max_shift() : 0;
+ return CompressedKlassPointers::max_shift();
}
#endif // _LP64
diff --git a/src/hotspot/share/oops/compressedKlass.cpp b/src/hotspot/share/oops/compressedKlass.cpp
index b32d10c74d2..f642a162adb 100644
--- a/src/hotspot/share/oops/compressedKlass.cpp
+++ b/src/hotspot/share/oops/compressedKlass.cpp
@@ -46,9 +46,10 @@ size_t CompressedKlassPointers::_protection_zone_size = 0;
size_t CompressedKlassPointers::max_klass_range_size() {
#ifdef _LP64
- const size_t encoding_allows = nth_bit(narrow_klass_pointer_bits() + max_shift());
- constexpr size_t cap = 4 * G;
- return MIN2(encoding_allows, cap);
+ // The max klass range is determined by narrow Klass pointer bits and max shift:
+ // - With UseCompactObjectHeaders: 22-bit + shift 10 = 4GB
+ // - Without UseCompactObjectHeaders (legacy): 32-bit + shift 3 = 32GB
+ return nth_bit(narrow_klass_pointer_bits() + max_shift());
#else
// 32-bit: only 32-bit "narrow" Klass pointers allowed. If we ever support smaller narrow
// Klass pointers here, coding needs to be revised.
-------------
PR Comment: https://git.openjdk.org/jdk/pull/29494#issuecomment-3842105081
More information about the hotspot-dev
mailing list