RFR: 8363986: Heap region in CDS archive is not at deterministic address [v2]

[Vladimir Kozlov]

On Fri, 14 Nov 2025 07:04:09 GMT, Ioi Lam <iklam at openjdk.org> wrote:

>> **Overview**
>> 
>> This PR fixes the problem where the JDK build is not reproducible because the `lib/server/classes*.jsa` files do not always put the heap objects at the same addresses. This bug affects only `+UseCompressedOops`.
>> 
>> This bug is in generic code and is not specific to any platform. We hadn't hit this bug because our build platforms always allocated the heap at the same location. However, this is no longer true with macOS 26, which puts the heap at random locations.
>> 
>> **The fix**
>> 
>> (1) In `ArchiveHeapWriter::init()`, we check if we need deterministic heap contents.
>> 
>> (2) In `ArchiveHeapWriter::set_requested_address_range()`, if deterministic heap contents are needed, we always put the archived heap objects to just below `0x100000000`, so that we always write the archived oops into the CDS archive with  zero-based, zero-shift encoding.
>> 
>> Please see comments in the above two functions for more details.
>
> Ioi Lam has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits:
> 
>  - clean up of old C++ class name ArchiveHeapWriter
>  - @ashu-mehra comment -- simplified the criteria for _is_writing_deterministic_heap
>  - Merge branch 'master' into 8363986-heap-region-in-aot-cache-not-determinitic-address
>  - More clean up
>  - fixed aot map
>  - clean up
>  - 8363986: Heap region in CDS archive is not at deterministic address

src/hotspot/share/cds/aotMappedHeapWriter.cpp line 109:

> 107:       // optimizations such as AOT compiled code require the same compressed oop
> 108:       // encoding to be used in the assembly phase and production run, so we cannot
> 109:       // force (zero-based, no shift) encoding.

In Leyden we do force encoding `(base, shift)` with `-XX:+UseCompatibleCompressedOops` flag in Leyden when AOT code is generated and used:
https://github.com/openjdk/leyden/blob/premain/src/hotspot/share/oops/compressedOops.cpp#L51

The flag is enabled by default when AOT code is used:
https://github.com/openjdk/leyden/blob/premain/src/hotspot/share/cds/cdsConfig.cpp#L121

It allows to use different heap size in production runs.

src/hotspot/share/cds/aotMappedHeapWriter.cpp line 588:

> 586: //     In the production run, if different COOPS encodings are used:
> 587: //         - The heap contents needs to be relocated.
> 588: //         - AOTCodeCache will be disabled.

Not true by default. See my first comment.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/28052#discussion_r2539402767
PR Review Comment: https://git.openjdk.org/jdk/pull/28052#discussion_r2539412737