[jdk21u-dev] RFR: 8354922: ZGC: Use MAP_FIXED_NOREPLACE when reserving memory

[Sergey Chernyshev]

On Fri, 14 Nov 2025 18:31:20 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Hi all,
>> 
>> Please review the backport of JDK-8354922 to JDK 21.
>> 
>> There are versions of the Linux kernel that do not honor the address hint when mmapping memory without MAP_FIXED, that clobbers the older mappings overlapped with the requested range. A safer MAP_FIXED_NOREPLACE flag is used with ZGC since Java 25.
>> 
>> JDK 21 crashes with -XX:+UseZGC with RANDMMAP [1] kernel patch enabled, while JDK 25 works as expected.
>> 
>> 
>> $ ./25.0.1/bin/java -XX:+UseZGC -version
>> openjdk version "25.0.1" 2025-10-21 LTS
>> OpenJDK Runtime Environment (build 25.0.1+11-LTS)
>> OpenJDK 64-Bit Server VM (build 25.0.1+11-LTS, mixed mode, sharing)
>> 
>> $ ./21.0.9/bin/java -XX:+UseZGC -version
>> [0.070s][error][gc] Failed to reserve enough address space for Java heap
>> Error: Could not create the Java Virtual Machine.
>> Error: A fatal exception has occurred. Program will exit.
>> 
>> 
>> ZGC relies on kernel that respects address hints in mmap system call, which is not the case with RANDMMAP enabled kernels. ZVirtualMemoryManager::pd_reserve() always returns false at line 50 because the mmap returned value never matches the requested address.
>> 
>> https://github.com/openjdk/jdk21u-dev/blob/634d3fa7f147fc6fb1a7765755a3feed82ffaf50/src/hotspot/os/posix/gc/z/zVirtualMemory_posix.cpp#L40-L51
>> 
>> Since Linux 4.17 there's MAP_FIXED_NOREPLACE flag in mmap, that satisfies the `addr` requests and reports failure when the requested range overlaps a pre-existing mapping.
>> 
>> The backport isn't clean. JDK 21 doesn't have JDK-8350441 that renamed zVirtualMemory_posix.cpp to zVirtualMemoryManager_posix.cpp, and JDK-8341692 that removed non-generational mode in ZGC. In absense of JDK-8341692 the same approach was taken on gc/x/ version of pd_reserve().
>> 
>> It is also proposed to backport JDK-8313319 that prevents unnecessary mmap-munmap cycle, that will follow in a separate PR (#2476 ).
>> 
>> [1] https://pax.grsecurity.net/docs/randmmap.txt
>
> This looks good to me.

Thanks @stefank

-------------

PR Comment: https://git.openjdk.org/jdk21u-dev/pull/2475#issuecomment-3534064041