RFR: 8375598: VM crashes with "assert((labs(val) & 0xFFFFFFFF00000000) == 0 || dest == (address)-1) failed: must be 32bit offset or -1" when using too high value for NonNMethodCodeHeapSize

[Martin Doerr]

On Tue, 20 Jan 2026 14:50:28 GMT, Guanqiang Han <ghan at openjdk.org> wrote:

> Please review this change. Thanks!
> 
> **Description:**
> 
> On x86/x64, near calls/jumps use 32-bit signed PC-relative displacements. With SegmentedCodeCache enabled, a very large NonNMethodCodeHeapSize can inflate the derived ReservedCodeCacheSize, causing the code cache span to
> exceed the reach of 32-bit relative branches. This may later lead to relocation failures (e.g. "must be 32bit offset") when installing nmethods.
> https://github.com/openjdk/jdk/blob/037040129e82958bd023e0b24d962627e8653710/src/hotspot/cpu/x86/nativeInst_x86.hpp#L433-L440
> 
> **Fix:**
> 
> Add an x86-specific validation in CodeCache::initialize_heaps() after final segment alignment. If the computed code cache size exceeds max_jint bytes, abort VM initialization with a clear error message that includes the segment sizes, instead of failing later during compilation/relocation.
> 
> **Test:**
> 
> GHA

Please note that other problems related to modified `ReservedCodeCacheSize` have been reported, too: https://github.com/openjdk/jdk/pull/28658#issuecomment-3782072965
I think we should find a solution which solves all the problems. We should be able to avoid enlarging `ReservedCodeCacheSize`.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/29324#issuecomment-3810739340