RFR: 8351334: [ubsan] memoryReserver.cpp:552:60: runtime error: applying non-zero offset 1073741824 to null pointer

[Afshin Zafari]

On Wed, 27 Aug 2025 14:19:23 GMT, Johan Sjölen <jsjolen at openjdk.org> wrote:

>> The minimum acceptable value was 0 where using it as address was problematic according to UBSAN.
>> The acceptable value is changed to 64K.
>> 
>> Tests:
>> linux-x64 tier1
>
> It seems to me like we can fix the ubsan issue by avoiding the cast into pointers and computing through `size_t` as far as possible, and converting into pointer when necessary.
> 
> For example
> 
> ```c++
> size_t aligned_heap_base_min_address = align_up(HeapBaseMinAddress, alignment);
> size_t noaccess_prefix = ((aligned_heap_base_min_address + size) > OopEncodingHeapMax) ? noaccess_prefix_size : 0;
> 
>   if (!FLAG_IS_DEFAULT(HeapBaseMinAddress)) {
>     reserved = try_reserve_memory(size + noaccess_prefix, alignment, page_size, static_cast<char*>(aligned_heap_base_min_address));
>     if (reserved.base() != static_cast<char*>(aligned_heap_base_min_address)) { // Enforce this exact address.
>       release(reserved);
>       reserved = {};
>     }
>   }
> 
> 
> The semantics of reserving a `null` base address is still preserved as meaning "don't care where you put it" then. Someone from GC needs to look at this and see whether or not an undefined (implicitly 0?) HeapBaseMinAddress is meant to have that type of meaning.

Thanks for your comments @jdksjolen and @xmas92. The discussion here is exactly what I meant by running UBSAN checks. It seems that more bugs get discovered.

To answer 'when/how does this happen?', this issue can be reproduced  by running `runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java`, in which invalid values of options will be tested, particularly here `0` for `HeapBaseMinAddress`.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26955#issuecomment-3236511021