RFR: 8351334: [ubsan] memoryReserver.cpp:552:60: runtime error: applying non-zero offset 1073741824 to null pointer

[Johan Sjölen]

On Wed, 27 Aug 2025 11:24:07 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

> The minimum acceptable value was 0 where using it as address was problematic according to UBSAN.
> The acceptable value is changed to 64K.
> 
> Tests:
> linux-x64 tier1

It seems to me like we can fix the ubsan issue by avoiding the cast into pointers and computing through `size_t` as far as possible, and converting into pointer when necessary.

For example

```c++
size_t aligned_heap_base_min_address = align_up(HeapBaseMinAddress, alignment);
size_t noaccess_prefix = ((aligned_heap_base_min_address + size) > OopEncodingHeapMax) ? noaccess_prefix_size : 0;

  if (!FLAG_IS_DEFAULT(HeapBaseMinAddress)) {
    reserved = try_reserve_memory(size + noaccess_prefix, alignment, page_size, static_cast<char*>(aligned_heap_base_min_address));
    if (reserved.base() != static_cast<char*>(aligned_heap_base_min_address)) { // Enforce this exact address.
      release(reserved);
      reserved = {};
    }
  }


The semantics of reserving a `null` base address is still preserved as meaning "don't care where you put it" then. Someone from GC needs to look at this and see whether or not an undefined (implicitly 0?) HeapBaseMinAddress is meant to have that type of meaning.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26955#issuecomment-3228406340