RFR: 8351334: [ubsan] memoryReserver.cpp:552:60: runtime error: applying non-zero offset 1073741824 to null pointer [v3]

[Afshin Zafari]

On Mon, 15 Sep 2025 08:42:36 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> src/hotspot/share/gc/shared/jvmFlagConstraintsGC.cpp line 288:
>> 
>>> 286:   // If an overflow happened in Arguments::set_heap_size(), MaxHeapSize will have too large a value.
>>> 287:   // Check for this by ensuring that MaxHeapSize plus the requested min base address still fit within max_uintx.
>>> 288:   if (value + MaxHeapSize < MaxHeapSize) {// overflow
>> 
>> Can we perform the overflow check via subtraction from the max value instead? I know that unsigned types do wrap around, but at a first glance this will look wrong and make the reader stop and think.
>> 
>> Style: Space between { and //
>> 
>> 
>> if (std::numeric_limits<size_t>::max() - value < MaxHeapSize) { // overflow
>
> Using ` a < a - b` for checking overflow, does not work when `a - b` is negative and it will become a very large number when cast to unsigned.
> 
> Space added.

misunderstood. Done.

>> src/hotspot/share/memory/memoryReserver.cpp line 560:
>> 
>>> 558:   if (!FLAG_IS_DEFAULT(HeapBaseMinAddress)) {
>>> 559:     reserved = try_reserve_memory(size + noaccess_prefix, alignment, page_size, (char *)aligned_heap_base_min_address);
>>> 560:     if (reserved.base() != (char *)aligned_heap_base_min_address) { // Enforce this exact address.
>> 
>> Style: Please hug the * to the char for the casts
>
> Done.
> There are some preexisting cases like this. Should I change them too?

Done

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26955#discussion_r2477746409
PR Review Comment: https://git.openjdk.org/jdk/pull/26955#discussion_r2477747887