RFR: 8349554: [UBSAN] os::attempt_reserve_memory_between reported applying non-zero offset to non-null pointer produced null pointer [v2]

[SendaoYan]

On Fri, 7 Feb 2025 16:16:47 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> SendaoYan has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   hi_end should not less or equals to bytes.
>
> The following is not a request to change your PR. It's merely an explanation of what I think could be done to this function to get rid of some of the comparisons of bytes/ranges with pointers:
> 
> 
>   char* lower_limit = MAX2(min, absolute_min);
>   char* upper_limit = MIN2(max, absolute_max);
> 
>   // Precondition check
>   if (lower_limit >= upper_limit) {
>     return nullptr; // no need to go on
>   }
> 
>   // Calculate first attach points
>   assert(alignment_adjusted < std::numeric_limits<uintptr_t>::max() - (uintptr_t)upper_limit, "overflow precondition");
>   char* const lo_att = align_up(lower_limit, alignment_adjusted);
> 
>   if (lo_att >= upper_limit) {
>     // no attachment point within limits
>     return nullptr;
>   }
> 
>   if (bytes < size_t(upper_limit - lo_att)) {
>     // no attachment point that can accommodate the request
>     return nullptr;
>   }
> 
>   // Now we are guaranteed to have an attachment point that can
>   // accommodate the request
>   char* hi_att = align_down(upper_limit - bytes, alignment_adjusted);
>   assert(hi_att >= lo_att, "checked above");
> 
> 
> This is completely untested

Thanks @stefank @tstuefe for the suggestions and reviews.

The additional test has finish and no new failure.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23508#issuecomment-2645535307