RFR: 8349554: [UBSAN] os::attempt_reserve_memory_between reported applying non-zero offset to non-null pointer produced null pointer

[SendaoYan]

On Fri, 7 Feb 2025 08:50:22 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

> Would silence the compiler

- if ((uintptr_t)hi_end < bytes) {
+ if ((uintptr_t)hi_end <= bytes) {


Yes.



> Or maybe even use the lowest attach point instead of nullptr:

uintptr_t max_range = hi_end - lo_att;
if (max_range < bytes) {


`hi_end` less than `lo_att` in some cases, `hi_end - lo_att` subtraction will overflow, and save a bigger value to `max_range`, so `if (max_range < bytes)` return false.

Should we change like below:


-  if ((uintptr_t)hi_end < bytes) {
+  uintptr_t max_range = hi_end - lo_att;
+  if (max_range < bytes || hi_end < lo_att) {

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23508#issuecomment-2642598127