RFR: 8332865: ubsan: os::attempt_reserve_memory_between reports overflow

[Matthias Baesken]

When running by ubsan-enabled binaries on Linux x86_64, os::attempt_reserve_memory_between reports overflows.

"runtime error: pointer index expression with base 0x000000001000 overflowed to 0xfffffffffffff000"

This coding triggers the ubsan issue


  char* const hi_att = align_down(MIN2(max, absolute_max) - bytes, alignment_adjusted);
  if (hi_att > max) {
    return nullptr; // overflow
  }


However the function already contains overflow handling, so probably it is sufficient to add an attribute to the function os::attempt_reserve_memory_between to disable ubsan checks for this function.

-------------

Commit messages:
 - JDK-8332865

Changes: https://git.openjdk.org/jdk/pull/19543/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19543&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8332865
  Stats: 3 lines in 1 file changed: 3 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/19543.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/19543/head:pull/19543

PR: https://git.openjdk.org/jdk/pull/19543