RFR: 8332865: ubsan: os::attempt_reserve_memory_between reports overflow

[Matthias Baesken]

On Tue, 4 Jun 2024 15:19:11 GMT, Matthias Baesken <mbaesken at openjdk.org> wrote:

> When running by ubsan-enabled binaries on Linux x86_64, os::attempt_reserve_memory_between reports overflows.
> This happens in the  :tier1 tests ( gtest/LargePageGtests_use-large-pages.jtr )
> 
> 
> "runtime error: pointer index expression with base 0x000000001000 overflowed to 0xfffffffffffff000"
> 
> This coding triggers the ubsan issue
> 
> 
>   char* const hi_att = align_down(MIN2(max, absolute_max) - bytes, alignment_adjusted);
>   if (hi_att > max) {
>     return nullptr; // overflow
>   }
> 
> 
> However the function already contains overflow handling, so probably it is sufficient to add an attribute to the function os::attempt_reserve_memory_between to disable ubsan checks for this function.

I just ran the test with -Xlog:os+map=debug and got the same debug output.

> The fix is simple: ...

Thanks for suggesting the fix;  should I just add this to the PR instead of disabling ubsan for the method ?


> and after squashing about a zillion unrelated ubsan errors

Yeah there are unfortunately still a few ones remaining (I opened already JBS issues for most of them so the situation improves slowly) .

-------------

PR Comment: https://git.openjdk.org/jdk/pull/19543#issuecomment-2149982941