RFR: 8357155: [asan] ZGC does not work

[Axel Boldt-Christmas]

On Fri, 30 May 2025 12:18:46 GMT, Matthias Baesken <mbaesken at openjdk.org> wrote:

> Many (all?) ZGC related jtreg tests do not work when the JDK is built with address sanitizer asan enabled (configure flag --enable-asan).
> This can be seen on SUSE Linux x86_64 and also on ppc64le , opt binaries were used.
> It has been suggested to do a workaround -  'But I think that simply adapting the zAddress_[...].cpp implementations to always select the largest heap base would go a long way for providing ASAN compatibility.'
> This seems to work nicely on x86_64 and ppc64le,  however the zgc related tests still fail on Linux aarch64 (should I exclude this platform from my patch?) .

x86 change looks good. I did realise a problem with the PPC implementation. And maybe there is something we could try on aarch64.

src/hotspot/cpu/aarch64/gc/z/zAddress_aarch64.cpp line 98:

> 96:   const size_t max_address_offset_bits = valid_max_address_offset_bits - 3;
> 97: #ifdef ADDRESS_SANITIZER
> 98:   return max_address_offset_bits;

I was hoping this could work for Linux with 47/48 bit aarch64 VMA. But it is unclear how ASAN selects its mappings on such platforms. 

On 39/42 bit VMA returning `MIN2(valid_max_address_offset_bits, 44)` as I suggested in the PPC function may be a better best effort, as we are using addresses where we actually probed that reservations could be possible).  Or even `MIN2(valid_max_address_offset_bits - 1, 44)`. Feel free to try it out, but I think this is otherwise an alright approach until we implement a better heap base selection strategy where we can test multiple base candidates.

src/hotspot/cpu/ppc/gc/z/zAddress_ppc.cpp line 95:

> 93:   const size_t max_address_offset_bits = valid_max_address_offset_bits - 3;
> 94: #ifdef ADDRESS_SANITIZER
> 95:   return max_address_offset_bits;

I think this actually has to be 
```c++
  return MIN2(valid_max_address_offset_bits, 44);


Because the way we probe we may otherwise return 45 here. Which could result in more than 44 bits in a ZOffset which our internal data structures cannot handle. Hopefully this still works for ASAN on PPC. (The `-3` is a left over from non-generational ZGC). Aarch64 could do the same, but it does not have this issue as it starts its probing at bit 46, not bit 47.

_Side note: This makes me realise that there probably is a bug here on PPC and RISCV if running on a NUMA machine with more than 8 TB heap. As after ZGlobalsPointers::min_address_offset_request() was introduced we can return 45 from this function._

-------------

Changes requested by aboldtch (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/25549#pullrequestreview-2881567488
PR Review Comment: https://git.openjdk.org/jdk/pull/25549#discussion_r2116093093
PR Review Comment: https://git.openjdk.org/jdk/pull/25549#discussion_r2116053030