RFR: 8369393: NMT: poison the canaries of malloc header under ASAN build [v7]

[Afshin Zafari]

On Tue, 14 Oct 2025 11:50:43 GMT, Johan Sjölen <jsjolen at openjdk.org> wrote:

> Ideas for names:
> 
> * AsanPoisonSuppressor
> * AsanUnpoisonGuard
> * AsanUnpoisonScope

`AsanPoisoningHelper` is used. It is used both for suppressing and register/unregister memory regions.

> src/hotspot/share/nmt/mallocHeader.hpp line 108:
> 
>> 106:     ASAN_UNPOISON_MEMORY_REGION(addr, sizeof(T));
>> 107:   }
>> 108: };
> 
> Wrap the bodies of these functions with the `#ifdef ASAN`. As these defs are in the header, they'll be visible to any usages by the compiler, and so can be optimized away when ASAN is not in use. This helps simplify the rest of the code.
> 
> The name `Poisoner` is unfortunate, as it's technically `Unpoisoning` the memory region. You can also have the `_memory` be `T*` and have the class take `T*` as its argument and do the casting inside of the class body instead. This avoids unnecessary clutter in the user-code.
> 
> Perhaps rename `_memory` to `_memory_region` to mimic the wording used in the ASAN macros?
> 
> Use `reinterpret_cast` instead of C-style cast, with the intent of providing the full meaning of the cast.
> 
> 
> `register` and `unregister` memory, perhaps they should be poison and unpoison?

Done

> src/hotspot/share/nmt/mallocHeader.hpp line 120:
> 
>> 118:   static void unregister_memory(char* addr) { }
>> 119:   ~AsanPoisoner() { }
>> 120: };
> 
> I think you can delete this definition

Kept. to be used in gtests.

> src/hotspot/share/nmt/mallocHeader.hpp line 130:
> 
>> 128:   using SizeType = void;
>> 129:   NOT_LP64(using AltCanaryType = void;)
>> 130: #endif
> 
> Surely this won't work, as we're using some of these types as return types in functions?

return values removed.

> src/hotspot/share/nmt/mallocTracker.cpp line 191:
> 
>> 189:   assert(((size_t)memblock & (sizeof(size_t) * 2 - 1)) == 0, "Alignment check");
>> 190: 
>> 191: 
> 
> Style: Added in space, remove

Done.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/27685#issuecomment-3401744335
PR Review Comment: https://git.openjdk.org/jdk/pull/27685#discussion_r2429112495
PR Review Comment: https://git.openjdk.org/jdk/pull/27685#discussion_r2429114941
PR Review Comment: https://git.openjdk.org/jdk/pull/27685#discussion_r2429117080
PR Review Comment: https://git.openjdk.org/jdk/pull/27685#discussion_r2429111374