RFR: 8369393: NMT: poison the canaries of malloc header under ASAN build [v20]

[Thomas Stuefe]

On Mon, 10 Nov 2025 08:46:41 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> NMT can detect malloc'd memory corruption using canary tests at header and footer of every memory region. This can only be done at free time of the memory where NNT checks the canaries and report error if they are not as expected.
>> In this PR, the canary parts also are poisoned using ASAN API to get notified whenever a read/write op is done. on the canary parts. `_size` member of the malloc header is also poisoned, since it is used for finding the footer address.
>
> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
> 
>   order of includes corrected.

Just to understand this right: We already notice buffer over- and underflows, right? So this is just about catching the case where someone writes into the malloc header but does *not* overwrite the actual real malloc area?

Makes sense, since we also want to catch one-byte-overwrites. Though I would probably just disable NMT and do it that way.

src/hotspot/share/nmt/mallocHeader.hpp line 126:

> 124:   inline static OutTypeParam resolve_checked_impl(InTypeParam memblock);
> 125: 
> 126:   void asan_poison_self() {

Why this complex? Why not just poison/unpoison the whole header in one call? You could simplify quite a bit that way.

-------------

PR Review: https://git.openjdk.org/jdk/pull/27685#pullrequestreview-3442286623
PR Review Comment: https://git.openjdk.org/jdk/pull/27685#discussion_r2509799450