RFR: 8369393: NMT: poison the canaries of malloc header under ASAN build [v20]

[Thomas Stuefe]

On Fri, 14 Nov 2025 10:57:22 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Wouldn't this lead to more branches in the code, and questions like "why is this poisoned, but not _size?" etc? I wouldn't call the `_size` field in the header frequently accessed (once at malloc, once at free, any I'm missing?), it's more than 0 I guess :-).
>> 
>> I haven't measured, so grains of salt and so on, but the cost of ASAN is already high, I don't think any additional cost to poisoning/unpoisoning NMT headers at malloc/free is going to add too much of a perf hit.
>
> Hm, okay. I refreshed my memory, and yes, we only access the header on malloc and free. But then, what exactly is the purpose of selectively poisoning size and the canary, but leaving the rest unpoisoned? Either we must have perfect coverage - in which case we should poison the whole header - or we are happy with a 99% solution, in which case it's sufficient to poison only the canaries. Either way would be preferable to what the patch does now, which feels a bit arbitrary.
> 
> Also, since we only access the header on os::malloc/realloc/free, we can move up the RAII unpoisoning helper to those functions and reduce the number of invocations from 16 to three call sites.

I take back what I wrote. From looking into how UNPOISON is implemented, we dive down into libsanitizer for every single call to unpoison, and that function is non-trivial. So I would prefer to protect the malloc header in its whole.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/27685#discussion_r2527132246