RFR: 8369393: NMT: poison the canaries of malloc header under ASAN build [v20]

[Thomas Stuefe]

On Fri, 14 Nov 2025 11:44:26 GMT, Johan Sjölen <jsjolen at openjdk.org> wrote:

> > We use ASAN in production to find real issues in sometimes large applications that may run a while before hitting an error.
> 
> TIL. I didn't know that anyone ran it in prod.
> 

Not in a production server. In production as in "trying to find errors in real production scenarios" where the JVM lives for a while. Think a long-running server app that only at certain points in its load hits errors. To do these tests, the app still needs to be somewhat reasonably fast. (like with JIT- most real-world scenarios are difficult to reproduce without a JIT, since you never even get there).

> I think (un)poisoning the whole header with one call is fine, probably preferable.
> 
> > Also, since we only access the header on os::malloc/realloc/free, we can move up the RAII unpoisoning helper to those functions and reduce the number of invocations from 16 to three call sites.
> 
> It does move the responsibility of unpoisoning from an internal detail to something the caller needs to be aware of and do. Not saying I'm against it, but I want to see what that looks like first.

True, but then, what is worse: sprinkling the code with so many unpoison calls - including the runtime overhead - or adding these calls to the three places where it matters, and that need to be aware of header logic anyway.

Side note: templates are powerful, but have a downside in that they make it hard for IDEs to analyze code. And to grapple with large code bases, a simple grep is not enough. For example, I saw that "Show Call Hierarchy" command in my IDE stopped working when invoked for `check_block_integrity` because `MallocHeader::resolve_checked_impl` is now templatized. We do this to handle both const and non-const pointers. In that case, is the benefit really worth it, compared to a simple const_cast?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/27685#discussion_r2527473067