RFR: JDK-8298445: Add LeakSanitizer support in HotSpot [v2]

[Justin King]

On Fri, 3 Feb 2023 06:16:06 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> Thanks for the explanation. Do I understand correctly: You register address ranges with LSAN, and at certain points (program exit?) the memory is examined for pointers that point still to unreleased memory? Does LSAN instrument malloc and free?

LSan intercepts all malloc-related functions and is its own allocator. It normally performs leak checking at program exit, but we disable this behavior. We do not particularly care about leaks that happen as a result of the shutdown process. Instead we trigger leak checking extremely early in the JVM shutdown process, immediately after the synchronization lock but before the shutdown logic actually occurs. When LSan performs leak checking, the entire process is paused except the thread performing the leak checking. LSan is aware of all memory that has been allocated up until that point, let's call it set `A`, its goal is to prove that all pointers (the ones returned by malloc and friends) are present. It considers other memory allocated by malloc, the mutable data section of the executable, our registered root regions, and some others. During scanning if it finds the pointer to an allocation in set `A`, besides from an allocation itself, it marks it. It must be the pointer returned b
 y malloc and friends itself, not a pointer to a subregion of the allocation. After the scanning process, leaks are all the allocations in set `A` that have not been marked.

> Metaspace objects hold pointers to malloced memory. Metaspace Objects can leak, that's accepted. So these pointers would leak too. Would that not give us tons of false positives?

No. LSan only cares that it can find pointers to malloc'ed memory, not that it can find pointers to Metaspace objects. The Metaspace is registered as a root region. It does however take into account ASan poisoning, which we added earlier to Metaspace. LSan will not scan poisoned memory regions that are part of root regions. We only poison in Metaspace when deallocation occurs and unpoison when allocation occurs.

So there are no false positives there.

-------------

PR: https://git.openjdk.org/jdk/pull/12229