RFR: JDK-8298908: Instrument Metaspace for ASan [v12]

[Ioi Lam]

On Tue, 10 Jan 2023 23:17:06 GMT, Justin King <jcking at openjdk.org> wrote:

>> This change instruments Metaspace for ASan. Metaspace allocates memory using `mmap`/`munmap` which ASan is not aware of. Fortunately ASan supports applications [manually poisoning/unpoisoning memory](https://github.com/google/sanitizers/wiki/AddressSanitizerManualPoisoning). ASan is able to detect poisoned memory, similar to `use-after-free`, and will raise an error similarly called `use-after-poison`. This provides and extra layer of defense and confidence.
>> 
>> The header `sanitizers/address.h` defines macros for poisoning/unpoisoning memory regions. These macros can be used regardless of build mode. When ASan is not available, they are implemented using a NOOP approach which still compiles the arguments but does so such that they will be stripped out by the compiler due to being unreachable. This helps with maintenance.
>> 
>> This also has the added benefit of making [LSan](https://bugs.openjdk.org/browse/JDK-8298445) more accurate and deterministic, as LSan will not look for pointers to malloc memory in poisoned memory regions.
>> 
>> IMO the benefit of doing this greatly outweighs the cost.
>
> Justin King has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Update sanitizers/address.h based on review
>   
>   Signed-off-by: Justin King <jcking at google.com>

src/hotspot/share/memory/metaspace/chunkManager.cpp line 308:

> 306:   }
> 307: 
> 308:   if (enlarged) {

What's the reason of calling the ASAN functions outside of the lock?

Will we have a race condition if two threads call ChunkManager::attempt_enlarge_chunk() at the same time and get to here out of order? @tstuefe

-------------

PR: https://git.openjdk.org/jdk/pull/11702