RFR: 8368365: ASAN errors should produce hs-err files and core dumps [v2]

[David Holmes]

On Tue, 23 Sep 2025 10:43:11 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> When we run with ASAN enabled and ASAN catches an error, it reports, then stops the JVM. hs-err files and crash dumps at that point would be incredibly useful, though. The ASAN error report itself is seldom enlightening since it only contains native stacks.
>> 
>> This patch makes it possible to get hs-err files and subsequent crash dumps by using the error callback functionality of ASAN. It registers a callback which gets called when ASAN catches an error. We then - carefully - print out the ASAN report to stderr (as ASAN itself would have done) and proceeed to end the JVM with a `fatal()`, which gives us an hs-err file, a callstack at that point including Java frames, and - if enabled - core dumps.
>> 
>> Tested on Fedora 42 and Debian 12, both manually and by running the new companion jtreg test.
>
> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Update address.cpp

Seems reasonable to me but I am not an ASAN user so ...

-------------

PR Review: https://git.openjdk.org/jdk/pull/27446#pullrequestreview-3260354072