RFR: 8368365: ASAN errors should produce hs-err files and core dumps [v4]

[Dean Long]

On Wed, 24 Sep 2025 14:23:45 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> When we run with ASAN enabled and ASAN catches an error, it reports, then stops the JVM. hs-err files and crash dumps at that point would be incredibly useful, though. The ASAN error report itself is seldom enlightening since it only contains native stacks.
>> 
>> This patch makes it possible to get hs-err files and subsequent crash dumps by using the error callback functionality of ASAN. It registers a callback which gets called when ASAN catches an error. We then - carefully - print out the ASAN report to stderr (as ASAN itself would have done) and proceeed to end the JVM with a `fatal()`, which gives us an hs-err file, a callstack at that point including Java frames, and - if enabled - core dumps.
>> 
>> Tested on Fedora 42 and Debian 12, both manually and by running the new companion jtreg test.
>
> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix

Actually, what I was trying to get at with my question was whether we need the callback if abort_on_error=1 is set.  If abort_on_error=1 is set, it will call abort(), which should send a SIGABRT and then cause an hs_err file to get generated.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/27446#issuecomment-3331019879