RFR: 8368365: ASAN errors should produce hs-err files and core dumps [v5]

[David Holmes]

On Sat, 27 Sep 2025 06:48:36 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> When we run with ASAN enabled and ASAN catches an error, it reports, then stops the JVM. hs-err files and crash dumps at that point would be incredibly useful, though. The ASAN error report itself is seldom enlightening since it only contains native stacks.
>> 
>> After this patch, the JVM will always produce hs-err files when an ASAN-report happens. It will *only* produce core files if ASAN_OPTIONS `disable_coredump=0` and `abort_on_error=1` and the JVM option `CreateCoredumpOnCrash` had not been disabled (and the limit for core file size is high enough etc, all the usual restrictions on OS level still apply).
>> 
>> This means that ASAN builds, by default, will continue to *not* generate cores, since ASAN default options inhibit that. See detail in the comments below.
>> 
>> ---
>> 
>> Tested on Fedora 42 and Debian 12, both manually and by running the new companion jtreg test.
>
> Thomas Stuefe has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Update address.cpp
>  - Comment reference to asan_interface.h

src/hotspot/share/sanitizers/address.cpp line 62:

> 60:     log_warning(asan)("*** Failed to install JVM callback for ASAN. ASAN errors will not generate hs-err files. ***");
> 61:     return;
> 62:   }

Doesn't this just mean ASAN is not present at runtime? It isn't really an error installing the callback.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/27446#discussion_r2386486400