RFR: 8368365: ASAN errors should produce hs-err files and core dumps [v7]

[Thomas Stuefe]

On Mon, 29 Sep 2025 08:36:15 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> When we run with ASAN enabled and ASAN catches an error, it reports, then stops the JVM. hs-err files and crash dumps at that point would be incredibly useful, though. The ASAN error report itself is seldom enlightening since it only contains native stacks.
>> 
>> After this patch, the JVM will always produce hs-err files when an ASAN-report happens. It will *only* produce core files if ASAN_OPTIONS `disable_coredump=0` and `abort_on_error=1` and the JVM option `CreateCoredumpOnCrash` had not been disabled (and the limit for core file size is high enough etc, all the usual restrictions on OS level still apply).
>> 
>> This means that ASAN builds, by default, will continue to *not* generate cores, since ASAN default options inhibit that. See detail in the comments below.
>> 
>> ---
>> 
>> Tested on Fedora 42 and Debian 12, both manually and by running the new companion jtreg test.
>
> Thomas Stuefe has refreshed the contents of this pull request, and previous commits have been removed. Incremental views are not available.

Sorry for the spam and the force push. Something went wrong with my final merge from master.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/27446#issuecomment-3345696306