RFR: 8360791: [ubsan] Adjust signal handling when using clang

[Matthias Baesken]

On Fri, 27 Jun 2025 09:30:19 GMT, Matthias Baesken <mbaesken at openjdk.org> wrote:

> A couple of tests e.g. VendorInfoPluginsTest but also some Hotspot like runtime/ErrorHandling/CreateCoredumpOnCrash.java put (write) to special addresses like 0 to provoke crashs.
> However this does not work well with ubsan-enabled binaries on the clang based platforms (macOS, AIX).
> The mentioned tests generate a SIGSEGV.
> 
> Some other tests generate a SIGBUS, e.g.
> 
> runtime/memory/ReserveMemory.java
> runtime/memory/ReadFromNoaccessArea.java
> 
> and this leads to similar issues with ubsan-enabled binaries.
> 
> We should adjust the signal handling with the sanitizer options, how to do this is documented here :
> https://github.com/google/sanitizers/wiki/SanitizerCommonFlags

> I wonder if this should be unconditional, rather than conditional on __clang__.

At least it did not do harm on Linux x86_64/gcc when making it unconditional.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26015#issuecomment-3014119542