RFR: 8357826: Avoid running some jtreg tests when asan is configured [v5]

[Thomas Stuefe]

On Fri, 6 Jun 2025 11:34:04 GMT, Matthias Baesken <mbaesken at openjdk.org> wrote:

>> There are a couple of jtreg tests, especially in the HS area, with very special assumptions about memory layout/sizes .
>> Those fail when the address sanitizer is configured ( --enable-asan ).
>> The change adds a way to tag those tests with 'requires' so that they can be avoided easily when running jtreg tests with ASAN enabled.
>> Adjusting the tests for "pleasing" the sanitizer is not always desired (if possible for some tests it can be done later) .
>> While at it, also same is also added for ubsan .
>
> Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision:
> 
>   exclude AbsPathInImage test from asan

Like @dholmes-ora , I would appreciate a short comment explaining the reasons for these exclusions. If those are bugs that can be fixed, its better to have a JBS issue describing the problem in order to evenutally fix it, and possibly problemlist the test (for asan) in the meantime. 

Note that even if you don't know the exact reasons why e.g. libjsig + asan fails, and have no time to investigate it, its okay to make that a JBS task as in "Investigate whether libjsig tests can work with asan". Those are good starter issues for folks with a bit of time at their hands.

test/hotspot/jtreg/runtime/XCheckJniJsig/XCheckJSig.java line 32:

> 30:  *          java.management
> 31:  * @requires os.family == "linux" | os.family == "mac"
> 32:  * @requires !vm.asan

I would like to understand this. What part of asan interferes with the libjsig? Asan interposes memory APIs, libjsig interposes signal functions, both should be able to coexist.

 If its a bug, we should have a JBS issue to (eventually) fix it, and problemlist the test for asan in the meantime. Otherwise, a short comment explaining the issue would be good.

test/hotspot/jtreg/serviceability/dcmd/vm/SystemDumpMapTest.java line 39:

> 37:  * @library /test/lib
> 38:  * @requires (os.family == "linux" | os.family == "windows" | os.family == "mac")
> 39:  * @requires !vm.asan

If this is easy to fix, e.g. like Jiangli did for the static build, I would rather have a fix inside the test than to completely disable it. Otherwise, a short comment would be good explaining why the test cannot run with asan.

Preexisting: I have no clue why this is excluded for riscv quemu, but SystemMapTest is not, both tests are functionally almost identical.

-------------

PR Review: https://git.openjdk.org/jdk/pull/25575#pullrequestreview-2905989713
PR Review Comment: https://git.openjdk.org/jdk/pull/25575#discussion_r2132815020
PR Review Comment: https://git.openjdk.org/jdk/pull/25575#discussion_r2132822857