RFR: 8343503: Problemlist support fastdebug/slowdebug build

[David Holmes]

On Mon, 4 Nov 2024 03:31:40 GMT, SendaoYan <syan at openjdk.org> wrote:

> Hi all,
> Sometimes some tests only fails or crash run by fastdebug/slowdebug jdk binary, such as `java/lang/Thread/jni/AttachCurrentThread/AttachTest.java#id1` which has been recorded by [JDK-8343244](https://bugs.openjdk.org/browse/JDK-8343244). To make less CI noisy, we can disable these tests by add tag such as `@requires vm.debug == false`, but I think put the releated tests to Problemlist is a better way before the root cause failure has been fixed.
> Thus I think it's necessary support Problemlist for fastdebug/slowdebug build, the Problemlist only work for fastdebug/slowdebug build.
> By the way, the timeout factor with fastdebug/slowdebug should larger than release build, this PR also make jtreg timeoutFactor to 10 if the tested jdk is fastdebug/slowdebug build.

Changes requested by dholmes (Reviewer).

make/RunTests.gmk line 860:

> 858: 
> 859:   ifneq ($$(DEBUG_LEVEL), release)
> 860:     JTREG_AUTO_PROBLEM_LISTS += ProblemList-debug.txt

We (Oracle) typically do not need this because of the way we run our tests. Hotspot tests are always run on fastdebug in lower tiers and only some tests run on release bits in higher tiers. So the fact the ProblemList affects fastdebug and release is not really an issue. Conversely, JDK tests are run on release bits in lower tiers, and only in some cases run on fastdebug in later tiers.

My main concern here is that we end up with too many potential ProblemList files that have to be managed and maintained. And, IMO jtreg should be enhanced so that the PL syntax allows more flexibility over the conditions under which a test is to be excluded.

make/RunTests.gmk line 861:

> 859:   ifneq ($$(DEBUG_LEVEL), release)
> 860:     JTREG_AUTO_PROBLEM_LISTS += ProblemList-debug.txt
> 861:     JTREG_AUTO_TIMEOUT_FACTOR := 10

Please don't do this. We set timeout factors as determined by how the tests will be run in our CI and this could interfere with that.

-------------

PR Review: https://git.openjdk.org/jdk/pull/21862#pullrequestreview-2412098944
PR Review Comment: https://git.openjdk.org/jdk/pull/21862#discussion_r1827173036
PR Review Comment: https://git.openjdk.org/jdk/pull/21862#discussion_r1827170973