RFR: 8294142: make test should report only on executed tests
Aleksey Shipilev
shade at openjdk.org
Fri Sep 23 11:15:47 UTC 2022
This should help to speed up tests significantly. Currently, if we run "make test" with a subset of tests, JTReg would still read the entirety of test root to report on tests that were not run. Even with current suite of tests it gets expensive. If you add more tests to suite -- for example, mounting a large test archive like pre-generated fuzzer tests -- it starts to take a lot of time.
We might default to `-report:executed`. My own CI -- that has millions of tests mounted in `test/` -- was running with `JTREG="OPTIONS=-report:executed"` for years now. I think that should be the default way to run the tests in JDK tree.
There is a wrinkle with jtreg retries: we need to collate together the reports for successful retries from multiple runs. Fortunately, we can then fall back to `-report:all-executed`. [CODETOOLS-7903323](https://bugs.openjdk.org/browse/CODETOOLS-7903323) is poised to provide a new reporting option, `-report:files`, which would make this whole business easier, we can upgrade to it once minimum jtreg version is bumped. This RFE provides faster tests today, and is backportable to update releases.
Motivational example:
$ time CONF=linux-x86_64-server-release make run-test TEST=gc/epsilon/TestHelloWorld.java
# Default JDK tree
# Baseline
real 0m8.813s
user 0m20.846s
sys 0m3.841s
# Patched
real 0m5.926s ; about 1.5x faster
user 0m14.637s
sys 0m3.451s
# +100K fuzzer tests in tree
# Baseline
real 3m22.978s
user 3m33.417s
sys 0m11.197s
# Patched
real 0m8.415s ; about 23x faster
user 0m15.623s
sys 0m4.746s
-------------
Commit messages:
- Fix
Changes: https://git.openjdk.org/jdk/pull/10405/files
Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=10405&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8294142
Stats: 11 lines in 1 file changed: 8 ins; 2 del; 1 mod
Patch: https://git.openjdk.org/jdk/pull/10405.diff
Fetch: git fetch https://git.openjdk.org/jdk pull/10405/head:pull/10405
PR: https://git.openjdk.org/jdk/pull/10405
More information about the build-dev
mailing list