RFR: 8294403: [REDO] make test should report only on executed tests [v2]

Aleksey Shipilev shade at openjdk.org
Wed Jan 4 09:15:18 UTC 2023


> This should help to speed up tests significantly. Currently, if we run "make test" with a subset of tests, JTReg would still read the entirety of test root to report on tests that were not run. Even with current suite of tests it gets expensive. If you add more tests to suite -- for example, mounting a large test archive like pre-generated fuzzer tests -- it starts to take a lot of time.
> 
> The reasonable default for older jtreg is `-report:executed`. My own CI -- that has millions of tests mounted in `test/` -- was running with `JTREG="OPTIONS=-report:executed"` for years now. [CODETOOLS-7903323](https://bugs.openjdk.org/browse/CODETOOLS-7903323) provides a new reporting option, `-report:files`, which makes this whole business easier. This requires new jtreg.
> 
> Motivational improvements:
> 
> 
> $ time CONF=linux-x86_64-server-release make run-test TEST=gc/epsilon/TestHelloWorld.java
> 
> # Default JDK tree
> 
> # Baseline
> real	0m9.086s
> user	0m22.857s
> sys	0m4.202s
> 
> # Patched
> real	0m6.406s   ; +40% faster
> user	0m17.156s
> sys	0m3.193s
> 
> # +100K fuzzer tests in tree
> 
> # Baseline
> real	3m1.997s
> user	3m16.643s
> sys	0m10.490s
> 
> # Patched
> real	0m8.919s   ; 20x faster
> user	0m17.904s
> sys	0m4.860s
> 
> 
> Additional testing:
>  - [x] Ad-hoc timing tests (see above)
>  - [x] Reproducer from [CODETOOLS-7903331](https://bugs.openjdk.org/browse/CODETOOLS-7903331)
>  - [x] Eyeballing the test results on passing tests with REPEAT_COUNT > 1
>  - [x] Eyeballing the test results on intermittently failing tests with RETRY_COUNT > 1

Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:

  Copyright years

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/11824/files
  - new: https://git.openjdk.org/jdk/pull/11824/files/a803958f..a82ac6cc

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=11824&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=11824&range=00-01

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/11824.diff
  Fetch: git fetch https://git.openjdk.org/jdk pull/11824/head:pull/11824

PR: https://git.openjdk.org/jdk/pull/11824



More information about the build-dev mailing list