Fwd: Re: running tests

[Alan Bateman]

On 16/04/2012 20:20, Jonathan Gibbons wrote:
> :
>
> -------- Original Message --------
> Subject: 	Re: running tests
> Date: 	Mon, 16 Apr 2012 07:13:55 +0200
> From: 	Fredrik Öhrström <oehrstroem at gmail.com>
> To: 	Jonathan Gibbons <jonathan.gibbons at oracle.com>
> CC: 	build-infra-dev at openjdk.java.net
>
>
>
> How about this?
>
> make quicktest
> # Runs a fast subset that verifies the basic
> # functionality of all modules. Should complete
> # in less than a minute.
>
> make test
> # Runs all tests! Takes a long time.
>
> make performance_tests
> # Test performance benchmarks
> make gc_tests
> # Test garbage collectors
> make codegen_tests
> # Test the compilers c1/c2/c3.... (oh, do I wish there was only one)
>
> Any more groups of tests Jon?
>
> //Fredrik
>
Just to add to Jon's comments, in jdk/test/Makefile there are 20 or so 
jdk* targets that map to various subsets of the library tests ("make 
jdk_util" runs the java/util/** tests for example). As Jon also 
mentioned, they are in these batches so that the execution can be 
partitioned horizontally across machines (make/jprt.properties shows how 
they are grouped into test sets in that system).

We don't have keywords on the regression tests that would allow for 
cross cutting targets such as "quicktest" or "performance_tests". Also 
performance tests is a somewhat special case as they need highly stable 
environments to be useful (we don't check them into the jdk repository 
although it would be nice to have them available in OpenJDK).

On execution/through-put then the tests that run in agent VM mode can 
also run with concurrency > 1. The Makefile makes it clear which run in 
agent VM mode and it has a CONCURRENCY option so it can be specified to 
the make command ("make CONCURRENCY=4 jdk_util" for example).

Hope this is useful,

-Alan