RFR: 8024140: [TESTBUG] Profile based regression test groups for jdk repo

David Holmes david.holmes at oracle.com
Tue Sep 3 03:02:58 UTC 2013


webrev:

http://cr.openjdk.java.net/~dholmes/8024140/webrev/

Similar to what was recently done on the hotspot side, this introduces 
profile-based test groups for the jdk regression tests.

The primary groups are:
  - jdk
  - jre
  - compact3
  - compact2
  - compact2_minimal
  - compact1
  - compact1_minimal

The minimal VM is only supported on compact1 and compact2.

The groups are defined hierarchically in two forms:
- The need_xxx groups list all the tests that have a dependency on a 
specific profile. This is either because it tests a feature in that 
profile, or the test infrastructure uses a feature in that profile.
- The primary groups are defined in terms of the other primary groups 
combined with the needs_xxx groups (including and excluding them as 
appropriate). For example the jre can run all tests from compact3, plus 
those from needs_jre, but excluding those from need_jdk.

The bottom group defines all the actual tests to be considered, simply 
by listing the top-level test directories, and then excluding all the 
needs_xxx groups.

To select a group of tests you use :<group-name>

Eg to run only those tests that can run on compact1 use:

jtreg :compact1

Of course you still need to point jtreg at the right kind of runtime 
image (and give it a full JDK as the compile-jdk!); and if testing the 
minimal VM you need to tell jtreg to select it using -javaoptions:-minimal

These top-level groups are not as useful standalone as they are for 
hotspot due to the fact that there are so many regression tests that are 
problematic in one way or another. In most cases what is desired is a 
way to run sub-set of tests for a given profile eg all jdk_core tests 
that can run on compact1. At present this would require a new group to 
be defined for each permutation and that is not scalable. I hope that in 
the near future jtreg will allow you to implicitly define such ad-hoc 
groups via the command-line, simply by listing groups to include and 
groups to exclude.

The full jtreg group facility is only available in the most recent jtreg 
builds, so you will need to grab the latest nightly build, or latest 
sources.

Note: once this is in place, anyone writing regression tests will need 
to be aware of whether that test is limited to certain profiles and 
update the group file accordingly. Sometimes it is not the item being 
tested that determines the minimum needed profile, but the test 
infrastructure eg if it uses XML.

See TEST.groups for more information.

Note: The initial group definitions proposed here are not complete. 
There are over 4000 tests of the "desktop" that can not be readily 
executed manually and it may be that a number of these tests will 
require a JDK rather than the full JRE. Our full test processes will 
discover this and update the lists as needed. (Or we fix the tests to 
not need full JDKs.)

Thanks,
David



More information about the core-libs-dev mailing list