RFR: 8272914: Create hotspot:tier2 and hotspot:tier3 test groups [v2]

[David Holmes]

On Tue, 24 Aug 2021 17:52:51 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Currently, no tests run in `hotspot:tier2` and `hotspot:tier3` groups, yet some groups have the "tier2" and "tier3" in their names. As we move tests from `hotspot:tier1`, they need to land in higher tiers. Therefore, we need `hotspot:tier2` and `hotspot:tier3`.
>> 
>> I had to exclude `InvocationTests` from `tier3`, because they run for a very long time.
>> 
>> Sample runs:
>> 
>> 
>> TEST                                              TOTAL  PASS  FAIL ERROR   
>> jtreg:test/hotspot/jtreg:tier2                      383   383     0     0   
>> 
>> real	10m9.008s
>> user	418m16.348s
>> sys	37m16.874s
>> 
>> 
>> 
>> TEST                                              TOTAL  PASS  FAIL ERROR   
>> jtreg:test/hotspot/jtreg:tier3                       72    72     0     0   
>> 
>> real	6m51.729s
>> user	368m37.880s
>> sys	4m26.574s
>> 
>> 
>> These also hook up properly to global `tier2` and `tier3`.
>> 
>> Additional testing:
>>  - [x] Linux x86_64 fastdebug `hotspot:tier2`
>>  - [x] Linux x86_64 fastdebug `hotspot:tier3`
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Filter InvocationTests in tier3, leaving the existing group alone

Hi Aleksey,

A couple of queries below.

Also just to be very clear when Oracle refer to tier1, tier2 etc there is only an approximate mapping to the jtreg groups with those names. Our tier levels also have varied platform support, can include tests not already grouped in a tierN_x, and can run previous tier tests with different flags. So it is important to note that after this change if you refer to "tier3" testing and I refer to "tier3" testing then we are not referring to the exact same thing.

Cheers,
David

test/hotspot/jtreg/TEST.groups line 448:

> 446: 
> 447: tier2 = \
> 448:   -:tier1 \

Not certain but I think the excludes have to come last. Also in theory shouldn't be needed because the tier2_x group should already exclude the tier1_x group.

test/hotspot/jtreg/TEST.groups line 457:

> 455:   -:tier2 \
> 456:   :hotspot_tier3_runtime \
> 457:   -runtime/InvocationTests \

Why are these excluded?

-------------

PR: https://git.openjdk.java.net/jdk/pull/5241