RFR: 8273314: Add tier4 test groups

[Aleksey Shipilev]

On Sat, 4 Sep 2021 03:13:58 GMT, Igor Ignatyev <iignatyev at openjdk.org> wrote:

>>> > <...> I have excluded `vmTestbase` and `hotspot:tier4`<...>  I have also excluded `applications` from `hotspot:tier4` <...>
>>> 
>>> assuming the goal of tier4 is to catch the rest of the tests, I don't think we should exclude `vmTestbase`, `applications` or any other tests from tier4. unless you also want to create tier5 for them.
>> 
>> Apart from practicality of using `tier4`, I think `vmTestbase` and `applications` are separate test suites in their own right. `tier4` is catching all the assorted Hotspot tests that are not part of larger suites. Makes sense?
>
>> > > <...> I have excluded `vmTestbase` and `hotspot:tier4`<...>  I have also excluded `applications` from `hotspot:tier4` <...>
>> > 
>> > 
>> > assuming the goal of tier4 is to catch the rest of the tests, I don't think we should exclude `vmTestbase`, `applications` or any other tests from tier4. unless you also want to create tier5 for them.
>> 
>> Apart from practicality of using `tier4`, I think `vmTestbase` and `applications` are separate test suites in their own right. `tier4` is catching all the assorted Hotspot tests that are not part of larger suites. Makes sense?
> 
> to some extent. I agree that `applications` tests can/should be seen as a separate test suite, yet I differ on `vmTestbase` as the end goal for `vmTestbase` tests is (and always was) to become just another test within hotspot jtreg test suite, hence I don't think we should treat them any different than other jtreg tests. there is also a plan (which I need to formalize and share w/ a broader audience) to rearrange `vmTestbase` tests so they will be placed within the corresponding component subdirectories, which would bring us closer to the end goal and at the same time make it slightly harder to select all `vmTestbase` tests.
> 
> -- Igor

@iignatev: OK, I reinstated `vmTestbase` and left `applications` out of `hotspot:tier4`.
@dholmes-ora: Generally speaking, all `tierX` definitions are rather arbitrary, as there seem to be nothing intrinsic about the tests to be in a particular tier. In other words, what `tierX` consists of is a matter of agreement. I'd say `hotspot:tier4` is "all assorted Hotspot tests that are not application-specific suites"
@mrserb: Yes, I ran this on a headless config, so headful tests were skipped, apparently. I'll try to arrange runs on my desktop.

Please see new commit. As `hotspot:tier4` is now much longer, it would take me a while to verify everything works there.

-------------

PR: https://git.openjdk.java.net/jdk/pull/5357