RFR: 8273314: Add tier4 test groups [v3]

[Igor Ignatyev]

On Mon, 6 Sep 2021 13:22:03 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> During the review of JDK-8272914 that added hotspot:tier{2,3} groups, @iignatev suggested to create tier4 groups that capture all tests not in tiers{1,2,3}. I have excluded `vmTestbase` and `hotspot:tier4,` because they take 10+ hours on my highly parallel machine. I have also excluded `applications` from `hotspot:tier4`, because they require test dependencies (e.g. jcstress).
>> 
>> Sample run:
>> 
>> 
>> ==============================
>> Test summary
>> ==============================
>>    TEST                                              TOTAL  PASS  FAIL ERROR   
>>>> jtreg:test/hotspot/jtreg:tier4                      426   425     1     0 <<
>>>> jtreg:test/jdk:tier4                               2891  2885     4     2 <<
>>    jtreg:test/langtools:tier4                            0     0     0     0   
>>    jtreg:test/jaxp:tier4                                 0     0     0     0   
>> ==============================
>> 
>> real	64m13.994s
>> user	1462m1.213s
>> sys	39m38.032s
>> 
>> 
>> There are interesting test failures on my machine, which I would address separately.
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Drop applications and fix the comment

> _Mailing list message from [David Holmes](mailto:david.holmes at oracle.com) on [hotspot-dev](mailto:hotspot-dev at mail.openjdk.java.net):_
> 
> On 7/09/2021 1:17 am, Aleksey Shipilev wrote:
> 
> > @dholmes-ora: Generally speaking, all `tierX` definitions are rather arbitrary, as there seem to be nothing intrinsic about the tests to be in a particular tier. In other words, what `tierX` consists of is a matter of agreement. I'd say `hotspot:tier4` is "all assorted Hotspot tests that are not application-specific suites"
> 
> The difference is that your previous work just consolidated the existing
> subsystem tier 1-3 definitions, but here you are choosing to define "all
> the rest" as tier 4. I don't think it is actually helpful/useful to
> anyone - and it bears no resemblance whatsoever to what we call "tier
> 4", so that will just lead to unnecessary confusion IMO.

@dholmes-ora , although I fully agree that this might lead to some misunderstanding b/w Oracle and non-Oracle folks, I don't see how it's different from the previous patch, which introduced `hotspot:tier2` and `hotspot:tier3`. even if we reduce `tierN` to just a set of tests, the test groups added by 8272914 bear as much resemblance to the test sets used in Oracle's tier2-3 as the suggested `hotspot:tier4` groups in this patch to the actual `tier4` definition used in Oracle's internal system, e.g. `hotspot:tier2` group has 0 tests from `test/hotspot/jtreg/compiler`, but Oracle's `tier2` does include a number of  `test/hotspot/jtreg/compiler` tests (which aren't part of `:tier1`). I believe that this patch actually moves us closer to a convergence point, as the union of `hotspot:tier1` -- `hotspot:tier4` test groups is very close to the test sets used in hotspot parts of Oracle's `tier1`  -- `tier4` definitions. 

Thanks,
-- Igor

-------------

PR: https://git.openjdk.java.net/jdk/pull/5357