RFR(XS): 8200187: Exclude 3 long-running tests from tier1

[Leonid Mesnik]

Hi

There is no strict rules and time budgets for hotspot tiers. It is assumed that faster tests should be executed earlier tiers. The order and total impact of of these test depends on how they are executed.  

Here is time of execution of tier1 groups with and without these excluded tests:
The whole :tier1 time reduced from 40 to 30 minutes (on dedicated HW with 32 core)
Time for :tier1_serviceability reduced from 15 to 6 min (in HS CI)
Time for :tier1_gc_1 reduced from 12 to 8-10 min  (in HS CI)

The benefits of exclusion of gc/g1/ihop/TestIHOPErgo.java are not so significant. However it is the only one GC stress test executed in tier1 while intention is to don’t run stress testing in tier1.

Tests
serviceability/sa/ClhsdbScanOops.java
serviceability/sa/TestHeapDumpForLargeArray.java
are now hotspot_tier3_runtime which includes all hotspot_serviceability tests which are not a part of tier1. They are executed as a part of tier3 now as well as most of serviceability tests.

Test gc/g1/ihop/TestIHOPErgo.java in now in group :hotspot_gc only and also it is marked as “stress” test. So it is executed with all other GC stress tests

Leonid

> On Mar 25, 2018, at 3:48 PM, David Holmes <david.holmes at oracle.com> wrote:
> 
> Hi Leonid,
> 
> On 24/03/2018 9:31 AM, Leonid Mesnik wrote:
>> Hi
>> Could you please review following fix which exclude following tests from tier1 testing:
>> serviceability/sa/ClhsdbScanOops.java
>> serviceability/sa/TestHeapDumpForLargeArray.java
>> gc/g1/ihop/TestIHOPErgo.java
>> Each of them takes more then 5 minutes to complete and significantly increase overall time to complete tier1.
> 
> I'd need to see a much more detailed analysis of all the tests run, the order they run and the execution times to ascertain what impact this actually has on overall test execution time.
> 
> But assuming 5 minutes is too long for tier1, this seems okay. But the tests must still be run in some other tier, and I'm not clear where that would be now? I would expect them to move to tier 3 perhaps, depending on what the time criteria for tier 3 is.
> 
> Thanks,
> David
> 
>> Please let me know if there are any reasons to run these tests in tier1 despite on their execution time.
>> webrev:http://cr.openjdk.java.net/~lmesnik/8200187/webrev.00/
>> bug: https://bugs.openjdk.java.net/browse/JDK-8200187
>> Leonid