[8u] RFR: 8222737: [TESTBUG] Allow for tier 1 like testing in OpenJDK 8u

[Aleksey Shipilev]

On 6/27/19 1:20 PM, Severin Gehwolf wrote:
> On Thu, 2019-06-27 at 11:58 +0200, Aleksey Shipilev wrote:
>> On 5/22/19 6:34 PM, Severin Gehwolf wrote:
>> hotspot:
>>
>> *) Test exclusion question. Do we really want to exclude these tests from tier1? I would rather see
>> them included and make them fail in regular runs. This would put pressure on fixing them for 8u...
>>
>>  150 # Right now tier1 runs all hotspot tests except RTM tests,
>>  151 # and one serviceability test which seems flakey.
>>  152 hotspot_tier1 = \
>>  153   :jdk \
>>  154   -compiler/rtm \
>>  155   -serviceability/sa/jmap-hashcode/Test8028623.java
> 
> I'm not sure about it. We could exclude nothing and go from there.
> However, the more tests we have failing from the get-go the less useful
> it is to enforce tier1 tests to always pass for JDK 8u (bad
> signal/noise ratio). I'd be OK for Test8028623.java, but the
> compiler/rtm tests, I'm not so sure... Thoughts?

From my perspective, new test profile would inevitably show some (new) failures.

We would need to work to fix those failures before we can rely on low SNR in tier1. Ignoring tests
may skip over the tests that need attention during that transitional period. We don't ignore the
tests that are failing right now in fastdebug, right? Same thing here.

In other words, I would rather see tier1 suite to land, and then work on fixing/ignoring tests once
we understand how/if they are fixable.

-- 
Thanks,
-Aleksey