[8u] RFR: 8222737: [TESTBUG] Allow for tier 1 like testing in OpenJDK 8u

[Severin Gehwolf]

On Thu, 2019-06-27 at 13:27 +0200, Aleksey Shipilev wrote:
> On 6/27/19 1:20 PM, Severin Gehwolf wrote:
> > On Thu, 2019-06-27 at 11:58 +0200, Aleksey Shipilev wrote:
> > > On 5/22/19 6:34 PM, Severin Gehwolf wrote:
> > > hotspot:
> > > 
> > > *) Test exclusion question. Do we really want to exclude these tests from tier1? I would rather see
> > > them included and make them fail in regular runs. This would put pressure on fixing them for 8u...
> > > 
> > >  150 # Right now tier1 runs all hotspot tests except RTM tests,
> > >  151 # and one serviceability test which seems flakey.
> > >  152 hotspot_tier1 = \
> > >  153   :jdk \
> > >  154   -compiler/rtm \
> > >  155   -serviceability/sa/jmap-hashcode/Test8028623.java
> > 
> > I'm not sure about it. We could exclude nothing and go from there.
> > However, the more tests we have failing from the get-go the less useful
> > it is to enforce tier1 tests to always pass for JDK 8u (bad
> > signal/noise ratio). I'd be OK for Test8028623.java, but the
> > compiler/rtm tests, I'm not so sure... Thoughts?
> 
> From my perspective, new test profile would inevitably show some (new) failures.
> 
> We would need to work to fix those failures before we can rely on low SNR in tier1. Ignoring tests
> may skip over the tests that need attention during that transitional period. We don't ignore the
> tests that are failing right now in fastdebug, right? Same thing here.
> 
> In other words, I would rather see tier1 suite to land, and then work on fixing/ignoring tests once
> we understand how/if they are fixable.

OK. Updated langtools/hotspot webrevs with no exclusions. jdk had no
exclusions to begin with:

http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/03/langtools/webrev/
http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/03/hotspot/webrev/

Thoughts?

Thanks,
Severin