Test policy follow-up, third testing tier

[Joseph D. Darcy]

On 6/24/2015 2:00 PM, Alejandro E Murillo wrote:
>
>
> On 6/23/2015 4:48 PM, joe darcy wrote:
>> Hello,
>>
>> As a reminder, JDK 9 is now using a tiered testing policy where about 
>> 9,000 tests have been placed into one of two tiers, with tier 1 tests 
>> having a stricter policy on passing than tier 2. [1] The overall 
>> policy is that tier 1 tests should always pass in master and that 
>> integrations, both dev -> master and hotspot main -> dev, should 
>> preserve the property that all tier 1 tests pass. In addition, only a 
>> limited number of tier 2 tests should fail.
> Why is not everyone pushing to dev required to make sure tier1 is not 
> broken?

In part, because we don't have a generally available pre-push build and 
test system for the JDK. Second, the tree of integration forests is 
designed to isolate dev (and dev is meant to isolate master) from bad 
changes getting propagated. IMO, the notion of "bad change" should now 
explicitly include tier 1 test failures. Since integrations are involved 
procedures which already involve a lot of building and testing, I think 
this is a reasonable constraint to include. That said, another part of 
the policy is that if an bad individual changeset is pushed to dev by a 
developer, the situation is expected to be addressed promptly, within 
one day or less.

Thanks,

-Joe