Test policy follow-up, third testing tier
Joseph D. Darcy
joe.darcy at oracle.com
Wed Jun 24 21:52:43 UTC 2015
On 6/24/2015 2:00 PM, Alejandro E Murillo wrote:
>
>
> On 6/23/2015 4:48 PM, joe darcy wrote:
>> Hello,
>>
>> As a reminder, JDK 9 is now using a tiered testing policy where about
>> 9,000 tests have been placed into one of two tiers, with tier 1 tests
>> having a stricter policy on passing than tier 2. [1] The overall
>> policy is that tier 1 tests should always pass in master and that
>> integrations, both dev -> master and hotspot main -> dev, should
>> preserve the property that all tier 1 tests pass. In addition, only a
>> limited number of tier 2 tests should fail.
> Why is not everyone pushing to dev required to make sure tier1 is not
> broken?
In part, because we don't have a generally available pre-push build and
test system for the JDK. Second, the tree of integration forests is
designed to isolate dev (and dev is meant to isolate master) from bad
changes getting propagated. IMO, the notion of "bad change" should now
explicitly include tier 1 test failures. Since integrations are involved
procedures which already involve a lot of building and testing, I think
this is a reasonable constraint to include. That said, another part of
the policy is that if an bad individual changeset is pushed to dev by a
developer, the situation is expected to be addressed promptly, within
one day or less.
Thanks,
-Joe
More information about the jdk9-dev
mailing list