stable subset of jtreg tests?

[Joseph D. Darcy]

PS For reference, see the tiered testing policy discussion for JDK 9 
earlier this year:

     "Proposed new policies for JDK 9 regression tests: tiered testing, 
intermittent failures, and randomness,"
http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-March/001991.html

and the subsequent refinements and amendments:

     "Proposed new policies for JDK 9 regression tests: tiered testing, 
intermittent failures, and randomness,"
http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-April/002164.html

     "Test policy follow-up, third testing tier,"
http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-June/002325.html

Implementing these polices has found and corrected a number of lingering 
test and product issues including:

     JDK-8022224: Rare bug in JISAutodetect charset detected by 
FindDecoderBugs test
     JDK-6854417: TESTBUG: java/util/regex/RegExTest.java fails 
intermittently

Cheers,

-Joe

On 9/16/2015 5:25 PM, Joseph D. Darcy wrote:
> Dropping the jtreg alias.
>
> On 9/16/2015 5:18 PM, Martin Buchholz wrote:
>> Just now for the first time I ran all the jtreg tests in the 
>> "jdk_stable" test set using
>> make test TEST=jdk_stable
>> and I got around 50 failures out of 5000.
>> Is that expected?
>>
>> Instead, I expect that with all the work on the test suite y'all have 
>> been doing, that the expected failure rate of something with a name 
>> like "jdk_stable" would be 0.
>>
>
> For JDK 9 dev, the tier 1 tests across repos should be stable. 
> Currently, only the langtools and jdk repos have non-empty tier 1 
> tests sets defined.
>
> If you run
>
>     jtreg ... -exclude:ProblemList.txt -k:intermittent :tier1
>
> there are some tier 1 test that are known to fail intermittently, but 
> at this point with very low frequency (at least with the testing I'm 
> exposed to).
>
> HTH,
>
> -Joe
>