RFR(M): JDK-8028160: [TESTBUG] Exclude failing (runtime) jtreg tests using @ignore

[Christian Tornqvist]

Hi David,

The known failure is not really part of this, since this will only hide the failures when running in Aurora and not when running locally.

In Hotspot we don't have a ProblemList.txt and after some discussion we decided to not use it but rather use @ignore since you won't need to specify an exclude list when running the jtreg tests, keeping it as simple as possible.

Thanks,
Christian

-----Original Message-----
From: hotspot-runtime-dev-bounces at openjdk.java.net [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of David Holmes
Sent: Sunday, November 24, 2013 7:49 PM
To: Mikhailo Seledtsov
Cc: hotspot-runtime-dev; GEORGE_TRIANTAFILLOU
Subject: Re: RFR(M): JDK-8028160: [TESTBUG] Exclude failing (runtime) jtreg tests using @ignore

I don't understand the reason for doing this. If the tests fail because of a genuine problem then there should be a bug for that problem and the test just becomes a "known failure". If that is undesirable why not just add them to the ProblemList.txt file?

It seems wrong to me to have any tests marked as @ignore.

David

On 23/11/2013 8:28 AM, Mikhailo Seledtsov wrote:
>      This is a review request on behalf of George Triantafillou
>
>      JBS: https://bugs.openjdk.java.net/browse/JDK-8028160
>
> Webrev:http://cr.openjdk.java.net/~mseledtsov/sponsored/george.trianta
> fillou/8028160/webrev.00/ 
> <http://cr.openjdk.java.net/%7Emseledtsov/sponsored/george.triantafill
> ou/8028160/webrev.00/>
>
>      Please review this test stabilization change. The goal of this 
> change is to ignore all known test issues
>      in jtreg runtime tests (which will be fixed in the future), in 
> order to achieve a full pass of jtreg runtime tests
>      Each of the ignored tests has a corresponding bug filed, which is
> p4 or p5 and is scheduled to be fixed beyond jdk8 release (jdk8 update 
> X or jdk9).
>
>      Please note that there 2 jtreg runtime tests that are known to 
> fail and were not ignored, because there is a product (non-test) bug 
> associated with these tests.
>
> Thank you,
> Misha