RFR(M): JDK-8028160: [TESTBUG] Exclude failing (runtime) jtreg tests using @ignore

[Christian Tornqvist]

Hi Leonid,

The benefits of using @ignore instead of problemlist are many so we went with using @ignore instead of introducing a problemlist.txt for hotspot. This is something that can be discussed when looking at how to improve jtreg in JDK9 if we feel that this doesn't meet all our requirements. 

Thanks,
Christian

-----Original Message-----
From: hotspot-runtime-dev-bounces at openjdk.java.net [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of Leonid Mesnik
Sent: Monday, November 25, 2013 11:37 AM
To: David Holmes
Cc: hotspot-runtime-dev; GEORGE_TRIANTAFILLOU
Subject: Re: RFR(M): JDK-8028160: [TESTBUG] Exclude failing (runtime) jtreg tests using @ignore

David

I agree with Alan. I also wanted to ask about ProblemsList.txt for exclusion. However it seems we don't use it for hotspot.
We could file RFE to make it and support on all levels of testing however it requires much more effort in different systems (JPRT, Aurora, make, other?).

Does it make a sense to do it for in JDK9?

Leonid
On 11/25/2013 05:01 PM, Alan Bateman wrote:
> On 25/11/2013 00:49, David Holmes wrote:
>> I don't understand the reason for doing this. If the tests fail 
>> because of a genuine problem then there should be a bug for that 
>> problem and the test just becomes a "known failure". If that is 
>> undesirable why not just add them to the ProblemList.txt file?
>>
>> It seems wrong to me to have any tests marked as @ignore.
> Looking at hotspot/test/Makefile then it looks like jtreg is not run 
> with an exclude list so it doesn't look like there is a way to exclude 
> tests that way. In any case, @ignore or an exclude list (like
> ProblemList.txt) amount to the same thing in that the test is not run. 
> The advantage of an exclude list over @ignore is that you can exclude 
> tests on specific platforms (which is really useful for some of the 
> library tests).
>
> -Alan.


--
Leonid Mesnik
JVM SQE