RFR(M): JDK-8028160: [TESTBUG] Exclude failing (runtime) jtreg tests using @ignore
christian.tornqvist at oracle.com
Tue Nov 26 12:39:55 PST 2013
This looks good, thanks for fixing this.
From: Mikhailo Seledtsov [mailto:mikhailo.seledtsov at oracle.com]
Sent: Monday, November 25, 2013 6:18 PM
To: Christian Tornqvist
Cc: 'Leonid Mesnik'; 'David Holmes'; 'hotspot-runtime-dev'; 'GEORGE_TRIANTAFILLOU'
Subject: Re: RFR(M): JDK-8028160: [TESTBUG] Exclude failing (runtime) jtreg tests using @ignore
The updated webrev has been posted at:
George has updated the bug numbers that follow the @ignore tag according to the feedback.
On 11/25/2013 1:32 PM, Christian Tornqvist wrote:
> Hi Leonid,
> The benefits of using @ignore instead of problemlist are many so we went with using @ignore instead of introducing a problemlist.txt for hotspot. This is something that can be discussed when looking at how to improve jtreg in JDK9 if we feel that this doesn't meet all our requirements.
> -----Original Message-----
> From: hotspot-runtime-dev-bounces at openjdk.java.net
> [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of
> Leonid Mesnik
> Sent: Monday, November 25, 2013 11:37 AM
> To: David Holmes
> Cc: hotspot-runtime-dev; GEORGE_TRIANTAFILLOU
> Subject: Re: RFR(M): JDK-8028160: [TESTBUG] Exclude failing (runtime)
> jtreg tests using @ignore
> I agree with Alan. I also wanted to ask about ProblemsList.txt for exclusion. However it seems we don't use it for hotspot.
> We could file RFE to make it and support on all levels of testing however it requires much more effort in different systems (JPRT, Aurora, make, other?).
> Does it make a sense to do it for in JDK9?
> On 11/25/2013 05:01 PM, Alan Bateman wrote:
>> On 25/11/2013 00:49, David Holmes wrote:
>>> I don't understand the reason for doing this. If the tests fail
>>> because of a genuine problem then there should be a bug for that
>>> problem and the test just becomes a "known failure". If that is
>>> undesirable why not just add them to the ProblemList.txt file?
>>> It seems wrong to me to have any tests marked as @ignore.
>> Looking at hotspot/test/Makefile then it looks like jtreg is not run
>> with an exclude list so it doesn't look like there is a way to
>> exclude tests that way. In any case, @ignore or an exclude list (like
>> ProblemList.txt) amount to the same thing in that the test is not run.
>> The advantage of an exclude list over @ignore is that you can exclude
>> tests on specific platforms (which is really useful for some of the
>> library tests).
> Leonid Mesnik
> JVM SQE
More information about the hotspot-runtime-dev