jtreg adds some postfixes into test names

Vitaly Provodin vitaly.provodin at jetbrains.com
Thu Jun 1 03:08:28 UTC 2017


Hi Jon,

Thanks for your reply.

Unfortunately, I am not sure it is that case your wrote.
My particular case is the file containing one test description (one tag @test). The test was taken from http://hg.openjdk.java.net/jdk9/dev/jdk/file/148a2cbe49a4/test/java/awt/Focus/AutoRequestFocusTest/AutoRequestFocusSetVisibleTest.java <http://hg.openjdk.java.net/jdk9/dev/jdk/file/148a2cbe49a4/test/java/awt/Focus/AutoRequestFocusTest/AutoRequestFocusSetVisibleTest.java> and it was not changed.

$ cat test/java/awt/Focus/AutoRequestFocusTest/AutoRequestFocusSetVisibleTest.java | grep "@test"
  @test

I also tried to write in description something like

$ cat test/java/awt/Focus/AutoRequestFocusTest/AutoRequestFocusSetVisibleTest.java | grep "@test"
  @test      test description

in both cases there is only one tag @test and both cases produce the same result:

./jtreg/bin/jtreg -exclude:test/jbProblemsList.txt -ignore:quiet -v -a -xml -testjdk:jbsdk/jdk/Contents/Home test/java/awt/Focus/AutoRequestFocusTest/AutoRequestFocusSetVisibleTest.java
XML output  to /Users/builduser/9d35cf148e807375/JTwork
runner starting test: java/awt/Focus/AutoRequestFocusTest/AutoRequestFocusSetVisibleTest.java#id0
runner finished test: java/awt/Focus/AutoRequestFocusTest/AutoRequestFocusSetVisibleTest.java#id0
Failed. Execution failed: `main' threw exception: TestFailedException: Test failed: the window shouldn't gain focus but it did!
Test results: failed: 1
Report written to /Users/builduser/9d35cf148e807375/JTreport/html/report.html
Results written to /Users/builduser/9d35cf148e807375/JTwork
Error: Some tests failed or other problems occurred.


Thanks,
Vitaly


> On 31 May 2017, at 22:11, Jonathan Gibbons <jonathan.gibbons at oracle.com> wrote:
> 
> Test names are never changed during test execution.
> Postfixes, as you call them, are added when the test is read when you have multiple test descriptions (comments beginning @test) in the same file. 
> They've always been added for the 2nd and subsequent test descriptions in a file; the recent change is to add them for the first test in a file when the file has multiple tests. This is for uniformity and to be able to disambiguate between specifying "all the tests in a file" vs. "the first test in a file."
> 
> Note that putting multiple test descriptions in a single file has never been formally supported, because of the potential ambiguity that was (finally) resolved by the recent change.
> -- Jon
> 
> On 5/30/17 8:06 PM, Vitaly Provodin wrote:
>> Hi all,
>> 
>> Running JDK regression tests via jtreg I detected that names of some tests can be changed during test execution. Namely, some postfix is added at the end of test name like follows:
>> ===================================8<-------------------------------------------------------------
>> #Mon May 29 18:56:12 MSK 2017
>> #-----testdescription-----
>> $file=/opt/buildAgent/work/9d35cf148e807375/test/java/awt/Focus/AutoRequestFocusTest/AutoRequestFocusSetVisibleTest.java
>> $root=/opt/buildAgent/work/9d35cf148e807375/test
>> author=anton.tarasov\: area\=awt.focus
>> id=id0
>> keywords=bug6187066
>> library=../../regtesthelpers
>> run=USER_SPECIFIED build Util\nUSER_SPECIFIED main AutoRequestFocusSetVisibleTest\n
>> source=AutoRequestFocusSetVisibleTest.java
>> title=Tests the Window.autoRequestFocus property for the Window.setVisible() method.
>> 
>> #-----environment-----
>> 
>> #-----testresult-----
>> compileJDK=/Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home
>> description=file\:/opt/buildAgent/work/9d35cf148e807375/test/java/awt/Focus/AutoRequestFocusTest/AutoRequestFocusSetVisibleTest.java\#id0
>> elapsed=7463 0\:00\:07.463
>> end=Mon May 29 18\:56\:12 MSK 2017
>> environment=regtest
>> ===================================8<——————————————————————————————
>> 
>> And jtreg output:
>> runner starting test: java/awt/Focus/AutoRequestFocusTest/AutoRequestFocusSetVisibleTest.java#id0
>> runner finished test: java/awt/Focus/AutoRequestFocusTest/AutoRequestFocusSetVisibleTest.java#id0
>> Failed. Execution failed: `main' threw exception: TestFailedException: Test failed: the window shouldn't               gain focus but it did!
>> 
>> As you can see the test java/awt/Focus/AutoRequestFocusTest/AutoRequestFocusSetVisibleTest.java became java/awt/Focus/AutoRequestFocusTest/AutoRequestFocusSetVisibleTest.java\#id0 ("\#id0” was added).
>> Please note the issue was observed intermittently.
>> 
>> The matter was that this test was included into my exclude list, nevertheless it was executed because of this postfix (most probably new test name does not correspond to the name included into exclude list).
>> I looked through documentation on jtreg but could not find any explanations for this behaviour.
>> My suspicious was concurrent execution of the tests but I could not find any symptoms confirming that.
>> 
>> My original intentions was to exclude some tests from test runs but I could not do it because of this issue. I tried to use regular expressions in test names adding them into exclude list but it did not help.
>> 
>> Here is command line used to start test runs:
>> 
>> $ jtreg -exclude:test/jbProblemsList.txt -ignore:quiet -v -a -xml -compilejdk:'%env.JDK_18_x64%' -testjdk:'%jtreg.testjdk%' -timeoutFactor:2 test/java/awt || echo 'Execution complete’
>> 
>> Could you please help me in resolving the following questions? why does jtreg add some postfixes? how can I exclude such tests from test runs?
>> 
>> 
>> Thanks in advance, 
>> Vitaly
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/jtreg-use/attachments/20170601/7cbccb74/attachment.html>


More information about the jtreg-use mailing list