<Swing Dev> [12] RFR JDK-8213181:Updation of ProblemList.txt for removal of passing swing test

Sergey Bylokhov sergey.bylokhov at oracle.com
Tue Nov 6 00:59:04 UTC 2018


> > So shouldn't this test just be marked @headful ?
> I can mark it headful if Sergey is ok with it.

This will skip the test in our nightly, but actually does not solve the existed bug in the test.

> fails last time due to some update dialog or antivirus popup)
> javax/swing/JButton/8151303/PressedIconTest.java 8198689 macosx-all*
> javax/swing/JWindow/ShapedAndTranslucentWindows/ShapedPerPixelTranslucentGradient.java

> >> I guess there will be many more tests that will fail if we try to
> run 
> >> for 100 iterations and Problemlist will swell, so I guess we should
> 
> >> keep only those tests in ProblemList that fail if we run
> individually.

The ProblemList excludes the tests from the nightly and testing after commit,
the rest of tests(which are headless and not in this list) are quite stable.

> > Automated tests that are being run on a regular basis need to pass 
> > reliably.
> But we do not run each test in an batch for 100 iterations to find out
> 
> if it is reliable (should we?).

I do that, our sqe run the tests for 1000 iterations.


>   I guess we just run it individually in standalone mode and if it
> pass 
> everytime, we consider it reliable, and all these 5 tests pass if run


I guess the problem here is that these tests were added to the problem list without
investigation of root cause, at least It was not checked that it was a setup issue like wrong "Display color profile".

As a solution I suggest to check the source code of the tests, and try to find some issues which may affect
the stability of the tests, then run these tests in a bash loop for 100 iterations to prove that they
really stable.


More information about the swing-dev mailing list