<Swing Dev> [12] RFR JDK-8213181:Updation of ProblemList.txt for removal of passing swing test

[Prasanta Sadhukhan]

Since some of the tests are marked generic-all and it's difficult to run 
100 iterations on all platforms, I concentrated on the tests that are 
marked failing on one platform.

I found that these tests are passing when ran for 100 iterations on windows

javax/swing/JComboBox/6607130/bug6607130.java
javax/swing/JPopupMenu/6675802/bug6675802.java
javax/swing/plaf/basic/BasicTextUI/8001470/bug8001470.java
javax/swing/plaf/basic/Test6984643.java
javax/swing/text/JTextComponent/5074573/bug5074573.java
javax/swing/SwingWorker/6493680/bug6493680.java
javax/swing/UIDefaults/8149879/InternalResourceBundle.java
javax/swing/PopupFactory/8048506/bug8048506.java

and these on mac

javax/swing/JButton/8151303/PressedIconTest.java
javax/swing/JWindow/ShapedAndTranslucentWindows/ShapedPerPixelTranslucentGradient.java

Proposed webrev:
http://cr.openjdk.java.net/~psadhukhan/8213181/webrev.0/

Regards
Prasanta
On 06-Nov-18 6:29 AM, Sergey Bylokhov wrote:
>>> So shouldn't this test just be marked @headful ?
>> I can mark it headful if Sergey is ok with it.
> This will skip the test in our nightly, but actually does not solve the existed bug in the test.
>
>> fails last time due to some update dialog or antivirus popup)
>> javax/swing/JButton/8151303/PressedIconTest.java 8198689 macosx-all*
>> javax/swing/JWindow/ShapedAndTranslucentWindows/ShapedPerPixelTranslucentGradient.java
>>>> I guess there will be many more tests that will fail if we try to
>> run
>>>> for 100 iterations and Problemlist will swell, so I guess we should
>>>> keep only those tests in ProblemList that fail if we run
>> individually.
> The ProblemList excludes the tests from the nightly and testing after commit,
> the rest of tests(which are headless and not in this list) are quite stable.
>
>>> Automated tests that are being run on a regular basis need to pass
>>> reliably.
>> But we do not run each test in an batch for 100 iterations to find out
>>
>> if it is reliable (should we?).
> I do that, our sqe run the tests for 1000 iterations.
>
>
>>    I guess we just run it individually in standalone mode and if it
>> pass
>> everytime, we consider it reliable, and all these 5 tests pass if run
>
> I guess the problem here is that these tests were added to the problem list without
> investigation of root cause, at least It was not checked that it was a setup issue like wrong "Display color profile".
>
> As a solution I suggest to check the source code of the tests, and try to find some issues which may affect
> the stability of the tests, then run these tests in a bash loop for 100 iterations to prove that they
> really stable.