<Swing Dev> [12] RFR JDK-8213181:Updation of ProblemList.txt for removal of passing swing test

[Prasanta Sadhukhan]

On 04-Nov-18 12:06 AM, Philip Race wrote:
>
>
> On 11/2/18, 9:12 PM, Prasanta Sadhukhan wrote:
>> Ok. I can retain the 1st test javax/swing/border/Test6981576.java 
>> 8198339 generic-all
>
> So shouldn't this test just be marked @headful ?
I can mark it headful if Sergey is ok with it.
>>
>> Can I remove the rest?
If Sergey is not ok with removing the tests he has added
-javax/swing/border/Test6981576.java 8198339 generic-all*
javax/swing/UITest/UITest.java 8198392 generic-all
javax/swing/JCheckBox/8032667/bug8032667_image_diff.java 8199063 macosx-all*

can I just remove the tests I have added in last sprint and passed in 
this sprint (passed for me in 10.13 and manajit in 10.12, probably it 
fails last time due to some update dialog or antivirus popup)
javax/swing/JButton/8151303/PressedIconTest.java 8198689 macosx-all*
javax/swing/JWindow/ShapedAndTranslucentWindows/ShapedPerPixelTranslucentGradient.java 
8198667 macosx-all
>>
>> I guess there will be many more tests that will fail if we try to run 
>> for 100 iterations and Problemlist will swell, so I guess we should 
>> keep only those tests in ProblemList that fail if we run individually.
>
> Automated tests that are being run on a regular basis need to pass 
> reliably.
But we do not run each test in an batch for 100 iterations to find out 
if it is reliable (should we?).
  I guess we just run it individually in standalone mode and if it pass 
everytime, we consider it reliable, and all these 5 tests pass if run 
individually .

Regards
Prasanta
> We could of course look at why they are unreliable and fix.
> That is the actual goal here. More reliable automated tests.
>
> -phil.
>>
>> Regards
>> Prasanta
>> On 03-Nov-18 1:31 AM, Sergey Bylokhov wrote:
>>> These tests were added to the problem list, if they fail at least 
>>> once per the hundred executions.
>>>
>>> For example the first test which you remove from the list:
>>> javax/swing/border/Test6981576.java 8198339 generic-all*
>>>
>>> It creates a frame which may produce HeadlessException, but 
>>> sometimes the
>>> test completes successfully because it uses invokeLater()
>>> which is not necessary completes before the end of the main().
>>>
>>> On 02/11/2018 02:30, Prasanta Sadhukhan wrote:
>>>> Hi Sergey,
>>>>
>>>> Checked on mach5 and these tests passed.
>>>>
>>>> Regards
>>>> Prasanta
>>>> On 31-Oct-18 11:38 PM, Sergey Bylokhov wrote:
>>>>> Hi, Prasanta.
>>>>>
>>>>> Some of them were added to the problem list as part of the
>>>>> tests cleanup for the headless mode, which is used in mach5.
>>>>> Did you check that these tests works fine in headless and mach5?
>>>>>
>>>>> On 31/10/2018 02:52, Prasanta Sadhukhan wrote:
>>>>>> Hi All,
>>>>>>
>>>>>> Please review an updation of ProblemList for removal of tests 
>>>>>> which are now passing.
>>>>>>
>>>>>> These 5 tests are being removed from ProblemList as they were 
>>>>>> tested on windows7,10, ubuntu 16.04,18.04, mac 10.12/10.13 and 
>>>>>> oel7.5 and found to be passing.
>>>>>>
>>>>>> diff -r edc729e2ee36 test/jdk/ProblemList.txt
>>>>>> --- a/test/jdk/ProblemList.txt  Tue Oct 30 15:51:10 2018 -0700
>>>>>> +++ b/test/jdk/ProblemList.txt  Wed Oct 31 15:18:01 2018 +0530
>>>>>> @@ -717,16 +717,13 @@
>>>>>>
>>>>>>   com/sun/java/swing/plaf/windows/Test8173145.java 8198334 
>>>>>> windows-all
>>>>>>
>>>>>> *-javax/swing/border/Test6981576.java 8198339 generic-all*
>>>>>>   javax/swing/JComponent/7154030/bug7154030.java 7190978 generic-all
>>>>>> javax/swing/JComboBox/ConsumedKeyTest/ConsumedKeyTest.java 
>>>>>> 8067986 generic-all
>>>>>>   javax/swing/JComponent/6683775/bug6683775.java 8172337 generic-all
>>>>>>   javax/swing/JComboBox/6236162/bug6236162.java 8028707 
>>>>>> windows-all,macosx-all
>>>>>>   javax/swing/text/html/parser/Test8017492.java 8022535 generic-all
>>>>>> *-javax/swing/JButton/8151303/PressedIconTest.java 8198689 
>>>>>> macosx-all*
>>>>>> javax/swing/JWindow/ShapedAndTranslucentWindows/PerPixelTranslucentCanvas.java 
>>>>>> 8081476 windows-all,macosx-all
>>>>>> javax/swing/JWindow/ShapedAndTranslucentWindows/PerPixelTranslucentSwing.java 
>>>>>> 8194128 macosx-all
>>>>>> *-javax/swing/JWindow/ShapedAndTranslucentWindows/ShapedPerPixelTranslucentGradient.java 
>>>>>> 8198667 macosx-all*
>>>>>> javax/swing/JWindow/ShapedAndTranslucentWindows/SetShapeAndClickSwing.java 
>>>>>> 8013450 macosx-all
>>>>>> javax/swing/JWindow/ShapedAndTranslucentWindows/TranslucentJComboBox.java 
>>>>>> 8024627 macosx-all
>>>>>>   # The next test below is an intermittent failure
>>>>>> @@ -779,7 +776,6 @@
>>>>>>   javax/swing/JPopupMenu/4966112/bug4966112.java 8064915 macosx-all
>>>>>>   javax/swing/MultiUIDefaults/Test6860438.java 8198391 generic-all
>>>>>>   javax/swing/MultiUIDefaults/4300666/bug4300666.java 7105119 
>>>>>> macosx-all
>>>>>> *-javax/swing/UITest/UITest.java 8198392 generic-all*
>>>>>> javax/swing/plaf/basic/BasicComboBoxEditor/Test8015336.java 
>>>>>> 8198394 generic-all
>>>>>> javax/swing/plaf/metal/MetalLookAndFeel/Test8039750.java 8198395 
>>>>>> generic-all
>>>>>>   javax/swing/text/DevanagariEditor.java 8198397 generic-all
>>>>>> @@ -797,7 +793,6 @@
>>>>>> javax/swing/PopupFactory/6276087/NonOpaquePopupMenuTest.java 
>>>>>> 8065099 macosx-all
>>>>>>   javax/swing/DataTransfer/8059739/bug8059739.java 8199074 
>>>>>> generic-all
>>>>>>   javax/swing/GroupLayout/6613904/bug6613904.java 8199072 
>>>>>> generic-all
>>>>>> *-javax/swing/JCheckBox/8032667/bug8032667_image_diff.java 
>>>>>> 8199063 macosx-all*
>>>>>>   javax/swing/JComboBox/7031551/bug7031551.java 8199056 generic-all
>>>>>>   javax/swing/JScrollBar/6924059/bug6924059.java 8199078 generic-all
>>>>>>   javax/swing/JTree/8003830/bug8003830.java 8199057 generic-all
>>>>>>
>>>>>> Regards
>>>>>> Prasanta
>>>>>
>>>>>
>>>>
>>>
>>>
>>