RFR: 8150145: javax/xml/jaxp/unittest/common/TransformationWarningsTest.java and ValidationWarningsTest.java failed intermittently without any error message

[Joe Wang]

Hi Aleksej,

Yes, the change looks good with the System Property and StreamSource. As 
we discussed, apparently the processor won't issue warnings in other 
cases (with SAXSource). Thanks for being patient with me, going back and 
forth with test runs.

Cheers,
Joe

On 8/31/16, 2:58 PM, Aleks Efimov wrote:
> On 31/08/16 23:55, Joe Wang wrote:
>>
>>
>> On 8/31/16, 12:08 PM, Aleks Efimov wrote:
>>> Joe,
>>>
>>> (answers in-lined)
>>>
>>>
>>> On 31/08/16 21:39, Joe Wang wrote:
>>>>
>>>>
>>>> On 8/31/16, 10:51 AM, Aleks Efimov wrote:
>>>>> Hi Joe,
>>>>>
>>>>> Thank you for reviewing the changes. I found one more 
>>>>> inconsistency with these tests:
>>>>>
>>>>> The TestSAXDriver class is not compiled by default now and because 
>>>>> of that the default SAX driver were used and the original issue 
>>>>> was not reproduced properly. New webrev can be found here:
>>>>>
>>>>> http://cr.openjdk.java.net/~aefimov/8150145/01
>>>>>
>>>>> About intermittent failure: The test was added with 'othervm' 
>>>>> initially, but few months ago the transformation test was failing 
>>>>> due to concurrency issue in PackageEntry::package_exports_do ( 
>>>>> JDK-8152404). Now failures are not reproducible with latest JDK9 
>>>>> builds and latest version of the tests.
>>>>
>>>> Did you mean it was later changed to othervm? The initial check-in 
>>>> record shows it wasn't [1]
>>>
>>> Yes, you're right - it was added without othervm initially. I was 
>>> looking at the different changeset *sigh*...
>>
>> Our memories can elude us, esp. after playing with it so many times :-)
>>>
>>>>
>>>> othervm provents the potential interference by other tests that 
>>>> also set the system property, and therefore removes the need for a 
>>>> further refactoring of the code. So this is not required, but you 
>>>> may remove the SAX driver setting and use a new instance instead, e.g.
>>>>     SAXSource saxSource = new SAXSource(new TestSAXDriver(), new 
>>>> InputSource(new StringReader(xml)));
>>> Thanks for the suggestion. Even with all sources changed to 
>>> SAXSource() and with explicitly specified TestSAXDriver  the warning 
>>> generated by Validation test can't be reproduced. Hopefully, it is 
>>> ok to leave "org.xml.sax.driver" property setup in-place.
>>
>> It's okay now it's running in othervm. I was saying that the system 
>> property might have been the cause for the intermittent failure. I'm 
>> not sure I understand what you meant that the Validation test can't 
>> be reproduced. Did you mean no warning was generated or not more than 
>> 1 warning generated?
> Ok, agree that without the othervm the system property might have been 
> the cause of the intermittent failures.
> By "Validation test can't be reproduced" I meant that 0 warnings were 
> generated during ValidationWarningsTest run when the property setting 
> was removed and all sources were replaced with
> "new SAXSource(new TestSAXDriver() ..."
>
>>
>> Thanks,
>> Joe
>>
>>>
>>>>
>>>> [1] 
>>>> http://hg.openjdk.java.net/jdk9/jdk9/jaxp/file/6aa83d55614a/test/javax/xml/jaxp/unittest/common/TransformationWarningsTest.java
>>>>
>>>> Thanks,
>>>> Joe
>>>>>
>>>>> -Aleksej
>>>>>
>>>>>
>>>>> On 31/08/16 19:25, Joe Wang wrote:
>>>>>> Hi Aleksej,
>>>>>>
>>>>>> It's good to put the tests back online. Thanks for the diligent 
>>>>>> work! I believe the change that made the tests run in othervm 
>>>>>> could have fixed the intermittent issue. But adding debugging 
>>>>>> code can always be helpful in case of failures.
>>>>>>
>>>>>> Thanks,
>>>>>> Joe
>>>>>>
>>>>>> On 8/30/16, 12:01 PM, Aleks Efimov wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> Please, help to review the tests fix.
>>>>>>> Webrev: http://cr.openjdk.java.net/~aefimov/8150145/00/
>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8150145
>>>>>>>
>>>>>>> The list of changes:
>>>>>>> 1. Two tests were modified to print exception occurred during 
>>>>>>> execution. Before that these tests were failing silently.
>>>>>>> 2. The tests were slightly modified to correctly run with 
>>>>>>> security manager run mode recently added to JAXP tests 
>>>>>>> (JDK-8067170).
>>>>>>> 3. TransformationWarningsTest were modified to synchronize the 
>>>>>>> TransformerFactory instantiation.
>>>>>>>
>>>>>>> The modified tests were executed 2000+ times (alongside to other 
>>>>>>> tests from jaxp/test/javax/xml/jaxp/unittest/common) on 
>>>>>>> linux-x64 and there were no failures observed. According to this 
>>>>>>> result and that the tests were modified not to fail silently. I 
>>>>>>> would like to remove these two tests from the jaxp problem list.
>>>>>>> If the tests will continue to fail on some configurations (JPRT 
>>>>>>> shows no failures for few runs though) the proposed changes will 
>>>>>>> help to diagnose the cause of failures.
>>>>>>>
>>>>>>> With Best Regards,
>>>>>>> Aleksej
>>>>>>>
>>>>>
>>>
>