RFR: 8150145: javax/xml/jaxp/unittest/common/TransformationWarningsTest.java and ValidationWarningsTest.java failed intermittently without any error message

[Aleks Efimov]

On 31/08/16 23:55, Joe Wang wrote:
>
>
> On 8/31/16, 12:08 PM, Aleks Efimov wrote:
>> Joe,
>>
>> (answers in-lined)
>>
>>
>> On 31/08/16 21:39, Joe Wang wrote:
>>>
>>>
>>> On 8/31/16, 10:51 AM, Aleks Efimov wrote:
>>>> Hi Joe,
>>>>
>>>> Thank you for reviewing the changes. I found one more inconsistency 
>>>> with these tests:
>>>>
>>>> The TestSAXDriver class is not compiled by default now and because 
>>>> of that the default SAX driver were used and the original issue was 
>>>> not reproduced properly. New webrev can be found here:
>>>>
>>>> http://cr.openjdk.java.net/~aefimov/8150145/01
>>>>
>>>> About intermittent failure: The test was added with 'othervm' 
>>>> initially, but few months ago the transformation test was failing 
>>>> due to concurrency issue in PackageEntry::package_exports_do ( 
>>>> JDK-8152404). Now failures are not reproducible with latest JDK9 
>>>> builds and latest version of the tests.
>>>
>>> Did you mean it was later changed to othervm? The initial check-in 
>>> record shows it wasn't [1]
>>
>> Yes, you're right - it was added without othervm initially. I was 
>> looking at the different changeset *sigh*...
>
> Our memories can elude us, esp. after playing with it so many times :-)
>>
>>>
>>> othervm provents the potential interference by other tests that also 
>>> set the system property, and therefore removes the need for a 
>>> further refactoring of the code. So this is not required, but you 
>>> may remove the SAX driver setting and use a new instance instead, e.g.
>>>     SAXSource saxSource = new SAXSource(new TestSAXDriver(), new 
>>> InputSource(new StringReader(xml)));
>> Thanks for the suggestion. Even with all sources changed to 
>> SAXSource() and with explicitly specified TestSAXDriver  the warning 
>> generated by Validation test can't be reproduced. Hopefully, it is ok 
>> to leave "org.xml.sax.driver" property setup in-place.
>
> It's okay now it's running in othervm. I was saying that the system 
> property might have been the cause for the intermittent failure. I'm 
> not sure I understand what you meant that the Validation test can't be 
> reproduced. Did you mean no warning was generated or not more than 1 
> warning generated?
Ok, agree that without the othervm the system property might have been 
the cause of the intermittent failures.
By "Validation test can't be reproduced" I meant that 0 warnings were 
generated during ValidationWarningsTest run when the property setting 
was removed and all sources were replaced with
"new SAXSource(new TestSAXDriver() ..."

>
> Thanks,
> Joe
>
>>
>>>
>>> [1] 
>>> http://hg.openjdk.java.net/jdk9/jdk9/jaxp/file/6aa83d55614a/test/javax/xml/jaxp/unittest/common/TransformationWarningsTest.java
>>>
>>> Thanks,
>>> Joe
>>>>
>>>> -Aleksej
>>>>
>>>>
>>>> On 31/08/16 19:25, Joe Wang wrote:
>>>>> Hi Aleksej,
>>>>>
>>>>> It's good to put the tests back online. Thanks for the diligent 
>>>>> work! I believe the change that made the tests run in othervm 
>>>>> could have fixed the intermittent issue. But adding debugging code 
>>>>> can always be helpful in case of failures.
>>>>>
>>>>> Thanks,
>>>>> Joe
>>>>>
>>>>> On 8/30/16, 12:01 PM, Aleks Efimov wrote:
>>>>>> Hello,
>>>>>>
>>>>>> Please, help to review the tests fix.
>>>>>> Webrev: http://cr.openjdk.java.net/~aefimov/8150145/00/
>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8150145
>>>>>>
>>>>>> The list of changes:
>>>>>> 1. Two tests were modified to print exception occurred during 
>>>>>> execution. Before that these tests were failing silently.
>>>>>> 2. The tests were slightly modified to correctly run with 
>>>>>> security manager run mode recently added to JAXP tests 
>>>>>> (JDK-8067170).
>>>>>> 3. TransformationWarningsTest were modified to synchronize the 
>>>>>> TransformerFactory instantiation.
>>>>>>
>>>>>> The modified tests were executed 2000+ times (alongside to other 
>>>>>> tests from jaxp/test/javax/xml/jaxp/unittest/common) on linux-x64 
>>>>>> and there were no failures observed. According to this result and 
>>>>>> that the tests were modified not to fail silently. I would like 
>>>>>> to remove these two tests from the jaxp problem list.
>>>>>> If the tests will continue to fail on some configurations (JPRT 
>>>>>> shows no failures for few runs though) the proposed changes will 
>>>>>> help to diagnose the cause of failures.
>>>>>>
>>>>>> With Best Regards,
>>>>>> Aleksej
>>>>>>
>>>>
>>