Intermittent test failure

[Paul Sandoz]

On Mar 16, 2013, at 1:51 AM, Jonathan Gibbons <jonathan.gibbons at oracle.com> wrote:
> The following lambda test is failing intermittently:
> 
> tests/org/openjdk/tests/java/util/stream/SpliteratorLateBindingFailTestTest.java
> 

Oh, i have not observed that, will try to dig deeper to find the source of non-determinism.


> I've attached the .jtr file from when it failed. There's a fair amount of tracing info switched on because I've been working on jtreg, but the interesting lines are:
> 

We need better reporting of results when using data providers. I cannot tell which sub-test are failing because the parameter values passed to the test are not logged. The "description" parameter is the important one.

In the test ng xml output for example:

          <params>
            <param index="0">
              <value>
                <![CDATA[new java.util.ArrayDeque.spliterator() REMOVE {size=4}]]>
              </value>
            </param>
            <param index="1">
              <value>
                <![CDATA[org.openjdk.tests.java.util.stream.SpliteratorLateBindingFailTestTest$SpliteratorDataBuilder$$Lambda$3 at 2d38eb89]]>
              </value>
            </param>
          </params>


> 
> test public void org.openjdk.tests.java.util.stream.SpliteratorLateBindingFailTestTest.testFailFastTestWithTryAdvance(java.lang.String,java.util.function.Supplier): failure
> java.lang.AssertionError: No Exception was thrown, expected an Exception of java.util.ConcurrentModificationException to be thrown expected object to not be null
>    at org.testng.Assert.fail(Assert.java:89)
>    at org.testng.Assert.assertNotNull(Assert.java:399)
>    at org.openjdk.tests.java.util.stream.SpliteratorLateBindingFailTestTest.executeAndCatch(SpliteratorLateBindingFailTestTest.java:340)
>    at org.openjdk.tests.java.util.stream.SpliteratorLateBindingFailTestTest.executeAndCatch(SpliteratorLateBindingFailTestTest.java:328)
>    at org.openjdk.tests.java.util.stream.SpliteratorLateBindingFailTestTest.testFailFastTestWithTryAdvance(SpliteratorLateBindingFailTestTest.java:275)
>    at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>    at java.lang.reflect.Method.invoke(Method.java:487)
>    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
> 
> As a minor point, it would also help if there was more grammatical assistance to parser the text of the AssertionError message :-)
> 

Grrr... it's annoying the way test ng appends stuff to the assertion message.

Paul.