[rfc][icedtea-web] KnownToFailInBrowsers annotation patch
Jiri Vanek
jvanek at redhat.com
Wed Apr 3 05:58:38 PDT 2013
I think this can go in as it is. Just minor nits which you can apply before push.
Please also apply the new annotation where valid (as new changeset). you can push this without review.
On 03/18/2013 04:11 PM, Jana Fabrikova wrote:
> Hi Jiri and Adam,
>
> as Jiri suggested I have modified the annotation (this time no new annotation KnownToFailInBrowsers,
> only a new optional parameter Browsers[] failsIn() added to the old KnownToFail annotation).
>
> I have modified the JunitLikeXmlOutputListener and LessVerboseTextListener also, they identify the
> combinations of testcase+browser that are declared to fail and write known-to-fail="true" into the
> xml file, which is then processed by the same xslt as usual.
>
> I have run one reproducer (modification of JToJSEval) in opera, midori and epiphany with a
> combination of failing/passing testcases.
>
> The results in terminal were (screenshot1):
>
> -For the testcase that really failed in midori and epiphany but was annotated with
> @KnownToFail(failsIn={}) :
>
> Passed: JToJSEvalTest.AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test - opera
> - WARNING This test is known to fail, but have passed!
> FAILED: AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test - midori(JToJSEvalTest) JToJSEval: the J
> and JS outputs are not equal!
> - This test is known to fail
> FAILED: AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test - epiphany(JToJSEvalTest) JToJSEval: the J
> and JS outputs are not equal!
> - This test is known to fail
>
> -For the testcase that really failed in midori and epiphany but was annotated with @KnownToFail :
>
> Passed: JToJSEvalTest.AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test - opera
> - WARNING This test is known to fail, but have passed!
> FAILED: AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test - midori(JToJSEvalTest) JToJSEval:
> the J and JS outputs are not equal!
> - This test is known to fail
> FAILED: AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test - epiphany(JToJSEvalTest)
> JToJSEval: the J and JS outputs are not equal!
> - This test is known to fail
>
> -For the testcase that really failed in midori and epiphany and was annotated with
> @KnownToFail(failsIn={Browsers.midori, Browsers.epiphany})
>
> Passed: JToJSEvalTest.AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test - opera
> FAILED: AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test - midori(JToJSEvalTest) JToJSEval: the J and
> JS outputs are not equal!
> - This test is known to fail
> FAILED: AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test - epiphany(JToJSEvalTest) JToJSEval: the J
> and JS outputs are not equal!
> - This test is known to fail
>
> -For the testcase that really failed in opera (only because opera crashed, it does not normally
> fail) and was not annotated:
>
> FAILED: AppletJToJSEval_typeof_Test - opera(JToJSEvalTest) JToJSEvalTest stdout should contain
> JToJSEval applet initialized. but it didnt.
> Passed: JToJSEvalTest.AppletJToJSEval_typeof_Test - midori
> Passed: JToJSEvalTest.AppletJToJSEval_typeof_Test - epiphany
>
> -For the testcase that passes in all browsers and was not annotated:
>
> Passed: JToJSEvalTest.AppletJToJSEval_location_Test - opera
> Passed: JToJSEvalTest.AppletJToJSEval_location_Test - midori
> Passed: JToJSEvalTest.AppletJToJSEval_location_Test - epiphany
>
>
> -Total summary:
>
> Total tests run: 15; From those : 8 known to fail
> Test known to fail: passed: 2; failed: 6; ignored: 0
> Test results: passed: 8; failed: 7; ignored: 0
>
>
> The results in html are attached as index_reproducers.html and screenshot2).
>
>
> Please see the attached patch,
> cheers,
> Jana
>
>
> On 03/13/2013 05:45 PM, Jiri Vanek wrote:
>> On 03/13/2013 05:13 PM, Jana Fabrikova wrote:
>>> Hello,
>>>
>>> I am sending for your consideration a patch that introduces the
>>> @KnownToFailInBrowsers annotation, which may be useful in some of the
>>> reproducers (namely passing arrays from JS->J does work in opera and
>>> firefox and does not work in google-chrome, chromium, midori and
>>> epiphany, and i think it is more problem of the browsers than of
>>> icedtea-web plugin).
>>>
>>> The annotation is detected in
>>> junit-runner/JunitLikeXmlOutputListener.java and
>>> junit-runner/LessVerboseTextListener.java and reflected in the console
>>> and xml output of run-netx-dist-tests.
>>>
>>> Cheers,
>>> Jana
>>>
>>> 2013-03-13 Jana Fabrikova <jfabriko at redhat.com>
>>>
>>> *
>>> /tests/test-extensions/net/sourceforge/jnlp/annotations/KnownToFailInBrowsers.java
>>>
>>> the implementation of new annotation, which has an array of
>>> browsers of type Browsers[] named failsIn
>>>
>>> * /tests/junit-runner/JunitLikeXmlOutputListener.java
>>> in method (testDone) the testcases that are known to fail in
>>> current browser are detected in addition to the tests that are
>>> k2f in all browsers
>>>
>>> * /tests/junit-runner/LessVerboseTextListener.java
>>> added method (getK2FinB) reading the annotion,
>>> in method (printK2F) the testcases that are known to fail in
>>> current browser are detected in addition to the tests that are
>>> k2f in all browsers
>>
>> hmhm... I'm not sure If new type of annotation is needed. I would prefer
>> extension of KnownTofail as it is.
>> Now it have no parameter. In your new implementation it will have
>> optional Browsers[] failsIn(). If it will be null/empty, then it will
>> behave as we are used now. Otherwise it will be more sophisticated
>> What do you think?
>>
>> For the implementation itself - uless I missed something, I'm terribly
>> missing the browser recognition handling in xml/plaintext generation
>> and in xslt sheet.
>> Also some example usage of annotaion would be nice and as a top candy
>> the screenshots of outputs can be attached.
>>
>>
>> J.
>
>
> annotk2f-version2.patch
>
>
> diff --git a/tests/junit-runner/JunitLikeXmlOutputListener.java b/tests/junit-runner/JunitLikeXmlOutputListener.java
> --- a/tests/junit-runner/JunitLikeXmlOutputListener.java
> +++ b/tests/junit-runner/JunitLikeXmlOutputListener.java
> @@ -20,10 +20,12 @@
> import java.util.Map;
> import java.util.Map.Entry;
> import java.util.Set;
> +import java.util.Arrays;
> import java.util.concurrent.TimeUnit;
> import net.sourceforge.jnlp.annotations.Bug;
> import net.sourceforge.jnlp.annotations.KnownToFail;
> import net.sourceforge.jnlp.annotations.Remote;
> +import net.sourceforge.jnlp.browsertesting.Browsers;
>
>
> import org.junit.internal.JUnitSystem;
> @@ -201,17 +203,30 @@
> testcaseAtts.put(TEST_IGNORED_ATTRIBUTE, Boolean.TRUE.toString());
> }
> KnownToFail k2f = LessVerboseTextListener.getAnnotation(testClass, testMethod.getName(), KnownToFail.class);
> + boolean knownToFailInThisBrowser = false;
Is this name correct? I think it mark that test fails. No matter if "for all" or "for one browser".
So maybe better name?
> Remote remote = LessVerboseTextListener.getAnnotation(testClass, testMethod.getName(), Remote.class);
> - if (k2f != null) {
> - testcaseAtts.put(K2F, Boolean.TRUE.toString());
> - }
> + if (k2f != null) {
> + //determine if k2f in the current browser
> +
> + Browsers[] br = k2f.failsIn();
> + if(0 == br.length){//the KnownToFail annotation without optional parameter
> + knownToFailInThisBrowser = true;
> + }else{
> + for(Browsers b : br){
> + if(description.toString().contains(b.toString())){
> + knownToFailInThisBrowser = true;
> + }
> + }
> + }
> + }
> + if( knownToFailInThisBrowser ) testcaseAtts.put(K2F, Boolean.TRUE.toString());
> if (remote != null) {
> testcaseAtts.put(REMOTE, Boolean.TRUE.toString());
>
> }
> openElement(TEST_ELEMENT, testcaseAtts);
> if (testFailed != null) {
> - if (k2f != null) {
> + if (knownToFailInThisBrowser) {
> failedK2F++;
> }
> Map<String, String> errorAtts = new HashMap<String, String>(3);
> @@ -226,7 +241,7 @@
>
> writeElement(TEST_ERROR_ELEMENT, testFailed.getTrace(), errorAtts);
> } else {
> - if (k2f != null) {
> + if (knownToFailInThisBrowser) {
> if (ignored) {
> ignoredK2F++;
> } else {
> @@ -265,25 +280,25 @@
> classStats.put(description.getClassName(), classStat);
> }
> classStat.total++;
> - if (k2f != null) {
> + if (knownToFailInThisBrowser) {
> classStat.totalK2F++;
> }
> classStat.time += testTime;
> if (testFailed == null) {
> if (ignored) {
> classStat.ignored++;
> - if (k2f != null) {
> + if (knownToFailInThisBrowser) {
> classStat.ignoredK2F++;
> }
> } else {
> classStat.passed++;
> - if (k2f != null) {
> + if (knownToFailInThisBrowser) {
> classStat.passedK2F++;
> }
> }
> } else {
> classStat.failed++;
> - if (k2f != null) {
> + if (knownToFailInThisBrowser) {
> classStat.failedK2F++;
> }
> }
> diff --git a/tests/junit-runner/LessVerboseTextListener.java b/tests/junit-runner/LessVerboseTextListener.java
> --- a/tests/junit-runner/LessVerboseTextListener.java
> +++ b/tests/junit-runner/LessVerboseTextListener.java
> @@ -10,6 +10,7 @@
> import java.lang.reflect.Method;
> import net.sourceforge.jnlp.annotations.KnownToFail;
> import net.sourceforge.jnlp.annotations.Remote;
> +import net.sourceforge.jnlp.browsertesting.Browsers;
>
> import org.junit.internal.JUnitSystem;
> import org.junit.runner.Description;
> @@ -74,24 +75,39 @@
> private void printK2F(PrintStream writer, Boolean failed, Description description) {
> try {
> KnownToFail k2f = getK2F(description);
> - if (k2f != null) {
> - totalK2F++;
> - if (failed != null) {
> - if (failed) {
> - failedK2F++;
> - } else {
> - passedK2F++;
> - }
> - } else {
> - ignoredK2F++;
> - }
> - if (failed != null && !failed) {
> - writer.println(" - WARNING This test is known to fail, but have passed!");
> - } else {
> - writer.println(" - This test is known to fail");
> - }
> + boolean knownToFailInThisBrowser = false;
again.
> + if (k2f != null){
> + //determine if k2f in the current browser
> + Browsers[] br = k2f.failsIn();
> + if(0 == br.length){ //@KnownToFail with default optional parameter failsIn={}
> + knownToFailInThisBrowser = true;
> + }else{
> + for(Browsers b : br){
> + if(description.toString().contains(b.toString())){
> + knownToFailInThisBrowser = true;
> + }
> + }
> + }
> }
>
> + if( knownToFailInThisBrowser ){
> + totalK2F++;
> + if (failed != null) {
> + if (failed) {
> + failedK2F++;
> + } else {
> + passedK2F++;
> + }
> + } else {
> + ignoredK2F++;
> + }
> + if (failed != null && !failed) {
> + writer.println(" - WARNING This test is known to fail, but have passed!");
> + } else {
> + writer.println(" - This test is known to fail");
> + }
> + }
> +
>
> } catch (Exception ex) {
> ex.printStackTrace();
>
> diff --git a/tests/test-extensions/net/sourceforge/jnlp/annotations/KnownToFail.java b/tests/test-extensions/net/sourceforge/jnlp/annotations/KnownToFail.java
> --- a/tests/test-extensions/net/sourceforge/jnlp/annotations/KnownToFail.java
> +++ b/tests/test-extensions/net/sourceforge/jnlp/annotations/KnownToFail.java
> @@ -41,6 +41,7 @@
> import java.lang.annotation.Retention;
> import java.lang.annotation.RetentionPolicy;
> import java.lang.annotation.Target;
> +import net.sourceforge.jnlp.browsertesting.Browsers;
>
> /**
> * <p>
> @@ -52,10 +53,15 @@
> * This annotation is meant for adding tests for bugs before the fix is
> * implemented.
> * </p>
> + * <p>
> + * The meaning of optional parameter failsIn is either a list of
> + * browsers where the test fails, or a default value - an empty array {},
> + * default value means that the test fails always.
> + * </p>
> */
>
> @Target({ElementType.METHOD,ElementType.TYPE})
> @Retention(RetentionPolicy.RUNTIME)
> public @interface KnownToFail {
> -
> + public Browsers[] failsIn() default {};
> }
>
>
> index_reproducers.html
>
>
> Date:
>
> Mon Mar 18 15:35:34 CET 2013
>
>
> Result: (88s)
>
>
> In brackets are KnownToFail values if any
>
> TOTAL:
> 15 (8)
> passed:
> 8 (2)
> failed:
> 7 (6)
> ignored:
> 0
>
>
> Classes: show/hide
>
> JToJSEvalTest (88272ms):
> <imap://jvanek@redhat.com:993/fetch%3EUID%3E/mlists/disto-pkg-dev%3E274267#JToJSEvalTest>
>
> TOTAL:
> 15 (8)
> passed:
> 8 (2)
> failed:
> 7 (6)
> ignored:
> 0
>
> ----------------------------------------------------------------------------------------------------
>
>
> Individual results:
>
> NoneTraceAllTraces
> JToJSEvalTest
> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test%20-%20opera>
> -
> AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test - opera
> PASSED (4.9260s) *- This test is known to fail*
> JToJSEvalTest
> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test%20-%20midori>
> -
> AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test - midori
> FAILED (4.5260s) *- This test is known to fail*
> java.lang.AssertionError - JToJSEval: the J and JS outputs are not equal!StackTrace
>
> java.lang.AssertionError: JToJSEval: the J and JS outputs are not equal!
> at org.junit.Assert.fail(Assert.java:91)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at JToJSEvalTest.evaluateStdoutContents(JToJSEvalTest.java:80)
> at JToJSEvalTest.javaToJSEvalTest(JToJSEvalTest.java:87)
> at JToJSEvalTest.AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test(JToJSEvalTest.java:102)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runLeaf(BrowserTestRunner.java:162)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChildX(BrowserTestRunner.java:137)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:96)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:58)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
> at CommandLine.runMain(CommandLine.java:49)
> at CommandLine.runMainAndExit(CommandLine.java:28)
> at CommandLine.main(CommandLine.java:24)
>
> JToJSEvalTest
> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test%20-%20epiphany>
> -
> AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test - epiphany
> FAILED (4.5160s) *- This test is known to fail*
> java.lang.AssertionError - JToJSEval: the J and JS outputs are not equal!StackTrace
>
> java.lang.AssertionError: JToJSEval: the J and JS outputs are not equal!
> at org.junit.Assert.fail(Assert.java:91)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at JToJSEvalTest.evaluateStdoutContents(JToJSEvalTest.java:80)
> at JToJSEvalTest.javaToJSEvalTest(JToJSEvalTest.java:87)
> at JToJSEvalTest.AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test(JToJSEvalTest.java:102)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runLeaf(BrowserTestRunner.java:162)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChildX(BrowserTestRunner.java:137)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:96)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:58)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
> at CommandLine.runMain(CommandLine.java:49)
> at CommandLine.runMainAndExit(CommandLine.java:28)
> at CommandLine.main(CommandLine.java:24)
>
> JToJSEvalTest
> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test%20-%20opera>
> -
> AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test - opera
> PASSED (5.1110s) *- This test is known to fail*
> JToJSEvalTest
> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test%20-%20midori>
> -
> AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test - midori
> FAILED (4.7250s) *- This test is known to fail*
> java.lang.AssertionError - JToJSEval: the J and JS outputs are not equal!StackTrace
>
> java.lang.AssertionError: JToJSEval: the J and JS outputs are not equal!
> at org.junit.Assert.fail(Assert.java:91)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at JToJSEvalTest.evaluateStdoutContents(JToJSEvalTest.java:80)
> at JToJSEvalTest.javaToJSEvalTest(JToJSEvalTest.java:87)
> at JToJSEvalTest.AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test(JToJSEvalTest.java:110)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runLeaf(BrowserTestRunner.java:162)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChildX(BrowserTestRunner.java:137)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:96)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:58)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
> at CommandLine.runMain(CommandLine.java:49)
> at CommandLine.runMainAndExit(CommandLine.java:28)
> at CommandLine.main(CommandLine.java:24)
>
> JToJSEvalTest
> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test%20-%20epiphany>
> -
> AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test - epiphany
> FAILED (4.7180s) *- This test is known to fail*
> java.lang.AssertionError - JToJSEval: the J and JS outputs are not equal!StackTrace
>
> java.lang.AssertionError: JToJSEval: the J and JS outputs are not equal!
> at org.junit.Assert.fail(Assert.java:91)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at JToJSEvalTest.evaluateStdoutContents(JToJSEvalTest.java:80)
> at JToJSEvalTest.javaToJSEvalTest(JToJSEvalTest.java:87)
> at JToJSEvalTest.AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test(JToJSEvalTest.java:110)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runLeaf(BrowserTestRunner.java:162)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChildX(BrowserTestRunner.java:137)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:96)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:58)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
> at CommandLine.runMain(CommandLine.java:49)
> at CommandLine.runMainAndExit(CommandLine.java:28)
> at CommandLine.main(CommandLine.java:24)
>
> JToJSEvalTest
> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test%20-%20opera>
> -
> AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test - opera
> PASSED (5.3160s)
> JToJSEvalTest
> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test%20-%20midori>
> -
> AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test - midori
> FAILED (4.5170s) *- This test is known to fail*
> java.lang.AssertionError - JToJSEval: the J and JS outputs are not equal!StackTrace
>
> java.lang.AssertionError: JToJSEval: the J and JS outputs are not equal!
> at org.junit.Assert.fail(Assert.java:91)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at JToJSEvalTest.evaluateStdoutContents(JToJSEvalTest.java:80)
> at JToJSEvalTest.javaToJSEvalTest(JToJSEvalTest.java:87)
> at JToJSEvalTest.AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test(JToJSEvalTest.java:118)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runLeaf(BrowserTestRunner.java:162)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChildX(BrowserTestRunner.java:137)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:96)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:58)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
> at CommandLine.runMain(CommandLine.java:49)
> at CommandLine.runMainAndExit(CommandLine.java:28)
> at CommandLine.main(CommandLine.java:24)
>
> JToJSEvalTest
> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test%20-%20epiphany>
> -
> AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test - epiphany
> FAILED (4.6170s) *- This test is known to fail*
> java.lang.AssertionError - JToJSEval: the J and JS outputs are not equal!StackTrace
>
> java.lang.AssertionError: JToJSEval: the J and JS outputs are not equal!
> at org.junit.Assert.fail(Assert.java:91)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at JToJSEvalTest.evaluateStdoutContents(JToJSEvalTest.java:80)
> at JToJSEvalTest.javaToJSEvalTest(JToJSEvalTest.java:87)
> at JToJSEvalTest.AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test(JToJSEvalTest.java:118)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runLeaf(BrowserTestRunner.java:162)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChildX(BrowserTestRunner.java:137)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:96)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:58)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
> at CommandLine.runMain(CommandLine.java:49)
> at CommandLine.runMainAndExit(CommandLine.java:28)
> at CommandLine.main(CommandLine.java:24)
>
> JToJSEvalTest <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_typeof_Test%20-%20opera>
> -
> AppletJToJSEval_typeof_Test - opera
> FAILED (21.6470s)
> java.lang.AssertionError - JToJSEvalTest stdout should contain JToJSEval applet initialized. but it
> didnt.StackTrace
>
> java.lang.AssertionError: JToJSEvalTest stdout should contain JToJSEval applet initialized. but it didnt.
> at org.junit.Assert.fail(Assert.java:91)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at JToJSEvalTest.evaluateStdoutContents(JToJSEvalTest.java:68)
> at JToJSEvalTest.javaToJSEvalTest(JToJSEvalTest.java:87)
> at JToJSEvalTest.AppletJToJSEval_typeof_Test(JToJSEvalTest.java:125)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runLeaf(BrowserTestRunner.java:162)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChildX(BrowserTestRunner.java:137)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:96)
> at net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:58)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
> at CommandLine.runMain(CommandLine.java:49)
> at CommandLine.runMainAndExit(CommandLine.java:28)
> at CommandLine.main(CommandLine.java:24)
>
> JToJSEvalTest <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_typeof_Test%20-%20midori>
> -
> AppletJToJSEval_typeof_Test - midori
> PASSED (4.5120s)
> JToJSEvalTest <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_typeof_Test%20-%20epiphany>
> -
> AppletJToJSEval_typeof_Test - epiphany
> PASSED (4.6080s)
> JToJSEvalTest <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_location_Test%20-%20opera>
> -
> AppletJToJSEval_location_Test - opera
> PASSED (5.1070s)
> JToJSEvalTest <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_location_Test%20-%20midori>
> -
> AppletJToJSEval_location_Test - midori
> PASSED (4.7130s)
> JToJSEvalTest <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_location_Test%20-%20epiphany>
> -
> AppletJToJSEval_location_Test - epiphany
> PASSED (4.7130s)
> STD-OUT - Show/hide
>
> @sout@
>
> STD-ERR - Show/hide
>
> @serr@
>
>
> screenshot1.png
>
>
>
> screenshot2.png
>
>
So assuming this apply, build, and output is correct, and knwntofail for browser are really errors
on browser side, thanx and go on :)
J.
More information about the distro-pkg-dev
mailing list