[rfc][icedtea-web] KnownToFailInBrowsers annotation patch
Jana Fabrikova
jfabriko at redhat.com
Wed Apr 3 11:20:32 PDT 2013
Thank you for the review. I have added the annotation to the repository
and to all the relevant methods in the following reproducers:
-JavascriptGet (to 2 testcases)
-JavascriptFuncParam (to 1 testcase)
-JavascriptSet (no need, all the tests pass)
-JavascriptFuncReturn (no need, all the tests pass)
cheers,
Jana
On 04/03/2013 02:58 PM, Jiri Vanek wrote:
> I think this can go in as it is. Just minor nits which you can apply
> before push.
> Please also apply the new annotation where valid (as new changeset).
> you can push this without review.
>
> On 03/18/2013 04:11 PM, Jana Fabrikova wrote:
>> Hi Jiri and Adam,
>>
>> as Jiri suggested I have modified the annotation (this time no new
>> annotation KnownToFailInBrowsers,
>> only a new optional parameter Browsers[] failsIn() added to the old
>> KnownToFail annotation).
>>
>> I have modified the JunitLikeXmlOutputListener and
>> LessVerboseTextListener also, they identify the
>> combinations of testcase+browser that are declared to fail and write
>> known-to-fail="true" into the
>> xml file, which is then processed by the same xslt as usual.
>>
>> I have run one reproducer (modification of JToJSEval) in opera, midori
>> and epiphany with a
>> combination of failing/passing testcases.
>>
>> The results in terminal were (screenshot1):
>>
>> -For the testcase that really failed in midori and epiphany but was
>> annotated with
>> @KnownToFail(failsIn={}) :
>>
>> Passed:
>> JToJSEvalTest.AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test - opera
>> - WARNING This test is known to fail, but have passed!
>> FAILED: AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test -
>> midori(JToJSEvalTest) JToJSEval: the J
>> and JS outputs are not equal!
>> - This test is known to fail
>> FAILED: AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test -
>> epiphany(JToJSEvalTest) JToJSEval: the J
>> and JS outputs are not equal!
>> - This test is known to fail
>>
>> -For the testcase that really failed in midori and epiphany but was
>> annotated with @KnownToFail :
>>
>> Passed:
>> JToJSEvalTest.AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test
>> - opera
>> - WARNING This test is known to fail, but have passed!
>> FAILED: AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test -
>> midori(JToJSEvalTest) JToJSEval:
>> the J and JS outputs are not equal!
>> - This test is known to fail
>> FAILED: AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test -
>> epiphany(JToJSEvalTest)
>> JToJSEval: the J and JS outputs are not equal!
>> - This test is known to fail
>>
>> -For the testcase that really failed in midori and epiphany and was
>> annotated with
>> @KnownToFail(failsIn={Browsers.midori, Browsers.epiphany})
>>
>> Passed: JToJSEvalTest.AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test
>> - opera
>> FAILED: AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test -
>> midori(JToJSEvalTest) JToJSEval: the J and
>> JS outputs are not equal!
>> - This test is known to fail
>> FAILED: AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test -
>> epiphany(JToJSEvalTest) JToJSEval: the J
>> and JS outputs are not equal!
>> - This test is known to fail
>>
>> -For the testcase that really failed in opera (only because opera
>> crashed, it does not normally
>> fail) and was not annotated:
>>
>> FAILED: AppletJToJSEval_typeof_Test - opera(JToJSEvalTest)
>> JToJSEvalTest stdout should contain
>> JToJSEval applet initialized. but it didnt.
>> Passed: JToJSEvalTest.AppletJToJSEval_typeof_Test - midori
>> Passed: JToJSEvalTest.AppletJToJSEval_typeof_Test - epiphany
>>
>> -For the testcase that passes in all browsers and was not annotated:
>>
>> Passed: JToJSEvalTest.AppletJToJSEval_location_Test - opera
>> Passed: JToJSEvalTest.AppletJToJSEval_location_Test - midori
>> Passed: JToJSEvalTest.AppletJToJSEval_location_Test - epiphany
>>
>>
>> -Total summary:
>>
>> Total tests run: 15; From those : 8 known to fail
>> Test known to fail: passed: 2; failed: 6; ignored: 0
>> Test results: passed: 8; failed: 7; ignored: 0
>>
>>
>> The results in html are attached as index_reproducers.html and
>> screenshot2).
>>
>>
>> Please see the attached patch,
>> cheers,
>> Jana
>>
>>
>> On 03/13/2013 05:45 PM, Jiri Vanek wrote:
>>> On 03/13/2013 05:13 PM, Jana Fabrikova wrote:
>>>> Hello,
>>>>
>>>> I am sending for your consideration a patch that introduces the
>>>> @KnownToFailInBrowsers annotation, which may be useful in some of the
>>>> reproducers (namely passing arrays from JS->J does work in opera and
>>>> firefox and does not work in google-chrome, chromium, midori and
>>>> epiphany, and i think it is more problem of the browsers than of
>>>> icedtea-web plugin).
>>>>
>>>> The annotation is detected in
>>>> junit-runner/JunitLikeXmlOutputListener.java and
>>>> junit-runner/LessVerboseTextListener.java and reflected in the console
>>>> and xml output of run-netx-dist-tests.
>>>>
>>>> Cheers,
>>>> Jana
>>>>
>>>> 2013-03-13 Jana Fabrikova <jfabriko at redhat.com>
>>>>
>>>> *
>>>> /tests/test-extensions/net/sourceforge/jnlp/annotations/KnownToFailInBrowsers.java
>>>>
>>>>
>>>> the implementation of new annotation, which has an array of
>>>> browsers of type Browsers[] named failsIn
>>>>
>>>> * /tests/junit-runner/JunitLikeXmlOutputListener.java
>>>> in method (testDone) the testcases that are known to fail in
>>>> current browser are detected in addition to the tests that are
>>>> k2f in all browsers
>>>>
>>>> * /tests/junit-runner/LessVerboseTextListener.java
>>>> added method (getK2FinB) reading the annotion,
>>>> in method (printK2F) the testcases that are known to fail in
>>>> current browser are detected in addition to the tests that are
>>>> k2f in all browsers
>>>
>>> hmhm... I'm not sure If new type of annotation is needed. I would prefer
>>> extension of KnownTofail as it is.
>>> Now it have no parameter. In your new implementation it will have
>>> optional Browsers[] failsIn(). If it will be null/empty, then it will
>>> behave as we are used now. Otherwise it will be more sophisticated
>>> What do you think?
>>>
>>> For the implementation itself - uless I missed something, I'm terribly
>>> missing the browser recognition handling in xml/plaintext generation
>>> and in xslt sheet.
>>> Also some example usage of annotaion would be nice and as a top candy
>>> the screenshots of outputs can be attached.
>>>
>>>
>>> J.
>>
>>
>> annotk2f-version2.patch
>>
>>
>> diff --git a/tests/junit-runner/JunitLikeXmlOutputListener.java
>> b/tests/junit-runner/JunitLikeXmlOutputListener.java
>> --- a/tests/junit-runner/JunitLikeXmlOutputListener.java
>> +++ b/tests/junit-runner/JunitLikeXmlOutputListener.java
>> @@ -20,10 +20,12 @@
>> import java.util.Map;
>> import java.util.Map.Entry;
>> import java.util.Set;
>> +import java.util.Arrays;
>> import java.util.concurrent.TimeUnit;
>> import net.sourceforge.jnlp.annotations.Bug;
>> import net.sourceforge.jnlp.annotations.KnownToFail;
>> import net.sourceforge.jnlp.annotations.Remote;
>> +import net.sourceforge.jnlp.browsertesting.Browsers;
>>
>>
>> import org.junit.internal.JUnitSystem;
>> @@ -201,17 +203,30 @@
>> testcaseAtts.put(TEST_IGNORED_ATTRIBUTE,
>> Boolean.TRUE.toString());
>> }
>> KnownToFail k2f =
>> LessVerboseTextListener.getAnnotation(testClass, testMethod.getName(),
>> KnownToFail.class);
>> + boolean knownToFailInThisBrowser = false;
>
> Is this name correct? I think it mark that test fails. No matter if "for
> all" or "for one browser". So maybe better name?
>> Remote remote =
>> LessVerboseTextListener.getAnnotation(testClass, testMethod.getName(),
>> Remote.class);
>> - if (k2f != null) {
>> - testcaseAtts.put(K2F, Boolean.TRUE.toString());
>> - }
>> + if (k2f != null) {
>> + //determine if k2f in the current browser
>> +
>> + Browsers[] br = k2f.failsIn();
>> + if(0 == br.length){//the KnownToFail annotation without
>> optional parameter
>> + knownToFailInThisBrowser = true;
>> + }else{
>> + for(Browsers b : br){
>> + if(description.toString().contains(b.toString())){
>> + knownToFailInThisBrowser = true;
>> + }
>> + }
>> + }
>> + }
>> + if( knownToFailInThisBrowser ) testcaseAtts.put(K2F,
>> Boolean.TRUE.toString());
>> if (remote != null) {
>> testcaseAtts.put(REMOTE, Boolean.TRUE.toString());
>>
>> }
>> openElement(TEST_ELEMENT, testcaseAtts);
>> if (testFailed != null) {
>> - if (k2f != null) {
>> + if (knownToFailInThisBrowser) {
>> failedK2F++;
>> }
>> Map<String, String> errorAtts = new HashMap<String,
>> String>(3);
>> @@ -226,7 +241,7 @@
>>
>> writeElement(TEST_ERROR_ELEMENT, testFailed.getTrace(),
>> errorAtts);
>> } else {
>> - if (k2f != null) {
>> + if (knownToFailInThisBrowser) {
>> if (ignored) {
>> ignoredK2F++;
>> } else {
>> @@ -265,25 +280,25 @@
>> classStats.put(description.getClassName(), classStat);
>> }
>> classStat.total++;
>> - if (k2f != null) {
>> + if (knownToFailInThisBrowser) {
>> classStat.totalK2F++;
>> }
>> classStat.time += testTime;
>> if (testFailed == null) {
>> if (ignored) {
>> classStat.ignored++;
>> - if (k2f != null) {
>> + if (knownToFailInThisBrowser) {
>> classStat.ignoredK2F++;
>> }
>> } else {
>> classStat.passed++;
>> - if (k2f != null) {
>> + if (knownToFailInThisBrowser) {
>> classStat.passedK2F++;
>> }
>> }
>> } else {
>> classStat.failed++;
>> - if (k2f != null) {
>> + if (knownToFailInThisBrowser) {
>> classStat.failedK2F++;
>> }
>> }
>> diff --git a/tests/junit-runner/LessVerboseTextListener.java
>> b/tests/junit-runner/LessVerboseTextListener.java
>> --- a/tests/junit-runner/LessVerboseTextListener.java
>> +++ b/tests/junit-runner/LessVerboseTextListener.java
>> @@ -10,6 +10,7 @@
>> import java.lang.reflect.Method;
>> import net.sourceforge.jnlp.annotations.KnownToFail;
>> import net.sourceforge.jnlp.annotations.Remote;
>> +import net.sourceforge.jnlp.browsertesting.Browsers;
>>
>> import org.junit.internal.JUnitSystem;
>> import org.junit.runner.Description;
>> @@ -74,24 +75,39 @@
>> private void printK2F(PrintStream writer, Boolean failed,
>> Description description) {
>> try {
>> KnownToFail k2f = getK2F(description);
>> - if (k2f != null) {
>> - totalK2F++;
>> - if (failed != null) {
>> - if (failed) {
>> - failedK2F++;
>> - } else {
>> - passedK2F++;
>> - }
>> - } else {
>> - ignoredK2F++;
>> - }
>> - if (failed != null && !failed) {
>> - writer.println(" - WARNING This test is known to
>> fail, but have passed!");
>> - } else {
>> - writer.println(" - This test is known to fail");
>> - }
>> + boolean knownToFailInThisBrowser = false;
>
> again.
>> + if (k2f != null){
>> + //determine if k2f in the current browser
>> + Browsers[] br = k2f.failsIn();
>> + if(0 == br.length){ //@KnownToFail with default
>> optional parameter failsIn={}
>> + knownToFailInThisBrowser = true;
>> + }else{
>> + for(Browsers b : br){
>> +
>> if(description.toString().contains(b.toString())){
>> + knownToFailInThisBrowser = true;
>> + }
>> + }
>> + }
>> }
>>
>> + if( knownToFailInThisBrowser ){
>> + totalK2F++;
>> + if (failed != null) {
>> + if (failed) {
>> + failedK2F++;
>> + } else {
>> + passedK2F++;
>> + }
>> + } else {
>> + ignoredK2F++;
>> + }
>> + if (failed != null && !failed) {
>> + writer.println(" - WARNING This test is known to
>> fail, but have passed!");
>> + } else {
>> + writer.println(" - This test is known to fail");
>> + }
>> + }
>> +
>>
>> } catch (Exception ex) {
>> ex.printStackTrace();
>>
>> diff --git
>> a/tests/test-extensions/net/sourceforge/jnlp/annotations/KnownToFail.java
>> b/tests/test-extensions/net/sourceforge/jnlp/annotations/KnownToFail.java
>> ---
>> a/tests/test-extensions/net/sourceforge/jnlp/annotations/KnownToFail.java
>> +++
>> b/tests/test-extensions/net/sourceforge/jnlp/annotations/KnownToFail.java
>> @@ -41,6 +41,7 @@
>> import java.lang.annotation.Retention;
>> import java.lang.annotation.RetentionPolicy;
>> import java.lang.annotation.Target;
>> +import net.sourceforge.jnlp.browsertesting.Browsers;
>>
>> /**
>> * <p>
>> @@ -52,10 +53,15 @@
>> * This annotation is meant for adding tests for bugs before the fix is
>> * implemented.
>> * </p>
>> + * <p>
>> + * The meaning of optional parameter failsIn is either a list of
>> + * browsers where the test fails, or a default value - an empty array
>> {},
>> + * default value means that the test fails always.
>> + * </p>
>> */
>>
>> @Target({ElementType.METHOD,ElementType.TYPE})
>> @Retention(RetentionPolicy.RUNTIME)
>> public @interface KnownToFail {
>> -
>> + public Browsers[] failsIn() default {};
>> }
>>
>>
>> index_reproducers.html
>>
>>
>> Date:
>>
>> Mon Mar 18 15:35:34 CET 2013
>>
>>
>> Result: (88s)
>>
>>
>> In brackets are KnownToFail values if any
>>
>> TOTAL:
>> 15 (8)
>> passed:
>> 8 (2)
>> failed:
>> 7 (6)
>> ignored:
>> 0
>>
>>
>> Classes: show/hide
>>
>> JToJSEvalTest (88272ms):
>> <imap://jvanek@redhat.com:993/fetch%3EUID%3E/mlists/disto-pkg-dev%3E274267#JToJSEvalTest>
>>
>>
>> TOTAL:
>> 15 (8)
>> passed:
>> 8 (2)
>> failed:
>> 7 (6)
>> ignored:
>> 0
>>
>> ----------------------------------------------------------------------------------------------------
>>
>>
>>
>> Individual results:
>>
>> NoneTraceAllTraces
>> JToJSEvalTest
>> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test%20-%20opera>
>>
>> -
>> AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test - opera
>> PASSED (4.9260s) *- This test is known to fail*
>> JToJSEvalTest
>> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test%20-%20midori>
>>
>> -
>> AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test - midori
>> FAILED (4.5260s) *- This test is known to fail*
>> java.lang.AssertionError - JToJSEval: the J and JS outputs are not
>> equal!StackTrace
>>
>> java.lang.AssertionError: JToJSEval: the J and JS outputs are not equal!
>> at org.junit.Assert.fail(Assert.java:91)
>> at org.junit.Assert.assertTrue(Assert.java:43)
>> at JToJSEvalTest.evaluateStdoutContents(JToJSEvalTest.java:80)
>> at JToJSEvalTest.javaToJSEvalTest(JToJSEvalTest.java:87)
>> at
>> JToJSEvalTest.AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test(JToJSEvalTest.java:102)
>>
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>> at java.lang.reflect.Method.invoke(Method.java:601)
>> at
>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
>>
>> at
>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>>
>> at
>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
>>
>> at
>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runLeaf(BrowserTestRunner.java:162)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChildX(BrowserTestRunner.java:137)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:96)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:58)
>>
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
>> at org.junit.runners.Suite.runChild(Suite.java:128)
>> at org.junit.runners.Suite.runChild(Suite.java:24)
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
>> at CommandLine.runMain(CommandLine.java:49)
>> at CommandLine.runMainAndExit(CommandLine.java:28)
>> at CommandLine.main(CommandLine.java:24)
>>
>> JToJSEvalTest
>> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test%20-%20epiphany>
>>
>> -
>> AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test - epiphany
>> FAILED (4.5160s) *- This test is known to fail*
>> java.lang.AssertionError - JToJSEval: the J and JS outputs are not
>> equal!StackTrace
>>
>> java.lang.AssertionError: JToJSEval: the J and JS outputs are not equal!
>> at org.junit.Assert.fail(Assert.java:91)
>> at org.junit.Assert.assertTrue(Assert.java:43)
>> at JToJSEvalTest.evaluateStdoutContents(JToJSEvalTest.java:80)
>> at JToJSEvalTest.javaToJSEvalTest(JToJSEvalTest.java:87)
>> at
>> JToJSEvalTest.AppletJToJSEval_AnnotatedAsFailsInAll_1plus1_Test(JToJSEvalTest.java:102)
>>
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>> at java.lang.reflect.Method.invoke(Method.java:601)
>> at
>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
>>
>> at
>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>>
>> at
>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
>>
>> at
>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runLeaf(BrowserTestRunner.java:162)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChildX(BrowserTestRunner.java:137)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:96)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:58)
>>
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
>> at org.junit.runners.Suite.runChild(Suite.java:128)
>> at org.junit.runners.Suite.runChild(Suite.java:24)
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
>> at CommandLine.runMain(CommandLine.java:49)
>> at CommandLine.runMainAndExit(CommandLine.java:28)
>> at CommandLine.main(CommandLine.java:24)
>>
>> JToJSEvalTest
>> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test%20-%20opera>
>>
>> -
>> AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test - opera
>> PASSED (5.1110s) *- This test is known to fail*
>> JToJSEvalTest
>> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test%20-%20midori>
>>
>> -
>> AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test - midori
>> FAILED (4.7250s) *- This test is known to fail*
>> java.lang.AssertionError - JToJSEval: the J and JS outputs are not
>> equal!StackTrace
>>
>> java.lang.AssertionError: JToJSEval: the J and JS outputs are not equal!
>> at org.junit.Assert.fail(Assert.java:91)
>> at org.junit.Assert.assertTrue(Assert.java:43)
>> at JToJSEvalTest.evaluateStdoutContents(JToJSEvalTest.java:80)
>> at JToJSEvalTest.javaToJSEvalTest(JToJSEvalTest.java:87)
>> at
>> JToJSEvalTest.AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test(JToJSEvalTest.java:110)
>>
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>> at java.lang.reflect.Method.invoke(Method.java:601)
>> at
>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
>>
>> at
>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>>
>> at
>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
>>
>> at
>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runLeaf(BrowserTestRunner.java:162)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChildX(BrowserTestRunner.java:137)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:96)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:58)
>>
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
>> at org.junit.runners.Suite.runChild(Suite.java:128)
>> at org.junit.runners.Suite.runChild(Suite.java:24)
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
>> at CommandLine.runMain(CommandLine.java:49)
>> at CommandLine.runMainAndExit(CommandLine.java:28)
>> at CommandLine.main(CommandLine.java:24)
>>
>> JToJSEvalTest
>> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test%20-%20epiphany>
>>
>> -
>> AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test - epiphany
>> FAILED (4.7180s) *- This test is known to fail*
>> java.lang.AssertionError - JToJSEval: the J and JS outputs are not
>> equal!StackTrace
>>
>> java.lang.AssertionError: JToJSEval: the J and JS outputs are not equal!
>> at org.junit.Assert.fail(Assert.java:91)
>> at org.junit.Assert.assertTrue(Assert.java:43)
>> at JToJSEvalTest.evaluateStdoutContents(JToJSEvalTest.java:80)
>> at JToJSEvalTest.javaToJSEvalTest(JToJSEvalTest.java:87)
>> at
>> JToJSEvalTest.AppletJToJSEval_AnnotatedAsDefaultFailsInAll_1plus1_Test(JToJSEvalTest.java:110)
>>
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>> at java.lang.reflect.Method.invoke(Method.java:601)
>> at
>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
>>
>> at
>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>>
>> at
>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
>>
>> at
>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runLeaf(BrowserTestRunner.java:162)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChildX(BrowserTestRunner.java:137)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:96)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:58)
>>
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
>> at org.junit.runners.Suite.runChild(Suite.java:128)
>> at org.junit.runners.Suite.runChild(Suite.java:24)
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
>> at CommandLine.runMain(CommandLine.java:49)
>> at CommandLine.runMainAndExit(CommandLine.java:28)
>> at CommandLine.main(CommandLine.java:24)
>>
>> JToJSEvalTest
>> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test%20-%20opera>
>>
>> -
>> AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test - opera
>> PASSED (5.3160s)
>> JToJSEvalTest
>> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test%20-%20midori>
>>
>> -
>> AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test - midori
>> FAILED (4.5170s) *- This test is known to fail*
>> java.lang.AssertionError - JToJSEval: the J and JS outputs are not
>> equal!StackTrace
>>
>> java.lang.AssertionError: JToJSEval: the J and JS outputs are not equal!
>> at org.junit.Assert.fail(Assert.java:91)
>> at org.junit.Assert.assertTrue(Assert.java:43)
>> at JToJSEvalTest.evaluateStdoutContents(JToJSEvalTest.java:80)
>> at JToJSEvalTest.javaToJSEvalTest(JToJSEvalTest.java:87)
>> at
>> JToJSEvalTest.AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test(JToJSEvalTest.java:118)
>>
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>> at java.lang.reflect.Method.invoke(Method.java:601)
>> at
>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
>>
>> at
>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>>
>> at
>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
>>
>> at
>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runLeaf(BrowserTestRunner.java:162)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChildX(BrowserTestRunner.java:137)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:96)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:58)
>>
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
>> at org.junit.runners.Suite.runChild(Suite.java:128)
>> at org.junit.runners.Suite.runChild(Suite.java:24)
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
>> at CommandLine.runMain(CommandLine.java:49)
>> at CommandLine.runMainAndExit(CommandLine.java:28)
>> at CommandLine.main(CommandLine.java:24)
>>
>> JToJSEvalTest
>> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test%20-%20epiphany>
>>
>> -
>> AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test - epiphany
>> FAILED (4.6170s) *- This test is known to fail*
>> java.lang.AssertionError - JToJSEval: the J and JS outputs are not
>> equal!StackTrace
>>
>> java.lang.AssertionError: JToJSEval: the J and JS outputs are not equal!
>> at org.junit.Assert.fail(Assert.java:91)
>> at org.junit.Assert.assertTrue(Assert.java:43)
>> at JToJSEvalTest.evaluateStdoutContents(JToJSEvalTest.java:80)
>> at JToJSEvalTest.javaToJSEvalTest(JToJSEvalTest.java:87)
>> at
>> JToJSEvalTest.AppletJToJSEval_FailsInSomeBrowsers_1plus1_Test(JToJSEvalTest.java:118)
>>
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>> at java.lang.reflect.Method.invoke(Method.java:601)
>> at
>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
>>
>> at
>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>>
>> at
>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
>>
>> at
>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runLeaf(BrowserTestRunner.java:162)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChildX(BrowserTestRunner.java:137)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:96)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:58)
>>
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
>> at org.junit.runners.Suite.runChild(Suite.java:128)
>> at org.junit.runners.Suite.runChild(Suite.java:24)
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
>> at CommandLine.runMain(CommandLine.java:49)
>> at CommandLine.runMainAndExit(CommandLine.java:28)
>> at CommandLine.main(CommandLine.java:24)
>>
>> JToJSEvalTest
>> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_typeof_Test%20-%20opera>
>>
>> -
>> AppletJToJSEval_typeof_Test - opera
>> FAILED (21.6470s)
>> java.lang.AssertionError - JToJSEvalTest stdout should contain
>> JToJSEval applet initialized. but it
>> didnt.StackTrace
>>
>> java.lang.AssertionError: JToJSEvalTest stdout should contain
>> JToJSEval applet initialized. but it didnt.
>> at org.junit.Assert.fail(Assert.java:91)
>> at org.junit.Assert.assertTrue(Assert.java:43)
>> at JToJSEvalTest.evaluateStdoutContents(JToJSEvalTest.java:68)
>> at JToJSEvalTest.javaToJSEvalTest(JToJSEvalTest.java:87)
>> at JToJSEvalTest.AppletJToJSEval_typeof_Test(JToJSEvalTest.java:125)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>> at java.lang.reflect.Method.invoke(Method.java:601)
>> at
>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
>>
>> at
>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>>
>> at
>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
>>
>> at
>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runLeaf(BrowserTestRunner.java:162)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChildX(BrowserTestRunner.java:137)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:96)
>>
>> at
>> net.sourceforge.jnlp.browsertesting.BrowserTestRunner.runChild(BrowserTestRunner.java:58)
>>
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
>> at org.junit.runners.Suite.runChild(Suite.java:128)
>> at org.junit.runners.Suite.runChild(Suite.java:24)
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
>> at CommandLine.runMain(CommandLine.java:49)
>> at CommandLine.runMainAndExit(CommandLine.java:28)
>> at CommandLine.main(CommandLine.java:24)
>>
>> JToJSEvalTest
>> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_typeof_Test%20-%20midori>
>>
>> -
>> AppletJToJSEval_typeof_Test - midori
>> PASSED (4.5120s)
>> JToJSEvalTest
>> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_typeof_Test%20-%20epiphany>
>>
>> -
>> AppletJToJSEval_typeof_Test - epiphany
>> PASSED (4.6080s)
>> JToJSEvalTest
>> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_location_Test%20-%20opera>
>>
>> -
>> AppletJToJSEval_location_Test - opera
>> PASSED (5.1070s)
>> JToJSEvalTest
>> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_location_Test%20-%20midori>
>>
>> -
>> AppletJToJSEval_location_Test - midori
>> PASSED (4.7130s)
>> JToJSEvalTest
>> <logs_reproducers.html#JToJSEvalTest.AppletJToJSEval_location_Test%20-%20epiphany>
>>
>> -
>> AppletJToJSEval_location_Test - epiphany
>> PASSED (4.7130s)
>> STD-OUT - Show/hide
>>
>> @sout@
>>
>> STD-ERR - Show/hide
>>
>> @serr@
>>
>>
>> screenshot1.png
>>
>>
>>
>> screenshot2.png
>>
>>
> So assuming this apply, build, and output is correct, and knwntofail for
> browser are really errors on browser side, thanx and go on :)
>
> J.
More information about the distro-pkg-dev
mailing list