[rfc] [icedtea-web] finally and proudly... TestInBrowsers annotation

[Pavel Tisnovsky]

Hi Jiri,

this quite a big change looks ok for me. I'm ok to push it, but please read following notes:

1) there are some typos (as usual :-):
unknown valkue
          ^^^
+IT_SET_GLOBAL_BROWSERTESTS_BHAVIOUR
                             ^^^ (on three or four places)

 seted
    ^^
+     * @return - bianry from where to lunch current browser
                  ^^^^^

+     * This is public just for testign purposes!
                              ^^^^^^^

returned browsers array msut not be null
                        ^^^^
Created from emppty there
             ^^^^^^
browser's plugins  shoud points to



2) There's not param named "tib", just a "testIn":

+     * @param tib
+     * @return all matching browser, if browser do not exists, this is ignored and run is silently continued
+     */
+    public List<Browser> translateAnnotationSilently(Browsers[] testIn) {



3) you mean injectBrowserCatched here?:
+        injcetBrowserCatched(method, Browsers.none);



4) I'm not sure how well this method will work in case of 32bit Browser + 32bit ITweb on 64bit system (which should work
AFAIK):
+    public String getDefaultPluginExpectedLocation() {
+        if (System.getProperty("os.arch").contains("64")) {
+            return "/usr/lib64/"+fsdir+"/plugins";
+        } else {
+            return "/usr/lib/"+fsdir+"/plugins";
+
+        }
+    }

Maybe it would be better to check, which JDK is running? (32/64bit)


Cheers,
Pavel


Jiri Vanek wrote:
> As introduced in  "Extending jnlp reproducers for html+applet
> reproducers" from 20.3 2012, after make links, configurable browsers,
> whitelist, multiple signatures, custom makefiles, gtk3 (midori and
> epiphany), knowntofail,  logging and many other improvements here is
> final version @TestInBrowser annotation on "future" head. This patch
> need to have applied  "[rfc][icedtea-web] finish to get rid of std
> out/err in testsuites" patch (as connection between log-id and
> browser-id is very close)
> 
> Other logic remains same (just adapted original text):
> 
> At first all available (supported and available) browsers are discovered
> by configure or set by user. Make links can ten ensure that all used
> browsers are configured properly
> 
> So just before running reproducers those links created (bad luck that
> for opera this linking need roots privileges :( ) are ensured and used
> browsers are passed  into testing framework by
> -Dused.browsers=$(FIREFOX):$(CHROMIUM):$(CHROME):$(OPERA):$(MIDORI):$(EPIPHANY)
> 
> 
> This variable is inside tranlsated by BrowserFactory to proxies
> representing individual browser (providing theirs binary, theirs plugin
> dirs, theirs individual settings.
> 
> Not-likely in original one, one global switch is available for cofigure
> - --with-browser-tests -
> yes - default behaviour, all browser tests behave as designed
> no - ignore all browser tests
> all - is handling all @TestInBrowsers as Browsers.all (good for daily
> report)
> one  - is handling all @TestInBrowsers as Browsers.one  (good for quick
> tests report), with exception  when some set of concrete browsers is
> set, then this "one" is selected from them
> 
> Tests are invoked in browser by lunching already introduced
> server.executeBrowser(resource).
> But there is little bit of magic  around this procedure - annotation
> @TestInBrowsers(testIn={...}) Where values are singletons pointing to
> individual concrete proxies. or special case ONE, which represents any
> one of configured browsers - so method will be invoked just once times
> with this browser injected, NONE - which is mostly just for testing
> purposes and will inject non existing browser. Or ALL which will inject
> all configured browsers.
> In case that ALL returns zero of browsers, or individual requested
> proxies are not configured, or when requested is one ONE  and no Browser
> is returned, then NONE browser is injected. (causing none-browser result)
> 
> This injection is very real, and if Eg FIREFOX,CHROMIUM and OPERA are
> requested by annotation, then the testing method is really launched
> three times - once for each browser. if ALL is requested by annotation,
> then method is really iterated for each configured browser.
> This injection is also propagated into reporting, where is displayed
> method - browser. When no browser is configured, then all "browser"
> tests will fail as for "unset_browser" (which is representation of
> mentioned NONE)
> This injection and iterating over methods is done inside custom junit
> runner BrowserTestRunner, which is invoking the tested method several
> times, and is always injecting desired browser into its  ServerAccess
> (included by inheriting of BrowserTest). So the only thing which final
> reproducer writer is to do is:
>  - prepare reproducer as is common now
>  - make his testcase to extends BrowserTest
>  - annotate methods which have something to do with browser which
> browsers they should be tested against
>      - there is still possibility to work without this annotation and
> get browsers directly from factory or to inject custom browsers, or to
> run whatever he wants :)
> -the port handling and other deployment stuff have been used form
> current jnlp reproducers
> 
> 
> 
> browserAnnotation-impl
> 
>     introduced possibility to run comfortably applets+html reproducers
>     * Makefile.am: used BROWSER_TESTS_MODIFICATION variable to pass
> global switch from configure
>     * acinclude.m4: (IT_SET_GLOBAL_BROWSERTESTS_BHAVIOUR) new method
> handling --with-browser-tests
>     * configure.ac: used IT_SET_GLOBAL_BROWSERTESTS_BHAVIOUR
>     switch and passing  BROWSER_TESTS_MODIFICATION variable to Makefile.
>     * tests/netx/jnlp_testsengine/net/sourceforge/jnlp/ServerAccess.java:
>     (currentBrowser) variable holding injected browser for ServerAccess
> instance
>     (loggedBrowser) static variable keeping id of (currentBrowser) for
> static logging purposes
>     (modifyMethodWithForBrowser) new function changing the name of
> method to "method - browser" for logging purposes
>     (getBrowserLocation) - returning path to process to be launched when
> browser requested
>     (getBrowserParams) - gathering set default's browser settings
>     (set/getCurrentBrowsers) - set browser by id/return id of set browser
>     (set/getCurrentBrowser) - set browser instance /returns instance of
> current browser
>     (executeBrowser) family of methods now cooperate with above methods
> for default set browser
>     (executeBrowser(Browser) family to work with implicit browser
>     *
> tests/netx/jnlp_testsengine/net/sourceforge/jnlp/annotations/TestInBrowsers.java:
> annotation for determining which browser(s) to use with annotated method
>     *
> tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/Browser.java:
> interface for dealing with various browsers
>     *
> tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/BrowserFactory.java:
> 
>     singleton for mapping configured browsers x requested browser x
> browsers proxies
>     *
> tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/BrowserTest.java:
> 
>     Forefather of all testcases which have methods to do tests iniside
> browser. Is allowing correct annotation -> proxy trasnver to
> VirtualServer for selected method and is requesting custom runner from
> junit framework
>     *
> tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/BrowserTestRunner.java:
> 
>     custom test Ruuner which is responsible for translating annotation
> and run the method mutlipletimes for each requested browser and to name
> it properly
>     *
> tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/Browsers.java:
> enumeration of abstract browsers and theirs sets or subsets.
>     *
> tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/browsers/*:
> individual browsers proxies and theirs abstractions,namely:
>     *
> tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/browsers/Opera.java:
> 
>     *
> tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/browsers/Firefox.java:
> 
>     *
> tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/browsers/Chrome.java:
> 
>     *
> tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/browsers/Chromium.java:
> 
>     *
> tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/browsers/Midori.java:
> 
>     *
> tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/browsers/Epiphany.java:
> proxies for  browsers as name suggests
>     *
> tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/browsers/LinuxBrowser.java:
> 
>     abstract forefather for all browsers implementing Browser interface.
> Is setting /usr/bin as bin path, libjavaplugin.so as default plugin
> library name, intorducing stubs for methods (eg 32/64 bit libs)
>     *
> tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/browsers/MozillaFamilyLinuxBrowser:
> forefather for all browsers except Opera. Is unifying .mozilla/plugins
> directories
> 
> 
> browserAnnotation-tests:
> 
>     various test for browser engine
>     * tests/jnlp_tests/simple/AppletTest/resources/appletAutoTests.html:
> fixed missing parenthesis
>     *
> tests/jnlp_tests/simple/AppletTest/resources/appletAutoTests2.html: new
> test excluding XslowX for applets
>     * tests/jnlp_tests/simple/AppletTest/testcases/AppletTestTests:
>     (doubleChrome) test for ensuring that two chrome browsers launched
> behind themselves will not cause errors as they vere withhout
> criticalFixes patch
>      (AppletInBrowserTest) and (AppletInBrowserTestXslowX) testing
> methods for all browser
>     * tests/jnlp_tests/simple/deadlocktest/srcs/DeadlockTest.java:
> improved to print sometimes something out
>     *
> tests/jnlp_tests/simple/deadlocktest/testcases/DeadLockTestTest.java:
> (testDeadLockTestTerminatedBody) enchanced to ensure that not so much is
> lost when process is terminated, but showing that something can be lost
> (which is correct)
>     * tests/netx/jnlp_testsengine/net/sourceforge/jnlp/ResourcesTest.java:
>     (testNonExisitngBrowserWillNotDeadlock) testing taht when no browser
> is set, then no deadlock happens as happen before criticalFixes's
> deadlyException
>     (testUnexistingProcessWillFailRecognizedly)  is actually testing
> deadlyException
>     (testNonExisitngBrowserWillNotCauseMess) some but with annotation
> which was also harming output of tool little bit without
> TestInBrowsersAnnotation fixed.
>     (testBrowsers2) is testing all browsers configuration without
> annotation  and (testBrowser3) do the same configuration tests
> annotation driven
>     (testBrowser) body of above two methods. Is testing whether used
> browsers are correctly linked with latest build
>     (testBrowsers1) is testing parsing of -D variable
>     (testListeners) annotated that needs display
>     *
> tests/netx/unit/net/sourceforge/jnlp/runtime/CodeBaseClassLoaderTest.java:
> annotated with Bug annotation
> 
> 
> I hope that it will fulfil our needs as much as possible
>   J.
>