Extending jnlp reproducers for html+applet reproducers
Jiri Vanek
jvanek at redhat.com
Tue Mar 20 09:24:27 PDT 2012
Hi All!
As promised some time ago, and "proof of concepted" week ago, I have written small extension to
reproducers framework, which allows to launch applet reproducers inside browsers.
The hg diff in attachment is not considered for review (however any brave soul *can do this!*), but
is added for overview information. I will post individual parts of this one by one for next months.
Also individual parts into to which I have split browserTests.diff have small overleaps.
_concept_
At first all available (supported and avaiable) browsers are discovered by configure: eg
AC_CHECK_PROGS([OPERA],[opera],[], []) .... for opera, chrome, chromium and firefox.
*New make targets* for creating *symlinks* to --prefixed location were added
(stamps/user-links.stamp and stamps/global-links.stamp). Those depends on make install and
stamps/run-netx-dist-tests.stamp depends on global-links. Those targets are mentioned only for
testing purposes. Those ensure that tested browsers will use just built plugin. There is test for
this correctness inside reproducers (testBrowser family of tests inside
jnlp_testsengine/net/sourceforge/jnlp/ResourcesTest.java)
So just before running reproducers the links are created (bad luck that for opera this linking need
roots privileges :( )and used browsers are passed into testing framework by
-Dused.browsers=$(FIREFOX):$(CHROMIUM):$(CHROME):$(OPERA)
This variable is inside tranlsated by BrowserFactory to proxies representing individual browser
(providing theirs binary, theirs plugin dirs, theirs individual settings.
Tests are invoked in browser by lunching already introduced server.executeBrowser(resource).
But there is little bit of magic around this procedure - annotation @TestInBrowsers(testIn={...})
Where values are singletons pointing to individual concrete proxies. or special case ONE, which
represents any one of configured browsers - so method will be invoked just once times with this
browser injected, NONE - which is mostly just for testing purposes and will inject non existing
browser. Or ALL which will inject all configured browsers.
In case that ALL returns zero of browsers, or individual requested proxies are not configured, or
when requested is one ONE and no Browser is returned, then NONE browser is injected. (causing
none-browser result)
This injection is very real, and if Eg FIREFOX,CHROMIUM and OPERA are requested by annotation, then
the testing method is really launched three times - once for each browser. if ALL is requested by
annotation, then method is really iterated for each configured browser.
This injection is also propagated into reporting, where is displayed method - browser. When no
browser is configured, then all "browser" tests will fail as for "unset_browser" (which is
representation of mentioned NONE)
This injection and iterating over methods is done inside custom junit runner BrowserTestRunner,
which is invoking the tested method several times, and is always injecting desired browser into its
ServerAccess (included by inheriting of BrowserTest). So the only thing which final reproducer
writer is to do is:
- prepare reproducer as is common now
- instead of jnlp file prepare html file which will launch appelt
- make his testcase to extends BrowserTest
- annotate methods which have something to do with browser which browsers they should be tested
against
- there is still possibility to work without this annotation and get browsers directly from
factory or to inject custom browsers, or tu run whatever he wants :)
-the port handling and other deployment stuff have been used form current jnlp reproducers
_fixes_
There was bug in ProcessAssasin which leaked killed process. It acctual did not kill it, just left
it be and return control to main theread :-/
It is fixed by correctly added (!to proper place!) destroy() method of process. This leak have come
to light when I started to launch browsers.
_other minor and useful stuff_
*@Bug* annotation - can handle IDs of bugs or urls to web which contains more info about "why this
reproducer or unit test was added", are *nicely* propagated also to html report (see attached
report.zip). NAme convention is as we are used to from NEWS files.
*@NeedsDispaly* - annotation which tell that this method can not be run if no display is specified.
*UnitTests* also have benefits of Bug annotation and of *ServerAccess*. - added classpath dependence
*Blacklist* to exclude some reproducers is introduced (REPRODUCERS_CLASS_BLACKLIST)
*enabled decision weather to kill the process or just to skip him
*embeded virtual server now returns correct mimetypes (this was really mayor issue just wrongly
ignored by firefox)
*fied *leaking* proceses (missing process.destroy()
Changelogs: in order of expected order of to be applied
browserTests-criticalFixes:
fixing issue when process was not launched at all and when was killed but left behind hanging
*tests/netx/jnlp_testsengine/net/sourceforge/jnlp/ServerAccess.java:
(getContentOfStream) this method overlaoded with possibility to specify encoding (I needed to set
aASCI in one test)
(deadlyException) field intoruced in ThreadedProcess to record exception caused by impassibility of
launching the process. And so process have been null without any sign why.
(TinyHtmlServer) now correctly returns known mime types
(ProcessAssasin) can now skip or smoothly (and finally correctly) destroy its process, and all his
logging messages were done null-proof (as deadlyException now allows)
Asynchronous (ContentReader) have been silenced when complaining about closed streams by Assassin.
browserTests-MakeLinks:
Added detection of installed browsers and added targets to create symbolic links from install dir
to browsers' plugin directories. Primarily for testing purposes
*Makefile.am: (clean-local) optionally delets links' stamps
(stamps/user-links.stamp) with alias (links) - new target for creating symlinks for all users. One
must be root to execute this target.
(stamps/global-links.stamp) with alias (user-links) - new target for creating symlinks for logged
user only. Because opera is missing this feature, quite useless.
*configure.ac: added basic check whether and which browsers are installed
browserTests-blacklist:
Added possibility to exclude some classes) from being processed by reproducers engine. Usefull when
testing just one testcase or when interfaces or abstract classes are included (throwing
initialization exception)
*Makefile.am: (REPRODUCERS_CLASS_BLACKLIST) new variable holding the file with list
(REPRODUCERS_CLASS_NAMES) added loop through REPRODUCERS_CLASS_BLACKLIST file which contains
regular expressions which, when matched, causing file not to be included in list.
*netx-dist-tests-blacklist: new file containing expressions.
browserTests-BugAnnotation:
Intorduced new annotations Bug (to connect test/reproducer with documentation) and NeedsDisplay
which tells the launching engine thet this particualr test needs Display. BAsed on Pavel's ideas and
jtreg experiences
*Makefile.am: (JUNIT_RUNNER_JAR), (stamps/run-netx-unit-tests.stamp) and
(stamps/run-unit-test-code-coverage.stamp) are now dependent on (stamps/netx-dist-tests-compile) and
classpaths inside them have been enriched for JNLP_TESTS_ENGINE_DIRwhich contains definitions of
those annotations
*tests/jnlp_tests/simple/CheckServices/testcases/CheckServicesTests.java: and
*tests/jnlp_tests/simple/ManifestedJar1/testcases/ManifestedJar1Test.java: and
*tests/jnlp_tests/simple/Spaces can be everywhere/testcases/SpacesCanBeEverywhereTests.java: filled
Bug annotations
*tests/junit-runner/JunitLikeXmlOutputListener.java: made to understan Bug annotation
*tests/netx/jnlp_testsengine/net/sourceforge/jnlp/annotations/NeedsDisplay.java: and
*tests/netx/jnlp_testsengine/net/sourceforge/jnlp/annotations/Bug.java: annotations definitions
*tests/report-styles/jreport.xsl" made nice links from bug annotation prepared by
JunitLikeXmlOutputListener
browserTests-TestInBrowsersAnnotation
introduced possibility to run comfortably applets+html reproducers
*tests/netx/jnlp_testsengine/net/sourceforge/jnlp/ServerAccess.java:
(currentBrowser) variable holding injected browser
(getBrowserLocation) - returning path to process to be launched when browser requested
(getBrowserParams) - gathering set default's browser settings
(set/getCurrentBrowsers) - set browser by id/return id of set browser
(set/getCurrentBrowser) - set browser instance /returns instance of current browser
(executeBrowser) family of methods now cooperate with above methods for default set browser
(executeBrowser(Browser) family to work with implicit browser
*tests/netx/jnlp_testsengine/net/sourceforge/jnlp/annotations/TestInBrowsers.java: annotation for
determining which browser(s) to use with annotated method
*tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/Browser.java: interface for
dealing with various browsers
*tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/BrowserFactory.java:
singleton for mapping configured browsers x requested browser x browsers proxies
*tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/BrowserTest.java:
Forefather of all testcases which have methods to do tests iniside browser. Is allowing correct
annotation -> proxy trasnver to VirtualServer for selected method and is requesting custom runner
from junit framework
*tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/BrowserTestRunner.java:
custom test Ruuner which is responsible for translating annotation and run the method mutlipletimes
for each requested browser and to name it properly
*tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/Browsers.java: enumeration of
abstract browsers and theirs sets or subsets.
tests/netx/jnlp_testsengine/net/sourceforge/jnlp/browsertesting/browsers/*: individual browsers proxies
browserTests-tests:
various test for browser engine
*tests/jnlp_tests/simple/AppletTest/resources/appletAutoTests.html: fixed missing parenthesis
*tests/jnlp_tests/simple/AppletTest/resources/appletAutoTests2.html: new test excluding XslowX for
applets
*tests/jnlp_tests/simple/AppletTest/testcases/AppletTestTests:
(doubleChrome) test for ensuring that two chrome browsers launched behind themselves will not cause
errors as they vere withhout criticalFixes patch
(AppletInBrowserTest) and (AppletInBrowserTestXslowX) testing methods for all browser
*/tests/jnlp_tests/simple/deadlocktest/srcs/DeadlockTest.java: improved to print sometimes
something out
*tests/jnlp_tests/simple/deadlocktest/testcases/DeadLockTestTest.java:
(testDeadLockTestTerminatedBody) enchanced to ensure that not so much is lost when process is
terminated, but showing that something can be lost (which is correct)
*tests/netx/jnlp_testsengine/net/sourceforge/jnlp/ResourcesTest.java:
(testNonExisitngBrowserWillNotDeadlock) testing taht when no browser is set, then no deadlock
happens as happend before criticalFixes's deadlyException
(testUnexistingProcessWillFailRecognizedly) is actually testing deadlyException
(testNonExisitngBrowserWillNotCauseMess) some but with annotation which was also harmiing output of
tool little bit without TestInBrowsersAnnotation fixed.
(testBrowsers2) is testing all browsers configuration without annotation and (testBrowser3) do the
same configuration tests annotation driven
(testBrowser) body of above two methods. Is testing whether used browsers are correctly linked with
latest build
(testBrowsers1) is testing parsing of -D variable
(testListeners) annotated that needs display
*tests/netx/unit/net/sourceforge/jnlp/runtime/CodeBaseClassLoaderTest.java: annotated with Bug
annotation
browserTests is gathering all previous patches and can be applied and tested
See samples in atached archive!-)
However.... Omair have raised an opinion against this approach (damn.. after it was written) -
http://mail.openjdk.java.net/pipermail/distro-pkg-dev/2012-March/017768.html
Personally I disagree with searching for new engine and I think it is worthy to keep maintain our
framework (as I have made quite progress on it) and is designed exactly for our needs. If it will be
extended for awt robot (soon) and for remote tests (faaar future) it will be quite really nice!-)
And maintaining it is quite fun O:)
I'm also not aware of any framework matching our needs at least from 50% and bending something so
much ...I don't like this idea.
Looking forward to opinions!
Best regards and good luck
J.
Ps: mjw is cced just for case it need approve to distro-pkg O:)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: report.zip
Type: application/zip
Size: 10967 bytes
Desc: not available
Url : http://mail.openjdk.java.net/pipermail/distro-pkg-dev/attachments/20120320/8ef6bdfe/report.zip
-------------- next part --------------
A non-text attachment was scrubbed...
Name: browserTests-TestInBrowsersAnnotation.diff
Type: text/x-patch
Size: 23759 bytes
Desc: not available
Url : http://mail.openjdk.java.net/pipermail/distro-pkg-dev/attachments/20120320/8ef6bdfe/browserTests-TestInBrowsersAnnotation.diff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: browserTests-criticalFixes.diff
Type: text/x-patch
Size: 14198 bytes
Desc: not available
Url : http://mail.openjdk.java.net/pipermail/distro-pkg-dev/attachments/20120320/8ef6bdfe/browserTests-criticalFixes.diff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: browserTests-BugAnnotation.diff
Type: text/x-patch
Size: 17588 bytes
Desc: not available
Url : http://mail.openjdk.java.net/pipermail/distro-pkg-dev/attachments/20120320/8ef6bdfe/browserTests-BugAnnotation.diff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: browserTests-blacklist.diff
Type: text/x-patch
Size: 2815 bytes
Desc: not available
Url : http://mail.openjdk.java.net/pipermail/distro-pkg-dev/attachments/20120320/8ef6bdfe/browserTests-blacklist.diff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: browserTests-MakeLinks.diff
Type: text/x-patch
Size: 3911 bytes
Desc: not available
Url : http://mail.openjdk.java.net/pipermail/distro-pkg-dev/attachments/20120320/8ef6bdfe/browserTests-MakeLinks.diff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: browserTests-tests.diff
Type: text/x-patch
Size: 28292 bytes
Desc: not available
Url : http://mail.openjdk.java.net/pipermail/distro-pkg-dev/attachments/20120320/8ef6bdfe/browserTests-tests.diff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: browserTests.diff
Type: text/x-patch
Size: 89998 bytes
Desc: not available
Url : http://mail.openjdk.java.net/pipermail/distro-pkg-dev/attachments/20120320/8ef6bdfe/browserTests.diff
More information about the distro-pkg-dev
mailing list