From Jonathan.Gibbons at Sun.COM  Wed Jan 14 13:38:12 2009
From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons)
Date: Wed, 14 Jan 2009 13:38:12 -0800
Subject: Allowing partial jtreg samejvm safe runs
In-Reply-To: <1227460674.29485.26.camel@hermans.wildebeest.org>
References: <1227460674.29485.26.camel@hermans.wildebeest.org>
Message-ID: <496E5B44.2090208@sun.com>

Mark,

Thanks for the idea behind this patch, and for the patch too.

To you and other jtreg users,

I'd be interested in comments regarding the relative merits of
white-listing vs black-listing.  

I'd also be interested in comments regarding the feature of
only using the whitelist when running the whole test suite.
An alternative proposal would be to only use the whitelist
in a default mode, where neither -othervm or -samevm is
defined.  Specifying either switch would disable the whitelist
(or blacklist?) lookup.

-- Jon




Mark Wielaard wrote:
> Hi,
>
> Jonathan Gibbons has been extending jtreg so that tests can be run in
> the same vm. He also made it so that most langtools tests are run that
> way. That brings an enormous speed up. Since running all jdk tests takes
> ages it would be great if we could use this same speedup there.
>
> Problem is that -samevm only works if all tests have been adapted to
> behave well (or marked as needing /othervm). So my idea was to add a
> "whitelist mechanism" so you can indicate which (sub)dirs are ready to
> be run with samevm.
>
> Luckily jtreg has mostly been setup for this already. All Actions ask
> their Script whether or not they should be running in an othervm. So we
> hook into that to check the scripts location against a whitelist.
>
> I choose the TEST.ROOT file as the location of the whitelist. Since that
> is also the root of the test suite. I made the code so that the
> whitelist is only used when running the full test suite from the root.
> That way you can still experiment with -samevm or -othervm when running
> subtrees of tests.
>
> Also CCed jdk-regtest at sun.com since that address is mentioned in the
> TEST.ROOT file with an request to contact them if you change anything in
> that file.
>
> Below is the implementation as I checked into icedtea6 (patch also
> attached).
>
> 2008-11-23  Mark Wielaard  <mark at klomp.org>
>
>     * test/jtreg/com/sun/javatest/regtest/Main.java
>     (createParameters): Set same jvm safe dirs when non-null.
>     (getSameJVMSafeDirs): New method that reads samejvmsafe property
>     from TEST.ROOT.
>     (sameJVMSafeDirs): New private field.
>     * test/jtreg/com/sun/javatest/regtest/RegressionParameters.java
>     (SAME_JVM_SAFE_DIRS): New static final field.
>     (load): Read same jvm safe dirs.
>     (save0): Write same jvm safe dirs.
>     (getSameJVMSafeDirs): New method.
>     (setSameJVMSafeDirs): New method.
>     * test/jtreg/com/sun/javatest/regtest/RegressionScript.java
>     (run): Set testDirPath.
>     (isOtherJVM): Take same jvm safe into account.
>     (isSameJVMSafe): New method.
>     (testDirPath): New private field.
>
> For icedtea6 I did a quick scan of the openjdk/jdk/test subdirs and
> listed an initial set that work with -samejvm and added a patch to
> augment the TEST.ROOT file (attached).
>
> 2008-11-23  Mark Wielaard  <mark at klomp.org>
>
>     * patches/icedtea-samejvm-safe.patch: New patch.
>     * Makefile.am (ICEDTEA_PATCHES): Add new patch.
>     (check-jdk): Run jtreg with -samevm.
>     * HACKING: Document new patch.
>
> Comments on the idea and the implementation very appreciated.
>
> With this 'make check-jdk' takes a little bit less than 3 hours on my
> machine now. It was almost 3.5 hours, so it already saves a lot of time.
> But we need to make much more tests work with samevm to bring the total
> test running time down to something an ordinary developer would be
> comfortable with running every day.
>
> We have to see whether or not that is really possible. It seems jdi, rmi
> and awt take up a lot of the time needed to run all tests, and I am not
> sure those tests can be speed up much.
>
> Cheers,
>
> Mark
>