Passing time factor to tests run under jtreg

[Kelly O'Hair]

The jdk/test/Makefile that runs jtreg for JPRT (and the TL Nightly now?), adds a -timefactor:2
option I think, you could have the test/Makefile detect a slower system and adjust the timefactor on the
command line there.  Or is that what you are suggesting?

I know it's a bit crude to use Makefiles as an interface for testing, but there is a big advantage to
JPRT, Nightly testing, and developers being able to run the tests the exact same way.

-kto

On Nov 16, 2011, at 5:08 AM, Gary Adams wrote:

> I'm investigating the current jdk tests with timeouts to see if we 
> can make them more reliable for slower machines. As a a first step,
> I want to see if the jtreg command line arguments can be made 
> visible to the individual test.
> 
> Second I want to explore the information about the target machine that
> can help adjust time limits from the time sensitive tests.
> 
> -------- Original Message --------
> Subject:	Re: Passing time factor to tests run under jtreg
> Date:	Tue, 15 Nov 2011 22:45:03 +0000
> From:	Alan Bateman <Alan.Bateman at oracle.com>
> To:	gary Adams <Gary.Adams at Oracle.COM>
> CC:	core-libs-dev at openjdk.java.net
> 
> Gary - this might be something to bring up on the jtreg-use list. 
> Ideally the tests wouldn't have any hardcoded timeouts but sometimes 
> there isn't any other choice.
> 
> -Alan
> 
> On 15/11/2011 20:14, Gary Adams wrote:
> >  I've been scanning a number of the slow machine test
> > bugs that are reported and wanted to check to see if
> > anyone has looked into time dependencies in the regression
> > tests previously. From what I've been able to learn so far
> > individual bugs can use the "timeout" parameter to indicate to
> > the test harness an expected time to run.
> >
> > The test harness has command line arguments where it can
> > filter out tests that take too long (timelimit) or can apply a 
> > multiplier to
> > to the timeout when conditions are known to slow down the process
> > (timeoutFactor). e.g. 8X for a slow machine or running with -Xcomp
> >
> > I see that there are some wrappers that can be applied around running
> > a particular test to allow processing before main(). Could this mechanism
> > be exploited so the harness command line options could be made known
> > to the time dependent tests as command line arguments or as system
> > properties?
> >
> > My thought is the current timeout granularity is too large and only 
> > applies
> > to the full test execution. If a test knew that a timeoutFactor was to 
> > be applied,
> > it could internally adjust the time dependent delays appropriately. e.g.
> > not every sleep(), await(), join() with timeouts  would need the 
> > timeoutFactor
> > applied.
> >
> > Before any test could be updated the information would need to be 
> > available
> > from the test context.
> >
> > Any feedback/pointers appreciated!
> >
> >
> > See
> >  timeoutFactorArg
> >     jtreg/src/share/classes/com/sun/javatest/regtest/Main.java
> >  runOtherJVM()
> >     jtreg/src/share/classes/com/sun/javatest/regtest/MainAction.java
> >  maxTimeoutValue
> >     
> > jtreg/src/share/classes/com/sun/javatest/regtest/RegressionParameters.java
> >
> >
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/jtreg-use/attachments/20111116/ed3e4882/attachment-0001.html