Fwd: Re: Passing time factor to tests run under jtreg

[Gary Adams]

Hi Jon,
    I don't think it is as simple as designating a length of time
a test is allowed to run. In the scenario, I mentioned below
a test needed 5 seconds to perform it's main task and 1 second to
clean up. Internally both operations were manged with separate
timeouts.

   The jtreg harness allows for timefactor=2 to allow for 2*(5+1)
or 12 seconds to complete, but the test as written really needs
(2*5) + (2*1) seconds to run correctly. e.g. it has to allow for 2 seconds
in the cleanup on the slower machine.

   If we can provide the (6 seconds * 2 timefactor) to the test it
could divide up the time between subtasks, but that may be a fairly
complicated computation over a large number of tests currently using
hard coded timeouts.

Gary

=======
With respect, my sense is that this is somewhat misdirected
micromanagement.  It's not the time factor that the test should know,
but the actual allotted time.   Given a certain period of time, a test
could breakdown the allotted time into intervals for each step of its
processing.

Would that work for what you have in mind?

-- Jon

On 11/16/11 8:08 AM, Gary Adams wrote:
> I'm investigating the current jdk tests with timeouts to see if we
> can make them more reliable for slower machines. As a a first step,
> I want to see if the jtreg command line arguments can be made
> visible to the individual test.
>
> Second I want to explore the information about the target machine that
> can help adjust time limits from the time sensitive tests.
>
> -------- Original Message --------
> Subject: 	Re: Passing time factor to tests run under jtreg
> Date: 	Tue, 15 Nov 2011 22:45:03 +0000
> From: 	Alan Bateman <Alan.Bateman at oracle.com>
> To: 	gary Adams <Gary.Adams at Oracle.COM>
> CC: 	core-libs-dev at openjdk.java.net
>
>
>
> Gary - this might be something to bring up on the jtreg-use list.
> Ideally the tests wouldn't have any hardcoded timeouts but sometimes
> there isn't any other choice.
>
> -Alan
>
> On 15/11/2011 20:14, Gary Adams wrote:
> >   I've been scanning a number of the slow machine test
> >  bugs that are reported and wanted to check to see if
> >  anyone has looked into time dependencies in the regression
> >  tests previously. From what I've been able to learn so far
> >  individual bugs can use the "timeout" parameter to indicate to
> >  the test harness an expected time to run.
> >
> >  The test harness has command line arguments where it can
> >  filter out tests that take too long (timelimit) or can apply a
> >  multiplier to
> >  to the timeout when conditions are known to slow down the process
> >  (timeoutFactor). e.g. 8X for a slow machine or running with -Xcomp
> >
> >  I see that there are some wrappers that can be applied around running
> >  a particular test to allow processing before main(). Could this mechanism
> >  be exploited so the harness command line options could be made known
> >  to the time dependent tests as command line arguments or as system
> >  properties?
> >
> >  My thought is the current timeout granularity is too large and only
> >  applies
> >  to the full test execution. If a test knew that a timeoutFactor was to
> >  be applied,
> >  it could internally adjust the time dependent delays appropriately. e.g.
> >  not every sleep(), await(), join() with timeouts  would need the
> >  timeoutFactor
> >  applied.
> >
> >  Before any test could be updated the information would need to be
> >  available
> >  from the test context.
> >
> >  Any feedback/pointers appreciated!
> >
> >
> >  See
> >   timeoutFactorArg
> >      jtreg/src/share/classes/com/sun/javatest/regtest/Main.java
> >   runOtherJVM()
> >      jtreg/src/share/classes/com/sun/javatest/regtest/MainAction.java
> >   maxTimeoutValue
> >
> >  jtreg/src/share/classes/com/sun/javatest/regtest/RegressionParameters.java
> >
> >
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/jtreg-use/attachments/20111116/91a95759/attachment.html