Passing time factor to tests run under jtreg

[Gary Adams]

  I'm not concerned how the timefactor is set.
It could be from makefiles or could be from
developer running jtreg from the command line.

What I'm talking about is the actual test code
being informed that the timefactor is being applied.
e.g. communicated from the test harness to the test case

A typical test using timeouts internally has some expectation
how long particular tasks will take. It might  use 5 seconds for
threads to be set up and then 1 second delay while each
thread is shut down.

An overall timefactor setting will still catch the runaway test,
but in some cases the test will terminate prematurely because
individual steps were not alloted sufficient time.



On 11/16/11 11:21 AM, Kelly O'Hair wrote:
> The jdk/test/Makefile that runs jtreg for JPRT (and the TL Nightly now?), adds 
> a -timefactor:2
> option I think, you could have the test/Makefile detect a slower system and 
> adjust the timefactor on the
> command line there.  Or is that what you are suggesting?
>
> I know it's a bit crude to use Makefiles as an interface for testing, but 
> there is a big advantage to
> JPRT, Nightly testing, and developers being able to run the tests the exact 
> same way.
>
> -kto
>
> On Nov 16, 2011, at 5:08 AM, Gary Adams wrote:
>
>> I'm investigating the current jdk tests with timeouts to see if we
>> can make them more reliable for slower machines. As a a first step,
>> I want to see if the jtreg command line arguments can be made
>> visible to the individual test.
>>
>> Second I want to explore the information about the target machine that
>> can help adjust time limits from the time sensitive tests.
>>
>> -------- Original Message --------
>> Subject: 	Re: Passing time factor to tests run under jtreg
>> Date: 	Tue, 15 Nov 2011 22:45:03 +0000
>> From: 	Alan Bateman <Alan.Bateman at oracle.com>
>> To: 	gary Adams <Gary.Adams at Oracle.COM>
>> CC: 	core-libs-dev at openjdk.java.net
>>
>>
>>
>> Gary - this might be something to bring up on the jtreg-use list.
>> Ideally the tests wouldn't have any hardcoded timeouts but sometimes
>> there isn't any other choice.
>>
>> -Alan
>>
>> On 15/11/2011 20:14, Gary Adams wrote:
>> >   I've been scanning a number of the slow machine test
>> >  bugs that are reported and wanted to check to see if
>> >  anyone has looked into time dependencies in the regression
>> >  tests previously. From what I've been able to learn so far
>> >  individual bugs can use the "timeout" parameter to indicate to
>> >  the test harness an expected time to run.
>> >
>> >  The test harness has command line arguments where it can
>> >  filter out tests that take too long (timelimit) or can apply a
>> >  multiplier to
>> >  to the timeout when conditions are known to slow down the process
>> >  (timeoutFactor). e.g. 8X for a slow machine or running with -Xcomp
>> >
>> >  I see that there are some wrappers that can be applied around running
>> >  a particular test to allow processing before main(). Could this mechanism
>> >  be exploited so the harness command line options could be made known
>> >  to the time dependent tests as command line arguments or as system
>> >  properties?
>> >
>> >  My thought is the current timeout granularity is too large and only
>> >  applies
>> >  to the full test execution. If a test knew that a timeoutFactor was to
>> >  be applied,
>> >  it could internally adjust the time dependent delays appropriately. e.g.
>> >  not every sleep(), await(), join() with timeouts  would need the
>> >  timeoutFactor
>> >  applied.
>> >
>> >  Before any test could be updated the information would need to be
>> >  available
>> >  from the test context.
>> >
>> >  Any feedback/pointers appreciated!
>> >
>> >
>> >  See
>> >   timeoutFactorArg
>> >      jtreg/src/share/classes/com/sun/javatest/regtest/Main.java
>> >   runOtherJVM()
>> >      jtreg/src/share/classes/com/sun/javatest/regtest/MainAction.java
>> >   maxTimeoutValue
>> >
>> >  jtreg/src/share/classes/com/sun/javatest/regtest/RegressionParameters.java
>> >
>> >
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/jtreg-use/attachments/20111116/a509e2c4/attachment.html