Passing time factor to tests run under jtreg

[Jonathan Gibbons]

Gary,

With respect, my sense is that this is somewhat misdirected 
micromanagement.  It's not the time factor that the test should know, 
but the actual allotted time.   Given a certain period of time, a test 
could breakdown the allotted time into intervals for each step of its 
processing.

Would that work for what you have in mind?

-- Jon


On 11/16/2011 08:36 AM, Gary Adams wrote:
> I'm not concerned how the timefactor is set.
> It could be from makefiles or could be from
> developer running jtreg from the command line.
>
> What I'm talking about is the actual test code
> being informed that the timefactor is being applied.
> e.g. communicated from the test harness to the test case
>
> A typical test using timeouts internally has some expectation
> how long particular tasks will take. It might  use 5 seconds for
> threads to be set up and then 1 second delay while each
> thread is shut down.
>
> An overall timefactor setting will still catch the runaway test,
> but in some cases the test will terminate prematurely because
> individual steps were not alloted sufficient time.
>
>
>
> On 11/16/11 11:21 AM, Kelly O'Hair wrote:
>> The jdk/test/Makefile that runs jtreg for JPRT (and the TL Nightly 
>> now?), adds a -timefactor:2
>> option I think, you could have the test/Makefile detect a slower 
>> system and adjust the timefactor on the
>> command line there.  Or is that what you are suggesting?
>>
>> I know it's a bit crude to use Makefiles as an interface for testing, 
>> but there is a big advantage to
>> JPRT, Nightly testing, and developers being able to run the tests the 
>> exact same way.
>>
>> -kto
>>
>> On Nov 16, 2011, at 5:08 AM, Gary Adams wrote:
>>
>>> I'm investigating the current jdk tests with timeouts to see if we
>>> can make them more reliable for slower machines. As a a first step,
>>> I want to see if the jtreg command line arguments can be made
>>> visible to the individual test.
>>>
>>> Second I want to explore the information about the target machine that
>>> can help adjust time limits from the time sensitive tests.
>>>
>>> -------- Original Message --------
>>> Subject: 	Re: Passing time factor to tests run under jtreg
>>> Date: 	Tue, 15 Nov 2011 22:45:03 +0000
>>> From: 	Alan Bateman <Alan.Bateman at oracle.com>
>>> To: 	gary Adams <Gary.Adams at Oracle.COM>
>>> CC: 	core-libs-dev at openjdk.java.net
>>>
>>>
>>>
>>> Gary - this might be something to bring up on the jtreg-use list.
>>> Ideally the tests wouldn't have any hardcoded timeouts but sometimes
>>> there isn't any other choice.
>>>
>>> -Alan
>>>
>>> On 15/11/2011 20:14, Gary Adams wrote:
>>> >   I've been scanning a number of the slow machine test
>>> >  bugs that are reported and wanted to check to see if
>>> >  anyone has looked into time dependencies in the regression
>>> >  tests previously. From what I've been able to learn so far
>>> >  individual bugs can use the "timeout" parameter to indicate to
>>> >  the test harness an expected time to run.
>>> >
>>> >  The test harness has command line arguments where it can
>>> >  filter out tests that take too long (timelimit) or can apply a
>>> >  multiplier to
>>> >  to the timeout when conditions are known to slow down the process
>>> >  (timeoutFactor). e.g. 8X for a slow machine or running with -Xcomp
>>> >
>>> >  I see that there are some wrappers that can be applied around running
>>> >  a particular test to allow processing before main(). Could this mechanism
>>> >  be exploited so the harness command line options could be made known
>>> >  to the time dependent tests as command line arguments or as system
>>> >  properties?
>>> >
>>> >  My thought is the current timeout granularity is too large and only
>>> >  applies
>>> >  to the full test execution. If a test knew that a timeoutFactor was to
>>> >  be applied,
>>> >  it could internally adjust the time dependent delays appropriately. e.g.
>>> >  not every sleep(), await(), join() with timeouts  would need the
>>> >  timeoutFactor
>>> >  applied.
>>> >
>>> >  Before any test could be updated the information would need to be
>>> >  available
>>> >  from the test context.
>>> >
>>> >  Any feedback/pointers appreciated!
>>> >
>>> >
>>> >  See
>>> >   timeoutFactorArg
>>> >      jtreg/src/share/classes/com/sun/javatest/regtest/Main.java
>>> >   runOtherJVM()
>>> >      jtreg/src/share/classes/com/sun/javatest/regtest/MainAction.java
>>> >   maxTimeoutValue
>>> >
>>> >  jtreg/src/share/classes/com/sun/javatest/regtest/RegressionParameters.java
>>> >
>>> >
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/jtreg-use/attachments/20111116/1052b6a9/attachment-0001.html