Fwd: Re: Passing time factor to tests run under jtreg

[Jonathan Gibbons]

I don't understand your reasoning/math in your 2nd para below. Did you 
mean 5 + (2*1) ??    Otherwise, according to my high school
math, 2*(5+1) == (2*5) + (2*1).

Timeouts are not meant to be reached in normal use. They are intended to 
catch runaway tests, doing stuff like "while(true);". Therefore, I don't 
think we should be micromanaging time too much.    If the test is 
written to take 6 seconds on a typical machine, consisting of 5 seconds 
activity and 1 second cleanup, then on a slower machine it should be 
acceptable to use timefactor=2 and give it 12 seconds. Internally, the 
test should not need to know it is running on a slower machine; it 
should proceed to do the same activity as before, and if it now takes 5 
seconds activity + (2*1) cleanup, well,   5 + 2*1 < 2*(5 + 1) and the 
test will continue to pass.

-- Jon



On 11/16/2011 02:08 PM, Gary Adams wrote:
> Hi Jon,
>    I don't think it is as simple as designating a length of time
> a test is allowed to run. In the scenario, I mentioned below
> a test needed 5 seconds to perform it's main task and 1 second to
> clean up. Internally both operations were manged with separate
> timeouts.
>
>   The jtreg harness allows for timefactor=2 to allow for 2*(5+1)
> or 12 seconds to complete, but the test as written really needs
> (2*5) + (2*1) seconds to run correctly. e.g. it has to allow for 2 seconds
> in the cleanup on the slower machine.
>
>   If we can provide the (6 seconds * 2 timefactor) to the test it
> could divide up the time between subtasks, but that may be a fairly
> complicated computation over a large number of tests currently using
> hard coded timeouts.
>
> Gary
> =======
> With respect, my sense is that this is somewhat misdirected
> micromanagement.  It's not the time factor that the test should know,
> but the actual allotted time.   Given a certain period of time, a test
> could breakdown the allotted time into intervals for each step of its
> processing.
>
> Would that work for what you have in mind?
>
> -- Jon
> On 11/16/11 8:08 AM, Gary Adams wrote:
>> I'm investigating the current jdk tests with timeouts to see if we
>> can make them more reliable for slower machines. As a a first step,
>> I want to see if the jtreg command line arguments can be made
>> visible to the individual test.
>>
>> Second I want to explore the information about the target machine that
>> can help adjust time limits from the time sensitive tests.
>>
>> -------- Original Message --------
>> Subject: 	Re: Passing time factor to tests run under jtreg
>> Date: 	Tue, 15 Nov 2011 22:45:03 +0000
>> From: 	Alan Bateman <Alan.Bateman at oracle.com>
>> To: 	gary Adams <Gary.Adams at Oracle.COM>
>> CC: 	core-libs-dev at openjdk.java.net
>>
>>
>>
>> Gary - this might be something to bring up on the jtreg-use list.
>> Ideally the tests wouldn't have any hardcoded timeouts but sometimes
>> there isn't any other choice.
>>
>> -Alan
>>
>> On 15/11/2011 20:14, Gary Adams wrote:
>> >   I've been scanning a number of the slow machine test
>> >  bugs that are reported and wanted to check to see if
>> >  anyone has looked into time dependencies in the regression
>> >  tests previously. From what I've been able to learn so far
>> >  individual bugs can use the "timeout" parameter to indicate to
>> >  the test harness an expected time to run.
>> >
>> >  The test harness has command line arguments where it can
>> >  filter out tests that take too long (timelimit) or can apply a
>> >  multiplier to
>> >  to the timeout when conditions are known to slow down the process
>> >  (timeoutFactor). e.g. 8X for a slow machine or running with -Xcomp
>> >
>> >  I see that there are some wrappers that can be applied around running
>> >  a particular test to allow processing before main(). Could this mechanism
>> >  be exploited so the harness command line options could be made known
>> >  to the time dependent tests as command line arguments or as system
>> >  properties?
>> >
>> >  My thought is the current timeout granularity is too large and only
>> >  applies
>> >  to the full test execution. If a test knew that a timeoutFactor was to
>> >  be applied,
>> >  it could internally adjust the time dependent delays appropriately. e.g.
>> >  not every sleep(), await(), join() with timeouts  would need the
>> >  timeoutFactor
>> >  applied.
>> >
>> >  Before any test could be updated the information would need to be
>> >  available
>> >  from the test context.
>> >
>> >  Any feedback/pointers appreciated!
>> >
>> >
>> >  See
>> >   timeoutFactorArg
>> >      jtreg/src/share/classes/com/sun/javatest/regtest/Main.java
>> >   runOtherJVM()
>> >      jtreg/src/share/classes/com/sun/javatest/regtest/MainAction.java
>> >   maxTimeoutValue
>> >
>> >  jtreg/src/share/classes/com/sun/javatest/regtest/RegressionParameters.java
>> >
>> >
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/jtreg-use/attachments/20111116/d32a5492/attachment.html