RFR: 8356171: Increase timeout for testcases as preparation for change of default timeout factor

[Leo Korinth]

On Fri, 9 May 2025 08:58:15 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> test/jdk/java/awt/font/NumericShaper/MTTest.java
>> 
>> - * @run main/timeout=300/othervm MTTest
>> + * @run main/timeout=1200/othervm MTTest
>> 
>> I'm puzzling over why you saw this test fail with timeout = 300 .. or perhaps you saw it fail with 0.7 ? Which would amount to 210 seconds .. that might just be enough to cause it to fail because if you look at the whole test you'll see it wants the core loops of the test to run for 180 seconds.
>> 
>> https://openjdk.github.io/cr/?repo=jdk&pr=25122&range=00#new-144-test/jdk/java/awt/font/NumericShaper/MTTest.java
>> 
>> So 300 was fine, and 1200 isn't needed.
>
>> test/jdk/java/awt/font/NumericShaper/MTTest.java
>> 
>>     * * @run main/timeout=300/othervm MTTest
>> 
>> 
>>     * * @run main/timeout=1200/othervm MTTest
>> 
>> 
>> I'm puzzling over why you saw this test fail with timeout = 300 .. or perhaps you saw it fail with 0.7 ? Which would amount to 210 seconds .. that might just be enough to cause it to fail because if you look at the whole test you'll see it wants the core loops of the test to run for 180 seconds.
>> 
>> https://openjdk.github.io/cr/?repo=jdk&pr=25122&range=00#new-144-test/jdk/java/awt/font/NumericShaper/MTTest.java
>> 
>> So 300 was fine, and 1200 isn't needed.
> 
> I started with a timeout factor less than `0.7` but I got hindered by CODETOOLS-7903937. That is probably the reason. Maybe I should change the timeout to 400? I think it is reasonable to handle a timeout factor somewhat less than 1 to weed out tight test cases. But maybe 300 is good enough?

> @lkorinth Moving to a TIMEOUT_FACTOR of 1 seems a good goal. Would it be possible to expand a bit on what repeat testing was done to identify the tests to add /timeout ? If I read it correctly, any tests using /timeout=N have been to bumped to 4*N so no change if the scaling is adjusted in the future. Most tests don't use /timeout so I assume many runs were done to identify the tests that would timeout with if there was no scaling. Test machines vary, as does the test execution times when running concurrently with other tests, so I think it would help if you could say a bit more, even to confirm that it was many test runs.

The code was run as it currently looks (with a timeout factor of `0.7`), what timeout factor do you think I should use to test, and for how many times?

At the moment I am awaiting jtreg 7.6, I therefore guess the timeout factor change to `1` will happen after the fork.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/25122#issuecomment-2865784064