RFR: 8356171: Increase timeout for testcases as preparation for change of default timeout factor

[Alan Bateman]

On Thu, 8 May 2025 16:43:10 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> This change tries to add timeout to individual testcases so that I am able to run them with a timeout factor of 1 in the future (JDK-8260555).
>> 
>> The first commit changes the timeout factor to 0.7, so that I can run tests and test the change (it will finally be changed to 1.0 in JDK-8260555). The next commit excludes some junit/testng tests where I can currently not change the timeout factor (CODETOOLS-7903961). Both these commits will be reverted before integrating the change. I will also apply copyright updates after the review.
>> 
>> In addition to changing the timeout factor, I am also using a library call to parse the timeout factor from the java properties (I can not use the library function everywhere as jtreg does not allow me to add @library notations to non testcase files).
>> 
>> My approach has been to run all test, and afterwards updating those that fails due to a timeout factor. The amount of updated testcases is huge, and my strategy has been to quadruple the timeout if I could not directly see that less was needed (thus the timeout will be the same after JDK-8260555 is implemented). In a few places I have added a bit more timeout so that it will work with the 0.7 timeout factor.
>> 
>> These fixes have been created when I have plown through testcases:
>> JDK-8352719: Add an equals sign to the modules statement
>> JDK-8352709: Remove bad timing annotations from WhileOpTest.java
>> JDK-8352074: Test MemoryLeak.java seems not to test what it is supposed to test
>> CODETOOLS-7903937: JTREG uses timeout factor on socket timeout but not on KEEPALIVE
>> CODETOOLS-7903961: Make default timeout configurable
>> 
>> Sometime in the future I will also fix:
>> 8260555: Change the default TIMEOUT_FACTOR from 4 to 1
>> 
>> for which I am awaiting:
>> CODETOOLS-7903961 that is fixed in jtreg 7.6, but we are still running 7.5.1+1
>> 
>> *After the review I will revert the two first commits, and update the copyrights*
>
> After I revert the two first commits and go back to a timeout factor of 4, I will run tier 1-8 again.

@lkorinth Moving to a TIMEOUT_FACTOR of 1 seems a good goal. Would it be possible to expand a bit on what repeat testing was done to identify the tests to add /timeout ?  If I read it correctly, any tests using /timeout=N have been to bumped to 4*N so no change. Most tests don't use /timeout so I assume many runs were done to identify the tests that would timeout with if there was no scaling. Test machines vary, as does the test execution times when running concurrently with other tests, so I think it would help if you could say a bit more, even to confirm that it was many test runs.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/25122#issuecomment-2865581927