RFR: 8366298: FDLeakTest sometimes takes minutes to complete on Linux [v3]

[Stefan Karlsson]

On Tue, 2 Sep 2025 08:03:27 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> While investigating [JDK-8260555](https://bugs.openjdk.org/browse/JDK-8260555), which lowers the timeout factor from 4 to 1, we found that FDLeakTest sometimes times out on Linux.
>> 
>> The reason is that the test performs a `fcntl` call for each and every potential file descriptor number. This can be a large number of calls and sometimes results in minutes-long test executions.
>> 
>> I propose that we fix this by limiting the max number of open file descriptors. This lowers the test execution time to about 1 second.
>> 
>> The test has two processes. One that executes the libFDLeaker.c code below as an agent in the test JVM, then it forks into a exeFDLeakTester.c, which reads the property `int max_fd = (int)sysconf(_SC_OPEN_MAX);`. The setting of `RLIMIT_NOFILE` to `100` lowers `max_fd` to `100`. I've verified this on both Linux and on macOS.
>> 
>> I've run the test manually on Linux and macOS and verified that it runs faster. I've also run this through tier1.
>
> Stefan Karlsson has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:
> 
>  - Remove jtreg timeouts
>  - Merge remote-tracking branch 'upstream/master' into 8366298_fdleaktest
>  - Add error logging
>  - 8366298: FDLeakTest sometimes takes minutes to complete on Linux

:D Thanks for nudging the Skara bots.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26979#issuecomment-3249726291