RFR: 8343285: java.lang.Process is unresponsive and CPU usage spikes to 100%

[Gerard Ziemski]

We lower the max value of allowed file descriptors on macOS to match that of Linux, i.e. 0x100000 (1,048,576).

Background:

On JDK22 and earlier the limit on maximum number of file descriptors allowed in Java process on macOS was 10,240. We increased it in jdk23 to unlimited value (MAX_LONG), as allowed by the os, but that caused a crash in KSH, so we then lowered it to MAX_INT (so it wouldn't overflow an int).

Then we noticed that we had code in jdwp agent that would iterate over all allowed file descriptors trying to close them all, which would time out internal tests, because it was taking so long. We fixed this in jdwp, so that we only close actual opened file descriptors - see https://github.com/openjdk/jdk/commit/a6632487863db5ff3136cdcc76b7440c15ce6be9#diff-1a48137c6688c91d10f931b3e37e4b961b24748fbcb2906d629807aea53db80fR71

However, there are other external jdk clients, that do the exact same thing and their java processes also appear to hang closing all allowed file descriptors.

Since `MaxFDLimit` is ON by default, we have no other choice, but to lower the value, so it doesn't cause regressions.  With this new max value, closing all file descriptors on an old x86_64 machine takes under 1 second on macOS (and Linux).

Testing: in progress...

-------------

Commit messages:
 - llimit max file descriptors to 0x100000, same as on Linux

Changes: https://git.openjdk.org/jdk/pull/21992/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21992&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8343285
  Stats: 7 lines in 1 file changed: 0 ins; 1 del; 6 mod
  Patch: https://git.openjdk.org/jdk/pull/21992.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21992/head:pull/21992

PR: https://git.openjdk.org/jdk/pull/21992