File descriptors not getting closed by JVM in latest MacBooks?

[H. Mijail]

Hi there, 

I'm a member of the Hyperledger Besu project's dev team. Our main product is open source, requires Java 11+, builds using gradle and has a relatively good suite of tests.

Recently (last few days, maybe 1 week) 3 members of our team had to get new Macbooks. For some reason, the project builds correctly on their machines but some of the tests fail. We have determined that the java processes that run the tests are not closing file descriptors, or at least they don't close them as fast as other macOS machines do. So a test fails when the process running it reaches ~10250 file descriptors, as reported by lsof. At that point, there is an IOException (Too many open files).

Our other 10+ Macbooks (1-2 years old) never even get close to opening such a high number of file descriptors while building and running the same tests.

This happens with every JDK we have tried - versions 11, 12 and 13. Oracle, OpenJDK, AdoptOpenJDK. 
2 of the problematic Macbooks run macOS Catalina 10.15, BUT at least one of the new Macbooks came with macOS 10.14.6 (18G95) and also showed the same problem.
The older Macbooks run everything perfectly in a variety of configurations: from macOS 10.14.6 with JDK 13 to macOS 10.15 with JDK 11.

We have managed to make the tests finish successfully in the affected Macbooks by lifting the limits of open files in the JVM. For this we had to make gradle pass the -XX:-MaxFDLimit option to the tester JVMs, plus rise the system's limits for file descriptors in launchd, *plus disable SIP*. Which is crazy, but removing any of the steps breaks the tests again.

But of course this is just a workaround; the proper fix would be to make the JVM close the FDs, just as it does in other Macbooks.

So my question is: any idea of what can be happening? Has anyone seen this kind of thing? Any recommendation of where to dig to find some more concrete bug to report?

If I had to diagnose this myself, I would try to get some more fine characterization of the number of FDs used by the JVM in each Macbook, probably with DTrace. But even if I get that smoking gun, I'm not sure I could do anything about it - apart from waiting for some (probably) OS fix?

Best,
--- Horacio