RFR: 8318302: ThreadCountLimit.java failed with "Native memory allocation (mprotect) failed to protect 16384 bytes for memory to guard stack pages"

[Stefan Karlsson]

On Thu, 22 Feb 2024 07:36:15 GMT, David Holmes <dholmes at openjdk.org> wrote:

> The test tries to create as many threads as possible in 5 seconds, with the intention of hitting platform limits on the maximum number of threads that can be created, so that we see that the VM reacts in a reasonable way. However on Linux, if the thread creation limit is high enough, and things run fast enough, then we can instead hit a fatal error when a mmap/mprotect returns E_NOMEM because the maximum number of memory mappings (default 65530) has been exceeded.
> 
> The fix to the test (courtesy @stefank ) is to re-exec the test on Linux with `ulimit -u` set to a much smaller value than the default (I chose 4096 as it still showed over 2000 threads were created before failure). This means we are unlikely to hit the default memory mapping limit before we hit the max threads limit.
> 
> I also updated the VM's OOM error message to include exceeding the maximum number of memory mappings as a possible cause (with a hint as to what to look for in the hs_err log).
> 
> Testing:
>  - ran the test 10x on all platforms in our CI (plus local testing)
> 
> Thanks

Thanks for handling this issue.

-------------

Marked as reviewed by stefank (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/17959#pullrequestreview-1895327648