RFR: 8318302: ThreadCountLimit.java failed with "Native memory allocation (mprotect) failed to protect 16384 bytes for memory to guard stack pages" [v2]

[Thomas Stuefe]

On Tue, 27 Feb 2024 05:55:55 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> The test tries to create as many threads as possible in 5 seconds, with the intention of hitting platform limits on the maximum number of threads that can be created, so that we see that the VM reacts in a reasonable way. However on Linux, if the thread creation limit is high enough, and things run fast enough, then we can instead hit a fatal error when a mmap/mprotect returns E_NOMEM because the maximum number of memory mappings (default 65530) has been exceeded.
>> 
>> The fix to the test (courtesy @stefank ) is to re-exec the test on Linux with `ulimit -u` set to a much smaller value than the default (I chose 4096 as it still showed over 2000 threads were created before failure). This means we are unlikely to hit the default memory mapping limit before we hit the max threads limit.
>> 
>> I also updated the VM's OOM error message to include exceeding the maximum number of memory mappings as a possible cause (with a hint as to what to look for in the hs_err log).
>> 
>> Testing:
>>  - ran the test 10x on all platforms in our CI (plus local testing)
>> 
>> Thanks
>
> David Holmes has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Invert logic to avoid negation.

So... if we reach the limit "max processes per user" we expect OOM, but if we reach vm.max_map_count, a fatal error is okay?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/17959#issuecomment-1965848918