RFR: 8253916: ResourceExhausted/resexhausted001 crashes on Linux-x64 [v2]

[Thomas Stuefe]

On Tue, 1 Dec 2020 14:16:14 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Give an error message rather than logging the error and then crashing later because the JVM can't detect stack overflow.  In a resource exhausted situation, thread creation is also failing.  This is the vm_exit_out_of_memory message printed:
>> 
>> `$ java -XX:+UseNewCode -version
>> [0.003s][warning][os,thread] Attempt to protect stack guard pages failed (0x00007f606b249000-0x00007f606b24d000).
>> 
>>  There is insufficient memory for the Java Runtime Environment to continue.
>>  Native memory allocation (mprotect) failed to protect 16384 bytes for memory to guard stack pages
>>  An error report file with more information is saved as:
>>  /16mprotect/hs_err_pid30596.log`
>> `
>
> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
> 
>  - Merge branch 'master' into mprotect
>  - Made resexhaused001.004 manual tests.  You can't reliably run these tests.
>  - 8253916: ResourceExhausted/resexhausted001 crashes on Linux-x64

After reading through the JBS comments I think this makes sense as an intermediate solution, since - as it was pointed out - running on without stack guards could cause crashes on later. The situation is not ideal but this change does not change much.

As far as native memory shortages go, whether or not they result in an OOM is random anyway. pthread_create may fail, which would result in an OOM, or it may cause any number of other APIs to fail, e.g. mmap, which would cause the VM to terminate.

The hotspot changes look good. I cannot comment on the changed test attributes.

src/hotspot/share/runtime/stackOverflow.cpp line 109:

> 107:   }
> 108: 
> 109:   log_debug(os, thread)("Thread " UINTX_FORMAT " stack guard pages activated: "

I always was curious about this uncommit. I wondered whether it was a fallback for platforms where mprotect would not work on thread stacks. But then, why warn unconditionally?

-------------

Marked as reviewed by stuefe (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/1540