RFR: 8253916: ResourceExhausted/resexhausted001 crashes on Linux-x64 [v2]

[Coleen Phillimore]

On Tue, 1 Dec 2020 20:16:05 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
>> 
>>  - Merge branch 'master' into mprotect
>>  - Made resexhaused001.004 manual tests.  You can't reliably run these tests.
>>  - 8253916: ResourceExhausted/resexhausted001 crashes on Linux-x64
>
> After reading through the JBS comments I think this makes sense as an intermediate solution, since - as it was pointed out - running on without stack guards could cause crashes on later. The situation is not ideal but this change does not change much.
> 
> As far as native memory shortages go, whether or not they result in an OOM is random anyway. pthread_create may fail, which would result in an OOM, or it may cause any number of other APIs to fail, e.g. mmap, which would cause the VM to terminate.
> 
> The hotspot changes look good. I cannot comment on the changed test attributes.

Thanks for reviewing this Thomas, and for more information about this failure.  I answered your comments in the JBS issue.  It might be possible to throw OOM instead of vm_exit_out_of_memory but it's not very clean where it'll propagate in the code, and for this low native memory situation, I don't think it's going to help the test make more progress.
This has been a longstanding problem with this test so this change makes a cleaner exit for an unlikely application which may also have the same problem.

-------------

PR: https://git.openjdk.java.net/jdk/pull/1540