[jdk11u-dev] RFR: 8312182: THPs cause huge RSS due to thread start timing issue [v2]

[Thomas Stuefe]

On Mon, 21 Aug 2023 12:16:15 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Unclean composite backport to fix JDK-8312182 - "THPs cause huge RSS due to thread start timing issue" (https://bugs.openjdk.org/browse/JDK-8312182)
>> 
>> Problem:
>> 
>> On a machine with transparent huge pages (THP) unconditionally enabled (/sys/kernel/mm/transparent_hugepage/enabled = "always"), the JVM may show a huge memory footprint (RSS) and degraded thread start performance.
>> 
>> The following factors make the problem more severe and more likely:
>> - thread stack size of 2M (on arm64 or x64) or larger
>> - many threads, or high thread creation churn
>> - a slow or overloaded machine (since part of the problem is timing-dependent)
>> 
>> For a detailed discussion of the underlying problem, please see https://github.com/openjdk/jdk/pull/14919.
>> 
>> ----------------
>> 
>> In jdk Head, the issue got fixed with a sequence of patches:
>> 
>> - [JDK-8303215](https://bugs.openjdk.org/browse/JDK-8303215) "Make thread stacks not use huge pages"
>> - [JDK-8312182](https://bugs.openjdk.org/browse/JDK-8312182) "THPs cause huge RSS due to thread start timing"
>> 
>> However, JDK-8312182 itself needed one preparatory fix:
>> - [JDK-8310233](https://bugs.openjdk.org/browse/JDK-8310233) "Fix THP detection on Linux"
>> 
>> and then we had several corner-case test problems which are fixed with:
>> - [JDK-8312394](https://bugs.openjdk.org/browse/JDK-8312394) "[linux] SIGSEGV if kernel was built without hugepage support"
>> - [JDK-8312620](https://bugs.openjdk.org/browse/JDK-8312620) "WSL Linux build crashes after JDK-8310233"
>> - [JDK-8314139](https://bugs.openjdk.org/browse/JDK-8314139) "TEST_BUG: runtime/os/THPsInThreadStackPreventionTest.java could fail on machine with large number of cores"
>> 
>> and finally, we decided to rename the switch that allows to switch off the THP mitigation with a final patch:
>> - [JDK-8312585](https://bugs.openjdk.org/browse/JDK-8312585) "Rename DisableTHPStackMitigation flag to THPStackMitigation"
>> 
>> 
>> 
>> Instead of downporting these 7 patches verbatim, I prepared a composite patch containing only the necessary mitigation and mitigation tests.
>> 
>> This patch does:
>> - make sure that all thread stacks have at least one glibc guard page to prevent clustering of adjacent thread stacks into one VMA
>> - change the default size of stacks to be not aligned to 2MB to prevent intra-stack THPs from forming
>> 
>> The patch needs some infrastructure, but I downported only the necessary parts: the helper class "HugePages", which is used in head to scan the oper...
>
> Thomas Stuefe has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision:
> 
>  - Merge branch 'openjdk:master' into tstuefe-backport-84b325b8
>  - Backport 84b325b844c08809448a9c073a11443d9e3c3f8e

Friendly ping. It would be good to get this fixed in time for the next CPU.

-------------

PR Comment: https://git.openjdk.org/jdk11u-dev/pull/2086#issuecomment-1691660053