RFR: 8315923: pretouch_memory by atomic-add-0 fragments huge pages unexpectedly [v7]

[Thomas Stuefe]

On Fri, 20 Oct 2023 05:54:06 GMT, Liming Liu <duke at openjdk.org> wrote:

>> As described at [JDK-8315923](https://bugs.openjdk.org/browse/JDK-8315923), this patch uses madvise with MADV_POPULATE_WRITE to pretouch memory when supported (since kernel 5.14).
>> 
>> Ran the newly added jtreg test on 64c Neoverse-N1 machines with kernel 4.18, 5.13 and 6.1, and observed that transparent huge pages formed right after pretouch on kernel 6.1. Recorded the time spent on the test in *seconds* with `VERBOSE=time` as the table below, and got that the patch takes improvements when the system call is supported, while does not hurt if not supported:
>> 
>> <table>
>>   <tr>
>>     <th>Kernel</th>
>>     <th colspan="2"><tt>-XX:-TransparentHugePages</tt></th>
>>     <th colspan="2"><tt>-XX:+TransparentHugePages</tt></th>
>>   </tr>
>>   <tr><td></td><td>Unpatched</td><td>Patched</td><td>Unpatched</td><td>Patched</td></tr>
>>   <tr><td>4.18</td><td>11.30</td><td>11.30</td><td>0.25</td><td>0.25</td></tr>
>>   <tr><td>5.13</td><td>0.22</td><td>0.22</td><td>3.42</td><td>3.42</td></tr>
>>   <tr><td>6.1</td><td>0.27</td><td>0.33</td><td>3.54</td><td>0.33</td></tr>
>> </table>
>
> Liming Liu has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision:
> 
>   Make the jtreg test check the usage of THP

I am still a bit worried about concurrent usage of the touched memory. Not sure what to do there other than testing. Other than that, the patch is mechanically fine.

src/hotspot/share/runtime/os.cpp line 2119:

> 2117: void os::pretouch_memory_common(void* first, void* last, size_t page_size) {
> 2118:   assert(is_aligned(first, page_size), "pointer " PTR_FORMAT " is not page-aligned by %zu", p2i(first), page_size);
> 2119:   assert(is_aligned(last, page_size), "pointer " PTR_FORMAT " is not page-aligned by %zu", p2i(last), page_size);

New assertions, right? Affects all platforms, not only Linux. If possible, please don't change behavior on other platforms.

Have you checked all users of this function? AFAICT calling this with unaligned pointers would have been possible and would have worked.

test/hotspot/jtreg/runtime/os/TestTransparentHugePageUsage.java line 37:

> 35:  *                   -Xms24G -Xmx24G -XX:+AlwaysPreTouch
> 36:  *                   runtime.os.TestTransparentHugePageUsage
> 37:  */

Requiring the test to need 24G is a lot... why does it need to be so large?

What does the test test? The problem was that pre-touching the memory would allocate small pages, and then later khugepaged would fold them into large pages at its own leisure. Your patch prevents that, so now huge pages form faster? So, the success of the patch can be described by timing?

test/hotspot/jtreg/runtime/os/TestTransparentHugePageUsage.java line 74:

> 72:                     // cover all cases considered to be failures, but we can
> 73:                     // just say the non-usage of THP failes for sure.
> 74:                     System.exit(1);

Please throw RuntimeException for a test error.

-------------

PR Review: https://git.openjdk.org/jdk/pull/15781#pullrequestreview-1694848326
PR Review Comment: https://git.openjdk.org/jdk/pull/15781#discussion_r1370156676
PR Review Comment: https://git.openjdk.org/jdk/pull/15781#discussion_r1370191445
PR Review Comment: https://git.openjdk.org/jdk/pull/15781#discussion_r1370161355