RFR: 8340426: ZGC: Move defragment out of the allocation path [v2]

[Erik Österlund]

On Fri, 27 Sep 2024 08:34:19 GMT, Stefan Johansson <sjohanss at openjdk.org> wrote:

>> Please review this change to move defragmentation of small pages out of the allocation path,
>> 
>> **Summary**
>> In ZGC small pages are allocated at lower addresses while medium and large pages are allocated at higher addresses. Sometimes when memory usage is high or the distribution of pages in the cache demand it, small pages can be split out from medium or large pages. When this happens the small page is defragmented by remapping it to a lower address if it resides at a too high address. This is done to avoid fragmentation, but doing it in the allocation path comes with a cost since the remapping involves system calls.
>> 
>> This change moves the remapping away from the allocation path to when the pages are freed after being garbage collected. This can increase the fragmentation a bit, since small pages are allowed to reside at higher addresses for a period time. The expectation is that since small pages are frequently collected the period should be short enough to not cause any problems.
>> 
>> I've done detailed experiments with temporary JFR events to look at the cost of doing the defrag/remapping of pages. One problem is that by default we don't get a lot of defrags (unless we run out of memory), and to better investigate this I also altered the allocation path to force more defragmentation. These tests showed that moving defrag out of the allocation path not only have the positive effect of reducing the time spent allocating, but also spaced out the defragmentation a bit more.  The reason for this is that now we will defrag when a page is freed by the GC, instead of when the page is allocated and the tests show that often many threads allocate at the same time (and then also defrag at the same time). This in turn leads to many concurrent calls down to the system to remap the memory, and this increases the cost of the actual mapping (seen by adding JFR events).
>> 
>> **Additional testing**
>> 
>> - Functional testing in mach5 tier1-7
>> - Sanity performance testing in aurora
>
> Stefan Johansson has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Additional changes
>  - StefanK review

Looks good.

-------------

Marked as reviewed by eosterlund (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/21191#pullrequestreview-2347441445