RFR: 8339161: ZGC: Remove unused remembered sets

[Axel Boldt-Christmas]

On Wed, 11 Sep 2024 12:15:47 GMT, Joel Sikström <duke at openjdk.org> wrote:

> In ZGC, when a page becomes old it needs a remembered set (remset) which stores 2 bits per byte, a memory overhead of 3.125% (2/64) per page that stores an allocated remset.
> 
> When an old page is potentially freed and inserted in the page cache, it can later be re-used as a young page. In this case, the remset is still allocated even though the young page does not need it. This is especially noteworthy for long-running programs where pages are recycled for a long enough period to have a remset allocated for close to all pages.
> 
> The attached plot shows remset memory usage for pages that are "live" for a program that frequently recycles pages using a cache-mechanism. As remsets for young pages are unused, it should be considered wasted memory.
> 
> ![remset_waste](https://github.com/user-attachments/assets/2a60948b-9297-4554-8fb4-d9f527855c33)
> 
> The alternative solution that I propose in this PR deletes/frees remsets when an old page is inserted into the page cache, to not wast ememory. This would mean that remsets are only stored for old pages that are in use. Pages that are not in use or are young, should not have an allocated remset. With this change, the line showing remset usage for young pages would disappear in the plot above and the total memory usage dictated by the number of "live" old pages.
> 
> Below is a performance measurement of initializing vs. clearing remsets in the same cache program mentioned above. When deleting remsets, freeing is made by GC threads so there is no latency impact for mutators. Initializing remsets is on average ~2.3x slower than clearing, but also uses 3.125% less memory for pages that do not need a remset. ~89% of the measured initializations are made by GC threads and the rest by mutator threads.
> 
> |              | min (ms) | max (ms) | mean (ms)  |
> | ------------ | -------- | -------- | ---------- |
> | remset init  | 0.000292 | 0.706    | 0.00258083 |
> | remset clear | 0.000082 | 0.015    | 0.00111340 |
> 
> Tested with tiers 1-7 and local test making sure there are no remsets for young pages. SPECjbb2015 performance measurements show no statistically significant regression/improvement.

lgtm. Nicely done.

-------------

Marked as reviewed by aboldtch (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/20947#pullrequestreview-2313491254