Troubeshooting scanned cards in remembered sets

[Thomas Schatzl]

Hi,

On 23.02.20 22:54, Rémi p wrote:
> I am looking into reducing the young collection pause times in my Java 
> 11 application, particularly in the Scan RS step. AFAIK the duration of 
> this phase is linear in the sum of sizes of the remembered sets of the 
> regions in the collection set (=old->young region references).
> 
> Is there a built-in or recommended way to troubleshoot where all the 
> old->young references are coming from? I was thinking about patching the 
> hotspot code to log the object types of randomly sampled old->young 
> links in order to give me some hints, but I am wondering if there is an 
> easier/less intrusive solution.
> 

At this time there is no way official way to map remembered set entries 
to particular objects. You can get remembered set sizes for each region 
with gc+remset=trace and setting G1SummarizeRSetStatsPeriod to something 
 > 0; the reason for this convoluted process is that in jdk11 getting 
remembered set statistics is expensive (fixed in 14 [2]).

If you want to proceed in that direction, I recommend adding the 
sampling in the "rebuild remembered sets" concurrent phase [3]; hook 
into somewhere in the path starting with 
G1RebuildRemSetHeapRegionClosure::scan_for_references().

What you could try in 11 to decrease remembered set memory usage (and 
typically Scan RS time a bit) is increasing the use of sparse PRTs by 
manually increasing the thresholds when using them. I.e. set 
G1RSetSparseRegionEntries to something much higher than the defaults 
(4/8/12/16/20/24 entries for 1-32M regions respectively).

The JDK-8223162 change [0] in JDK13 does that for you, selecting better 
defaults.

Another option is to increase default region size, but I assume you 
already maxed it out (but you did not describe your previous attempts).

If you can try jdk14, we reimplemented the remembered set scan work 
distribution method [1], which can yield very significant pause time 
improvements. We've seen pause times reduced to 60% of baseline (jdk13 
to jdk14) on benchmarks with very large remembered sets.

Hth,
   Thomas

[0] https://bugs.openjdk.java.net/browse/JDK-8223162
[1] https://bugs.openjdk.java.net/browse/JDK-8213108
[2] https://bugs.openjdk.java.net/browse/JDK-8233919
[3] https://bugs.openjdk.java.net/browse/JDK-8180415