G1gc compaction algorithm

[Thomas Schatzl]

Hi,

On Thu, 2014-07-17 at 13:31 +0300, Martin Makundi wrote:
>         
>         
>         > -server -XX:InitiatingHeapOccupancyPercent=0 -XX:
>         > +UnlockExperimentalVMOptions
>         -XX:G1MixedGCLiveThresholdPercent=10


>         Just setting the default (65) will give a more reasonable
>         "expected"
>         heap size. (~20G)
>         
>         Depending on length and amount of humongous allocation bursts,
>         you also
>         want to increase -XX:InitiatingHeapOccupancyPercent to
>         something larger
>         than G1MixedGCLiveThresholdPercent, otherwise concurrent
>         marking will
>         run all the time.
> 
> 
> Is it a problem that concurrent marking runs all the time? It's bit
> unclear what it means, our goal is to force gc earn its keep all the
> time, no idle time. However, we are unaware if this low
> G1MixedGCLiveThresholdPercent will affect adversely some other gc
> features.

Constant concurrent marking decreases throughput. If you do not mind the
performance loss it can be acceptable. 

If you evacuate a region with higher occupancy, you need to copy around
more data. However, G1 will automatically take less regions to meet the
pause time goal. It may need more mixed gcs in sequence to reclaim the
same amount of memory (it will always take the ones which are "easiest"
first, which mostly coincides to ones that are less occupied).

*Up to* G1MixedGCCountTarget mixed gcs iirc.

>         You may also need to increase
>         G1MixedGCLiveThresholdPercent if this buffer of 10G is too
>         small.
>
>         >  -XX:G1OldCSetRegionThresholdPercent=85 -Xss4096k
>         -XX:MaxPermSize=512m
>         > -XX:G1HeapWastePercent=1 -XX:PermSize=512m -Xms20G -Xmx30G
>         -Xnoclassgc
>
>         -Xnoclassgc disables all class unloading, even during full
>         gc. 
>
> Class unloading halted the whole system for several minutes every time
> classes unload so we disabled it. It occurred very often and made the
> system practically unusable, so we disabled it completely. 

Okay. That seems extreme though.

>         If you
>         notice increasing "Ext root scan time" over time, this setting
>         is set
>         wrongly. Note that 7u55 can only do class unloading at full
>         gc. Only
>         8u40 and later will also do this at concurrent mark.

[...]
>         
>         >  -XX:GCPauseIntervalMillis=1000 -XX:+PrintGCDetails -XX:
>         +PrintHeapAtGC
>         
>         
>         MaxGCPauseMillis=75 in combination with
>         GCPauseIntervalMillis=1000 seems
>         to be a tough target, at least for 7uX. Does your application
>         really
>         need such a low pause time? It may be achievable.
> 
> 
> It's a web application so basically user will experience significant
> inconvenience in recurringly over 100-200 ms pauses. I am not sure how
> this target time translates to actual user experience so 75 is
> somewhat a safe choice.
>  
>         From the log, already collecting the young generation breaks
>         that pause
>         time goal. Try -XX:G1NewSizePercent=1 to allow smaller young
>         generation
>         sizes.
> 
> 
> How does this work together with the adaptive sizing?

*Roughly*, the next collection's young gen size will be (where X is the
current heap size)

MAX(G1NewSizePercent * X, MIN(X / SurvivorRatio, f(pause-time)))

I.e. G1NewSizePercent bounds the minimum young gen size. So if the
second term is already very low, you might end up with a too large young
gen as in that g1 is not able to collect the young gen within the pause
time already. Or there is not enough time in the time goal to collect a
reasonable amount of old gen regions. Again there are some overrides for
the minimum amount of old gen regions to collect iirc.

Hth,
Thomas