G1gc compaction algorithm

[Thomas Schatzl]

Hi,

  some somewhat late response...

On Sat, 2014-07-19 at 05:38 +0300, Martin Makundi wrote:
> BTW: We have 16 cores and we probably need 4 for serving web users, is
> there a way we can utilize more efficiently the excess resources
> towards gc while keeping user experienced pause at minimum?

You can try assigning threads to the process using the usual OS tools
(cpuset, taskset or one of the other ways).

You also want to decrease the number of parallel gc threads according to
the number of assigned cpus, as the 7uX ergonomics do not check the
actual number of threads the process is allowed to run with.

Others reported improvements when setting the number of parallel gc
threads to the number of available cpu cores (not threads).

Again, this likely depends on your workload.
> 
> 
> 2014-07-17 22:21 GMT+03:00 Martin Makundi
> <martin.makundi at koodaripalvelut.com>:
>         
>                 There are 3 factors: MaxGCPauseMilli,
>                 MixedGCCountTarget, G1OldCSetRegionThresholdPercent
>                 
>                 The candidate regions is calculated by some algorithm.
>                 minimum regions = (candidate
>                 regions)/MixedGCCountTarget
>                 maximum regions = (heap regions)*
>                 G1OldCSetRegionThresholdPercent
>                 
>                 If the estimated mixed gc time is < MaxGCPauseMilli,
>                 g1 will try to add the candidate regions to cset while
>                 keeping the estimated time below MaxGCPauseMilli, as
>                 long as it is less than maximum regions, and
>                 reclaimable percentage higher than the waste limit.
>                 If the estimated mixed gc time is > MaxGCPauseMilli,
>                 g1 will add minimum regions to cset.
>                 
>         
>         
>         What's the science behind these equations (in short) or are
>         they purely ad-hoc?

There are a few reasons for the implementation and use of these
formulas, like backward compatibility (trying to shoehorn functionality
of other collectors on G1, or carrying along earlier wrong decisions),
trying to work around performance anomalies (or bugs :), trying to get
good performance considering a (mythical) average application without
too much tampering, the fact that in pause time control simply more
stuff can go wrong given the GC has no good idea about the application,
and others.

So yes, particularly the default values are somewhat ad-hoc based on
measurements on "representative" applications. As these and the
implementation change over time, they might not always completely fit
your application.

I think in particular the pause time control options are more hairy than
others.

Thomas