G1 with Solr - thread from dev at lucene.apache.org

[Thomas Schatzl]

Hi Shawn,

On Tue, 2014-12-23 at 14:04 -0700, Shawn Heisey wrote:
> On 12/23/2014 10:55 AM, Thomas Schatzl wrote:
> > Remove NewRatio. This will severely limit adaptiveness.
> > 
> >> -XX:+ParallelRefProcEnabled
> >> -XX:maxGCPauseMillis=200
> > 
> > Use "MaxGCPauseMillis" with capital M.
> > 
> >> -XX:+UseLargePages \
> >> -XX:+AggressiveOpts \
> >> "
> > 
> > I.e.
> > 
> > JVM_OPTS=" \
> >   -XX:+UseG1GC \
> >   -XX:+ParallelRefProcEnabled \
> >   -XX:MaxGCPauseMillis=200 \
> >   -XX:+UseLargePages \
> >   -XX:+AggressiveOpts
> 
> That's what I ultimately ended up with.  I don't have a lot of runtime
> yet, but this is looking REALLY good. 

Great to hear about field experience with G1 - a late Christmas present
for us particularly because they are good.

> It looks like parallel reference
> processing and waiting for a later Java 7 release were the secrets to
> using G1 effectively.

You really might want to try 8u40: the small logs you provided indicate
that there is at least some amount of large object allocation going on
("occupancy higher than threshold [...] cause: G1 Humongous
Allocation"). 8u40 added some special handling for those which allows
fast and cheap reclamation of these in some cases, which improves
throughput by decreasing GC frequency.

Nothing worrying I think given these logs, but these
allocations/messages seem frequent enough that it could be considered
useful.

> After I've got a few hours (and ultimately a few days) of runtime on the
> new settings, I will grab the log and graph it again.

Would be really nice to have them for analysis.

Maybe they could be used to tweak the threshold that starts concurrent
cycles to reduce the number of GCs. 

> 
> Many thanks for all your help!

Did not do anything yet other than removing all CMS flags :)

Thanks,
  Thomas