Using G1 with Apache Solr

[charlie hunt]

If on Linux, most definitely disable THP (transparent huge pages). You will likely not have a good experience with any GC with THP enabled.

charlie

> On Mar 25, 2015, at 1:05 PM, Kamran Khawaja <narmak101 at gmail.com> wrote:
> 
> Solr is being run on a CentOS 7 server.  Both the os and java are 64 bit.  I see that THP is enabled on the server.
> I'll have to discuss with the rest of my team about disabling THP and upgrading to java 8 but I'll post back when I have some results from my testing.
> 
> 
> Thanks,
> 
> --
> Kamran Khawaja
> 
> 
> On Wed, Mar 25, 2015 at 2:47 AM, Shawn Heisey <java at elyograg.org <mailto:java at elyograg.org>> wrote:
> On 3/24/2015 3:48 PM, Kamran Khawaja wrote:
> > I'm running Solr 4.7.2 with Java 7u75 with the following JVM params:
> >
> >     -verbose:gc
> >     -XX:+PrintGCDateStamps
> >     -XX:+PrintGCDetails
> >     -XX:+PrintAdaptiveSizePolicy
> >     -XX:+PrintReferenceGC
> >     -Xmx3072m
> >     -Xms3072m
> >     -XX:+UseG1GC
> >     -XX:+UseLargePages
> >     -XX:+AggressiveOpts
> >     -XX:+ParallelRefProcEnabled
> >     -XX:G1HeapRegionSize=8m
> >     -XX:InitiatingHeapOccupancyPercent=35
> >
> >
> > What I'm currently seeing is that many of the gc pauses are under an
> > acceptable 0.25 seconds but seeing way too many full GCs with an average
> > stop time of 3.2 seconds.
> >
> > You can find the gc logs
> > here: https://www.dropbox.com/s/v04b336v2k5l05e/g1_gc_7u75.log.gz?dl=0 <https://www.dropbox.com/s/v04b336v2k5l05e/g1_gc_7u75.log.gz?dl=0>
> >
> > I initially tested without specifying the HeapRegionSize but that
> > resulted in the "humongous" message in the gc logs and a ton of full gc
> > pauses.
> 
> When I replied the first time, I only sent it to Kamran.  I quickly
> realized that I'd made that error, but I did not remember that the
> original message was on this list, so I sent the reply again, assuming
> that I saw the original on the solr-user mailing list.  Now I am
> bringing the silliness full-circle by sending the same reply here.
> 
> Some additional info:
> 
> When I initially brought my settings up on this list a few months ago, I
> got the recommendation to try changing InitiatingHeapOccupancyPercent to
> 70-75 from the default of 45 ... so setting it to 35 might not be the
> best idea.  I do currently have it set to 75 (not reflected on the
> wiki), but I haven't done any further analysis.
> 
> I have now upgraded java on those machines to 8u40 with the following
> settings, I hope to have a useful gc.log soon for comparison purposes.
> 
> -XX:+UseG1GC -XX:+ParallelRefProcEnabled -XX:G1HeapRegionSize=8m
> -XX:MaxGCPauseMillis=250 -XX:InitiatingHeapOccupancyPercent=75
> -XX:+UseLargePages -XX:+AggressiveOpts
> 
> 
> ---- original reply ----
> 
> This is similar to the settings I've been working on that I've
> documented on my wiki page, with better results than you are seeing, and
> a larger heap than you have configured:
> 
> https://wiki.apache.org/solr/ShawnHeisey#G1_.28Garbage_First.29_Collector <https://wiki.apache.org/solr/ShawnHeisey#G1_.28Garbage_First.29_Collector>
> 
> You have one additional option that I don't --
> InitiatingHeapOccupancyPercent.  I would suggest running without that
> option to see how it affects your GC times.
> 
> I'm curious what OS you're running under, whether the OS and Java are
> 64-bit, and whether you have actually enabled huge pages in your
> operating system.  If it's Linux and you have enabled huge pages, have
> you turned off transparent huge pages as documented by Oracle:
> 
> https://blogs.oracle.com/linux/entry/performance_issues_with_transparent_huge <https://blogs.oracle.com/linux/entry/performance_issues_with_transparent_huge>
> 
> On my servers, I do *not* have huge pages configured in the operating
> system, so the UseLargePages java option isn't doing anything.
> 
> One final thing ... Oracle developers have claimed that Java 8u40 has
> some major improvements to the G1 collector, particularly for programs
> that allocate very large objects.  Can you try 8u40?
> 
> Thanks,
> Shawn
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net <mailto:hotspot-gc-use at openjdk.java.net>
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use <http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use>
> 
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20150325/bf4b6b59/attachment-0001.html>