Bug in G1GC it performs Full GC when code cache is full resulting in overkill

[Darius D.]

Hi,

since I see a reference to my old post with G1 problems, I felt the
need to share our success story with G1 collector.

We have been using G1 in production since ~1.7_04, the main reason
being that CMS was generating way too many FullGCs (in retrospective
this is already a hint what the real problem was ...).

It turned out that we were getting Full GCs due to humongous object
allocation failures, our application generates JSON objects sized in
quite a few megabytes and due to unfortunate design of web framework
this was causing plenty of reallocation during character encode phase
spraying heap in progress. Once proper instrumentation was in place
for G1GC in mid-late 2012, we were seeing humongous allocation
failures of some 30+ megabytes or so. No wonder that in a busy heap
there was not enough continuous space for object this large ( remember
realloc chain burned 16, 8, 4, 2 etc megabyte sized  chunks down to
half of G1HeapRegionSize).

So we set out to fix it:

1) We got performance patch merged into our open source web framework
that slashed reallocations and rewrote our own code that was
generating JSON String to limit reallocation.
2) Tuning  -XX:G1HeapRegionSize once we saw proper explanation of G1GC
in Monica's JavaOne presentation. For whatever reason for ~7-8GB heap
we were getting thousands of G1GC heap regions and we've arrived to
-XX:G1HeapRegionSize=16m after some testing.

The fact is after all this tuning G1GC has been performing amazingly
for us { -XX:G1HeapRegionSize=16m
-XX:InitiatingHeapOccupancyPercent=33 -XX:MaxGCPauseMillis=250
-XX:+UseG1GC -Xmx7168m -Xms7168m -XX:MaxPermSize=768m
-XX:ReservedCodeCacheSize=128m }. We haven't seen FullGC in production
for months now and we have to work really hard to generate them in
testing too. Our GC pauses are sub 0.3s, giving us desired web app
performance.


Keep up great work! :)


Darius.


On Sun, May 26, 2013 at 6:20 PM, Martin Makundi
<martin.makundi at koodaripalvelut.com> wrote:
> Sorry, forgot to mention, using:
>
> java version "1.7.0_21"
> Java(TM) SE Runtime Environment (build 1.7.0_21-b11)
> Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode)
>
> Linux version 3.0.1.stk64 (dfn at localhost.localdomain) (gcc version
> 4.5.1 20100924 (Red Hat 4.5.1-4) (GCC) ) #1 SMP Sat Aug 13 12:53:46
> EDT 2011
>
> -Dclassworlds.conf=/usr/share/maven/maven/bin/m2.conf
> -Dmaven.home=/usr/share/maven/maven
> -Duser.timezone=EET
> -XX:+AggressiveOpts
> -XX:+DisableExplicitGC
> -XX:+ParallelRefProcEnabled
> -XX:+PrintGCDateStamps
> -XX:+PrintGCDetails
> -XX:+PrintHeapAtGC
> -XX:+UseAdaptiveSizePolicy
> -XX:+UseCompressedOops
> -XX:+UseFastAccessorMethods
> -XX:+UseG1GC
> -XX:+UseGCOverheadLimit
> -XX:+UseNUMA
> -XX:+UseStringCache
> -XX:CMSInitiatingOccupancyFraction=70
> -XX:GCPauseIntervalMillis=10000
> -XX:InitiatingHeapOccupancyPercent=0
> -XX:MaxGCPauseMillis=500
> -XX:MaxPermSize=512m
> -XX:PermSize=512m
> -XX:ReservedCodeCacheSize=48m
> -Xloggc:gc.log
> -Xmaxf1
> -Xms30G
> -Xmx30G
> -Xnoclassgc
> -Xss4096k
>
>
> **
> Martin
>
> 2013/5/26 Charlie Hunt <chunt at salesforce.com>:
>> Which version of the JDK/JRE are you using?
>>
>> One of the links you referenced below was using JDK 6, where there is no official support for G1. The other link suggests it could have been RMI DGC or a System.gc().
>>
>>
>>
>> Sent from my iPhone
>>
>> On May 25, 2013, at 11:43 PM, "Martin Makundi" <martin.makundi at koodaripalvelut.com> wrote:
>>
>>> it occurs daily.
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use