JEP 291: Deprecate the Concurrent Mark Sweep (CMS) Garbage Collector

[Daniel Ennis]

Hello,
I've pretty much been the lead on GC research as it applies to Minecraft
servers. I spent a few months researching everything I could on GC tuning,
and analyzing my server using VisualGC inside of VisualVM.

I chose to study G1 as the CMS fragmentation issue constantly causes MC
servers problems with large spikes (which is extremely visible for a
realtime game) when the full collections trigger.

In the end, I was able to optimize G1 to provide 25ms pause times on lower
sized heaps such as 2G, and 40 to 100ms on 10G heap.

GC Logs:
2GB test environment:
https://gist.github.com/aikar/ee619c12103c6d1700e1349bd9e21792
10GB prod environment:
https://gist.github.com/aikar/5fd99dd4b8c4b573a39f5c99ce5dda00

Test Environment - 2GB heap:
Minecraft server with high Entity activity (monsters running AI engines,
collisions to each other) clustered in a single area.
Enough to cause performance problems with the server.



I wrote an article on all of my flag choices and why I chose them here:
https://mcflags.emc.gs (redirects to aquifermc blog post)

I use 10GB heap on my production server and average 50-100ms pauses with
the following flags


 -Dfile.encoding=UTF-8 -XX:+UseG1GC -XX:+UnlockExperimentalVMOptions
-XX:MaxGCPauseMillis=50 -XX:+DisableExplicitGC -XX:G1HeapRegionSize=8M
-XX:TargetSurvivorRatio=90 -XX:+AggressiveOpts
-XX:HeapDumpPath=crash-reports/ -XX:+HeapDumpOnOutOfMemoryError
-Xloggc:logs/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps
-XX:NumberOfGCLogFiles=1 -XX:GCLogFileSize=20M -XX:LargePageSizeInBytes=2M
-XX:+UseLargePages -XX:+AlwaysPreTouch -XX:+UseLargePagesInMetaspace
-XX:G1NewSizePercent=50 -XX:G1MaxNewSizePercent=80
-XX:InitiatingHeapOccupancyPercent=20 -XX:G1MixedGCLiveThresholdPercent=40
-XX:ParallelGCThreads=6 -DprintSaveStats=60 -Xmx10G -Xms10

The key here really is the Experimental options G1NewSizePercent and
G1MaxNewSizePercent. These instructions let me constrain G1's predictive
calculations. You give it a closer to normal window, and it still does its
predictive calculations inside of that window.

With this setup, most collections are young. A few are mixed to keep old
somewhat maintained.

I'm not confident I have the best G1MixedGCLiveThresholdPercent or
InitiatingHeapOccupancyPercent configurations for maintaining old, but
things are so buttery smooth for me now I haven't had justification to mess
with them.

My ultimate focus is on Young Generation. Minecraft servers allocate a ton
of short lived memory.

My end goal is to avoid EVER triggering a full collection. I focus on
keeping the intervals between young collections as long as possible, giving
old only enough room that it ultimately needs + room for unexpected load
(if I gain more than expected, then full collections are triggered)

If your allocation rate is high, and the memory is short lived, having less
space on Eden results in survivor filling up fast, and premature promotions.

Eden is cheap to collect, so my tuning aims to let the memory die while
still in Eden and before age 15 of survivor, providing these consistently
low pause times.

I believe that is the key to maintaining the low pauses, is to keep most
focus in young and avoid having to ever do much work in old.

Hope this helps!