JEP 248: Make G1 the Default Garbage Collector

[Andrew Haley]

On 07/30/2015 04:03 PM, charlie hunt wrote:
> 
> Not that I like to talk negatively of benchmarks, and I respect the
> SPEC organization … Let’s keep in mind SPECjvm is a throughput
> benchmark. It does not include latency or footprint metrics. It also
> has other weaknesses including (though not limited too); doesn’t run
> long enough, the workloads are rather trivial, can be tuned to the
> point where it can run with Parallel GC without experiencing a full
> GC. As an illustration, how many apps in the wild do you know run
> for 4 minutes and have rather predictable object allocation rates
> and predictable object lifetimes?

I absolutely agree with you.

Having said that, SPEC is not a simple benchmark, it's a bunch of
programs from a bunch of places, and they don't have much in common
beyond being self-contained and running for a few minutes.  I'm not
claiming that SPEC is the right thing to use to measure garbage
collection: I just want to make sure that when JDK9 comes out and
J. Random Blogger comes along with "Hey!  Java just got slower" we
have a good answer for them.  A good answer would be something along
the lines of "We know that, but G1 is in general better, and this is
why."

> A performance regression in javac is worthy of worrying. It is an
> important part of the developer use case. As you are probably aware,
> one thing we should be careful about is reporting only
> percentages. For example, a 4 second versus 5 second javac
> compilation comparison warrants a 20% difference. 

Sure.  I suppose I was guessing that javac is typical of a of
Java application which parses lots of files, creates a lot of
hash tables and suchlike, and generates some output.  I was also
assuming that javac is well-written, which is very important when
considering a benchmark.  We surely don't want to optimize for
badly-written code which pointlessly allocates a ton of objects!

> In a developer use case, that 1 second difference in javac
> compilation may not be as noticeable or worrisome compared to an 8
> minutes versus 10 minutes difference, (that too being a 20%
> difference).

OK, just for fun:

java -XX:+UseParallelGC -jar dist/CompilerSpeed.jar 480 12
12 threads, 480 seconds
Time: 481.35s, 13.38 compiles/s

java -XX:+UseG1GC -jar dist/CompilerSpeed.jar 480 12
12 threads, 480 seconds
Time: 481.00s, 11.49 compiles/s

So, G1 takes 16% longer when run for 8 minutes, which is less
significant.

> One data point you can take a look at is from a J1 presentation that
> shows an application migration from Parallel GC to G1:
> https://www.parleys.com/tutorial/garbage-first-garbage-collector-migration-expectations-advanced-tuning <https://www.parleys.com/tutorial/garbage-first-garbage-collector-migration-expectations-advanced-tuning>
> * Start at about the 7:00 minute mark through about the 29 minute mark.
> ** Also note that this is about 18 - 24 months old, and many enhancements have gone into G1 since then.
> 
> I realize this is just one application. And, fwiw, it is a more
> complex application than SPECjvm. ;-)

This is excellent, thank you.  In particular, it is made clear quite
early on that the Parallel GC, if it is suitable for your application,
probably has the best lowest overheads of any of the GCs.  But
throughput and overhead not all we care about, of course.  They tend
to be all that benchmarks care about because they're the easiest
things to measure.  The collector which is the default has to be the
best over a variety of metrics.

> In general what you find with G1 is in the face of fluctuating
> object allocation rates and object lifetimes, and applications that
> run for extended period of time, when taking into account
> throughput, latency and memory footprint, G1 tends to meet those
> goals better than Parallel GC.

Okay, so we could perhaps say (to J.R. Blogger) that we are not
surprised to find some throughput degradation in Java applications,
but in general, over a balance of criteria, G1 is likely to be a
better default, especially for large and long-lived programs.

And we need to prepare the ground when JDK9 is released for people who
will immediately measure it with their favourite benchmark.  At least
explain to them that they should use the same GC for the comparison.

Andrew.