Option -gc true considered harmful

[Aleksey Shipilev]

Hi,

On 05/02/2017 06:21 AM, Jens Wilke wrote:
> The result is that without the option -gc true the throughput gets slightly 
> higher. This is contrary to the expectation. Since during the benchmark 
> iteration a GC is triggered multiple times, doing a full GC outside the 
> benchmark measurements should increase throughput.

It is indeed harmful in many cases, as the help message says:

  -gc <bool>      Should JMH force GC between iterations? Forcing
                  the GC may help to lower the noise in GC-heavy benchmarks,
                  at the expense of jeopardizing GC ergonomics decisions.
                  Use with care. (default: false)

This is why it is "false" by default.

> My assumption is that the cause is in the current JMH code which forces the 
> GC. It waits until the GC is finished by examination of a JMX counter value, 
> however the CMS collector actually generates two GC events and two counter 
> increments like this:
> 
> 30.998: [GC (System.gc()) [PSYoungGen: 144096K->3840K(153088K)] ....
> 31.004: [Full GC (System.gc()) [PSYoungGen: 3840K->0K(153088K)]....
> 
> So probably the benchmark iteration starts while the full GC is in progress.
> NB: G1 only generates one event for a forced garbage collection.

Oh. Notice that in the log above, two events are just 6 milliseconds apart.

We might just wait a little before continuing then, like this:
 http://cr.openjdk.java.net/~shade/jmh/gc-wait-settle.patch

-Aleksey