RFR: 8325403: Add SystemGC JMH benchmarks [v2]

[Thomas Schatzl]

On Mon, 12 Feb 2024 13:23:40 GMT, Stefan Johansson <sjohanss at openjdk.org> wrote:

>> Please review this change to add some System GC JMH benchmarks.
>> 
>> **Summary**
>> When developing and improving the Full GC implementation for the different GCs, using `System.gc()` to test performance is sometime very useful. This change adds a set of System GC benchmarks using JMH. The tests all have the same structure, a `@Setup` function that generates heap contents and a `@Benchmark` function which only calls `System.gc()`. The different scenarios covers some basic use-cases but more can certainly be added going forward. 
>> 
>> Since the benchmark function is just a single call to trigger the GC it makes sense to use the single shot JMH mode and fork a new JVM for each run. The tests also specify default heap options to avoid triggering GC during object creation to give all runs more or less the same heap layout.
>> 
>> **Testing**
>> Local and aurora testing verifying that results are fairly stable.
>
> Stefan Johansson has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Restructure and rename

Would like to discuss the use of `AlwaysPreTouch` before pushing....

test/micro/org/openjdk/bench/vm/gc/system_gc/AllDead.java line 23:

> 21:  * questions.
> 22:  */
> 23: package org.openjdk.bench.vm.gc.system_gc;

I do not like a package name with an underscore in it, but I can live with that.

test/micro/org/openjdk/bench/vm/gc/system_gc/AllDead.java line 38:

> 36: import java.util.concurrent.TimeUnit;
> 37: 
> 38: @BenchmarkMode(Mode.SingleShotTime)

I had a discussion with Stefan about the usefulness of using single-shot mode (i.e. no warmup, no iterations).
This can make algorithms (like Serial GC MSC) look better than they are as parts of the algorithm may be dependent on existing dead objects (i.e. it needs to iterate over dead objects to find marked objects). However setting up a "fair" dead object distribution isn't that easy. One could simply just assume the previous' GC's dead objects are representative "fair", but one can argue endlessly about it.

Just something to consider when interpreting the results.

test/micro/org/openjdk/bench/vm/gc/system_gc/AllDead.java line 39:

> 37: 
> 38: @BenchmarkMode(Mode.SingleShotTime)
> 39: @Fork(value=25, jvmArgsAppend={"-Xmx5g", "-Xms5g", "-Xmn3g"})

Please use `-XX:+AlwaysPreTouch` by default. Otherwise some of the tests degrade to testing the memory paging.

E.g. `gc+cpu` logging gives

`[0,837s][info][gc,cpu   ] GC(0) User=0,31s Sys=0,15s Real=0,45s`

when running with serial gc, i.e. 1/3rd of time is spent outside paging in memory.

-------------

Changes requested by tschatzl (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/17784#pullrequestreview-1877719437
PR Review Comment: https://git.openjdk.org/jdk/pull/17784#discussion_r1487690053
PR Review Comment: https://git.openjdk.org/jdk/pull/17784#discussion_r1487698457
PR Review Comment: https://git.openjdk.org/jdk/pull/17784#discussion_r1487692508