RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental)

[Thomas Stuefe]

On Fri, 26 May 2023 20:46:29 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> OpenJDK Colleagues:
> 
> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314.
> 
> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`.  The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity.  Generational mode of Shenandoah resembles G1 in the following regards:
> 
> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion.
> 2. After old-generation marking completes, we perform a sequence of mixed collections.  Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information.
> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah.
> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation.  In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort.  Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects.  These regions tend to have smaller amounts of garbage, and are less likely to be collected.  If they survive a sufficient number of young-generation collections, the “survivor” regions are promoted into the old generation.
> 
> We expect to refine heuristics as we gain experience with more production workloads.  In the future, we plan to remove the “experimental” qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah.
> 
> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64.

I did a first read through the tests to check if any test changes affect traditional Shenandoah. To see if regression tests for non-generational are unchanged.

All good, I did not find anything noteworthy.

test/hotspot/jtreg/gc/shenandoah/TestEvilSyncBug.java line 33:

> 31:  * @modules java.base/jdk.internal.misc
> 32:  *          java.management
> 33:  * @run driver/timeout=480 TestEvilSyncBug -XX:ShenandoahGCHeuristics=aggressive

Probably fine, but why this change to non-generational testing? Will aggressive heuristic sharpen the test?

test/hotspot/jtreg/gc/shenandoah/mxbeans/TestChurnNotifications.java line 169:

> 167: 
> 168:                     MemoryUsage before = getUsage(mapBefore);
> 169:                     MemoryUsage after = getUsage(mapAfter);

This also changes test logic for traditional Shenandoah, but its harmless. 
Nit: more precise would be to require "Young Gen" pool to only exist for -XX:ShenandoahGCMode=generational.

test/hotspot/jtreg/gc/shenandoah/oom/TestAllocOutOfMemory.java line 23:

> 21:  * questions.
> 22:  *
> 23:  */

Three tests folded into one, but it does not look like functionality changed for testing traditional Shenandoah. Okay.

test/hotspot/jtreg/gc/shenandoah/oom/TestAllocOutOfMemory.java line 92:

> 90:             expectFailure("-Xmx16m",
> 91:                           "-XX:+UnlockExperimentalVMOptions",
> 92:                           "-XX:+UseShenandoahGC",

Nit: should not need UnlockExperimentalVMOptions anymore.

test/hotspot/jtreg/gc/shenandoah/oom/TestClassLoaderLeak.java line 132:

> 130:              {{"iu"},           {"adaptive", "aggressive"}},
> 131:              {{"passive"},      {"passive"}},
> 132:              {{"generational"}, {"adaptive"}}

Curious, here and in similar places, why only test adaptive heuristic for generational, if we test satb with all variants?

-------------

PR Review: https://git.openjdk.org/jdk/pull/14185#pullrequestreview-1455490699
PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1213164085
PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1213232252
PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1213243280
PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1213242053
PR Review Comment: https://git.openjdk.org/jdk/pull/14185#discussion_r1213244749