RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v16]

[Thomas Stuefe]

On Wed, 7 Jun 2023 16:51:38 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> OpenJDK Colleagues:
>> 
>> Please review this proposed integration of Generational mode for Shenandoah GC under https://bugs.openjdk.org/browse/JDK-8307314.
>> 
>> Generational mode of Shenandoah is enabled by adding `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a command line that already specifies ` -XX:+UseShenandoahGC`.  The implementation automatically adjusts the sizes of old generation and young generation to efficiently utilize the entire heap capacity.  Generational mode of Shenandoah resembles G1 in the following regards:
>> 
>> 1. Old-generation marking runs concurrently during the time that multiple young generation collections run to completion.
>> 2. After old-generation marking completes, we perform a sequence of mixed collections.  Each mixed collection combines collection of young generation with evacuation of a portion of the old-generation regions identified for collection based on old-generation marking information.
>> 3. Unlike G1, young-generation collections and evacuations are entirely concurrent, as with single-generation Shenandoah.
>> 4. As with single-generation Shenandoah, there is no explicit notion of eden and survivor space within the young generation.  In practice, regions that were most recently allocated tend to have large amounts of garbage and these regions tend to be collected with very little effort.  Young-generation objects that survive garbage collection tend to accumulate in regions that hold survivor objects.  These regions tend to have smaller amounts of garbage, and are less likely to be collected.  If they survive a sufficient number of young-generation collections, the “survivor” regions are promoted into the old generation.
>> 
>> We expect to refine heuristics as we gain experience with more production workloads.  In the future, we plan to remove the “experimental” qualifier from generational mode, at which time we expect that generational mode will become the default mode for Shenandoah.
>> 
>> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, HyperAlloc, and multiple AWS production workload simulators. We test on Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows x64.
>
> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 305 commits:
> 
>  - Merge branch 'master' of https://git.openjdk.org/jdk into merge-generational-shenandoah
>  - Simplify test logic, fail if name of Shenandoah young gen pool changes (#3)
>    
>  - Remove a few more unneeded copyright notices
>  - Remove more extraneous copyright notices
>  - JDK-8309322: [GenShen] TestAllocOutOfMemory#large failed
>    
>    When generational Shenandoah is used, there may be an additional
>    alignment related heap size adjustment that the test should be cognizant
>    of. Such alignment might also happen in the non-generational case, but
>    in this case the specific size used in the test was affected on machines
>    with larger than usual os page size settings.
>    
>    The alignment related adjustment would have affected all generational
>    collectors (except perhaps Gen Z). In the future, we might try and relax
>    this alignment constraint.alignment.
>  - Remove one more extraneous Amazon copyright
>  - Update copyright notices
>  - Improve efficiency of card-size alignment calculations
>  - Exit during initialization on unsupported platforms
>  - Remove an inappropriate copyright notice
>  - ... and 295 more: https://git.openjdk.org/jdk/compare/33bb64f2...612072a4

I won't be able to give reasonable input here in the short time left before RDP1. Nor am I the most qualified to do so. 

Just wanted to re-iterate that I see this rushed review with worry. Nothing I have not said already, and of course, it does not diminish the massive effort behind this JEP.

Process-wise, when talking about Lilliput integration we touched on the idea of moving RDP1 down to a sooner date. I still think this makes sense.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1581244415