RFR: 8377180: Shenandoah: make escalation from degen to full more conservative [v2]

[William Kemper]

On Wed, 4 Feb 2026 17:46:50 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> This PR merges into Traditional Shenandoah an optimization that was applied to Generational Shenandoah in September 2025 and has proven itself effective in that context.  See https://github.com/openjdk/jdk/pull/27456
>> 
>> The change is to not immediately escalate Degenerated GC to Full GC following "bad progress" by Degen GC.  The most common reason for bad degenerated progress is that most of the potential garbage was created following the start of the concurrent GC cycle which degenerated.  This "floating" garbage cannot be reclaimed by the degenerated cycle because of the SATB protocol.
>> 
>> Pursuing at least one additional concurrent cycle, even if that subsequent concurrent GC degenerates, is usually a much better approach than forcing immediate escalation to Full GC.  While escalation to full GC will resolve the out-of-memory condition, it imposes unnecessarily long stop-the-world pauses:
>> 
>>    Existing behavior not to scale:
>>        <- [       Conc GC       ] -> <- [ Degen    ] -> <- [            Full GC             ]
>>           [    experiences      ]       [  with    ]
>>           [     allocation      ]       [   bad    ]
>>           [      failure        ]       [ progress ]
>>                                         [ STW pause is 100 ms degen + 3s full GC            ]
>> 
>>    Revised behavior not to scale:
>>        <- [       Conc GC       ] -> <- [ Degen    ] -> <- [       Conc GC       ] -> [ Degen    ]
>>           [    experiences      ]       [  with    ]       [    experiences      ]    [  with    ]
>>           [     allocation      ]       [   bad    ]       [     allocation      ]    [  good    ]
>>           [      failure        ]       [ progress ]       [      failure        ]    [ progress ]
>>                                         [  100 ms  ]                                  [  100 ms  ] 
>>                                         [   pause  ]                                  [   pause  ]
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix gtest for ShenandoahCollectorPolicy

Looks good, suggest one small change.

src/hotspot/share/gc/shenandoah/shenandoahCollectorPolicy.hpp line 122:

> 120:   // Genshen will only upgrade to a full gc after the configured number of futile degenerated cycles.
> 121:   bool should_upgrade_degenerated_gc() const {
> 122:     return _consecutive_degenerated_gcs_without_progress >= GENERATIONAL_CONSECUTIVE_BAD_DEGEN_PROGRESS_THRESHOLD;

Shall we rename `GENERATIONAL_CONSECUTIVE_BAD_DEGEN_PROGRESS_THRESHOLD` as well? Perhaps `CONSECUTIVE_BAD_DEGEN_PROGRESS_THRESHOLD`?

-------------

Changes requested by wkemper (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/29574#pullrequestreview-3759449781
PR Review Comment: https://git.openjdk.org/jdk/pull/29574#discussion_r2771144740