RFR: 8306334: Handle preemption of old cycle between filling and bootstrap phases [v3]

[Kelvin Nilsen]

On Fri, 21 Apr 2023 17:42:21 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> In the case when a request to run a young cycle arrives _after_ making the old generation parseable but _before_ disallowing preemption, the preemption request would cause the young bootstrap cycle to be preempted (not cancelled). This is not expected and would result in the old marking phase beginning with stale oops in the young mark queues after the mark bitmaps were cleared. This ultimately triggers assertions that such objects should be marked.
>> 
>> With this change if we detect a cancellation between filling and bootstrapping we make a distinction between:
>> * A cancellation due to allocation failure. In this case the old cycle is interrupted to run a degenerated cycle.
>> * A cancellation due to a young cycle request. In this case, the old cycle is allowed to continue with the boostrap cycle, which is itself a concurrent young cycle.
>
> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Do not transition from FILLING to BOOTSTRAP before checking cancellation

FWIW, I have observed in various performance test workloads that the bootstrap GC takes "significantly longer" than a normal young GC.  Part of this may be the requirement to coalesce and fill.  Another part is probably that we are not ignoring references to old-gen during the concurrent mark effort.

In any case, one of the "heuristics" I have implemented in the expand-old-on-demand is to preceded every bootstrap GC with a young GC which is immediately followed by the bootstrap GC.  This reduces the chance that we'll run out of mutator memory while doing the bootstrap and also reduces the chance that a young-gc will "trigger" while we're doing c&f.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/262#pullrequestreview-1398421756