RFR: 8306334: Handle preemption of old cycle between filling and bootstrap phases [v2]

[Y. Srinivas Ramakrishna]

On Thu, 20 Apr 2023 22:51:53 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp line 536:
>> 
>>> 534:     }
>>> 535:     case ShenandoahOldGeneration::BOOTSTRAPPING: {
>>> 536:       // It is possible for a young generation request to preempt this nascent old
>> 
>> This comment is helpful. 
>> 
>> There is an ASCII art of an intended state transition diagram for the old generation states in `shenandoahOldGeneration.cpp`. It would be a good idea to update that diagram with possible premptible states and also the state transitions that might occur as a result of cancellations that might occur outside of the pre-emptible states. (I assume that cancellations can only interrupt and reset us out of pre-emptible states, not from any other, i.e. non-pre-emptible state.)
>> 
>> Similarly, at lines 444-458 above is another, presumably for Controller thread states. Does that figure get new transitions as a result of this change?
>> 
>> I see this documentation comment further below at lines 607-610:
>> 
>> 
>>   // We can only tolerate being cancelled during concurrent marking or during preparation for mixed
>>   // evacuation. This flag here (passed by reference) is used to control precisely where the regulator
>>   // is allowed to cancel a GC.
>>   ShenandoahOldGC gc(generation, _allow_old_preemption);
>> 
>> 
>> The comment says "cancel", but uses a flag `_allow_old_preemption`.  When I first saw this I was a bit confused, until I realized that cancellations will be see only in the pre-emptible states, and ignored in all other states until we have completed the work for that state and transition out of it, or if we are otherwise in a pre-emptible state.
>> 
>> Your description below hints at these interactions, I think, but it would be great if this is documented clearly somewhere (may be it is, somewhere in either old generation or in the control thread). It would also be nice to explain, in case this makes sense, if there any possible correspondences between states of the control thread and those of the old generation state.
>
> Yes, [the diagram ](https://github.com/openjdk/shenandoah/blob/master/src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp#L363) depicts preemption as the "YOUNG GC" boxes associated with "FILLING" and "MARKING". The comment on the diagram explains that an allocation failure may interrupt _any_ state and that they all lead to degenerated/full GCs. These were omitted from the diagram to reduce clutter.
> 
> The new state in the control thread isn't strictly necessary for this fix, but it prevents the regulator from trying to start a concurrent young cycle during the bootstrap cycle (which is a concurrent young cycle).
> 
> The implementation of all this could certainly stand some readability improvement. Essentially, there are _two_ reasons an old cycle may be cancelled:
> 1. Allocation failure. This can happen at any phase of the old cycle and always results in at least a degenerated cycle.
> 2. Preemption. If the regulator thread detects a need to run a young generation it may "preempt" the old cycle and run a young cycle. This does _not_ result in a degenerated cycle, but it pauses the old cycle. This requires some coordination with the old collection and so is only supported during the filling and marking phases.

Thank you for your description & responses above; I am slowly getting the hang of the use of the terms and the supporting code.

It would be good to use the terms "cancellation" and "preemption" uniformly everywhere. Cancellibility and Pre-emptibility seem, to me, to be properties of a phase (or state) of an old collection. Here is my understanding of these two terms. Cancellation and Pre-emption are actions that may be taken on phases (states) that are respectively cancellable or pre-emptible.

Cancellation is stronger than pre-emption in that we transition out of the old generation state where cancellation happened, and the old generation collection is commandeered because of that cancellation (in some cases, resulting in a degenerate collection and in other cases a full collection).

Pre-emption, on the other hand, merely "interrupts" (or suspends) an ongoing old generation collection while leaving it in the same state, for young collection to do its work. Persumably in the happy case, the old generation collection continues from its interruption point.

In the odd case, a pre-emption may be followed by a cancellation.

Pre-emption can happen in only a subset of the states, viz. "filling" and "marking". In the non-pre-emptible cases, the pre-emption happens when the work in the state is completed and the old generation collection has proceeded to a pre-emption point, which can be at the point of transition into the next state (either pre-emptible or not).
Cancellation must be, and is, supported in any state of the old generation collection, and subsequent actions reset the state of the old generation collection to an initial (idle) state. 

The term "degeneration" applies to a collection that follows an allocation failure (either by a mutator allocating in the young generation, or a mutator or GC thread allocating in the survivor space or in the old generation but not finding any space to do that allocation), which results in the cancellation of an ongoing collection cycle which would have normally produced that space (usually an old collection cycle when the allocation failure was while servicing a young collection, but could also be a young collection cycle when a mutator was unable to allocate in the young generation), and is followed by work done to complete that cycle with the failed allocation suspended until space has been freed up by that collection.

If I nailed down my understanding of the use of these terms a little, I would probably understand their relationship more easily. :-)

I'll follow up offline with y'all to make sure these are clear in my head.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/262#discussion_r1174180695