Aligning the Serial collector with ZGC

[Kirk Pepperdine]

Hi Thomas,

I wanted to respond to all of your comments but I thought better of it given one response deserves it’s own email. The focus is mostly on that one question.


> >
> > - Introduce an adaptive size policy that takes into account memory and
> > CPU pressure along with global memory pressure.
> >     - Heap should be large enough to minimize GC overhead but not
> > large enough to trigger OOM.
> 
> (probably meant "small enough" the second time)

I actually did mean large but in the context of OOM killer…. But to your point, smaller but avoid OOME is also a concern.

> 
> >     - Introduce -XX:SerialPressure=[0-100] to support this work.
> 
> (Fwiw, regards to the other discussion, I agree that if we have a flag with the same "meaning" across collectors it might be useful to use the same name).

I think we have deadly agreement on this one.

> 
> >     - introduce a smoothing algorythm to avoid excessive small
> > resizes.
> 
> One option is to split this further into parts:
> 
> * list what actions Serial GC could do in reaction to memory pressure on an abstract level, and which make sense; from that see what functionality is needed.

I built a chart some time ago and this is an expanded version of it.

GC Overhead
(Pause:mutator time)
Allocation Pressure
Global Memory Pressure
Action (Eden)
Action (Tenured)
(full collection only)
< target
Low
Low
shrink
shrink
< target
Low
Medium
shrink
shrink
< target
Low
High
shrink
shrink
< target
Medium
Low
hold
shrink
< target
Medium
Medium
shrink
shrink
< target
Medium
High
shrink
shrink
< target
High
Low
shrink
shrink
< target
High
Medium
shrink
shrink
< target
High
High
shrink
shrink
~= target
Low
Low
hold
hold
~= target
Low
Medium
hold
hold
~= target
Low
High
shrink
shrink
~= target
Medium
Low
hold
hold
~= target
Medium
Medium
hold
hold
~= target
Medium
High
shrink
shrink
~= target
High
Low
hold
hold
~= target
High
Medium
hold
hold
~= target
High
High
shrink
shrink
> target
Low
Low
expand
expand
> target
Low
Medium
expand
expand
> target
Low
High
hold
hold
> target
Medium
Low
expand
expand
> target
Medium
Medium
expand
expand
> target
Medium
High
hold
hold
> target
High
Low
expand
expand
> target
High
Medium
expand
expand
> target
High
High
hold
hold
 
 

Some of my thoughts used to construct the table.

GC Overhead tells us if the heap is under/appropriately/over sized.
Allocation pressure combined with the size of Eden drives the frequency of young generational collections
Global memory pressure is an measure of the availability of memory.
 
The goal of resizing is to hit a target GC Overhead threshold without risking either OutOfMemoryError or the OOM killer. Reducing Full GC activity requires one to provide enough tenured space to hold the Live Data Set (LDS) as well as minimizing the promotion of transients. Partial GC frequency is a function of the size of Eden and the allocation pressure. Controlling GC frequency is key to controlling the rate at which transients are promoted.
 
On Heap sizing.
Tenured maybe resized at the end of a tenured (full) collection. Eden and Survivor maybe resized at the end of either a tenured or partial collection. The size of Eden, Survivor and Tenured will be decided separately. Overall logic is the heap should have as much memory as it needs for the GC to run within overhead targets.
 
The live set size is used to determine the size of tenured. The heuristic is that tenured should be 1.5 to 2x * LDS. Tenured should be expanded or shrunk to meet this ratio. Expansion should only happen when there is memory to support it.
 
The decision to resize young is based on;
is the GC overhead target being met
the strength of the allocation pressure
the availability of global memory
 
Meeting the GC overhead target indicates that the heap is appropriately sized. Under this condition there is no pressure to resize unless there is a shortage of global memory. If this is the case, there should be a balance made between being a good neighbour by releasing memory and the risk/costs of higher GC overhead.
 
Having GC overhead being under target is an indication that the heap is oversized. In this case it should be safe to reduce the heap size and release memory back to OS.
 
Having GC overhead be higher than the target indicates that heap is undersized. In this case heap (and likely Eden in particular) should be expanded assuming there is enough global memory to support the expansion without risking an OOM killer event.
 
 
Allocation pressure combined with the size of Eden sets GC frequency. High GC frequency tends to drive up GC overhead. If allocation pressure is high and GC overhead is high then increasing the size of Eden should reduce GC overhead. Having both allocation pressure and GC overhead be low provides and opportunity to reduce heap size and return memory.
 
All of the resizing decisions need to be moderated by the availability of (global) memory. If global memory is scarce, then the decision should favour releasing (uncommitting) memory. This may come at the expense of higher GC overhead. Resizing to smaller pool sizes is not without risk and in the case of young, both high global memory pressure and high allocation pressure add to the risk.



> 
> * provide functionality that tries to keep some kind of GC/mutator time ratio; I would start with looking at G1 does because Serial GC's behaviour is probably closer to G1 than ZGC, but ymmv.
> (Obviously improvements are welcome :))

I would agree.
> 
> (This may not need to be exposed externally like some GCTimeRatio/GCCPUPercentage/whatever flag name)
> 
> * add functionality to calculate memory pressure from the environment; maybe in a containerized environment from a manageable flag as it does not have a global "pressure" view. This could probably taken from ZGC, at least partially

This is but one area where we are looking to “borrow” from.

> 
> * some transfer function that translates this external memory pressure, based on "GCPressure", (e.g. that "sigmoid" function plus lots of magic numbers) to reaction in the gc: e.g. change the gc/mutator pause time goal, start collections, uncommit memory...

We prototyped our own smoothing function but I’d defer to the sigmoid function as I’d prefer to share where ever possible.

> 
> * (probably) some background thread that continuously calculates and reacts on global pressure (uncommit memory, do a gc, resize heap, ...) because one probably does not want to wait for the next gc to react...

I’ve been trying to avoid an extra background thread and try to backload the work on the GC thread but I also recognize that an extra thread maybe necessary.

> 
> * do lots of testing to weed out corner cases
> 
> > - Introduce manageable flag SoftMaxHeapSize to define a target heap
> > size nd set the default max heap size to 100% of available.
> 
> I am a bit torn about SoftMaxHeapSize in Serial GC. What do you envision that Serial GC would do when the SoftMaxHeapSize has been reached, and what if old gen occupancy permanently stays above that value?

At the moment, SoftMaxHeapSize is an implementation in Z. I’d first like to pull a (rough) spec out of the implementation and then try to answer your question. It’s currently not clear to me how this should work with any collector.
> 
> The usefulness of SoftMaxHeapSize kind of relies on having a minimally invasive old gen collection that tries to get old gen usage back below that value.

Well, the LDS is what it is and running a speculative collection would likely clean up (prematurely) promoted transients… but that’s about it. Whereas it would clean both transients and floating garbage for the concurrent collectors. I’m not at fan of speculative collections given all of the time I’ve spent getting rid of them :-) IMO, a DGC triggered full collections was rarely necessary (all overhead with very little return). This also applied to the G1 patch that speculatively ran to counter to-space overflows and it also applied to running a young gen prior to remark with CMS collector. Long story sort, loads of extra overhead with very little to no payback.

> 
> Serial GC has no "minimally invasive" way to collect old generation. It is either Full GC or nothing. This is the only option for Serial, but always doing Full collections after reaching that threshold seems very heavy handed, expensive and undesirable to me (ymmv).
> 
> That reaction would follow the spirit of the flag though.
> 
> Maybe at the small heaps Serial GC targets, this makes sense, and full gc is not that costly anyway.

Yeah, for small heap this shouldn’t be a big deal. But this is one of the reasons why I believe we should treat young and old separately. We can cheaply and safely return memory from young gen and leave the sizing of tenured to when a full is really needed. I grant you that this may not be very timely but I’m not sure that we need this to happen on demand… I think we can wait for natural cycles to take their course. But, maybe I’m wrong on this point. We plan to experiment with this.
> 
> It might be useful to enumerate what actions could be performed on global pressure.

That’s in the table…

> 
> > - Add in the ability to uncommit memory (to reduce global memory
> > pressure).
> >
> 
> The following imo outlines a compdoneletely separate idea, and should be discussed separately:
> 
> >
> > While working through the details of this work I noted that there
> > appear  to opportunities to offer new defaults for other settings. For
> > example, [...]
> 
> That seems to be some more elaborate way of finding "optimal" generation size for a given heap size (which may follow from what the gc/mutator time ratio algorithm gives you).

I’m trying to apply my years of experience tuning 100s of collectors across 100s of applications.

> 
> >
> > For Eden the guiding metric is allocation rate. For Survivor it's life
> > cycle (age table). For Tenured it's live set size. Using these metrics
> > to determine size of the parts and use that to then calculate a max
> > heap size has almost always yielded lower GC overheads than setting a
> > heap size and then letting ratios size everything. This maybe a
> > separate piece of work
> 
> +1
> 
> > but the intent would be to have ergonomics calculate
> > optimal eden, survivor and tenured sizes. Each young collection is an
> > opportunity to resize Eden and Survivor whereas a full would be used
> > to resize Eden, Survivor and Tenured space. This may lead to the need
> > to ignore NewRatio and (the soft target) MaxGCPauseMillis.
> 
> Fwiw, the only collector that observes MaxGCPauseMillis is G1; in the context of Serial GC discussed further above I am confused.
> 
> Not sure if MaxGCPauseMillis would make sense in Serial GC given that you can't control Full GC pause length.

Agreed. Sizing in Serial is currently controlled by the number of non-daemon threads and that rarely changes. This implies that pause times are loosely a function of load and LDS size.

> 
> Also, in the context of G1 some of the statements above are hard to understand: e.g. the text seems to imply that there is a fixed ratio between eden and survivor which isn't really the case, at least not in the sense of Serial  GC.

Sorry for the confusion, I wasn’t trying to imply that the ratio is fixed. I was trying to do was introduce better default start settings When I’m tuning I tend to set the young to tenured ratio to 1 and then set the survivor ratio to 2. This allows me to collect as clean a signal from the collector as it possible. I would then make adjustments from that starting point. If we want to resize then I believe that this starting point would give ergonomics a better chance to stabilize at a more optimal place. 
> 
> Could you elaborate?
> 
> Even then, with Serial GC's fixed generation sizes fine-grained on-the-fly adaptation as somewhat suggested might be harder than usual.
> 
> Not against doing all that, but it really sounds like separate work.

I believe it might be as it feels like to falls into the category of auto-tuning.

> 
> >
> > As for testing. I’m currently looking at modifying HyperAlloc to add
> > ability to alter the shape of the load on the collector over time.
> >
> > All of this is still in it’s infancy and we’re open for guidance and
> > input.
> >
> > As for the work on G1, an initial patch as been submitted (URL above)
> > and is open for comments.
> >
> 
> The patch does not seem to implement AHS. It implements CurrentMaxHeapSize which might be what AHS uses to set max heap size.
> 
> To implement AHS for G1 roughly at least the following items need to be added/implemented/changed:
> 
> * remove the use of Min/MaxHeapFreeRatio for heap sizing. These flags completely disregard cpu and heap pressure based heap sizing (should also be removed from Serial GC - this means deprecating/obsoleting this flag as soon as the last user is gone).
> 
> * implement CurrentMaxHeapSize which is a (configurable) hard limit on how much the Java application may allocate (JDK-8204088) in support of AHS. As mentioned, that patch might be an initial discussion base.
> I do not think we need a JEP for that, but it gives you more publicity.
> 
> * implement SoftMaxHeapSize in the sense of ZGC where it uses it to guide IHOP (or ZGC's equivalent). Note that I am not sure that SoftMaxHeapSize is something absolutely necessary in the context of AHS, but may be a tool.
> 
> * the same background functionality as for serial: implement some mechanism to control the heap size based on the decisions of AHS; i.e. start collections to get to heap target, uncommit stuff/enqueue for uncommit etc.
> 
> Currently G1 only resizes the heap during Remark and Full GC which is too limiting to follow current "memory pressure". Maybe use/update Soft/CurrentMaxHeapSize as needed so that GC compacts the heap first; this may either be in the form of JDK-8238687 which uncommits at every gc, which is probably still too limiting for an AHS system.

Got it, I think we’re aiming to get all of this done it’s just not written here. But I appreciate the list as it’s helpful.

> 
> Probably other issues will crop up along the way.
> 
> * do lots of testing to weed out corner cases and hopefully not regress too much from current performance

I’m hoping that instead of regressing, we reduce GC interference. And happy to avoid a JEP but also happy to write one if it’s really needed.. and I don’t need nor want more publicity but thanks for the warning. ;-)

Kind regards,
Kirk

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20240927/74296741/attachment-0001.htm>