Further discussion on Adaptable Heap Sizing with G1

[Thomas Schatzl]

Hi,

On 11.10.24 09:16, Jonathan Joo wrote:
> Hi Thomas,
> 
>     I think what this suggestion overlooks is that a SoftMaxHeapSize that
>     guides used heap size will automatically guide committed size: i.e. if
>     G1 shrinks the used heap, G1 will automatically shrink (and keep) the
>     committed size.
> 
>     So ProposedHeapSize seems to be very similar to SoftMaxHeapSize.
> 
> 
> If I'm understanding this correctly - both ProposedHeapSize and (the 
> proposed version of) SoftMaxHeapSize have similar semantics, but 
> actually modify the heap in different ways. SoftMaxHeapSize helps us 
> determine when to start a concurrent mark, whereas ProposedHeapSize 
> doesn't actually trigger any GC directly, but affects the size of the 
> heap after a GC. Is that correct? Would it make sense then to have both 
> flags, where one helps set a trigger point for a GC, and one helps us 
> determine the heap size we are targeting after the GC? I might also be 
> missing some nuances here.

I think SoftMaxHeapSize (or actually either) will result in 
approximately the same behavior. The difference is in intrusiveness.

ProposedHeapSize forcefully attempts to decrease the committed heap size 
and then the rest of the "heap sizing machinery" follows, while 
SoftMaxHeapSize gives a target for the "heap sizing machinery" and 
committed heap size follows.

ProposedHeapSize has the following disadvantages (as implemented):

- since it forces committed heap size, I expect that in case you are 
close or above that target, you can get frequent uncommits/attempts to 
uncommit which waste cpu cycles.

Hopefully, by giving the heap sizing machinery a goal, it will itself 
determine a sustainable committed memory level without too frequent 
commits and uncommits.

- for some reason it does not allow less memory to be committed than 
proposed (but still larger than MinHeapSize). This can be inefficient 
wrt to memory usage.
I.e. it basically disables other heap sizing afaict.

- (that's more a nit) the use of "0" as special marker for 
SoftMaxHeapSize is unexpected.

This mechanism kind of feels like a very blunt tool to get the desired 
effect (a certain committed heap) without caring about other goals. It 
may be necessary to pull out the immediately un/commit hammer in some 
situations, and imho, let's not give that hammer to users as the first 
option to screw themselves.

> 
>       I.e. if I understand this correctly: allowing a higher GC overhead,
>     automatically shrinks the heap.
> 
> 
> Exactly - in practice, tuning this one parameter up (the target gc cpu 
> overhead) correlates with decreasing both the average as well as maximum 
> heap usage for a java program.
> 
>       I noticed the same with the patch attached to the SoftMaxHeapSize CR
>     (https://bugs.openjdk.org/browse/JDK-8236073
>     <https://bugs.openjdk.org/browse/JDK-8236073>) discounting effects of
>     Min/MaxHeapFreeRatio (i.e. if you remove it,
>     https://bugs.openjdk.org/browse/JDK-8238686
>     <https://bugs.openjdk.org/browse/JDK-8238686> explains the issue).
>     In practice, these two flags prohibit G1 from adjusting the heap unless
>     the SoftMaxHeapSize change is very large.
> 
> 
>     So I would prefer to only think of an alternative to SoftMaxHeapSize if
>     it has been shown that it does not work.
> 
> 
> Given that you have a much stronger mental model than I do of how all 
> these flags fit together in the context of G1 GC, perhaps it would be 
> helpful to schedule some time to chat in person! I think that would help 
> clarify things much more quickly than email. To be clear - I have no 
> reason to doubt that SoftMaxHeapSize does not work. On the other hand, 
> could we possibly make use of both flags? For example, could 
> SoftMaxHeapSize potentially be a good replacement for our periodic GC?

Not sure what periodic GC has to do with SoftMaxHeapSize.

> 
>     There is the nit that unlike in this implementation of ProposedHeapSize,
>     SoftMaxHeapSize will not cause uncommit below MinHeapSize. This is
>     another discussion on what to do about this issue - in a comment in
>     https://bugs.openjdk.org/browse/JDK-8236073
>     <https://bugs.openjdk.org/browse/JDK-8236073> it is proposed to make
>     MinHeapSize manageable. 
> 
> 
> How useful is MinHeapSize in practice? Do we need it, or can we just set 
> it to zero to avoid having to deal with it at all?

I think you are mixing AHS (give decent heap sizing in presence of 
external memory pressure) and getting "optimal" heap sizing (or iow 
"steering heap size" externally).

AHS is mostly about the user not doing/setting any heap sizes; in this 
case just having min heap size very low is just fine just as suggested 
in the JEP.

SoftMaxHeapSize (and ProposedHeapSize) is about the user setting a 
particular goal according to his whims. It is still interesting to set 
-Xms==-Xmx for e.g. fast startup or during heavy activity; however if an 
external system decides that it is useful to intermittently save memory 
up to a certain level, then follow that guidance.

The mechanism to internally follow that guidance can be used by AHS.


> 
>     I (still) believe that AHS and SoftMaxHeapSize/ProposedHeapSize are
>     somewhat orthogonal.
> 
>     AHS (https://openjdk.org/jeps/8329758
>     <https://openjdk.org/jeps/8329758>) is about finding a reasonable
>     heap size, and adjust on external "pressure". SoftMax/ProposedHeapSize
>     are manual external tunings.
> 
> 
>     Wdyt?
> 
> 
> I agree with the general idea - for us, we used a manual external flag 
> like ProposedHeapSize because we did not implement any of the AHS logic 
> in the JVM. (We had a separate AHS thread reading in container 
> information and then doing the calculations, then setting 
> ProposedHeapSize as a manageable flag.) The way I see it is that 
> SoftMax/ProposedHeapSize is the "output" of AHS, and then 
> SoftMax/ProposedHeapSize is the "input" for the JVM, after which the JVM 
> uses this input to adjust its behavior accordingly. Does that align with 
> how you see things?

As mentioned in the other thread, SoftMaxHeapSize can be used by AHS to 
get heap to a certain level (based on memory pressure), but there is 
also that external entity that can modify SoftMaxHeapSize to adjust VM 
behavior.

So ultimately there will be multiple inputs for target heap size (and 
probably I'm forgetting one or the other):

* External memory pressure (AHS) (*)

* CurrentMaxHeapSize

* SoftMaxHeapSize

* CPU usage (existing GCTimeRatio based policy)

* other *HeapSize flags

that need to be merged into some target heap level using some policy.

After knowing that level, the VM needs to decide on a proper reaction, 
which might be anything from just setting internal IHOP goal, to 
(un-)committing memory directly, to doing the appropriate garbage 
collection in a "timely" fashion (which is where the regular periodic 
gc/marking comes in) or anything inbetween.

(*) I am aware that the AHS JEP not only includes reaction on external 
memory pressure but also the merging of goals for different sources; 
some of them are ZGC specific. Some of them are already implemented in 
G1. So for this discussion it is imo useful to limit "AHS" in G1 context 
to things that G1 does not do. Ie. "return another goal based on 
external memory pressure", "min/max heap size defaults(!)", and "adjust 
adaptive sizing".

> If we do indeed implement AHS logic fully within the JVM, then we could 
> internally manage the sizing of the heap without exposing a manageable 
> flag. That being said, it seems to me that exposing this as a manageable 
> flag brings the additional benefit that one could plug in their own AHS 
> implementation that calculates target heap sizes with whatever data they 
> want (and then passes it into the JVM via the manageable flag).
> 
> Again, I wonder if meeting to discuss would be efficient, and then we 
> can update the mailing list with the results of our discussion. Let me 
> know your thoughts!

It's fine with me to meet to recap and discuss above; please suggest 
some time.

Hth,
   Thomas