RFR: 8236073: G1: Use SoftMaxHeapSize to guide GC heuristics [v5]

[Thomas Schatzl]

On Tue, 1 Apr 2025 20:57:36 GMT, Man Cao <manc at openjdk.org> wrote:

> > With the changes to `young_collection_expansion_amount()`, once we reach the `SoftMaxHeapSize`, we cannot expand the heap except during GC where expansion can happen without regard for `SoftMaxHeapSize`. Thus, after exceeding `SoftMaxHeapSize` we go into a phase of repeated GCs where we expand the heap almost one region at a time. Is this the expected effect of the `SoftMaxHeapSize` as implemented by this patch?
> 
> Yes. This is the expected behavior if user sets `SoftMaxHeapSize` too small. G1 will try its best to respect `SoftMaxHeapSize`, which could cause GC thrashing. However, it won't cause `OutOfMemoryError`. This problem is due to user's misconfiguration of `SoftMaxHeapSize`, which is similar to user misconfiguring `Xmx` to be too small.

The original patch on the CR only set the guidance for the marking. It did not interact with heap sizing directly at all like the change does. What is the reason for this change?

(Iirc, in tests long time ago, with that original patch, and also adapting `Min/MaxHeapFreeRatio`, did result the desired effect of G1/`SoftMaxHeapSize` decreasing the heap appropriately. Without it, the heap will almost never change, but that is expected how `Mindoes not work).

So similar to @walulyai I would strongly prefer for `SoftMaxHeapSize` not interfere that much with the application's performance. To me, this behavior is not "soft", and there seems to be general consensus internally about allowing unbounded cpu usage for GC. Afaiu in ZGC, if heap grows beyond `SoftMaxHeapSize`, GC activity can grow up to 25% of cpu usage (basically maxing out concurrent threads). That could be a reasonable guidance as well here.

GC thrashing will also prevent progress with marking, and actually cause more marking because of objects not having enough time to die. This just makes the situation worse until the heap gets scaled back to `SoftMaxHeapSize`.

However at the moment, changing the GC activity threshold internally will not automatically shrink the heap as you would expect, since currently shrinking is controlled by marking using the `Min/MaxHeapFreeRatio` flags.

That gets us back to (JDK-8238687)[https://bugs.openjdk.org/browse/JDK-8238687] and (JDK-8248324)[https://bugs.openjdk.org/browse/JDK-8248324]...

@walulyai is currently working on the former issue again, testing it, maybe you two could together on that to see whether basing this work on what @walulyai is cooking up is a better way forward, if needed modifying `gctimeratio` if we are above `SoftMaxHeapSize`?

Otherwise, if there really is need to get this functionality asap, even only making it a guide for the marking should at least give some effect (but I think without changing `Min/MaxHeapFreeRatio` at the same time there is not much effect anyway). But that is a fairly coarse and indirect way of getting the necessary effect to shrink the heap. We should not limit ourselves to what mainline provides at the moment.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/24211#issuecomment-2772493942