RFR: 8236073: G1: Use SoftMaxHeapSize to guide GC heuristics [v7]

[Man Cao]

On Wed, 2 Apr 2025 16:00:33 GMT, Man Cao <manc at openjdk.org> wrote:

>> Hi all,
>> 
>> I have implemented SoftMaxHeapSize for G1 as attached. It is completely reworked compared to [previous PR](https://github.com/openjdk/jdk/pull/20783), and excludes code for `CurrentMaxHeapSize`. I believe I have addressed all direct concerns from [previous email thread](https://mail.openjdk.org/pipermail/hotspot-gc-dev/2024-November/050214.html), such as:
>> 
>> - does not respect `MinHeapSize`;
>> - being too "blunt" and does not respect other G1 heuristics and flags for resizing, such as `MinHeapFreeRatio`, `MaxHeapFreeRatio`;
>> - does not affect heuristcs to trigger a concurrent cycle;
>> 
>> [This recent thread](https://mail.openjdk.org/pipermail/hotspot-gc-dev/2025-March/051619.html) also has some context.
>
> Man Cao has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix test failure on macos-aarch64 by using power-of-two sizes.

Re [Thomas' comment](#issuecomment-2772493942):

> The original patch on the CR only set the guidance for the marking. It did not interact with heap sizing directly at all like the change does. What is the reason for this change?

Because without changing heap sizing directly, setting `SoftMaxHeapSize` alone is ineffective to shrink the heap in most cases. E.g.,  the included test `test/hotspot/jtreg/gc/g1/TestSoftMaxHeapSize.java` will fail.

For other concerns, I think one fundamental issue is the precedence of heap sizing flags: should the JVM respect `SoftMaxHeapSize`  over `GCTimeRatio`/`MinHeapFreeRatio`/`MaxHeapFreeRatio`? My preference is yes, that `SoftMaxHeapSize` should have higher precedence, for the following reasons:

1. Users that set `SoftMaxHeapSize` expect it to be effective to limit heap size. The JVM should do its best to respect user's request. As [JDK-8222181](https://bugs.openjdk.org/browse/JDK-8222181) mentions: "When -XX:SoftMaxHeapSize is set, the GC should strive to not grow heap size beyond the specified size, unless the GC decides it's necessary to do so."  We might interpret "GC decides it's necessary" differently. I think the real necessary case is "the JVM will throw OutOfMemoryError if it does not grow the heap", instead of "the JVM will violate `MinHeapFreeRatio`/`MaxHeapFreeRatio`/`GCTimeRatio` if it does not grow the heap".

1. Having a single flag that makes G1 shrink heap more aggressively, is much more user-friendly than requiring users to tune 3 or more flags to achieve the same effect. As you mentioned, if `SoftMaxHeapSize` only guides marking, user has to also tune `MinHeapFreeRatio`/`MaxHeapFreeRatio` to make G1 shrink more aggressively. It is difficult to figure out a proper value for each flag. Moreover, if user wants to make G1 shrink to a specific heap size, it is a lot harder to achieve that through tuning `MinHeapFreeRatio`/`MaxHeapFreeRatio`.

1. Issues with expansion after young collections from `GCTimeRatio`. `MinHeapFreeRatio`/`MaxHeapFreeRatio` have no effect on how much G1 expands the heap after young collections. Users need to tune `GCTimeRatio` if they want to make G1 expand less aggressively, otherwise aggressive expansion would defeat the purpose of `SoftMaxHeapSize`. However, `GCTimeRatio` is not a manageable flag, so it cannot be changed at run time. If `SoftMaxHeapSize` has precedence, we don't need to bother making `GCTimeRatio` manageable and asking users to tune it at run time. (This is somewhat related to [JDK-8349978](https://bugs.openjdk.org/browse/JDK-8349978) and [email thread](https://mail.openjdk.org/pipermail/hotspot-gc-dev/2025-February/051004.html). )

> So similar to @walulyai I would strongly prefer for SoftMaxHeapSize not interfere that much with the application's performance.

If user sets a too small `SoftMaxHeapSize` and causes performance regression or GC thrashing, it is really user's misconfiguration, and they should take measures to adjust `SoftMaxHeapSize` based on workload. Also misconfiguring `GCTimeRatio`/`MinHeapFreeRatio`/`MaxHeapFreeRatio` could cause similar regressions (think of `-XX:GCTimeRatio=1 -XX:MinHeapFreeRatio=1 -XX:MaxHeapFreeRatio=1`).

However, I can see that `SoftMaxHeapSize` may be easier to misconfigure than the other 3 flags, because it does not adapt to changing live size by itself. I wonder if we could try reaching a middle ground (perhaps this is also what you suggests with ZGC's example of growing up to 25% of cpu usage?):
  - `SoftMaxHeapSize` still takes higher precedence over `GCTimeRatio`/`MinHeapFreeRatio`/`MaxHeapFreeRatio`.
  -  G1 could have an internal mechanism to detect GC thrashing, and expands heap above `SoftMaxHeapSize` if thrashing happens.

> That gets us back to [JDK-8238687](https://bugs.openjdk.org/browse/JDK-8238687) and [JDK-8248324](https://bugs.openjdk.org/browse/JDK-8248324)...

Yes, fixing these two issues would be great regardless of `SoftMaxHeapSize`.  However, they do not address the 3 issues above about flag precedence.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/24211#issuecomment-2774619383