Further discussion on Adaptable Heap Sizing with G1
Thomas Schatzl
thomas.schatzl at oracle.com
Thu Nov 14 13:29:23 UTC 2024
Hi Monica,
as far as I understand the suggested approach is good.
Just two comments: what Google provided with ProposedHeapSize has only
been part of the solution, so it obviously falls a bit short of what AHS
intends to do, but that was intended I think.
One might also see SoftMaxHeapSize (the flag) as a knob for the user to
set what he thinks is good, that is somehow merged with an internal
SoftMaxHeapSize. I hope this is not too confusing, if it is, then we can
ignore this comment for now.
Thanks for driving this forward,
Thomas
On 12.11.24 20:11, Monica Beckwith wrote:
> Hi everyone,
> Thank you all for the valuable and detailed discussion around AHS and
> heap management for G1. I wanted to share some thoughts that align with
> Thomas’s comments and clarify the best path forward, especially given
> the distinctions between AHS (Automatic Heap Sizing) and Google’s
> Adaptive Heap Sizing (AHS-Google). I’ve included simple diagrams to
> illustrate the technical flow and interactions of each approach.
> *1. Consolidating Around SoftMaxHeapSize for Dynamic, Adaptive Sizing*
> Thomas’s suggestion to prioritize SoftMaxHeapSize as the main dynamic
> driver aligns with my understanding of an effective AHS model. Using
> SoftMaxHeapSize in this way allows us to minimize the CPU overhead
> associated with frequent uncommit/commit cycles, which would be a
> potential risk with a more rigid setting like ProposedHeapSize. Here’s a
> basic illustration of how Automatic Heap Sizing (AHS) with
> SoftMaxHeapSize would work dynamically:
> +-----------------------------+
> | External Inputs |
> |-----------------------------|
> | - Global Memory Pressure |
> | - GCTimeRatio policy |
> | - Heap tunables via |
> | commandline |
> +-----------------------------+
> |
> v
> +-----------------------------+
> | Automatic Heap |
> | Sizing (AHS) |
> +-----------------------------+
> |
> v
> +-----------------------------+
> | SoftMaxHeapSize (Dynamic) |
> | - Guides heap size |
> | - Shrinks under pressure |
> | - Uses target heuristics |
> +-----------------------------+
> |
> v
> +-----------------------------+
> | JVM Heap Management |
> | - Adjusts committed memory |
> | - Controls expansions & |
> | contractions smoothly |
> +-----------------------------+
> By consolidating around SoftMaxHeapSize as the primary “target” flag, we
> create a more straightforward, adaptive, and consistent experience.
> *2. The AHS-Google Approach and Its Challenges*
> Google’s current Adaptive Heap Sizing (AHS-Google) approach uses
> ProposedHeapSize as a fixed committed size target. While this allows for
> setting a specific target for memory use, it introduces some challenges,
> particularly with forced uncommit/commit cycles that might ignore
> dynamic inputs. Here’s how this approach typically functions:
> +-----------------------------+
> | AHS-Google Logic |
> |-----------------------------|
> | - Periodic GC with target |
> | - Uses ProposedHeapSize as |
> | "optimal" committed size |
> +-----------------------------+
> |
> v
> +-----------------------------+
> | ProposedHeapSize (Fixed) |
> | - Forced committed memory |
> | - Overrides dynamic inputs |
> | - Can cause frequent |
> | uncommit/commit cycles |
> +-----------------------------+
> |
> v
> +-----------------------------+
> | JVM Heap Management |
> | - Follows set memory level |
> | - May ignore external |
> | pressure signals |
> +-----------------------------+
> A purely AHS-based approach would allow SoftMaxHeapSize to adjust
> dynamically in response to real-time signals without forcing committed
> memory levels. This avoids unnecessary CPU cycles and provides a more
> adaptive response to environmental changes, such as fluctuating memory
> demands in containerized and cloud environments.
> *3. Key Differences Between AHS and AHS-Google*
> In my understanding:
>
> * *AHS (Automatic Heap Sizing)*: Focuses on finding a reasonable heap
> size based on external memory pressure and dynamically adjusts
> according to environmental inputs. This aligns with Thomas’s point
> that AHS should allow for minimal user intervention and let dynamic
> factors guide heap behavior.
> * *AHS-Google*: Treats ProposedHeapSize as a fixed input, overriding
> dynamic adjustments. While this gives more explicit control, it
> limits adaptability and could introduce inefficiencies, as mentioned
> earlier.
>
> *4. Moving Forward with a Balanced, Dynamic AHS for G1*
> Based on the discussion, I suggest we focus on developing an AHS model
> that leverages SoftMaxHeapSize as the adaptable target, allowing the JVM
> to adjust based on real-time memory pressures and CPU usage. Integrating
> multiple inputs dynamically will create a robust model for managing
> “noisy neighbor” challenges—a very real need in today’s cloud and
> container scenarios and one that AHS is well-suited to manage, as
> highlighted in Erik’s recent JVMLS presentation.
> Thank you all again for the insightful conversation and technical
> contributions. I believe these steps will help us build a technically
> sound and stable AHS for G1.
> Please feel free to correct any misunderstandings or clarify any points
> where further alignment is needed.
> Regards,
> Monica
> <https://urldefense.com/v3/__https://outlook.office.com/bookwithme/user/6dc2f1f46dfd446aa456d1c1245cecd6@microsoft.com?anonymous&ep=bwmEmailSignature__;!!ACWV5N9M2RV99hQ!PbxUKlUyx9Ac7nM5qynv6cmm0TkRu4SwHBLdIakj1AtTw-4ORHeRBexOJhYzyo5P81TK8pEtcoZ3bNYGHPQIOt5I3C6J6D1j$> Book time to meet with me <https://urldefense.com/v3/__https://outlook.office.com/bookwithme/user/6dc2f1f46dfd446aa456d1c1245cecd6@microsoft.com?anonymous&ep=bwmEmailSignature__;!!ACWV5N9M2RV99hQ!PbxUKlUyx9Ac7nM5qynv6cmm0TkRu4SwHBLdIakj1AtTw-4ORHeRBexOJhYzyo5P81TK8pEtcoZ3bNYGHPQIOt5I3C6J6D1j$>
>
> *From:* hotspot-gc-dev <hotspot-gc-dev-retn at openjdk.org> *On Behalf Of
> *Jonathan Joo
> *Sent:* Thursday, October 17, 2024 7:11 PM
> *To:* Thomas Schatzl <thomas.schatzl at oracle.com>
> *Cc:* hotspot-gc-dev at openjdk.org
> *Subject:* Re: Further discussion on Adaptable Heap Sizing with G1
> Hi Thomas,
> The points you mentioned make sense to me! There are some nuances that
> I'd like to dig into further to make sure that we are aligned. I think
> to summarize - I'm not sure exactly how SoftMaxHeapSize is intended to
> work, whereas we have experimented with ProposedHeapSize at Google
> already, so I want to bridge my gap in understanding there.
> I appreciate you offering to meet and discuss! As far as meeting time -
> I'm currently in US Pacific time, but flexible in terms of when we meet.
> (I am generally awake from 9am-1am PT, so I am good to meet any time in
> that time period -- please let me know what time works best for you.)
> Tuesday and Thursday of the coming week I have the most availability,
> but if you have any other dates/times in mind, I can let you know
> whether that works for me.
> Best,
> ~ Jonathan
> On Mon, Oct 14, 2024 at 2:52 AM Thomas Schatzl
> <_thomas.schatzl at oracle.com_ <mailto:thomas.schatzl at oracle.com>> wrote:
> Hi,
>
> On 11.10.24 09:16, Jonathan Joo wrote:
>> Hi Thomas,
>>
>> I think what this suggestion overlooks is that a SoftMaxHeapSize that
>> guides used heap size will automatically guide committed size: i.e. if
>> G1 shrinks the used heap, G1 will automatically shrink (and keep) the
>> committed size.
>>
>> So ProposedHeapSize seems to be very similar to SoftMaxHeapSize.
>>
>>
>> If I'm understanding this correctly - both ProposedHeapSize and (the
>> proposed version of) SoftMaxHeapSize have similar semantics, but
>> actually modify the heap in different ways. SoftMaxHeapSize helps us
>> determine when to start a concurrent mark, whereas ProposedHeapSize
>> doesn't actually trigger any GC directly, but affects the size of the
>> heap after a GC. Is that correct? Would it make sense then to have both
>> flags, where one helps set a trigger point for a GC, and one helps us
>> determine the heap size we are targeting after the GC? I might also be
>> missing some nuances here.
>
> I think SoftMaxHeapSize (or actually either) will result in
> approximately the same behavior. The difference is in intrusiveness.
>
> ProposedHeapSize forcefully attempts to decrease the committed heap size
> and then the rest of the "heap sizing machinery" follows, while
> SoftMaxHeapSize gives a target for the "heap sizing machinery" and
> committed heap size follows.
>
> ProposedHeapSize has the following disadvantages (as implemented):
>
> - since it forces committed heap size, I expect that in case you are
> close or above that target, you can get frequent uncommits/attempts to
> uncommit which waste cpu cycles.
>
> Hopefully, by giving the heap sizing machinery a goal, it will itself
> determine a sustainable committed memory level without too frequent
> commits and uncommits.
>
> - for some reason it does not allow less memory to be committed than
> proposed (but still larger than MinHeapSize). This can be inefficient
> wrt to memory usage.
> I.e. it basically disables other heap sizing afaict.
>
> - (that's more a nit) the use of "0" as special marker for
> SoftMaxHeapSize is unexpected.
>
> This mechanism kind of feels like a very blunt tool to get the desired
> effect (a certain committed heap) without caring about other goals. It
> may be necessary to pull out the immediately un/commit hammer in some
> situations, and imho, let's not give that hammer to users as the first
> option to screw themselves.
>
>>
>> I.e. if I understand this correctly: allowing a higher GC overhead,
>> automatically shrinks the heap.
>>
>>
>> Exactly - in practice, tuning this one parameter up (the target gc cpu
>> overhead) correlates with decreasing both the average as well as maximum
>> heap usage for a java program.
>>
>> I noticed the same with the patch attached to the SoftMaxHeapSize CR
>> (_https://bugs.openjdk.org/browse/JDK-8236073_
> <https://urldefense.com/v3/__https://nam06.safelinks.protection.outlook.com/?url=https*3A*2F*2Fbugs.openjdk.org*2Fbrowse*2FJDK-8236073&data=05*7C02*7Cmonica.beckwith*40microsoft.com*7Cfdcf0de799d34a28c1a708dcef098c5b*7C72f988bf86f141af91ab2d7cd011db47*7C1*7C0*7C638648071506605286*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C0*7C*7C*7C&sdata=tDXYfCNOqe7cR*2FnHwmD*2F87tbeUvo06K6mpTVqKjaYmw*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU!!ACWV5N9M2RV99hQ!PbxUKlUyx9Ac7nM5qynv6cmm0TkRu4SwHBLdIakj1AtTw-4ORHeRBexOJhYzyo5P81TK8pEtcoZ3bNYGHPQIOt5I3GC08IEo$>
>> <_https://bugs.openjdk.org/browse/JDK-8236073_
> <https://urldefense.com/v3/__https://nam06.safelinks.protection.outlook.com/?url=https*3A*2F*2Fbugs.openjdk.org*2Fbrowse*2FJDK-8236073&data=05*7C02*7Cmonica.beckwith*40microsoft.com*7Cfdcf0de799d34a28c1a708dcef098c5b*7C72f988bf86f141af91ab2d7cd011db47*7C1*7C0*7C638648071506634149*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C0*7C*7C*7C&sdata=CE1feUMnxtgoxREf*2BaLK9yyv6a545238mDXDLUMcwF4*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!ACWV5N9M2RV99hQ!PbxUKlUyx9Ac7nM5qynv6cmm0TkRu4SwHBLdIakj1AtTw-4ORHeRBexOJhYzyo5P81TK8pEtcoZ3bNYGHPQIOt5I3GgGbjgP$>>) discounting effects of
>> Min/MaxHeapFreeRatio (i.e. if you remove it,
>> _https://bugs.openjdk.org/browse/JDK-8238686_
> <https://urldefense.com/v3/__https://nam06.safelinks.protection.outlook.com/?url=https*3A*2F*2Fbugs.openjdk.org*2Fbrowse*2FJDK-8238686&data=05*7C02*7Cmonica.beckwith*40microsoft.com*7Cfdcf0de799d34a28c1a708dcef098c5b*7C72f988bf86f141af91ab2d7cd011db47*7C1*7C0*7C638648071506653746*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C0*7C*7C*7C&sdata=mwYKt9FtJ72rRZYNgtXHRwHBCGRzgIGeeJj56WsQwvs*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUl!!ACWV5N9M2RV99hQ!PbxUKlUyx9Ac7nM5qynv6cmm0TkRu4SwHBLdIakj1AtTw-4ORHeRBexOJhYzyo5P81TK8pEtcoZ3bNYGHPQIOt5I3FBjJ2X6$>
>> <_https://bugs.openjdk.org/browse/JDK-8238686_
> <https://urldefense.com/v3/__https://nam06.safelinks.protection.outlook.com/?url=https*3A*2F*2Fbugs.openjdk.org*2Fbrowse*2FJDK-8238686&data=05*7C02*7Cmonica.beckwith*40microsoft.com*7Cfdcf0de799d34a28c1a708dcef098c5b*7C72f988bf86f141af91ab2d7cd011db47*7C1*7C0*7C638648071506670981*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C0*7C*7C*7C&sdata=qm7IT0*2BgGk*2FC6rEufD0KodFoKtLND40Hu72tyMXW28A*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU!!ACWV5N9M2RV99hQ!PbxUKlUyx9Ac7nM5qynv6cmm0TkRu4SwHBLdIakj1AtTw-4ORHeRBexOJhYzyo5P81TK8pEtcoZ3bNYGHPQIOt5I3A0vqBdc$>> explains the issue).
>> In practice, these two flags prohibit G1 from adjusting the heap unless
>> the SoftMaxHeapSize change is very large.
>>
>>
>> So I would prefer to only think of an alternative to SoftMaxHeapSize if
>> it has been shown that it does not work.
>>
>>
>> Given that you have a much stronger mental model than I do of how all
>> these flags fit together in the context of G1 GC, perhaps it would be
>> helpful to schedule some time to chat in person! I think that would help
>> clarify things much more quickly than email. To be clear - I have no
>> reason to doubt that SoftMaxHeapSize does not work. On the other hand,
>> could we possibly make use of both flags? For example, could
>> SoftMaxHeapSize potentially be a good replacement for our periodic GC?
>
> Not sure what periodic GC has to do with SoftMaxHeapSize.
>
>>
>> There is the nit that unlike in this implementation of ProposedHeapSize,
>> SoftMaxHeapSize will not cause uncommit below MinHeapSize. This is
>> another discussion on what to do about this issue - in a comment in
>> _https://bugs.openjdk.org/browse/JDK-8236073_
> <https://urldefense.com/v3/__https://nam06.safelinks.protection.outlook.com/?url=https*3A*2F*2Fbugs.openjdk.org*2Fbrowse*2FJDK-8236073&data=05*7C02*7Cmonica.beckwith*40microsoft.com*7Cfdcf0de799d34a28c1a708dcef098c5b*7C72f988bf86f141af91ab2d7cd011db47*7C1*7C0*7C638648071506688389*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C0*7C*7C*7C&sdata=U3SJVBO*2F0gtF4xZOLdGi2QMORV8DIht1FciPivWgMA8*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!ACWV5N9M2RV99hQ!PbxUKlUyx9Ac7nM5qynv6cmm0TkRu4SwHBLdIakj1AtTw-4ORHeRBexOJhYzyo5P81TK8pEtcoZ3bNYGHPQIOt5I3G39T64H$>
>> <_https://bugs.openjdk.org/browse/JDK-8236073_
> <https://urldefense.com/v3/__https://nam06.safelinks.protection.outlook.com/?url=https*3A*2F*2Fbugs.openjdk.org*2Fbrowse*2FJDK-8236073&data=05*7C02*7Cmonica.beckwith*40microsoft.com*7Cfdcf0de799d34a28c1a708dcef098c5b*7C72f988bf86f141af91ab2d7cd011db47*7C1*7C0*7C638648071506706310*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C0*7C*7C*7C&sdata=CMgyTApGCNiM6WoIWf*2F1T9GCXRIpgvaTQ08QJGhSajw*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!ACWV5N9M2RV99hQ!PbxUKlUyx9Ac7nM5qynv6cmm0TkRu4SwHBLdIakj1AtTw-4ORHeRBexOJhYzyo5P81TK8pEtcoZ3bNYGHPQIOt5I3KCjpF35$>> it is proposed to make
>> MinHeapSize manageable.
>>
>>
>> How useful is MinHeapSize in practice? Do we need it, or can we just set
>> it to zero to avoid having to deal with it at all?
>
> I think you are mixing AHS (give decent heap sizing in presence of
> external memory pressure) and getting "optimal" heap sizing (or iow
> "steering heap size" externally).
>
> AHS is mostly about the user not doing/setting any heap sizes; in this
> case just having min heap size very low is just fine just as suggested
> in the JEP.
>
> SoftMaxHeapSize (and ProposedHeapSize) is about the user setting a
> particular goal according to his whims. It is still interesting to set
> -Xms==-Xmx for e.g. fast startup or during heavy activity; however if an
> external system decides that it is useful to intermittently save memory
> up to a certain level, then follow that guidance.
>
> The mechanism to internally follow that guidance can be used by AHS.
>
>
>>
>> I (still) believe that AHS and SoftMaxHeapSize/ProposedHeapSize are
>> somewhat orthogonal.
>>
>> AHS (_https://openjdk.org/jeps/8329758_
> <https://urldefense.com/v3/__https://nam06.safelinks.protection.outlook.com/?url=https*3A*2F*2Fopenjdk.org*2Fjeps*2F8329758&data=05*7C02*7Cmonica.beckwith*40microsoft.com*7Cfdcf0de799d34a28c1a708dcef098c5b*7C72f988bf86f141af91ab2d7cd011db47*7C1*7C0*7C638648071506722980*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C0*7C*7C*7C&sdata=Aymh70Ju5C*2FX4hmtj2ri7EX4v*2F7NCA733kDBxAp*2FX8I*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUl!!ACWV5N9M2RV99hQ!PbxUKlUyx9Ac7nM5qynv6cmm0TkRu4SwHBLdIakj1AtTw-4ORHeRBexOJhYzyo5P81TK8pEtcoZ3bNYGHPQIOt5I3D8PsG6q$>
>> <_https://openjdk.org/jeps/8329758_
> <https://urldefense.com/v3/__https://nam06.safelinks.protection.outlook.com/?url=https*3A*2F*2Fopenjdk.org*2Fjeps*2F8329758&data=05*7C02*7Cmonica.beckwith*40microsoft.com*7Cfdcf0de799d34a28c1a708dcef098c5b*7C72f988bf86f141af91ab2d7cd011db47*7C1*7C0*7C638648071506741674*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C0*7C*7C*7C&sdata=quHGouKrnBKMDAHLTmnKVOGZ8*2FCDY0z79FiRL5xN*2BSg*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU!!ACWV5N9M2RV99hQ!PbxUKlUyx9Ac7nM5qynv6cmm0TkRu4SwHBLdIakj1AtTw-4ORHeRBexOJhYzyo5P81TK8pEtcoZ3bNYGHPQIOt5I3GaQLQ28$>>) is about finding a reasonable
>> heap size, and adjust on external "pressure". SoftMax/ProposedHeapSize
>> are manual external tunings.
>>
>>
>> Wdyt?
>>
>>
>> I agree with the general idea - for us, we used a manual external flag
>> like ProposedHeapSize because we did not implement any of the AHS logic
>> in the JVM. (We had a separate AHS thread reading in container
>> information and then doing the calculations, then setting
>> ProposedHeapSize as a manageable flag.) The way I see it is that
>> SoftMax/ProposedHeapSize is the "output" of AHS, and then
>> SoftMax/ProposedHeapSize is the "input" for the JVM, after which the JVM
>> uses this input to adjust its behavior accordingly. Does that align with
>> how you see things?
>
> As mentioned in the other thread, SoftMaxHeapSize can be used by AHS to
> get heap to a certain level (based on memory pressure), but there is
> also that external entity that can modify SoftMaxHeapSize to adjust VM
> behavior.
>
> So ultimately there will be multiple inputs for target heap size (and
> probably I'm forgetting one or the other):
>
> * External memory pressure (AHS) (*)
>
> * CurrentMaxHeapSize
>
> * SoftMaxHeapSize
>
> * CPU usage (existing GCTimeRatio based policy)
>
> * other *HeapSize flags
>
> that need to be merged into some target heap level using some policy.
>
> After knowing that level, the VM needs to decide on a proper reaction,
> which might be anything from just setting internal IHOP goal, to
> (un-)committing memory directly, to doing the appropriate garbage
> collection in a "timely" fashion (which is where the regular periodic
> gc/marking comes in) or anything inbetween.
>
> (*) I am aware that the AHS JEP not only includes reaction on external
> memory pressure but also the merging of goals for different sources;
> some of them are ZGC specific. Some of them are already implemented in
> G1. So for this discussion it is imo useful to limit "AHS" in G1 context
> to things that G1 does not do. Ie. "return another goal based on
> external memory pressure", "min/max heap size defaults(!)", and "adjust
> adaptive sizing".
>
>> If we do indeed implement AHS logic fully within the JVM, then we could
>> internally manage the sizing of the heap without exposing a manageable
>> flag. That being said, it seems to me that exposing this as a manageable
>> flag brings the additional benefit that one could plug in their own AHS
>> implementation that calculates target heap sizes with whatever data they
>> want (and then passes it into the JVM via the manageable flag).
>>
>> Again, I wonder if meeting to discuss would be efficient, and then we
>> can update the mailing list with the results of our discussion. Let me
>> know your thoughts!
>
> It's fine with me to meet to recap and discuss above; please suggest
> some time.
>
> Hth,
> Thomas
More information about the hotspot-gc-dev
mailing list