[External] : Re: Adaptable Heap Sizing for G1 GC

Tue Jun 18 19:14:29 UTC 2024

Hi Erik,

Thank you for the pointer - this does seem promising. I'll be on vacation
for the next week (+ a few days), but I'll look into this more when I get
back!

Best,

~ Jonathan

On Tue, Jun 18, 2024 at 5:01 AM Erik Osterlund <erik.osterlund at oracle.com>
wrote:

> Hi Jonathan,
>
> Regarding alternative 1, the JVM already does this. The
> os::available_memory() and os::physical_memory() APIs detect if we are
> running in a container, and then queries container limits. So if that’s the
> concern, then it would seem that we already have a solution. That’s what
> I’m using in my automatic heap sizing work for ZGC, and it’s working just
> fine. Am I missing anything?
>
> Thanks,
> /Erik
>
> On 15 Jun 2024, at 00:56, Jonathan Joo <jonathanjoo at google.com> wrote:
>
> Hi Erik,
>
> We had a similar vision with regards to not having to set heap sizes
> manually :) Agreed that with the currently proposed OpenJDK changes alone,
> there would be no effect for the user, just an entry point to allow for
> more intelligent heap sizing.
>
> We definitely do want to ship a policy that actually calculates and sets
> these flags, but I think a good point for discussion is *how* to ship such
> a policy. Note that as long as the two flags are introduced into the
> OpenJDK, there is always a way for people to modify the flags on their own
> and get AHS-like behavior. I guess the question is, to what extent do we
> want to take our current implementation of AHS logic, and move that from
> outside the JVM into the JVM?  I think there are a few different
> possibilities, given that currently, AHS relies on internal Google services
> to access all the data we need.
>
> 1. Try to replicate exactly the way AHS works using APIs available from
> within hotspot code. For example, querying container limit and fullness
> information in a way that can work in any generic container environment.
> (Is there a good way to obtain this?)
> 2. Come up with a potentially less complex, but general working solution
> that is maintained solely within the hotspot code. The cons of this is that
> Google's implementation and upstream's implementation will diverge, and so
> there is more maintenance overhead from our end. It also won't have as
> robust functionality as the solution we are using at Google.
> 3. Don't bother with importing any AHS logic into the OpenJDK, but instead
> simply open-source/publish our current policies. This would allow for
> people to adopt their own implementations of AHS to plug it in a way they
> see fit, or fiddle with our code and integrate it into their own
> environments. Though I agree that without access to a special launcher or
> other mechanism to run this code, this approach may have limited usefulness.
>
> I'm not as familiar with logistically how viable it would be to do these
> solutions. Would love to hear whether you think these approaches are
> viable, and/or any blockers you might foresee.
>
> Best,
>
> ~ Jonathan
>
>
> On Thu, Jun 13, 2024 at 4:17 AM Erik Osterlund <erik.osterlund at oracle.com>
> wrote:
>
>> Hi Jonathan,
>>
>> I’m currently working on automatic heap sizing for ZGC. My vision is that
>> users shouldn’t have to set heap sizes.
>> Would love to see that in G1 as well. What you are describing sounds like
>> it would do something similar.
>>
>> Having said that, it seems like the concrete changes you are proposing
>> for OpenJDK, would not actually
>> yield automatic heap sizing for the user. By the sound of it, you would
>> need your special launcher
>> with an extra thread that contains the actual heap sizing policy. The
>> proposed JVM changes are mostly for
>> being *able* to change the heap sizing policies externally, but without
>> any policy shipped that actually
>> changes it.
>>
>> While having a pluggable policy is great because anyone can put in their
>> own favourite policy, there
>> is also an obvious disadvantage that 99.9% of deployments won’t have any
>> special launcher or
>> supplier of an external heap sizing policy, or even know what we are
>> talking about. Therefore,
>> unless we also ship the policies, I unfortunately think that limits the
>> usefulness of the feature.
>> If, however, a policy was shipped so the heap can be sized automatically,
>> I think that would make it
>> much more widely useful.
>>
>> In my automatic heap sizing work, the goal is to ship both the mechanisms
>> and the policies needed
>> to automatically size (and resize) the heap, adapting to changing load
>> and environments. Are you
>> open to the idea of shipping a policy that actually changes the heap size
>> as well? It would be great
>> to be aligned on this, I think.
>>
>> Thanks,
>> /Erik
>>
>> On 13 Jun 2024, at 01:32, Jonathan Joo <jonathanjoo at google.com> wrote:
>>
>> Hello hotspot-dev and hotspot-gc-dev,
>>
>> I'd like to reopen discussion on Adaptable Heap Sizing (AHS) for the G1
>> Garbage Collector, since we now have some time to dedicate to bringing this
>> effort to the OpenJDK Community. Please see
>> https://mail.openjdk.org/pipermail/hotspot-gc-dev/2022-September/040096.html
>> for the original thread.
>>
>> The bullet points contained in the above link are still largely the same,
>> and we have made significant improvements to the service over the past few
>> years, and found success deploying it broadly across jobs internally. Now
>> that we feel the feature has matured, we'd like to introduce it to the
>> OpenJDK community in hopes that it can be adopted for broader use.
>>
>> In short - the goal of Adaptable Heap Sizing is to improve memory usage
>> and reduce OOMs for Java applications, especially those deployed in
>> containerized environments. The key insights are as follows:
>>
>>
>>    1. Applications with low memory requirements but configured with high
>>    RAM often use RAM unnecessarily. We can utilize GC CPU overhead metrics to
>>    help guide heap sizing, allowing for RAM savings in these scenarios.
>>    2. For Java applications running in containers, we can bound Java
>>    heap usage based on our knowledge of the current container memory usage as
>>    well as the current container size, to prevent container OOMs.
>>
>>
>> The implementation of AHS currently involves some fairly lightweight
>> changes to the JVM, through the introduction of two new manageable flags.
>> They are essentially the same as these two (open feature requests):
>>
>>    - https://bugs.openjdk.org/browse/JDK-8236073
>>    - https://bugs.openjdk.org/browse/JDK-8204088
>>
>>
>> In addition, we have a separate thread (outside of the JVM, in our custom
>> Java launcher) which reads in GC CPU overhead data and container
>> information, and calculates appropriate values for these two flags. We call
>> this the AHS worker thread, and this thread updates frequently (currently
>> every second). The vast majority of the AHS logic is in this worker thread
>> - the introduction of the new JVM flags above simply gives AHS a way to
>> tune GC heuristics given this additional information.
>>
>> Thomas Schatzl mentioned there is a similar-sounding effort going on in
>> ZGC <https://bugs.openjdk.org/browse/JDK-8329758>, and also there were
>> folks outside of Google who expressed interest in this project, so I think
>> it is an appropriate time to discuss this again on an open forum. Given the
>> positive results we've had deploying AHS internally at Google, we feel this
>> is a valuable feature to the broader Java community that should be able to
>> be leveraged by all to achieve more stable and efficient Java heap behavior
>> 🙂
>>
>> I'd appreciate hearing peoples' thoughts on this. Thank you!
>>
>> ~ Jonathan
>>
>> (P.S. For more information, a talk given about this project can be viewed
>> here
>> <https://urldefense.com/v3/__https://www.youtube.com/watch?v=qOt4vOkk49k__;!!ACWV5N9M2RV99hQ!Oszzs1LDHAorbfqM2bBWDnrhvBw1hegInNEpAMGdJ92V1Tx9b4dSwv_3WUXFd7IiXFG-jTYFw_dEdYAkSD1BICDT$>,
>> though it is somewhat dated.)
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20240618/29103297/attachment-0001.htm>