Adaptable Heap Sizing for G1 GC

[Jonathan Joo]

Hi Erik,

We had a similar vision with regards to not having to set heap sizes
manually :) Agreed that with the currently proposed OpenJDK changes alone,
there would be no effect for the user, just an entry point to allow for
more intelligent heap sizing.

We definitely do want to ship a policy that actually calculates and sets
these flags, but I think a good point for discussion is *how* to ship such
a policy. Note that as long as the two flags are introduced into the
OpenJDK, there is always a way for people to modify the flags on their own
and get AHS-like behavior. I guess the question is, to what extent do we
want to take our current implementation of AHS logic, and move that from
outside the JVM into the JVM?  I think there are a few different
possibilities, given that currently, AHS relies on internal Google services
to access all the data we need.

1. Try to replicate exactly the way AHS works using APIs available from
within hotspot code. For example, querying container limit and fullness
information in a way that can work in any generic container environment.
(Is there a good way to obtain this?)

2. Come up with a potentially less complex, but general working solution
that is maintained solely within the hotspot code. The cons of this is that
Google's implementation and upstream's implementation will diverge, and so
there is more maintenance overhead from our end. It also won't have as
robust functionality as the solution we are using at Google.

3. Don't bother with importing any AHS logic into the OpenJDK, but instead
simply open-source/publish our current policies. This would allow for
people to adopt their own implementations of AHS to plug it in a way they
see fit, or fiddle with our code and integrate it into their own
environments. Though I agree that without access to a special launcher or
other mechanism to run this code, this approach may have limited usefulness.

I'm not as familiar with logistically how viable it would be to do these
solutions. Would love to hear whether you think these approaches are
viable, and/or any blockers you might foresee.

Best,

~ Jonathan


On Thu, Jun 13, 2024 at 4:17 AM Erik Osterlund <erik.osterlund at oracle.com>
wrote:

> Hi Jonathan,
>
> I’m currently working on automatic heap sizing for ZGC. My vision is that
> users shouldn’t have to set heap sizes.
> Would love to see that in G1 as well. What you are describing sounds like
> it would do something similar.
>
> Having said that, it seems like the concrete changes you are proposing for
> OpenJDK, would not actually
> yield automatic heap sizing for the user. By the sound of it, you would
> need your special launcher
> with an extra thread that contains the actual heap sizing policy. The
> proposed JVM changes are mostly for
> being *able* to change the heap sizing policies externally, but without
> any policy shipped that actually
> changes it.
>
> While having a pluggable policy is great because anyone can put in their
> own favourite policy, there
> is also an obvious disadvantage that 99.9% of deployments won’t have any
> special launcher or
> supplier of an external heap sizing policy, or even know what we are
> talking about. Therefore,
> unless we also ship the policies, I unfortunately think that limits the
> usefulness of the feature.
> If, however, a policy was shipped so the heap can be sized automatically,
> I think that would make it
> much more widely useful.
>
> In my automatic heap sizing work, the goal is to ship both the mechanisms
> and the policies needed
> to automatically size (and resize) the heap, adapting to changing load and
> environments. Are you
> open to the idea of shipping a policy that actually changes the heap size
> as well? It would be great
> to be aligned on this, I think.
>
> Thanks,
> /Erik
>
> On 13 Jun 2024, at 01:32, Jonathan Joo <jonathanjoo at google.com> wrote:
>
> Hello hotspot-dev and hotspot-gc-dev,
>
> I'd like to reopen discussion on Adaptable Heap Sizing (AHS) for the G1
> Garbage Collector, since we now have some time to dedicate to bringing this
> effort to the OpenJDK Community. Please see
> https://mail.openjdk.org/pipermail/hotspot-gc-dev/2022-September/040096.html
> for the original thread.
>
> The bullet points contained in the above link are still largely the same,
> and we have made significant improvements to the service over the past few
> years, and found success deploying it broadly across jobs internally. Now
> that we feel the feature has matured, we'd like to introduce it to the
> OpenJDK community in hopes that it can be adopted for broader use.
>
> In short - the goal of Adaptable Heap Sizing is to improve memory usage
> and reduce OOMs for Java applications, especially those deployed in
> containerized environments. The key insights are as follows:
>
>
>    1. Applications with low memory requirements but configured with high
>    RAM often use RAM unnecessarily. We can utilize GC CPU overhead metrics to
>    help guide heap sizing, allowing for RAM savings in these scenarios.
>    2. For Java applications running in containers, we can bound Java heap
>    usage based on our knowledge of the current container memory usage as well
>    as the current container size, to prevent container OOMs.
>
>
> The implementation of AHS currently involves some fairly lightweight
> changes to the JVM, through the introduction of two new manageable flags.
> They are essentially the same as these two (open feature requests):
>
>    - https://bugs.openjdk.org/browse/JDK-8236073
>    - https://bugs.openjdk.org/browse/JDK-8204088
>
>
> In addition, we have a separate thread (outside of the JVM, in our custom
> Java launcher) which reads in GC CPU overhead data and container
> information, and calculates appropriate values for these two flags. We call
> this the AHS worker thread, and this thread updates frequently (currently
> every second). The vast majority of the AHS logic is in this worker thread
> - the introduction of the new JVM flags above simply gives AHS a way to
> tune GC heuristics given this additional information.
>
> Thomas Schatzl mentioned there is a similar-sounding effort going on in
> ZGC <https://bugs.openjdk.org/browse/JDK-8329758>, and also there were
> folks outside of Google who expressed interest in this project, so I think
> it is an appropriate time to discuss this again on an open forum. Given the
> positive results we've had deploying AHS internally at Google, we feel this
> is a valuable feature to the broader Java community that should be able to
> be leveraged by all to achieve more stable and efficient Java heap behavior
> 🙂
>
> I'd appreciate hearing peoples' thoughts on this. Thank you!
>
> ~ Jonathan
>
> (P.S. For more information, a talk given about this project can be viewed
> here <https://www.youtube.com/watch?v=qOt4vOkk49k>, though it is somewhat
> dated.)
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20240614/e6a68f7c/attachment-0001.htm>