A question about the present condition/plans of Dynamic Max Memory Limit

[Jonathan Joo]

Hi Kats,

I just wanted to weigh in here (as Thomas mentioned) and say that we at
Google are actively working on something fairly similar, and have seen
success with limiting heap expansions using a dynamic flag similar to
CurrentMaxHeapSize. We have begun using this functionality in containerized
environments to help prevent container OOMs. (If you are interested in
learning more about this project, feel free to check out this talk here:
https://www.youtube.com/watch?v=qOt4vOkk49k

Unfortunately, I haven't had the bandwidth to upstream the code (since it
is somewhat tied to Google's specific implementation). I do want to work on
upstreaming these changes and/or making progress on JEP-8204088, but I
don't see this happening for at least 6 months at the minimum. If this is
something you are actively interested in working on, then I am happy to
share my thoughts and experiences. (We can schedule a meeting to discuss,
etc.)

Thank you!

~ Jonathan

On Thu, Feb 2, 2023 at 4:36 AM Thomas Schatzl <thomas.schatzl at oracle.com>
wrote:

> Hi,
>
> On 02.02.23 12:31, Katsutomo Sekiguchi (Fujitsu) wrote:
> > Hi Thomas,
> >
> > Thank you for quick reply. I got the current situation.
> >
> > I believe that it needs a chunk of address space for MaxHeapSize even if
> > it only allocate heaps up to CurrentMaxHeapSize.
> > In other words, do you mean it reserve an address space of MaxHeapSize
> > and commit a physical memory to CurrentMaxHeapSize instead of MaxHeapSize
> > then control the amount of physical memory by changing
> CurrentMaxHeapSize?
> >
> > Maybe, I think we must figure out somehow to feed back current
> conditions like
> > heap usage as input to change CurrentMaxHeapSize.
>
>    making the heap expansion follow CurrentMaxHeapSize instead of
> MaxHeapSize is not that big of an issue. Existing code already handles
> committing and uncommitting just fine if you use the new variable.
>
> I.e. the only difference having a CurrentMaxHeapSize or not is instead
> of expanding until MaxHeapSize, only allow heap expansion (and
> allocation) up to CurrentMaxHeapSize.
>
> E.g. in the patch you gave [1], the hunk at @@ -1148,7 +1148,8 @@ in
> g1CollectedHeap.cpp limited that for heap expansion.
> It also handled other cases where G1 would try to expand.
>
> The real problems (to me) seem to be located in other users of "max heap
> size" value in the VM.
>
> An initial review of mine noticed quite a few issues and questions with
> that implementation ([2]) that ought to be addressed in one way or
> another; briefly looking over them the comments still seem very much
> applicable.
>
> [1] http://cr.openjdk.java.net/~tschatzl/jelastic/cmx/
> [2]
> https://mail.openjdk.org/pipermail/hotspot-gc-dev/2018-June/022472.html
>
> Hth,
>    Thomas
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20230202/c4f8a421/attachment.htm>