Heap Size Ergonomics in docker containers

[Bob Vandette]

> On Oct 27, 2017, at 9:54 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> Hi Bob,
> 
> 
> 
> On Wed, 2017-10-25 at 08:44 -0400, Bob Vandette wrote:
>>> On Oct 25, 2017, at 5:36 AM, Thomas Schatzl <thomas.schatzl at oracle.
>>> com> wrote:
>>> 
>>> On Tue, 2017-10-24 at 14:43 -0400, Bob Vandette wrote:
>>>> I’m trying to test the container support that I’m adding to 
>>>> JDK 18.3 and I’m running into an issue with the Heap size
>>>> calculation.
>>>> 
>>>> If I create a docker container that has 100MB of total memory on
>>>> a
>>>> 64-bit Linux box, I’m ending up with a 126MB Max Heap which
>>>> results
>>>> in the process getting killed by the OOM killer.
>>>> 
>>>> This is due to the fact that MaxHeapSize is 126MB and phys_mem
>>>> (100MB) is
>>>> not less than 50% of MinRAMPercentage of MaxHeapSize.
>>>> 
>>>> I would think in a small memory system, you don’t ever want the
>>>> Heap 
>>>> size to be more than somewhere around 70% of total RAM.  
>>>> 
>>>> A quick fix for my problem might be this, but this still
>>>> leaves some pathological cases where phys_mem is greater than but
>>>> close to MaxHeapSize.
>>>> 
>>>> if phys_mem < MaxHeapSize || phys_mem < MaxHeapSize *
>>>> MinRAMPercentage
>>>>    use phys_mem * MinRAMPercentage
>>>> 
>>>> I could improve on this with:
>>>> 
>>>> if MaxHeapSize > 70% of phys_mem || phys_mem < MaxHeapSize *
>>>> MinRAMPercentage
>>>>    use phys_mem * MinRAMPercentage
>>>> 
>>>> [...]
>>>> Let me know what you think.  
>>>> 
>>> 
>>>  that seems to be almost the same as making the default value of
>>> MaxHeapSize 70% smaller.
>> 
>> Not really.  That just creates the same problem if the containers
>> memory is set to some number near the lower value.
>> 
>>> Also that 70% only seems to serve the same
>>> purpose as MaxRAMPercentage, i.e. imo MaxRAMPercentage should have
>>> covered that.
>> 
>> I think it makes sense to have a MaxRAMPercentage with a
>> MinRAMPercentage. You don’t want to use the same % for large GB’s of
>> RAM as you’d use with 128M so I don’t agree that we should cover all
>> cases with MaxRAMPercentage.
> 
> The general consensus within in the gc team is that the less
> complicated any such heuristic is, the less surprising it is, the
> easier to explain, with less maintenance and so more favorable from our
> point of view. :)
> 
> The reasons are simple: this acknowledges the fact that you can't
> capture the needs of billions of users in any kind of manageable
> heuristic; there is also a limit on how complicated you want to make
> this particular heuristic, because the more complicated to make it,
> most likely you will get more corner cases for which you might want to
> expose knobs to the user; and at that point the ultimate fix to this
> issue is to tell to "set -Xmx" anyway.
> 
> A user is already going through the trouble of setting a max memory
> size limit for your container...
> 
> Also, we have never seen anyone setting Min/MaxRAMFraction ever except
> in error.
> 
> I do not think for this issue we should go down this road, making the
> heuristics even more complicated.
> 
>>> 
>>> I kind of question this line in the code:
>>> 
>>>       // Not-small physical memory, so require a heap at least
>>>       // as large as MaxHeapSize
>>>       reasonable_max = MAX2(reasonable_max, (julong)MaxHeapSize);
>>> 
>>> So we just found out a "reasonable max" as a part of the
>>> calculation, and then we ignore it…
>> 
>> I don’t think it’s ignored.  reasonable_max will be set to 25% of
>> phys_mem which could be larger than MaxHeapSize.
>> 
> 
> I filed JDK-8190283 [0], as this is a bug however you look at it.
> 
> The suggested fix for this is to use physmem*MinRAMFraction while it is
> below the default MaxHeapSize value, then switch over to
> physmem*MaxRAMFraction. Since physmem*MaxRAMFraction is at that point
> smaller than MaxHeapSize, keep using MaxHeapSize as long as
> physmem*MaxRAMFraction is smaller.
> As such, default MaxHeapSize would act as kind of switchover threshold
> between the two heuristics (that switchover point is pretty random as
> before) - that should ensure that the selected heap size is always
> smaller than the amount of physical memory, and fix the problem.
> 
> This would make the heap 50% (MinRAMFraction) of physical memory for
> machines with < 252M, 126M for machines with physical memory < 504M,
> and 25% (MaxRAMFraction) of physical memory otherwise.
> 
> A prototype patch is available at http://cr.openjdk.java.net/~tschatzl/
> 8190283/webrev/ (did not test that, I simply tried to implement the
> formula from a spreadsheet I used, but I think it shows the idea) -
> it's also very small.
> 
> What do you think?

You suggested fix looks fine to me.  According to my spreadsheat,
It results in more even and gradual increases in heap size over my proposal.

I’ll do some testing with different sizes and send the change out for review.

> 
> ------
> 
> Now there still may be the question whether the use case you presented,
> i.e. "with a "small heap" with 100MB use 70% of the available memory"
> is worth changing some additional default options for.
I would actually like to suggest that we change MinRAMPercentage to a value
a bit higher (possible 60%).  I’ve had complaints from the Container team
that they felt Java was not able to make use of available memory.  I gave them
the Percentage flavors of these flag so they could push the envelope a bit
but I think 50% is a bit too conservative.  Having said that, I don’t think
we should change that with this fix.

> 
> We think that we should not go that direction: the VM does not give
> outright bad values for heap size any more (50MB with default
> MinRAMFraction), providing safe and fairly good settings. There has
> been no indication on whether a 100MB container is that pervasive
> either.
> 
> Particularly a MinRAMFraction of 70% seems a bit too tight as default
> for the current default collector, i.e. G1. As we have seen it can, in
> extreme cases as a ballpark figure use another 20% of Java heap size
> just for its remembered sets (theoretical max is even higher), and
> other subsystems also need memory (compiler, runtime).
In my proposal, there was only one phys mem size where we selected
70% of memory for the heap (at 180MB) and that left 54MB avail.
If you look at the 100MB selection, we only have 50MB free.

In any case, your proposal works.

I also don’t think 100MB is something we should optimize for.  I just
wanted to keep us from picking a heap size too large for all low
numbers.  256MB is probably a better low end for the JFaas folks.

> 
> Note that I think that maybe G1 isn't the correct choice for such small
> heaps at the moment - serial or parallel would probably run as fine if
> not better with less overhead(*) - but that's also something we would
> not like to change haphazardly; and it is possible and might be better
> to improve G1 here in the future instead of changing defaults. (And if
> the user is already selecting the GC, he might as well set max heap
> size).

I agree that SerialGC is better for serverless.  This is what Amazon is using
with Lambda.


Thanks for spending time on this issue,
Bob.

> 
> Thanks,
>  Thomas
> 
> [0] https://bugs.openjdk.java.net/browse/JDK-8190283
> (*) haven't tried lately.
> 
>