Min vs Max vs Soft Memory Limits

[Per Liden]

Hi,

On 5/10/21 7:30 PM, Mike Rettig wrote:
> Thanks Per for your response. I added some additional questions below.
>
> On Mon, May 10, 2021 at 3:04 AM Per Liden <per.liden at oracle.com 
> <mailto:per.liden at oracle.com>> wrote:
>
>     Hi,
>
>     On 5/7/21 6:04 PM, Mike Rettig wrote:
>     > I've been using ZGC with JDK 16 for a few months now and have
>     been pleased
>     > with the performance. I converted a few apps from G1 to ZGC so
>     the heap
>     > could be increased without increasing pause times. ZGC has
>     performed well
>     > in these scenarios.
>
>     Good to hear!
>
>     >
>     > I've recently been modifying the ZGC options to optimize
>     performance and
>     > minimize the amount of vm option tweaking required for various
>     > environments.  The apps are deployed to a large range of servers
>     with
>     > different amounts of memory.  The apps can be greedy about using
>     memory so
>     > I don't have to worry about uncommitting memory. However, I'd
>     like the ZGC
>     > collector to find an optimal state without giving it all the
>     system memory
>     > at startup.
>
>     ZGC will typically try to use all the memory you give it, to space
>     out
>     GC cycles as much as possible, and thereby interfere as little as
>     possible. So, ZGC will not really try to find the optimal state (in
>     terms of memory usage) you're looking for here.
>
>
> Will the jvm option |ZAllocationSpikeTolerance influence the 
> greediness of ZGC?|

ZAllocationSpikeTolerance affects the amount of safety margin ZGC 
includes when it tries to the predict future allocation rates. I would 
not really recommend changing ZAllocationSpikeTolerance, unless you have 
a super spiky and unpredictable workload where the allocation rate 
fluctuates a lot.

> |
> |
>
>
>     >
>     > Here is an example config:
>     >
>     > -Xms1024m -XX:SoftMaxHeapSize=1024m -XX:-ZUncommit
>     >
>     > I know the app needs at least 1gb. I don't set the max heap
>     because all the
>     > servers are different. The jvm will default the max heap to 25%
>     of the
>     > system memory. I originally didn't set the soft max heap, but I was
>     > surprised when it defaulted to the max heap. How does ZGC handle
>     the soft
>     > heap in this configuration? If the app starts at 1gb but soon
>     needs 2gb
>     > does the soft heap grow with it or does zgc still try to stay
>     below 1gb
>     > since that was the initial value?
>
>     The SoftMaxHeapSize will stay at 1G, and if the heap grows to 2G,
>     then
>     ZGC will start doing GCs more often in order to try to get the heap
>     usage down to 1G again.
>
>
> Why doesn't it bump the soft heap to 2G when it is forced to grow and 
> uncommit is disabled? If ZGC generally performs better with a larger 
> heap and uncommitting memory is not an option, then why would it waste 
> cycles trying to keep memory usage below 1G? It seems keeping the soft 
> heap at 1G will cause ZGC to be less efficient.

SoftMaxHeapSize is an option to help keep memory usage down, while still 
allowing for the heap to temporarily grow to something larger to handle 
a workload spike.

However, it can also be used to make ZGC collect garbage more 
aggressively, as a way to increase the safety margin with regards to 
allocation stalls. For example, when memory usage isn't a concern, you 
have a large heap and no need to uncommit. But it still wouldn't make 
sense for ZGC to automatically bump SoftMaxHeapSize in that case, as 
that would go against what the user's configuration wanted to achieve.

>
>     > Would it be better not to set the soft heap and let it default
>     to the max heap?
>
>     In your case, I would recommend that you set Xmx to more than what
>     your
>     app typically needs, and then set SoftMaxHeapSize to what that
>     your app
>     works well with, and so that GC cycles happen at some reasonable
>     frequency. I would also recommend that you leave uncommit enabled.
>     That
>     way ZGC can grow the heap if there's a workload spike, but it will
>     later
>     return than memory once the spike is over.
>
>
> It is very difficult to determine what the "app works well with" for 
> every environment. I'd prefer not to create custom gc settings for 
> every server or desktop configuration that I deploy to. I might be 
> harming performance with my memory configurations. With g1, I usually 
> set the min/max memory and the pause target.  That is sufficient for 
> G1 to find a steady state that is a good compromise that still meets 
> the pause target. With G1, the pause times increase linearly with the 
> max heap so increasing the heap isn't really an option to decrease 
> pause times. However with ZGC I'd like the heap to grow to avoid any 
> and all allocation stalls. It seems the soft heap and the allocation 
> spike tolerance both factor into avoiding allocation stalls. Is that 
> true? Is there anything else I should look at to avoid allocation stalls?
>
> Would it be possible to specify the soft heap in a percentage of 
> system memory? That might make it easier in my scenario to find a good 
> balance of memory usage. Actually I think it would be helpful for all 
> memory based jvm options to allow for percentage based memory options 
> (1g=1 gigabyte, 10per= 10 percent of available memory).

You have the MaxRAMPercentage option to set the max heap size in 
percent, but there's no corresponding percent option for SoftMaxHeapSize 
at this time. As a workaround, you could of course have a start script 
that calculates how many megabytes corresponds to X% of the system RAM 
is and feed that number to SoftMaxHeapSize. But being able to set 
directly set SoftMaxHeapSize using percent might be a useful feature.

Finding a steady state is hard, since the GC doesn't know what is 
important to you and your workload (latency vs. memory usage, etc). By 
default/design, ZGC tries to use as much of the heap it was given to 
favour latency.

/Per

>
>     > Does the jvm grow the heap
>     > differently when the soft heap is set to the min heap instead of
>     the max
>     > heap when uncommit is disabled?
>
>     The way the heap grows will not be affected, SoftMaxHeapSize only
>     affects how often GCs happens.
>
>
>     I wrote a blog post about SoftMaxHeapSize some time ago. I'm not sure
>     the example I described there matches your use case, but you might
>     find
>     it useful nevertheless. https://malloc.se/blog/zgc-softmaxheapsize
>     <https://malloc.se/blog/zgc-softmaxheapsize>
>
>     cheers,
>     Per
>
>     >
>     > Mike
>     >
>