Min vs Max vs Soft Memory Limits

[Mike Rettig]

>> Finding a steady state is hard, since the GC doesn't know what is
important to you and your workload (latency vs. memory usage, etc).

Ideally ZGC would provide a set of options so that it knows the priorities
of the application. With G1, this is easy. You specify the min and max
memory and a max pause time target. G1 will adjust accordingly to meet
those goals. ZGC needs something similar to G1's max pause time. Since the
pause time is consistently below 1ms, the pause time target doesn't make
sense. Maybe ZGC needs a pause frequency target?

My goal with ZGC is to pause as infrequently as possible. I think that is a
common goal for developers. ZGC benchmarks often compare the avg and max
pause times compared to G1 but in some cases the overall pause time for ZGC
is greater than G1 because it pauses more frequently. Recently I've
disabled the proactive pauses. Why bother to do a collection when the heap
is 30% full when it can done in the same amount of time when the heap is at
80%?

On Wed, May 12, 2021 at 3:02 AM Per Liden <per.liden at oracle.com> wrote:

> Hi,
> On 5/10/21 7:30 PM, Mike Rettig wrote:
>
> Thanks Per for your response. I added some additional questions below.
>
> On Mon, May 10, 2021 at 3:04 AM Per Liden <per.liden at oracle.com> wrote:
>
>> Hi,
>>
>> On 5/7/21 6:04 PM, Mike Rettig wrote:
>> > I've been using ZGC with JDK 16 for a few months now and have been
>> pleased
>> > with the performance. I converted a few apps from G1 to ZGC so the heap
>> > could be increased without increasing pause times. ZGC has performed
>> well
>> > in these scenarios.
>>
>> Good to hear!
>>
>> >
>> > I've recently been modifying the ZGC options to optimize performance and
>> > minimize the amount of vm option tweaking required for various
>> > environments.  The apps are deployed to a large range of servers with
>> > different amounts of memory.  The apps can be greedy about using memory
>> so
>> > I don't have to worry about uncommitting memory. However, I'd like the
>> ZGC
>> > collector to find an optimal state without giving it all the system
>> memory
>> > at startup.
>>
>> ZGC will typically try to use all the memory you give it, to space out
>> GC cycles as much as possible, and thereby interfere as little as
>> possible. So, ZGC will not really try to find the optimal state (in
>> terms of memory usage) you're looking for here.
>>
>
> Will the jvm option ZAllocationSpikeTolerance influence the greediness of
> ZGC?
>
> ZAllocationSpikeTolerance affects the amount of safety margin ZGC includes
> when it tries to the predict future allocation rates. I would not really
> recommend changing ZAllocationSpikeTolerance, unless you have a super spiky
> and unpredictable workload where the allocation rate fluctuates a lot.
>
>
>
>> >
>> > Here is an example config:
>> >
>> > -Xms1024m -XX:SoftMaxHeapSize=1024m -XX:-ZUncommit
>> >
>> > I know the app needs at least 1gb. I don't set the max heap because all
>> the
>> > servers are different. The jvm will default the max heap to 25% of the
>> > system memory. I originally didn't set the soft max heap, but I was
>> > surprised when it defaulted to the max heap. How does ZGC handle the
>> soft
>> > heap in this configuration? If the app starts at 1gb but soon needs 2gb
>> > does the soft heap grow with it or does zgc still try to stay below 1gb
>> > since that was the initial value?
>>
>> The SoftMaxHeapSize will stay at 1G, and if the heap grows to 2G, then
>> ZGC will start doing GCs more often in order to try to get the heap
>> usage down to 1G again.
>>
>
> Why doesn't it bump the soft heap to 2G when it is forced to grow and
> uncommit is disabled? If ZGC generally performs better with a larger heap
> and uncommitting memory is not an option, then why would it waste cycles
> trying to keep memory usage below 1G? It seems keeping the soft heap at 1G
> will cause ZGC to be less efficient.
>
> SoftMaxHeapSize is an option to help keep memory usage down, while still
> allowing for the heap to temporarily grow to something larger to handle a
> workload spike.
>
> However, it can also be used to make ZGC collect garbage more
> aggressively, as a way to increase the safety margin with regards to
> allocation stalls. For example, when memory usage isn't a concern, you have
> a large heap and no need to uncommit. But it still wouldn't make sense for
> ZGC to automatically bump SoftMaxHeapSize in that case, as that would go
> against what the user's configuration wanted to achieve.
>
>
>
>>
>> > Would it be better not to set the soft heap and let it default to the
>> max heap?
>>
>> In your case, I would recommend that you set Xmx to more than what your
>> app typically needs, and then set SoftMaxHeapSize to what that your app
>> works well with, and so that GC cycles happen at some reasonable
>> frequency. I would also recommend that you leave uncommit enabled. That
>> way ZGC can grow the heap if there's a workload spike, but it will later
>> return than memory once the spike is over.
>>
>
> It is very difficult to determine what the "app works well with" for every
> environment. I'd prefer not to create custom gc settings for every server
> or desktop configuration that I deploy to. I might be harming performance
> with my memory configurations. With g1, I usually set the min/max memory
> and the pause target.  That is sufficient for G1 to find a steady state
> that is a good compromise that still meets the pause target. With G1, the
> pause times increase linearly with the max heap so increasing the heap
> isn't really an option to decrease pause times. However with ZGC I'd like
> the heap to grow to avoid any and all allocation stalls. It seems the soft
> heap and the allocation spike tolerance both factor into avoiding
> allocation stalls. Is that true? Is there anything else I should look at to
> avoid allocation stalls?
>
> Would it be possible to specify the soft heap in a percentage of system
> memory? That might make it easier in my scenario to find a good balance of
> memory usage. Actually I think it would be helpful for all memory based jvm
> options to allow for percentage based memory options (1g=1 gigabyte, 10per=
> 10 percent of available memory).
>
> You have the MaxRAMPercentage option to set the max heap size in percent,
> but there's no corresponding percent option for SoftMaxHeapSize at this
> time. As a workaround, you could of course have a start script that
> calculates how many megabytes corresponds to X% of the system RAM is and
> feed that number to SoftMaxHeapSize. But being able to set directly set
> SoftMaxHeapSize using percent might be a useful feature.
>
> Finding a steady state is hard, since the GC doesn't know what is
> important to you and your workload (latency vs. memory usage, etc). By
> default/design, ZGC tries to use as much of the heap it was given to favour
> latency.
>
> /Per
>
>
>
>>
>> > Does the jvm grow the heap
>> > differently when the soft heap is set to the min heap instead of the max
>> > heap when uncommit is disabled?
>>
>> The way the heap grows will not be affected, SoftMaxHeapSize only
>> affects how often GCs happens.
>>
>
>> I wrote a blog post about SoftMaxHeapSize some time ago. I'm not sure
>> the example I described there matches your use case, but you might find
>> it useful nevertheless. https://malloc.se/blog/zgc-softmaxheapsize
>>
>> cheers,
>> Per
>>
>> >
>> > Mike
>> >
>>
>