Re: G1 patch of elastic Java heap

Liang Mao maoliang.ml at alibaba-inc.com
Thu Oct 10 13:48:42 UTC 2019


Hi Thomas,

Thank you for the feedback.
You are right about some points that the present code seems to separate the heap into young 
and old gen pools. In OpenJDK8, there's no adaptive-ihop so fixed ihop and MaxNewSize can clearly separate
young gen and old gen. I'm also thinking about how to design it better in upstream of OpenJDK G1.

There is a tradeoff between memory and GC frequency. More frequent GC uses less memory. We found 
our online service applications keep large young generation for potential query traffic but most
 of time the young GC frequency is quite low. Memory can be easily saved by using smaller young gen.
In Shenandoah or ZGC, there is only 1 generation and it's straightforward to determine if memory is
wasted and can be returned. G1 has 2 generations, in remark phase MinHeapFreeRatio/MaxHeapFreeRatio
cannot tell the young generation is rather wasted for running 2 minutes without a young GC and we can 
return a lot of memory. Each generation's GC interval or time ratio spent on mutator/gc you mentioned
seems more intuitive.

The explicit limitation of generation may not be a good design from G1 GC's perspective. From the 
operation's point of view, it is easy for manipulating JVM. There is a simple relationship:
larger network traffic -> higher memory allocation rate -> larger young generation. So cluster
operation can easily set the young generation as 10% of max young gen size to every Java instance
if the network traffic is guanranteed to be below 10% for a period of time.

I'm not sticking to the current implementation to create clear boundary between young and old gen,
 especially for newer OpenJDK versions and I've been thinking of unifying the 2 generations' resizing 
within the single memory pool of heap along with Xms. The periodic uncommit mode does not strickly 
separate the young/old gen. Current implementation calculates the average GC interval and keep it in 
a certain range between a low bound and high bound and will immediately trigger an expansion if a 
single GC interval smaller than a threshould. We can use a similar policy to estimate a target young 
generation capacity and adjust the capacity of old generation after a concurrent cycle. The 2 parts
 together can be the target heap capacity. The capacity can vary between Xms and Xmx. The difference
 with current G1 is it can be resized in a young GC not only remark.

In order to do swift heap resizing we have to conquer the over head of memory request/release from OS.
 The memory unmap and map(including the page fault) cost significant time. So we use an intuitive way
to have a concurrent thread to do the map/unmap/pretouch. The free regions will be synchronized in GC
pause. In our applications, a typical G1 remark cost ~100ms of pause. I haven't tested latest G1 but 
based on our experimental data, the pause can be easily doubled if done considerable map/unmaps. 


All of above are our thoughts and the present implementation is kind of reference. Please let me know if 
I answered all your questions. Hope we can come to an agreement in some points and conceive a good design
 in latest G1 GC :)

Thanks,
Liang






------------------------------------------------------------------
From:Thomas Schatzl <thomas.schatzl at oracle.com>
Send Time:2019 Oct. 9 (Wed.) 22:12
To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>; hotspot-gc-dev <hotspot-gc-dev at openjdk.java.net>
Subject:Re: G1 patch of elastic Java heap

Hi,

   sorry for the late reply.

First, I have a more general question: lots of changes deal with 
providing options to separately change properties generations at 
runtime. Like if there were separate pools of young and old gen memory.

G1 is kind of built upon the idea that you pass a pause time goal and 
then modifies generation sizes and takes memory for the generations from 
a single memory pool as needed.

To me this indicates that automatic sizing is not working correctly, but 
there are many(?) use cases where it does not work as expected. This 
requires manual tuning in generation sizes for whatever reason.

Can you share your thoughts about this? There seems to be some bit of 
information missing to me - this is probably the reason for some of the 
dumb questions about the flags, and me being not too fond of them.

On 26.09.19 08:49, Liang Mao wrote:
> 
> Hi All,
> 
> Here is the user guide of G1ElasticHeap patch. Hope it will help to 
> understand.
> 
> G1ElasticHeap
> G1ElasticHeap is a GC feature to return memory of Java heap to OS to reduce the 
> 
> memory footprint of Java process. To enable this feature, you need to use G1 GC 
> 
> by options: -XX:+UseG1GC -XX:+G1ElasticHeap.
> 
> ## Usage
> There are 3 modes which can be enabled in G1ElasticHeap.
> ### 1. Periodic uncommit
> Memory will be uncommitted by periodic GC. To enable periodic uncommit, use option 
> 
> -XX:+ElasticHeapPeriodicUncommit or dynamically enable the option via jinfo:
> 
> `jinfo -flag +ElasticHeapPeriodicUncommit PID`

As far as I can tell, this setting periodically scans the heap for (too 
many?) uncommitted regions and, well, uncommits them.

Not completely sure if that is better than doing periodic gcs - as we do 
not expect to gain memory outside of a GC; in JDK12+ (I think) G1 alwasy 
uncommits at the remark pause which should give most of the benefits.

There *may* be reason to also try to uncommit after the last mixed GC, 
but not sure if uncommit is that urgent - to some degree the existing 
JEP 346: Promptly return unused committed memory from G1 
(https://openjdk.java.net/jeps/346) should cover some of the use cases. 
I.e. after some delay (and inactivity) there will be another Remark 
pause anyway.

The main reason why Remark has been chosen to uncommit memory is because 
we assume that the heap size at Remark (this is what adaptive IHOP 
shoots for) is the "target heap size".


> Related options:
> 
>> ElasticHeapPeriodicYGCIntervalMillis, 15000 \
> (target young GC interval 15 seconds in default) \
> (eg, if Java runs with MaxNewSize=4g, young GC every 30 seconds, G1ElasticHeap will keep 15s
>   GC interval and make a max 2g young generation to uncommit 2g memory)
> 
>> ElasticHeapPeriodicInitialMarkIntervalMillis, 3600000 \
> (Target initial mark interval, 1 hour in default. Unused memory of old generation will be uncommitted
>   after last mixed GC.)

This sesm to implement an unconditional concurrent cycle like with the 
CMSTriggerInterval flag for CMS.

Maybe there is a more clever alternative on triggering concurrent cycles 
like ZGC does based on the ratio between time spent by the mutator and 
the gc.

> 
>> ElasticHeapPeriodicUncommitStartupDelay, 300 \
> (Delay after startup to do memory uncommit, 300 seconds in default)
> 
>> ElasticHeapPeriodicMinYoungCommitPercent, 50 \
> (Percentage of young generation to keep, default 50% of the young generation will not be uncommitted)

See above about separating young/old.

> 
> ### 2. Generation limit
> To limit the young/old generation separately. Use jcmd or MXBean to enable.

I do not understand the reason for those, see above.

[...]
> 
> ### 3. Softmx mode
> Dynamically to limit the heap as a percentage of origin Xmx.
> 
> Use jcmd:
> 
> `jcmd PID ElasticHeap softmx_percent=60`
> 
> Use MXBean:
> 
> `elasticHeapMXBean.setSoftmxPercent(70);`

That one sounds good, and actually there is a flag SoftMaxHeapSize 
already in the VM. Only ZGC implements it though.

I think this idea matches the specifications in 
https://bugs.openjdk.java.net/browse/JDK-8222145 (i.e. as far as I can 
tell, the softmxpercent is a "soft"/target heap size), so I think this 
could be implemented under the SoftMaxHeapSize flag.

SoftMaxHeapSize is already manageable too, so could be modified already. 
Only the implementation is missing in G1 :)

> 
> ### Other G1ElasticHeap advanced options:
>> ElasticHeapMinYoungCommitPercent, 10 \
>   (Mininum percentage of young generation)
> 
>> ElasticHeapYGCIntervalMinMillis, 5000 \
>   (Mininum young GC interval)
> 
>> ElasticHeapInitialMarkIntervalMinMillis, 60000 \
> (Mininum initial mark interval)
> 
>> ElasticHeapEagerMixedGCIntervalMillis, 15000 \
> (Guaranteed mixed GC interval, to make sure the mixed will happen in time to uncommit memory after last mixed GC)

These options seem to be mostly useful for when the allocation rate of 
the mutator is not high enough to advance the collection cycle.

Would that feature provide the requested feature? Maybe it needs some 
minor improvement, but to me it seems very burdensome to specify so many 
options...

> 
>> ElasticHeapOldGenReservePercent, 5 \
> (To keep a mininum percentage of Xmx for old generation in the uncommitment after last mixed GC)

That seems to be related to some strict separation of young/old again.

> 
>> ElasticHeapPeriodicYGCIntervalCeilingPercent, 25 \
> ElasticHeapPeriodicYGCIntervalFloorPercent, 25 \
> (The actual young GC interval will fluctuate between \
> ElasticHeapPeriodicYGCIntervalMillis * (100 - ElasticHeapPeriodicYGCIntervalFloorPercent) / 100 and \
> ElasticHeapPeriodicYGCIntervalMillis * (100 + ElasticHeapPeriodicYGCIntervalCeilingPercent) / 100 )
> 

Thanks,
   Thomas



More information about the hotspot-gc-dev mailing list