RFC: Adaptively resize heap at any GC/SoftMaxHeapSize for G1

[Ruslan Synytsky]

>
>
> > question regarding the naming: did we agree on how this parameter should
> be
> > called? What happens when heap usage goes higher than SoftMaxHeapSize -
> > OOMError or JVM gets a little bit more memory? If JVM throws OOMError I
> > believe the right naming should be HardMaxHeapSize. Sorry in advance if I
> > missed this point in the previous conversations.
>
> SoftMaxHeapSize is not what you describe here - SoftMaxHeapSize is only
> an internal goal for heap sizing without guarantees. Hence the name
> *Soft*MaxHeapSize.
>
> See https://bugs.openjdk.java.net/browse/JDK-8222145.
>
> There has been no progress on anything like Current/HardMaxHeapSize.
>
Thomas and Liang, is there a possibility to easily add an optional
manageable parameter that regulates behaviour of JVM when memory
consumption goes above SoftMaxHeapSize? For example, by default JVM
allocates more memory when it can't keep memory usage
below SoftMaxHeapSize, and if the optional parameter was specified then JVM
throws OutOfMemoryError. In this case we will cover two cases with the same
code base.


>
> >
> > Also, some news regarding analysis automation of memory usage efficiency
> > I'm working on in the background. We built a relatively small script that
> > collects memory usage metrics from many containers running inside the
> same
> > large host machine. After executing it in one of our dev environments
> with
> > about 150 containers we got interesting results - the used heap is very
> > close to the committed heap while Xmx is much higher compared to
> committed
> > value. Please note, almost all containers use JEP 346 improvement or
> > javaagent which triggers GC at idle state in the older JDK versions.
> >
> > [image: Screenshot 2020-06-18 at 13.20.19.jpg] >
> > Zoomed
> >
> > [image: Screenshot 2020-06-18 at 14.40.18.jpg]
>
> While the screenshots have been scrubbed by the mailing list it's very
> nice to hear that you have had success with these approaches.
>
I plan to share more details on this soon with a link to dynamic charts for
a more convenient analysis.


> > However, enabling JMX ManagementAgent via jcmd and connecting to JVM
> with a
> > JMX client is a quite complex operation for getting such a simple metric
> > about heap memory usage. Also, some java processes may already
> > start ManagementAgent on a custom port with auth protection, so we can't
> > collect statistics from such java processes without contacting the
> > application owner (you can see the gaps on the chart). Do you know any
> > other way for collecting accurate heap usage statistics from a running
> java
> > process? We plan to run this analysis tool on productions with a large
> > number of containers, so we can get a more realistic picture.
> >
>
> Jcmd with the GC.heap_info command provides some information, probably
> not enough (I filed JDK-8248136) though. More information can be
> retrieved with the "VM.info" command, the detailed per-region printout
> which might be too much information.
>
> There is also JFR with its event streaming API that could be an option,
> however it is JDK14 only (https://openjdk.java.net/jeps/349).
>
> Finally, there is jstat to gather some information.
>
Unfortunately GC.heap_info and VM.info do not provide information about
COMMITTED heap. And jstat documentation
<https://docs.oracle.com/en/java/javase/11/tools/jstat.html> does not
mention options for collecting committed heap as well. Analyzing used and
max without understanding committed heap is useless in this context as the
"lost memory" is located between used and committed.

Thanks
Ruslan