RFC: Adaptively resize heap at any GC/SoftMaxHeapSize for G1

[Ruslan Synytsky]

Hi Thomas and Liang, thank you for moving this improvement forward. A quick
question regarding the naming: did we agree on how this parameter should be
called? What happens when heap usage goes higher than SoftMaxHeapSize -
OOMError or JVM gets a little bit more memory? If JVM throws OOMError I
believe the right naming should be HardMaxHeapSize. Sorry in advance if I
missed this point in the previous conversations.

Also, some news regarding analysis automation of memory usage efficiency
I'm working on in the background. We built a relatively small script that
collects memory usage metrics from many containers running inside the same
large host machine. After executing it in one of our dev environments with
about 150 containers we got interesting results - the used heap is very
close to the committed heap while Xmx is much higher compared to committed
value. Please note, almost all containers use JEP 346 improvement or
javaagent which triggers GC at idle state in the older JDK versions.

[image: Screenshot 2020-06-18 at 13.20.19.jpg]

Zoomed

[image: Screenshot 2020-06-18 at 14.40.18.jpg]
There is a challenge to get metrics from a running java process. I as
understand the most accurate metrics can be collected via MBean, for
example

JMXConnector c =
JMXConnectorFactory.newJMXConnector(createConnectionURL(host, port),
null);
c.connect();
MBeanServerConnection mbsc = c.getMBeanServerConnection();
MemoryMXBean mem = ManagementFactory.getPlatformMXBean(mbsc,
MemoryMXBean.class);
MemoryUsage heap = mem.*getHeapMemoryUsage*();

However, enabling JMX ManagementAgent via jcmd and connecting to JVM with a
JMX client is a quite complex operation for getting such a simple metric
about heap memory usage. Also, some java processes may already
start ManagementAgent on a custom port with auth protection, so we can't
collect statistics from such java processes without contacting the
application owner (you can see the gaps on the chart). Do you know any
other way for collecting accurate heap usage statistics from a running java
process? We plan to run this analysis tool on productions with a large
number of containers, so we can get a more realistic picture.

Thanks
-- 
Ruslan Synytsky