RFR: bug: Timely Reducing Unused Committed Memory

[Stefan Johansson]

On 2018-09-29 09:26, Ruslan Synytsky wrote:
> 
>     Another question, when
>     running in the cloud, what load is the user expecting us to compare
>     against, the overall system or the local container. I'm actually not
>     entirely sure what the getloadavg() call return in case of running in a
>     container.
> 
> Good question! It depends on the used container technology. In short, if 
> it’s a system container then it shows the load of the container, if it’s 
> an application container then the load of the host machine. There is an 
> article on a related topic 
> https://jelastic.com/blog/java-and-memory-limits-in-containers-lxc-docker-and-openvz/
> 
> Can we measure CPU usage of JVM process itself and use it for decisions?
> 
There is code currently only used by JFR that reports the CPU usage for 
the JVM process. So I guess this is another metric that also could be 
used to define idleness.

> 
> 
>      >
>      >> > Another thing that I don't fully
>      >> > understand is why the flags are manageable if there isn't
>     supposed
>      >> to be
>      >> > some external logic that sets them?
>      > Some advanced users, for example cloud platform or software vendors,
>      > will be able to apply an additional logic based on their custom
>     needs /
>      > specifics. Such flexibility enables more use cases and it helps to
>      > collect more feedback for the further default improvements.
> 
>     That's how I would expect it to be used as well, thanks for clarifying
>     your viewpoint.
> 
>      >>
>      >> >> We can also change the semantics of "idleness".  Currently it
>      >> checks the load.
>      >> >> I think that checking the allocation rate might be another good
>      >> option (instead of
>      >> >> load). The only corner case is  an application that does not
>      >> allocate but consumes
>      >> >> a lot of CPU. For this case, we might only trigger compaction at
>      >> most once because,
>      >> >> as it does not allocate memory, we will not get over committed
>      >> memory (i.e., the other
>      >> >> checks will prevent it). The opposite is also possible
>     (almost idle
>      >> application that allocates
>      >> >> a lot of memory) but in this scenario I don't think we want to
>      >> trigger an idle compaction.
>      >> >>
>      >>
>      >> > This is my main problem when it comes to determine "idleness",
>     for some
>      >> > applications allocation rate will be the correct metric, for
>     others it
>      >> > will be the load and for a third something different. It feels
>     like it
>      >> > is always possible to come up with a case that needs something
>      >> different.
>      > I would prefer to start with the most obvious one - based on CPU,
>     give
>      > it to try to more people by promoting the fact that JVM is
>     elastic now,
>      > and we will get more feedback that can be converted into an
>     additional
>      > logic later.
>      >
>     So basically, the first version would have two flags, one to turn on
>     periodic GCs (currently named GCFrequency) and one to control at which
>     average load (MaxLoadGC) these GCs will kick in?
> 
> I think it’s a good starting point.
> 
> 

Nice to hear.

We discussed this a bit further today and I'll try to summarize what we 
currently think is the way forward and what a first version should include:
* Periodic concurrent cycles (which after JDK-6490394 will uncommit 
memory) are on by default, but will only trigger if no other GC has 
occurred during the interval. The length of the interval should be 
controlled by a flag and if set to -1, the periodic GCs will be turned 
off. The name should probably be something other than GCFrequency, like 
PeriodicGCInterval or so. This will be similar to what currently is 
called CMSTriggerInterval for CMS.
* You can also control the periodic GCs by specifying a system average 
load threshold, which the current avg load must be under for a periodic 
GC to occur. This value will not be consider by default, and I think we 
should try to come up with at more describing name than MaxLoadGC.
* The code checking if a periodic GC should trigger will be added to the 
already existing G1YoungRemSetSamplingThread. This thread currently is 
pretty idle and there are ideas around removing it, but if that is done 
we can move the idle-checking code then.
* If you want the periodic GC to be a full collection, there will be a 
flag to control this.

Those were the key parts in our discussion, and I think Thomas plans to 
update his webrev in a few days or so. Does this still sound like a good 
starting point?

And also, other users having opinions are very welcome :)

Thanks,
Stefan
> 
> -- 
> Ruslan
> CEO @ Jelastic