RFR: bug: Timely Reducing Unused Committed Memory

[Stefan Johansson]

Hi Rodrigo,

Sorry for being a bit late into the discussion. We've had some internal 
discussions and realized that there are some questions that I need to 
bring up here.

I'm trying to better understand under what circumstances this feature is 
to be used and how a user should use the different flags to tweak it to 
their use case. To me it feels like GCFrequency would be enough to make 
sure that the VM returns memory on a timely basis. And if the flag is 
managed, it can be controlled to not do periodic GCs during high load. 
With that we get a good way to periodically try to reduce the committed 
heap.

The reason I ask is because I have a hard time seeing how we can 
implement a generic policy for when the system is idle. A policy that 
will apply well to most use cases. For some cases having the flags you 
propose might be good, but for other there might be a different set of 
options needed. If this is the case then maybe the logic and policy of 
when to do this can live outside the VM, while the code to periodically 
do GCs lives within the VM. What do you think about that? I understand 
the problems you've stated with having the policy outside that VM, but 
at least we have more information to act on there.

We know that many have asked for features similar to this one and it 
would be nice to get input from others on this to make sure we implement 
something that benefits the whole user base as much as possible. So 
anyone with a use case that could benefit from this, please chime in.

Regards,
Stefan



On 2018-09-07 17:37, Rodrigo Bruno wrote:
> Hi Per and Thomas,
> 
> thank you for your comments.
> 
> I think it is possible to implement this feature using the service 
> thread or using a separate thread.
> I see some pros and cons of having a separate thread:
> 
> Pros:
> - using the service thread exposes something that is G1 specific to the 
> rest of the JVM.
> Thus, using a separate thread, hides this feature from the outsite.
> 
> Cons:
> - Having a manageable timeout is a bit more tricky to implement in a 
> separate/dedicated thread.
> We need to be able to handle switch on and off. It might require some 
> variable pooling.
> - It requires some more memory.
> 
> Regardless of the path taken, I can prepare a new version of the patch 
> whenever we decide on this.
> 
> cheers,
> rodrigo
> 
> Per Liden <per.liden at oracle.com <mailto:per.liden at oracle.com>> escreveu 
> no dia sexta, 7/09/2018 à(s) 11:58:
> 
>     Hi Thomas,
> 
>     On 09/07/2018 10:10 AM, Thomas Schatzl wrote:
>     [...]
>      >    overnight I thought a bit of the implementation, and given the
>      > problem with heap usage of the new thread, and the requirement of
>     being
>      > able to turn on/off that feature by a managed variable, the best
>     change
>      > would probably reusing the service thread as you did in the initial
>      > change.
> 
>     I'm not convinced that this should be handled outside of G1. If there's
>     a need to have the flag manageable at runtime (is that really the
>     case?), you could just always start the G1DetectIdleThread and have it
>     check the flag. I wouldn't worry too much about the memory overhead for
>     the stack.
> 
>     cheers,
>     Per
>