<div dir="ltr">Hi Stefan,<br><br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>------------------------------<br>
<br>
Message: 3<br>
Date: Thu, 13 Sep 2018 14:30:37 +0200<br>
From: Stefan Johansson <<a href="mailto:stefan.johansson@oracle.com" target="_blank">stefan.johansson@oracle.com</a>><br>
To: <a href="mailto:hotspot-gc-dev@openjdk.java.net" target="_blank">hotspot-gc-dev@openjdk.java.net</a><br>
Subject: Re: RFR: bug: Timely Reducing Unused Committed Memory<br>
Message-ID: <<a href="mailto:57b10708-b870-cb98-0df0-fab650f431e4@oracle.com" target="_blank">57b10708-b870-cb98-0df0-fab650f431e4@oracle.com</a>><br>
Content-Type: text/plain; charset=utf-8; format=flowed<br>
<br>
Hi Rodrigo,<br>
<br>
Sorry for being a bit late into the discussion. We've had some internal <br>
discussions and realized that there are some questions that I need to <br>
bring up here.<br>
<br>
I'm trying to better understand under what circumstances this feature is <br>
to be used and how a user should use the different flags to tweak it to <br>
their use case. To me it feels like GCFrequency would be enough to make <br>
sure that the VM returns memory on a timely basis. And if the flag is <br>
managed, it can be controlled to not do periodic GCs during high load. <br>
With that we get a good way to periodically try to reduce the committed <br>
heap.<br>
<br>
The reason I ask is because I have a hard time seeing how we can <br>
implement a generic policy for when the system is idle. A policy that <br>
will apply well to most use cases. For some cases having the flags you <br>
propose might be good, but for other there might be a different set of <br>
options needed. If this is the case then maybe the logic and policy of <br>
when to do this can live outside the VM, while the code to periodically <br>
do GCs lives within the VM. What do you think about that? I understand <br>
the problems you've stated with having the policy outside that VM, but <br>
at least we have more information to act on there.<br>
<br></blockquote><div><br></div><div>I understand that it is hard to define what is idle. However, if we require the</div><div>user to provide one, I guess that most regular users that suffer from the problem</div><div>that this patch is trying to solve will simply not do it because it requires knowledge</div><div>and effort. If we provide an idle check that we think will benefit most users, then </div><div>we are probably helping a lot of users. For those that the default idle check is </div><div>not good enough, they can always disable this idle check and implement the idle</div><div>check logic it in an external tool.</div><div><br></div><div>We can also change the semantics of "idleness". Currently it checks the load.</div><div>I think that checking the allocation rate might be another good option (instead of</div><div>load). The only corner case is an application that does not allocate but consumes </div><div>a lot of CPU. For this case, we might only trigger compaction at most once because, </div><div>as it does not allocate memory, we will not get over committed memory (i.e., the other </div><div>checks will prevent it). The opposite is also possible (almost idle application that allocates </div><div>a lot of memory) but in this scenario I don't think we want to trigger an idle compaction.</div><div><br></div><div>Having said that, I am open to change this flag or even remove it as it is one of the</div><div>hardest to get right.</div><div><br></div><div>cheers,</div><div>rodrigo</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
We know that many have asked for features similar to this one and it <br>
would be nice to get input from others on this to make sure we implement <br>
something that benefits the whole user base as much as possible. So <br>
anyone with a use case that could benefit from this, please chime in.<br>
<br>
Regards,<br>
Stefan<br>
<br>
<br>
<br>
On 2018-09-07 17:37, Rodrigo Bruno wrote:<br>
> Hi Per and Thomas,<br>
> <br>
> thank you for your comments.<br>
> <br>
> I think it is possible to implement this feature using the service <br>
> thread or using a separate thread.<br>
> I see some pros and cons of having a separate thread:<br>
> <br>
> Pros:<br>
> - using the service thread exposes something that is G1 specific to the <br>
> rest of the JVM.<br>
> Thus, using a separate thread, hides this feature from the outsite.<br>
> <br>
> Cons:<br>
> - Having a manageable timeout is a bit more tricky to implement in a <br>
> separate/dedicated thread.<br>
> We need to be able to handle switch on and off. It might require some <br>
> variable pooling.<br>
> - It requires some more memory.<br>
> <br>
> Regardless of the path taken, I can prepare a new version of the patch <br>
> whenever we decide on this.<br>
> <br>
> cheers,<br>
> rodrigo<br>
> <br>
> Per Liden <<a href="mailto:per.liden@oracle.com" target="_blank">per.liden@oracle.com</a> <mailto:<a href="mailto:per.liden@oracle.com" target="_blank">per.liden@oracle.com</a>>> escreveu <br>
> no dia sexta, 7/09/2018 ?(s) 11:58:<br>
> <br>
> Hi Thomas,<br>
> <br>
> On 09/07/2018 10:10 AM, Thomas Schatzl wrote:<br>
> [...]<br>
> >? ? overnight I thought a bit of the implementation, and given the<br>
> > problem with heap usage of the new thread, and the requirement of<br>
> being<br>
> > able to turn on/off that feature by a managed variable, the best<br>
> change<br>
> > would probably reusing the service thread as you did in the initial<br>
> > change.<br>
> <br>
> I'm not convinced that this should be handled outside of G1. If there's<br>
> a need to have the flag manageable at runtime (is that really the<br>
> case?), you could just always start the G1DetectIdleThread and have it<br>
> check the flag. I wouldn't worry too much about the memory overhead for<br>
> the stack.<br>
> <br>
> cheers,<br>
> Per<br>
> <br>
<br>
<br>
------------------------------<br>
</blockquote></div></div>