Jon,<br><br>Thanks for your comments.<br><br><div class="gmail_quote">On Tue, Mar 16, 2010 at 8:04 PM, Jon Masamitsu <span dir="ltr"><<a href="mailto:Jon.Masamitsu@sun.com">Jon.Masamitsu@sun.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
I had forgotten the other half of this which was the behavior<br>
on System.gc(). I agree that it would be nice to have a way<br>
for the user to collapse the heap quickly if the user knows that<br>
the application is going to be inactive. The code you suggested<br>
only shrinks the heap. I think it would be more complete if it<br>
would also grow the heap according to MinHeapFreeRatio.<br></blockquote><div><br>In practice, the expansion side isn't a problem in my experience. i.e., it will expand eventually. Also, in the way the patch is written currently, the expansion is taken care of by PSAdaptiveSizePolicy which I think does a good job. That said, I agree that it'd be more complete if the MinHeapFreeRatio part is implemented from the point of view of seeing MinHeapFreeRatio and MaxHeapFreeRatio as a inseparable pair.<br>
<br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
I don't know the scenario where it would be used but I<br>
wouldn't be surprised if there is one. It would also make it<br>
practical to run UseParallelGC with UseAdaptiveSizePolicy<br>
turned off (the heap resizing would then be done similarly<br>
to the other collectors).<br></blockquote><div><br>One scenario that I have for system.gc() as a hint for shrinkage is where you have a daemon-like (background) process, running on your desktop/workstation, that performs occasional batch type jobs. After each job is done, it is highly likely that the process will be idle for a while until a next job comes. It isn't the best if the idle background process is occupying a large amount of (resident) memory. Another would be, if there is some server that does a batch type of job (so, latency isn't a priority) that could be idle for an extended period of time and there are other processes on the same machine that could benefit from more free memory.<br>
<br>Do you mean that if one runs with -XX:+UseParallelGC and -XX:-UseAdaptiveSizePolicy, the free ratio flags are honored by the parallel collector or the heap shrinks in practice some other ways?<br><br>Thanks,<br>Hiroshi<br>
<br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
Jon<br>
<font color="#888888"><br>
Jon Masamitsu wrote On 03/16/10 15:38,:<br>
</font><div><div></div><div class="h5"><br>
>Hiroshi,<br>
><br>
>The goal of UseAdaptiveSizePolicy with regard to performance is<br>
>to size the generations so as to achieve the throughput goal.<br>
>GC cost is reduced by reducing the frequency of collections<br>
>and that is achieved by more free space in the generations<br>
>(more space for more allocations before the next GC).<br>
><br>
>I can see how a user might want to limit the footprint. Without<br>
>your change I would tell the user to reduce the throughput goal.<br>
>Not a particularly friendly interface to limit footprint I'll<br>
>admit.<br>
><br>
>There is a method<br>
><br>
>PSAdaptiveSizePolicy::adjust_for_throughput()<br>
><br>
>which is called to calculate the amount that the heap<br>
>should grow in order to achieve a throughput goal.<br>
>If that method said not to increase the heap based<br>
>on the value of MaxHeapFreeRatio is that close to<br>
>what you want? It's not the same but it would be<br>
>a way of limiting the growth of the heap (stops<br>
>growing the heap when the free space after the collection<br>
>exceeds a threshold). I think it would fit in more<br>
>easily with the rest of UseAdativeSizePolicy. I think<br>
>it would be easier to explain too.<br>
><br>
>Jon<br>
><br>
>On 03/15/10 16:50, Hiroshi Yamauchi wrote:<br>
><br>
><br>
>>On Mon, Mar 15, 2010 at 12:03 PM, Jon Masamitsu <<a href="mailto:Jon.Masamitsu@sun.com">Jon.Masamitsu@sun.com</a><br>
>><mailto:<a href="mailto:Jon.Masamitsu@sun.com">Jon.Masamitsu@sun.com</a>>> wrote:<br>
>><br>
>> Hiroshi,<br>
>><br>
>> I'm looking at the code in psParallelCompact.cpp<br>
>><br>
>> 2076 bool free_ratio_in_effect = false;<br>
>> 2077 if ((UseFreeRatioForParallelGC ||<br>
>> 2078 (UseFreeRatioOnlyInSystemGCForParallelGC &&<br>
>> 2079 gc_cause == GCCause::_java_lang_system_gc))) {<br>
>> 2080 ParallelScavengeHeap* heap =<br>
>> (ParallelScavengeHeap*)Universe::heap();<br>
>> 2081 free_ratio_in_effect =<br>
>> heap->try_to_shrink_by_free_ratio(true);<br>
>> 2082 }<br>
>> 2083<br>
>> 2084 if (!free_ratio_in_effect && UseAdaptiveSizePolicy) {<br>
>><br>
>> If only UseFreeRatioForParallelGC is set to true, is it correct<br>
>> that the heap<br>
>><br>
>> a) will grow according to the original UseAdaptiveSizePolicy<br>
>> policy until MaxHeapFreeRatio is exceeded (i.e., until<br>
>> try_to_shrink_by_free_ratio() returns true)<br>
>> b) will then shrink until MaxHeapFreeRatio is no longer exceeded<br>
>> (likely just that once)<br>
>><br>
>> then start with a) again?<br>
>><br>
>><br>
>>I think that mostly matches my intention. The cyclical pattern<br>
>>(a->b->a->b->...) is assumed not to happen more often than necessary<br>
>>though. An expectation is that the value of MaxHeapFreeRatio is high<br>
>>enough (70 by default) that the b) only happens when the app is under<br>
>>reasonably light(er) load (or just idle.)<br>
>>UseFreeRatioOnlyInSystemGCForParallelGC (instead of<br>
>>UseFreeRatioForParallelGC ) is for apps that call System.gc() to give a<br>
>>hint about the need to have its heap shrunk and want to avoid heap<br>
>>shrinking otherwise.<br>
>><br>
>>An intuition is the following. To reduce the memory footprint, the free<br>
>>ratio logic holds back the heap expansion if the original adaptive size<br>
>>policy expands too aggressively, or overrides and shrink the heap if the<br>
>>original adaptive size policy leaves too much free space, in terms of<br>
>>the value of MaxHeapFreeRatio. I think it's not far from what flag<br>
>>MaxHeapFreeRatio originally meant. Otherwise, the heap size depends on<br>
>>the original adaptive size policy (for performance.) It's a tradeoff<br>
>>between performance and footprint.<br>
>><br>
>><br>
>> What type of application fits with such a combination of policies?<br>
>> Do you have a plots of how the heap sizing behaves (i.e., size of the<br>
>> generations vs. collections/time)?<br>
>><br>
>><br>
>>The reason to combine the two logics (size policies) isn't for<br>
>>particular applications. I don't get too surprised if you say it should<br>
>>be a separate size policy and leave the original adaptive size policy<br>
>>alone. My thinking is that users may want the same level of performance<br>
>>from the parallel collector (the adaptive size policy is enabled by<br>
>>default) unless a large part of the heap (> 70% by default) is free.<br>
>>This is my attempt to get the best of both. The flags are off by default<br>
>>since of course, not all applications need to have their heap shrunk<br>
>>this way.<br>
>><br>
>>No, I don't have plots.<br>
>><br>
>>Hiroshi<br>
>><br>
>><br>
>><br>
>> Jon<br>
>><br>
>> Hiroshi Yamauchi wrote On 03/10/10 15:17,:<br>
>><br>
>> > Hi,<br>
>> ><br>
>> > I'd like to have this patch<br>
>> ><br>
>> > <a href="http://cr.openjdk.java.net/%7Erasbold/69XXXXX/webrev.00/" target="_blank">http://cr.openjdk.java.net/~rasbold/69XXXXX/webrev.00/</a><br>
>> <<a href="http://cr.openjdk.java.net/%7Erasbold/69XXXXX/webrev.00/" target="_blank">http://cr.openjdk.java.net/%7Erasbold/69XXXXX/webrev.00/</a>><br>
>> > <<a href="http://cr.openjdk.java.net/%7Erasbold/69XXXXX/webrev.00/" target="_blank">http://cr.openjdk.java.net/%7Erasbold/69XXXXX/webrev.00/</a>><br>
>> ><br>
>> > contributed, if appropriate.<br>
>> ><br>
>> > It implements a simple heap shrinking logic based on the free ratio<br>
>> > (MaxHeapFreeRatio) in the parallel collector. The way it works is the<br>
>> > following: If the free ratio, (1.0 - <used> / <capacity>) * 100 ><br>
>> > MaxHeapFreeRatio, this logic kicks in. Otherwise, the (default)<br>
>> > adaptive size policy runs the show (as before). This feature is<br>
>> turned<br>
>> > off by default. There are two new flags to turn it on:<br>
>> ><br>
>> > UseFreeRatioForParallelGC - this enables it.<br>
>> > UseFreeRatioOnlyInSystemGCForParallelGC - this enables it only in<br>
>> > explicit System.gc() calls. This is more useful in apps that call<br>
>> > System.gc() when it's idle and don't want shrinking to happen at all<br>
>> > other times.<br>
>> ><br>
>> > In our tests, it appears to work well in a simple test and to help a<br>
>> > real app reduce its footprint (RSS) when it's idle.<br>
>> ><br>
>> > Thanks to Chuck Rasbold who uploaded the webrev on my behalf.<br>
>> ><br>
>> > Thanks,<br>
>> > Hiroshi<br>
>> ><br>
>><br>
>><br>
>><br>
>><br>
<br>
</div></div></blockquote></div><br>