NewRatio: to twiddle or not to twiddle

[kirk]

Hi Charles,

Peter, awesome explination. I use hpjmeter right now to do very gross 
analysis on gc logs. It doesn't tolerate all the Sun non-standard 
non-standard switches (-XX and as apposed to the standard non-standard 
switches -X) ;-) I've also signed up with Tony to work in gchisto, a 
tool that should be more tolerant to Sun gc logs (is there anyway we can 
settle on a standard format???) though that tool is still very limited. 
Eventually it should get rolled into visualvm but that is a future.

What I'm finding is that techniques that focus more on evacuation seem 
to work better than those that focus on collection compaction. This is 
one of the reasons I'm very interested in seeing G1 released into the 
wild. I believe that G1 will mix very well with dynamic languages 
because it primary logic is on evacuation. With G1, I don't believe it 
will matter if old fills up with transient objects that maybe should 
have died in young. In fact I'd bet money that G1 will excel under these 
conditions because old spaces will be evacuated based on who has the 
lowest live ratios. The biggest draw back that I can see will be copying 
costs. Even so, expect a win.

>
> -Xmn is a good one I had forgotten about, and answers my question 
> about how to increase NewRatio beyond 1.
use a combo of -Xmx and NewSize. That should pin things down.

- Kirk

> I guess there's really two tasks I'm looking into right now:
>
> 1. Discovering appropriate flags to tweak and "more appropriate" 
> defaults dynlangs might want to try
> 2. Exploring real-world dynlang applications and loads to refine those 
> better/best-guess defaults
>
> I'd say this round of emails is focused mostly on 1, since 2 is going 
> to vary more across languages. And I think we can only start to 
> explore 2 iff we know 1. So -Xmn goes on the list alongside NewRatio.
>
> Perhaps a brief illustration of the adaptive capabilities of the other 
> collectors would be useful here? Obviously for server applications 
> we'd hope HotSpot "does the right thing" for us already. Then perhaps 
> we can learn from that (and please, teach me how to learn, including 
> better tools and switches I can use to get the necessary GC metrics) 
> and apply what we've learned to command-line/client runs (which will 
> be more transient but not asimilar to the server runs in a language 
> like Ruby).
>
> - Charlie
>
> Peter B. Kessler wrote:
>> Why limit yourself to NewRatio?  The best you can get that way is 
>> half the heap for the young generation.  If you really want to a big 
>> young generation (to give your temporary objects time to die without 
>> even being looked at by the collector), use -Xmn (or -XX:NewSize= and 
>> -XX:MaxNewSize=) to set it directly.  Figure out what your live data 
>> size is and use that as the base size for the old generation.  Then 
>> figure out what kinds of pauses the young generation collections 
>> impose, and how much they promote, then amortize the eventual old 
>> generation collection time over as many young generation collections 
>> as you can give space to in the old generation.  Then make your total 
>> heap (-Xmx) as big as you can afford to get as big a young generation 
>> as that will allow.
>>
>>             ... peter
>>
>> Charles Oliver Nutter wrote:
>>> I've ben playing with JRuby on various benchmarks recently and found 
>>> that several object-intensive scripts run better if I set 
>>> NewRatio=1. Ruby, even more than Java, tends to generate lots and 
>>> lots of objects, especially considering that there's no unboxed 
>>> primitive numeric types (and no fixnums on the JVM...ahem ahem). So 
>>> my general theory is that:
>>>
>>> 1. A NewRatio of 1 allows all those extra transient objects to get 
>>> collected more quickly.
>>> 2. Too small a "new" generation causes transient objects to get 
>>> shoved into older generations, potentially snowballing and forcing 
>>> more comprehensive GC runs as time goes on.
>>>
>>> I'm curious whether this theory sounds reasonable, whether there's a 
>>> better way I can adapt hotspot to the memory demands of a dynamic 
>>> language like Ruby, and what other implications there are in setting 
>>> NewRatio to 1.
>>>
>>> Thoughts?
>>>
>>> (And please let me know if there's a better list to post this sort 
>>> of question to)
>>>
>>> - Charlie
>>
>
>

_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use