NewRatio behaviour and some interesting PSYoungGen

[Kirk Pepperdine]

On 2012-09-01, at 11:08 PM, Srinivas Ramakrishna <ysr1729 at gmail.com> wrote:

> 
> Hi Kirk -- Interesting questions... I will try and answer (albeit only partially) the last two questions  in your email, letting others address the other questions (as well as add to the following as appropriate).
> 
> On Fri, Aug 31, 2012 at 1:49 AM, Kirk Pepperdine <kirk at kodewerk.com> wrote:
> ... 
> 
> One other thing that I found interesting is that when using ParNew with CMS in combination with NewRatio. What I've seen is that when the VM is loaded in low ram Heap will resize young gen. However, for some unknown reason, the resize never happens when the VM is loaded higher up in memory. I first noticed this while running a benchmark at a conference. The demo worked as expected prior to my talk, failed during the talk, and then worked after the talk. The difference in the runs was that during the talk I had keynote and Chrome and a PDF reader and a few other things running. They were idle but they were (of course) consuming quite a bit of memory. After a little bit of experimentation I noted that the adaptivity of the JVM is some affected by where it lies in memory. In my case it wasn't swapping but maybe being higher ram causes enough extra work with the pointers that you end up with this side effect????? Pure speculation but would be interested in any thoughts.
> 
> I don't think that would be the case.
> 
> If you can recreate the conditions and run with a debug build you'd probably get more info. My guess is that you are short on swap as might happen when running with more active processes, you are unable to reserve swap for committing the expansion of the heap. So the expansion fails and you are unable to resize. The debug build of the JVM, from what i recall, issues a message indicating its inability to expand the heap. You might then try configuring more swap (and possibly RAM to avoid actually swapping) and see if the resize then happens with all of the programs in memory. What OS was this on?

This was running on OSX Snow Leopard, Lion and Mountain Lion. It's pretty consistent across all of the 6.0 and 7.0 versions of the JVM I've run.. and I'm intrigued by your explanation but I've been able to run VMs with much much larger heaps with lots of other things running. My heap setting (sorry I should have said this in the original email) -Xmx1g -XX:SurvivorRatio=1 -XX:NewRatio=1. I'll try running the bench using debug, with Linux and see how well it sticks.


> 
> 
> 
> One last thing, while spelunking through the code I ran into this.
> 
> 
> // This method currently does not expect to expand into eden (i.e.,
> // the virtual space boundaries is expected to be consistent
> // with the eden boundaries..
> void PSYoungGen::post_resize() {
>   assert_locked_or_safepoint(Heap_lock);
>   assert((eden_space()->bottom() < to_space()->bottom()) &&
>          (eden_space()->bottom() < from_space()->bottom()),
>          "Eden is assumed to be below the survivor spaces");
> 
>   MemRegion cmr((HeapWord*)virtual_space()->low(),
>                 (HeapWord*)virtual_space()->high());
>   Universe::heap()->barrier_set()->resize_covered_region(cmr);
>   space_invariants();
> }
> 
> The assertion, as I understand it, is asking if the bottom of to_space and from_space are both higher up in memory than the bottom of eden. Wouldn't you want to make sure that to or from is higher in memory than the top of eden?
> 
> You are right, but in fact the assert can and should be even stronger than even that, namely that the _end_ of  Eden (which is higher than its _top_ which is equal to its _bottom_, if Eden is empty, which it usually is at the end of a scavenge) is lower (smaller virtual address) than the _bottom_ of  the survivor spaces. Recall that top is the where new allocation occurs, and end is the absolute end of the space that new allocation cannot cross.

I've sloppily said top of eden when I should have said _end_of_Eden.
 
> 
> PS: Disclaimer: this is from memory, i haven't looked at the code in a while, just remembering the naming conventions used in the code as i remember it.

I think you've done quite well.

-- Kirk

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20120902/d5e58dd3/attachment.htm>