NewRatio behaviour and some interesting PSYoungGen

[Kirk Pepperdine]

Hi all,

I was spelunking about in PSYoungGen.cpp trying to sort out how the survivor space max size was being calculated with regards to how tenuring thresholds are being calculated. Point being that in this case I was trying to sort out why my app was rolling back the tenuring threshold about 90% of the time when there should have been plenty of been plenty of memory to accommodate incoming bytes. Some details, -Xmx36g -XX:NewRatio=4. Typical log record looks like this.

47877.205: [GC
Desired survivor size 80216064 bytes, new threshold 1 (max 15)
[PSYoungGen: 401014K->3647K(477568K)] 3983447K->3588145K(4129920K), 0.0097340 secs] [Times: user=0.13 sys=0.01, real=0.01 secs]

At one point in time when there was a higher demand for memory, survivor spaces where big enough to accommodate incoming and surviving bytes. But as demand drew down adaptive sizing drew down on heap size and that in turn drew down on survivor space sizing. The problem here is full gc pause times. I'd prefer not to fix the size of young and consequently survivor but I fear that unless I can stop the VM from undersizing survivor I will need to. The reason the heap is (too) big is that we are battling another issue to do with serialization (inadvertent construction of 1 gig byte arrays where only ~35K is ever used) and until that problem is solved the workaround is an excessively large heap with parallel collectors. In theory this combination should allow us to over size the heap and not take the penalty when things are working normally. In practice the premature promotion problem is causing too frequent full gcs with pause times that are longer than they should be. Scavenge pauses are acceptably short so no problems there.

Question is; is there is a way to give adaptive sizing a hint that survivor spaces need to larger while still allowing it to draw back on eden's capacity?


One other thing that I found interesting is that when using ParNew with CMS in combination with NewRatio. What I've seen is that when the VM is loaded in low ram Heap will resize young gen. However, for some unknown reason, the resize never happens when the VM is loaded higher up in memory. I first noticed this while running a benchmark at a conference. The demo worked as expected prior to my talk, failed during the talk, and then worked after the talk. The difference in the runs was that during the talk I had keynote and Chrome and a PDF reader and a few other things running. They were idle but they were (of course) consuming quite a bit of memory. After a little bit of experimentation I noted that the adaptivity of the JVM is some affected by where it lies in memory. In my case it wasn't swapping but maybe being higher ram causes enough extra work with the pointers that you end up with this side effect????? Pure speculation but would be interested in any thoughts.


One last thing, while spelunking through the code I ran into this.


// This method currently does not expect to expand into eden (i.e.,
// the virtual space boundaries is expected to be consistent
// with the eden boundaries..
void PSYoungGen::post_resize() {
  assert_locked_or_safepoint(Heap_lock);
  assert((eden_space()->bottom() < to_space()->bottom()) &&
         (eden_space()->bottom() < from_space()->bottom()),
         "Eden is assumed to be below the survivor spaces");

  MemRegion cmr((HeapWord*)virtual_space()->low(),
                (HeapWord*)virtual_space()->high());
  Universe::heap()->barrier_set()->resize_covered_region(cmr);
  space_invariants();
}

The assertion, as I understand it, is asking if the bottom of to_space and from_space are both higher up in memory than the bottom of eden. Wouldn't you want to make sure that to or from is higher in memory than the top of eden?

Regards,
Kirk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20120831/aed626ff/attachment.htm>