<div dir="ltr">If it were fragmentation only, then one wouldn't see free memory increase that dramatically after the compaction. All that the compaction should do is compact the objects while leaving free space in roughly the same ball park as before. Not the huge reduction from used memory of 1.92 GB to 50.7 MB. (Again, I am just reading yr email not paying attention to the logs.)<div>
<br></div><div>I'd suggest turning on class unloading and see how far you are able to get. Fragmentation might still get you, but the signal in that case would be clearer. Also, I think CMS has an option</div><div>that prints the cause of a collection and similarly for a concurrent mode failure (IIRC).</div>
<div>-- ramki</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Jul 14, 2014 at 6:31 PM, Elliot Barlas <span dir="ltr"><<a href="mailto:Elliot.Barlas@citrix.com" target="_blank">Elliot.Barlas@citrix.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">
<div style="direction:ltr;font-family:Tahoma;color:#000000;font-size:10pt">
<div>Alternatively, could my problem be due to fragmentation? Are there segments of free address space that are too small to use that could eventually be responsible for exhausting the old generation?</div>
<div><br>
</div>
<div>-Elliot</div>
<br>
<div style="font-family:Times New Roman;color:#000000;font-size:16px">
<hr>
<div style="direction:ltr"><font face="Tahoma" color="#000000"><b>From:</b> Kirk Pepperdine [<a href="mailto:kirk@kodewerk.com" target="_blank">kirk@kodewerk.com</a>]<br>
<b>Sent:</b> Wednesday, July 09, 2014 1:24 AM<div class=""><br>
<b>To:</b> Srinivas Ramakrishna<br>
<b>Cc:</b> Elliot Barlas; <a href="mailto:hotspot-gc-dev@openjdk.java.net" target="_blank">hotspot-gc-dev@openjdk.java.net</a> <a href="http://openjdk.java.net" target="_blank">openjdk.java.net</a><br>
<b>Subject:</b> Re: Continuous CMS Collections Followed By Concurrent Mode Failure<br>
</div></font><br>
</div><div><div class="h5">
<div></div>
<div>Hi Ramki,
<div><br>
</div>
<div>No need for flame proof suites… The (promotion failed) has been separated from the ParNew. I can easily adjust for this change… it’s just one more in a line of 100s of trivial changes that don’t add value to the GC logs… <sighs> ;-)</div>
<div><br>
</div>
<div>— Kirk</div>
<div><br>
<div>
<div>On Jul 9, 2014, at 3:45 AM, Srinivas Ramakrishna <<a href="mailto:ysr1729@gmail.com" target="_blank">ysr1729@gmail.com</a>> wrote:</div>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Jul 8, 2014 at 12:06 AM, Kirk Pepperdine <span dir="ltr">
<<a href="mailto:kirk@kodewerk.com" target="_blank">kirk@kodewerk.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">
<div>...</div>
<div>Ramki, should we file a bug for the ParNew (promotion failed) being corrupted by a CMS cycle?<br>
<div><br>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>Can you elaborate what precisely you meant by "corrupted (formatting)" below?</div>
<div><br>
</div>
<div>Perhaps you want what seems to be a missing newline or something? You should check with Jon or one of the HotSpot/GC folks on whether and how easy it would be to fix, or otherwise modify your parser. (Kevlar suit donned, for yr response ;-)</div>
<div><br>
</div>
<div>-- ramki<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">
<div></div>
<div>
<div>
<div>
<blockquote type="cite">
<div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<div style="direction:ltr;font-family:Tahoma;font-size:10pt">
<div style="font-family:'Times New Roman';font-size:16px">
<div><br>
</div>
<div>That was actually my guess. There is only 1 perm record in the file and although it shows that perm is grossly over-sized (ok, there is only one record ;-)) all of the recovery comes from the CMF which suggests perm is involved. All of the other CMS cycles
are clearly due to tenured never being below the initiating occupancy fraction. Even without a young collection the initial marks are constantly reporting an occupancy of >1920xxxK of 1929xxxK. Oddly enough the ParNew’s only once promoted enough to trip the
concurrent mode failure and oddly enough each of the CMS cycles (without the intervening ParNew) seem to recover about 1xxK bytes per cycle.</div>
<div><br>
</div>
<div>
<div>2014-06-10T22:56:18.793-0700: 4999527.565: [GC [1 CMS-initial-mark: 1920286K(1926784K)] 2051254K(2080128K), 0.3388330 secs] [Times: user=0.33 sys=0.00, real=0.33 secs] </div>
<div>…...</div>
<div>2014-06-10T22:56:26.242-0700: 4999535.014: [GC2014-06-10T22:56:26.242-0700: 4999535.014: [ParNew2014-06-10T22:56:26.256-0700: 4999535.028: [CMS-concurrent-abortable-preclean: 1.948/3.114 secs] [Times: user=1.93 sys=0.12, real=3.11 secs] </div>
<div> (promotion failed)</div>
<div>Desired survivor size 8716288 bytes, new threshold 6 (max 6)</div>
<div>- age 1: 1036320 bytes, 1036320 total</div>
<div>- age 2: 825248 bytes, 1861568 total</div>
<div>- age 3: 119024 bytes, 1980592 total</div>
<div>- age 4: 113784 bytes, 2094376 total</div>
<div>- age 5: 129024 bytes, 2223400 total</div>
<div>- age 6: 154976 bytes, 2378376 total</div>
<div>: 141769K->140729K(153344K), 0.3807730 secs]2014-06-10T22:56:26.623-0700: 4999535.395: [CMS</div>
<div> (concurrent mode failure): 1920816K->50773K(1926784K), 28.3938140 secs] 2062055K->50773K(2080128K), [CMS Perm : 48657K->41071K(262144K)], 28.7750370 secs] [Times: user=1.65 sys=0.03, real=28.78 secs] </div>
<div>
<div>2014-06-10T22:58:41.361-0700: 4999670.133: [GC2014-06-10T22:58:41.361-0700: 4999670.133: [ParNew</div>
<div>Desired survivor size 8716288 bytes, new threshold 6 (max 6)</div>
<div>- age 1: 1239232 bytes, 1239232 total</div>
<div>: 136320K->2239K(153344K), 0.0223680 secs] 187093K->53012K(2080128K), 0.0226340 secs] [Times: user=0.05 sys=0.00, real=0.02 secs] </div>
</div>
<div><br>
</div>
<div>Note the corrupted formatting of the ParNew (promotion failed).</div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div></div></div>
</div>
</div>
</blockquote></div><br></div>