CMS cycles triggered by Perm almost being full

[Mikael Gerdin]

Kirk,

On 10/02/2013 04:38 AM, Kirk Pepperdine wrote:
> Hi,
>
> I've just witnessed in 1.7.0_17-b02 (Solaris AMD) CMS cycles being
> initiated every 7.390 seconds. The system is idle and there are no
> foreground (ParNew) collections running. Perm occupancy is quite
> close to it's configured size so it's quite likely that the cause of
> the CMS cycle is this. However, Class unloading is not enabled and
> thus the CMS cycle doesn't "fix" the trigger by cleaning out perm or
> being able to enlarge it (configured size < max size) and there isn't
> any pressure for a Full collection (CMF??). Triggering a collection
> (System.gc()) of course "fixes" the problem (facilitates a perm space
> expansion).

When you do a System.gc() the perm generation will be resized to satisfy 
MinHeapFreeRatio, so the fact that the System.gc() causes perm gen to 
expand may not necessarily indicate that the cause for the CMS cycles 
was perm gen related.

>
> Ok, so there are work arounds for this but it really confused the
> person who contacted me with the problem and he's no slouch when it
> comes to GC. I've advised him to turn on perm space sweeping with
> class unloading. That said, it seems that CMS should know that it's
> not going to be able to fix the problem that triggered to to run and
> it should degrade into a CMF, reason perm space needs resizing. My
> questions are, have I missed something? Should this be filed as a
> bug? Or, is this as intended?

 From my understanding of  CMSCollector::shouldConcurrentCollect() it 
should only consider perm gen occupancy if +CMSClassUnloadingEnabled:

   if (CMSClassUnloadingEnabled && _permGen->should_concurrent_collect()) {
     bool res = update_should_unload_classes();
     if (res) {
       if (Verbose && PrintGCDetails) {
         gclog_or_tty->print_cr("CMS perm gen initiated");
       }
       return true;
     }
   }

Unfortunately, most of the logging from shouldConcurrentCollect seems to 
be guarded by the dreaded +Verbose flag (which is only available in 
*debug builds and enables all sorts of useless logging messages)

There seems to be some information that you can get out by setting 
+PrintCMSInitiationStatistics.
It appears that the CMSStats records the periodicity of ParNew 
collections and their expected promotion rate and tries to trigger CMS 
cycles proactively based on that data.
I'm not sure how well that code scales back if an application suddenly 
goes completely idle.

>
> On a side note I found the 7.390 second period to be an interesting
> distraction.

I think this could be related to the periodicity of earlier ParNew 
collections.

/Mikael

>
> Regards, Kirk
>