How to alert for heap fragmentation

[Todd Lipcon]

Hey Ramki,

Do you know if there's any plan to offer the FLS statistics as a metric via
JMX or some other interface in the future? It would be nice to be able to
monitor fragmentation without having to actually log and parse the gc logs.

-Todd

On Thu, Oct 11, 2012 at 7:50 PM, Srinivas Ramakrishna <ysr1729 at gmail.com>wrote:

> In the absence of fragmentation, one would normally expect the max chunk
> size of the CMS generation
> to stabilize at some reasonable value, say after some 10's of CMS GC
> cycles. If it doesn't, you should try
> and use a larger heap, or otherwise reshape the heap to reduce promotion
> rates. In my experience,
> CMS seems to work best if its "duty cycle" is of the order of 1-2 %, i.e.
> there are 50 to 100 times more
> scavenges during the interval that it's not running vs the interva during
> which it is running.
>
> Have Nagios grep the GC log file w/PrintFLSStatistics=2 for the string
> "Max  Chunk Size:" and pick the
> numeric component of every (4n+1)th match. The max chunk size will
> typically cycle within a small band,
> once it has stabilized, returning always to a high value following a CMS
> cycle's completion. If the upper envelope
> of this keeps steadily declining over some 10's of CMS GC cycles, then you
> are probably seeing fragmentation
> that will eventually succumb to fragmentation.
>
> You can probably calibrate a threshold for the upper envelope so that if
> it falls below that threshold you will
> be alerted by Nagios that a closer look is in order.
>
> At least something along those lines should work. The toughest part is
> designing your "filter" to detect the
> fall in the upper envelope. You will probably want to plot the metric,
> then see what kind of filter will detect
> the condition.... Sorry this isn't much concrete help, but hopefully it
> gives you some ideas to work in
> the right direction...
>
> -- ramki
>
> On Thu, Oct 11, 2012 at 4:27 PM, roz dev <rozdev29 at gmail.com> wrote:
>
>> Hi All
>>
>> I am using Java 6u23, with CMS GC. I see that sometime Application gets
>> paused for longer time because of excessive heap fragmentation.
>>
>> I have enabled PrintFLSStatistics flag and following is the log
>>
>>
>> 2012-10-09T15:38:44.724-0400: 52404.306: [GC Before GC:
>> Statistics for BinaryTreeDictionary:
>> ------------------------------------
>> Total Free Space: -668151027
>> Max   Chunk Size: 1976112973
>> Number of Blocks: 175445
>> Av.  Block  Size: 20672
>> Tree      Height: 78
>> Before GC:
>> Statistics for BinaryTreeDictionary:
>> ------------------------------------
>> Total Free Space: 10926
>> Max   Chunk Size: 1660
>> Number of Blocks: 22
>> Av.  Block  Size: 496
>> Tree      Height: 7
>>
>>
>> I would like to know from people about the way they track Heap
>> Fragmentation and how do we alert for this situation?
>>
>> We use Nagios and I am wondering if there is a way to parse these logs
>> and know the max chunk size so that we can alert for it.
>>
>> Any inputs are welcome.
>>
>> -Saroj
>>
>>
>>
>>
>> _______________________________________________
>> hotspot-gc-use mailing list
>> hotspot-gc-use at openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>>
>>
>
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>
>


-- 
Todd Lipcon
Software Engineer, Cloudera
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20121011/0b373cdd/attachment.html