A really weird GC problem (long GC pause)

[Zhenghua Gao]

Actually, there are some other java processes running on this machine(a
datanode, a mapreduce node).
So, how could I improve the GC pause in this scenario? Expand the physical
memory?



On Fri, Jul 13, 2012 at 1:52 AM, Vitaly Davidovich <vitalyd at gmail.com>wrote:

> Usually when you see things like this (low user and sys time but high
> real) it's because of heavy swapping.  You mention that you use 16gb out of
> 32 but is there nothing else running on this machine that could be tying up
> memory? This should be ruled out first, I'd say.
>
> Sent from my phone
> On Jul 12, 2012 1:42 PM, "Zhenghua Gao" <docete at gmail.com> wrote:
>
>> We run a Hbase cluster on ordinary machines (8 cores, 32GB memory).
>> Our RegionServer takes 16 GB (heap size) memory, and young GC takes less
>> than 1 second normally.
>> However, every now and then there is some periods that young GC may takes
>> hundreds seconds.
>>
>> We have 16GB heap size, and use ParNew and CMS for GC. The GC log is
>> listed below:
>>
>> 2012-07-11T19:41:00.387+0800: 12391.954: [GC Before GC:
>> Statistics for BinaryTreeDictionary:
>> ------------------------------------
>> Total Free Space: 1200692977
>> Max   Chunk Size: 1200692977
>> Number of Blocks: 1
>> Av.  Block  Size: 1200692977
>> Tree      Height: 1
>> Before GC:
>> Statistics for BinaryTreeDictionary:
>> ------------------------------------
>> Total Free Space: 1465991
>> Max   Chunk Size: 1464832
>> Number of Blocks: 4
>> Av.  Block  Size: 366497
>> Tree      Height: 3
>> 13209.843: [ParNew13209.911: [SoftReference, 0 refs, 0.0000120
>> secs]13209.911: [WeakReference, 93 refs, 0.0000150 secs]13209.911:
>> [FinalReference, 271 refs, 0.0009130 secs]13209.912: [PhantomReference, 10
>> refs, 0.0000110 secs]13209.912: [JNI Weak Reference, 0.0000030 secs]
>> Desired survivor size 26836992 bytes, new threshold 1 (max 15)
>> - age   1:   39360920 bytes,   39360920 total
>> : 471872K->52416K(471872K), 0.0692990 secs]
>> 7343322K->6979157K(16724800K)After GC:
>> Statistics for BinaryTreeDictionary:
>> ------------------------------------
>> Total Free Space: 1193619699
>> Max   Chunk Size: 1193619699
>> Number of Blocks: 1
>> Av.  Block  Size: 1193619699
>> Tree      Height: 1
>> After GC:
>> Statistics for BinaryTreeDictionary:
>> ------------------------------------
>> Total Free Space: 1465991
>> Max   Chunk Size: 1464832
>> Number of Blocks: 4
>> Av.  Block  Size: 366497
>> Tree      Height: 3
>> , 0.0695400 secs] [Times: user=0.47 sys=0.12, real=818.01 secs]
>>
>> Is there any clue that i can solve this issue? Thanks.
>>
>> _______________________________________________
>> hotspot-gc-use mailing list
>> hotspot-gc-use at openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20120723/146d6f9d/attachment.html