ParallelOldGC : Single threaded with one cpu 100% during full gc

[lohit]

Hello Devs,

We are trying to profile Full GC performance on our Java server.
Heap Size is about 80G, running on CentOS 5.5. As of now we use
ParallelOldGC for old generation compactions.
When we trigger fullGC by hand we see that it takes more than 2 minutes.
Running mpstat showed that for about half of the time only one CPU was
spinning 100% and all others were idle. (Box has 24 CPUs)
While this was happening we took stack trace and see that all threads are
waiting behind one thread whose trace look like below.

Question is the below expected. Even though documentation says
parallelOldGC uses multiple threads, are there cases when this kind of
serialization happens where all threads wait behind single thread.
I might have given very little information about problem, but let me know
if I could add any more information to know about this

Thread 18 (Thread 0x419f2940 (LWP 55314)):^M
#0  0x00007f051735e3e0 in
BitMap::get_next_one_offset_inline_aligned_right(unsigned long,
unsigned long) const ()^M
   from /usr/java/jdk1.6.0_24/jre/lib/amd64/server/libjvm.so^M
#1  0x00007f051735e09e in
ParMarkBitMap::live_words_in_range(HeapWord*, oopDesc*) const ()^M
   from /usr/java/jdk1.6.0_24/jre/lib/amd64/server/libjvm.so^M
#2  0x00007f0517398848 in ParallelCompactData::calc_new_pointer(HeapWord*) ()^M
   from /usr/java/jdk1.6.0_24/jre/lib/amd64/server/libjvm.so^M
#3  0x00007f05173413ec in
objArrayKlass::oop_update_pointers(ParCompactionManager*, oopDesc*)
()^M
   from /usr/java/jdk1.6.0_24/jre/lib/amd64/server/libjvm.so^M
#4  0x00007f051739d0a7 in
PSParallelCompact::update_deferred_objects(ParCompactionManager*,
PSParallelCompact::SpaceId) ()^M
   from /usr/java/jdk1.6.0_24/jre/lib/amd64/server/libjvm.so^M
#5  0x00007f051739c6c7 in PSParallelCompact::compact() () from
/usr/java/jdk1.6.0_24/jre/lib/amd64/server/libjvm.so^M
#6  0x00007f051739aeca in PSParallelCompact::invoke_no_policy(bool) ()^M
   from /usr/java/jdk1.6.0_24/jre/lib/amd64/server/libjvm.so^M
#7  0x00007f051739a845 in PSParallelCompact::invoke(bool) () from
/usr/java/jdk1.6.0_24/jre/lib/amd64/server/libjvm.so^M
#8  0x00007f05174a0bb0 in VM_ParallelGCSystemGC::doit() () from
/usr/java/jdk1.6.0_24/jre/lib/amd64/server/libjvm.so^M
#9  0x00007f05174ada5a in VM_Operation::evaluate() () from
/usr/java/jdk1.6.0_24/jre/lib/amd64/server/libjvm.so^M
(More stack frames follow...)^M


-- 
Have a Nice Day!
Lohit
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20121104/61557424/attachment.htm>