JVM Freeze, possibly in CMS

[Bob Cotton]

We've seen this now 3 times in production.

Server:
uname -a
Linux app2.rallydev.com 2.6.18-92.1.10.el5 #1 SMP Tue Aug 5 07:42:41 EDT
2008 x86_64 x86_64 x86_64 GNU/Linux

An 8 way 16GB machine

JVM:
java -version
java version "1.5.0_17"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_17-b04)
Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_17-b04, mixed mode)

JVM Args:

-Xms10g -Xmx10g -Xmn1g -XX:MaxPermSize=256m -XX:+UseConcMarkSweepGC
-XX:+ParallelRefProcEnabled -XX:+PrintGCApplicationStoppedTime
-XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCDetails
-XX:+PrintGCTimeStamps -XX:-TraceClassUnloading -Xloggc:gc.log
-Dsun.rmi.dgc.client.gcInterval=9223372036854775807
-Dsun.rmi.dgc.server.gcInterval=9223372036854775807
-Djboss.server.home.dir=/home/prod2/alm-gallium-220/alm-app-jboss/target/jboss4x/slmDomain
-Djboss.server.home.url=file:/home/prod2/alm-gallium-220/alm-app-jboss/target/jboss4x/slmDomain
-Djavax.management.builder.initial=org.jboss.system.server.jmx.MBeanServerBuilderImpl
-Djboss.platform.mbeanserver -Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=5002
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
-Doracle.jdbc.V8Compatible=true -Djava.awt.headless=true -
Djboss.partition.name=cluster-prod -disableassertions -Dprogram.name=run.sh
-Djava.endorsed.dirs=/usr/local/jboss/lib/endorsed


The last line in my gc.log looks like this:
62309.364: [GC 62309.364: [ParNew: 1046656K->0K(1047616K), 0.0351140 secs]
9406467K->8371032K(10484800K), 0.0354550 secs]
Total time for which application threads were stopped: 0.0374980 seconds
Application time: 0.0000520 seconds
62309.401: [GC [1 CMS-initial-mark: 8371032K(9437184K)] 8371174K(10484800K),
0.0116140 secs]
Total time for which application threads were stopped: 0.0128850 seconds
62309.413: [CMS-concurrent-mark-start]
62315.566: [CMS-concurrent-mark: 6.153/6.153 secs]
62315.566: [CMS-concurrent-preclean-start]
62315.870: [CMS-concurrent-preclean: 0.295/0.304 secs]
62315.870: [CMS-concurrent-abortable-preclean-start]
 CMS: abort preclean due to time 62316.940:
[CMS-concurrent-abortable-preclean: 0.551/1.070 secs]
Application time: 7.5270130 seconds
62316.942: [GC[YG occupancy: 633600 K (1047616 K)]62316.942: [Rescan
(parallel) , 0.4677820 secs]62317.410: [weak refs processing

then is stops.

I have the core. what other information should I be looking for?

How can I tell if i have excessive CMS heap fragmentation?

Thanks
 -Bob
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20090317/50ddebdf/attachment.htm>