occasionally rss usage surge
the.6th.month at gmail.com
the.6th.month at gmail.com
Mon Mar 12 05:49:10 PDT 2012
Hi,All:
I've recently encountered a memory usage surge of JVM, the JVM version is
jdk6, 6u26. What we see from the top command is as below:
top - 20:12:17 up 112 days, 8:26, 1 user, load average: 0.93, 1.13, 1.16
Tasks: 95 total, 2 running, 93 sleeping, 0 stopped, 0 zombie
Cpu(s): 11.1%us, 2.1%sy, 0.0%ni, 84.9%id, 0.7%wa, 0.0%hi, 0.4%si,
0.8%st
Mem: 6291456k total, 6277716k used, 13740k free, 10844k buffers
Swap: 4192924k total, 392692k used, 3800232k free, 336692k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
COMMAND
5349 root 19 0 6246m 5.3g 5320 S 53.6 88.2 933:38.43
java
8627 flume 19 0 1960m 55m 2368 S 0.0 0.9 3:26.98
java
8573 flume 18 0 1860m 17m 2052 S 0.0 0.3 0:01.97
java
1521 ntp 15 0 23412 5044 3916 S 0.0 0.1 0:06.33
ntpd
1719 root 34 19 254m 4772 1768 S 0.0 0.1 0:09.58
yum-updatesd
2060 root 17 0 88872 3268 2556 S 0.0 0.1 0:00.00
sshd
1482 root 15 0 154m 3228 1872 S 0.0 0.1 1:43.69
snmpd
2062 yue.liu 15 0 88872 1720 980 R 0.0 0.0 0:00.03
sshd
1589 postfix 15 0 54420 1688 1556 S 0.0 0.0 0:00.20
qmgr
1580 root 16 0 54180 1544 1456 S 0.0 0.0 0:00.59
master
2095 root 15 0 66104 1536 1196 S 0.0 0.0 0:00.01
bash
2063 yue.liu 16 0 66104 1520 1184 S 0.0 0.0 0:00.00
bash
It's weird that jvm takes up 5.3g memory, while the jvm heap size is
constrained to 4g as specified by JAVA_OPTS:
-Xms3800m -Xmx3800m -Xss256k -Xmn256m -XX:PermSize=256m -server
We also ran jmap to check the heap usage and didn't find any problem with
it:
Attaching to process ID 5349, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 20.1-b02
using thread-local object allocation.
Parallel GC with 4 thread(s)
Heap Configuration:
MinHeapFreeRatio = 40
MaxHeapFreeRatio = 70
MaxHeapSize = 3984588800 (3800.0MB)
NewSize = 268435456 (256.0MB)
MaxNewSize = 268435456 (256.0MB)
OldSize = 5439488 (5.1875MB)
NewRatio = 2
SurvivorRatio = 8
PermSize = 268435456 (256.0MB)
MaxPermSize = 268435456 (256.0MB)
Heap Usage:
PS Young Generation
Eden Space:
capacity = 159514624 (152.125MB)
used = 113902600 (108.62598419189453MB)
free = 45612024 (43.49901580810547MB)
71.40574145728482% used
>From Space:
capacity = 26083328 (24.875MB)
used = 26082200 (24.873924255371094MB)
free = 1128 (0.00107574462890625MB)
99.99567539847676% used
To Space:
capacity = 54460416 (51.9375MB)
used = 0 (0.0MB)
free = 54460416 (51.9375MB)
0.0% used
PS Old Generation
capacity = 3716153344 (3544.0MB)
used = 1924262008 (1835.119255065918MB)
free = 1791891336 (1708.880744934082MB)
51.78101735513312% used
PS Perm Generation
capacity = 268435456 (256.0MB)
used = 106493600 (101.56021118164062MB)
free = 161941856 (154.43978881835938MB)
39.67195749282837% used
Then I grep the smaps for RSS usage, and found some segments with high
memory residence:
grep Rss /proc/5349/smaps | awk '{if($(NF-1)>0) print $(NF-1)}' | sort -rn
| head -20
4091880
111036
65516
65508
65500
65492
65124
64276
64276
64012
62120
60664
59700
58076
57688
56144
55424
48584
42896
36512
the 4091880 and 111036 comes from $JAVA_HOME/bin/java, *and the other tens
of 60 megabytes usages, which sum up to 1g*, come from :
2aaabc000000-2aaabc008000 r-xs 00115000 ca:07 6488847
/home/q/java/jdk1.6.0_26/jre/lib/resources.jar
Size: 32 kB
Rss: 0 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 0 kB
Swap: 0 kB
Pss: 0 kB
2aaac0000000-2aaac4000000 rwxp 2aaac0000000 00:00 0
Size: 65536 kB
Rss: 51620 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 51620 kB
Swap: 12656 kB
Pss: 51620 kB
2aaac8000000-2aaacbff7000 rwxp 2aaac8000000 00:00 0
Size: 65500 kB
Rss: 65500 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 65500 kB
Swap: 0 kB
Pss: 65500 kB
2aaacc000000-2aaacfff9000 rwxp 2aaacc000000 00:00 0
Size: 65508 kB
Rss: 64000 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 64000 kB
Swap: 1508 kB
Pss: 64000 kB
2aaad0000000-2aaad3ff5000 rwxp 2aaad0000000 00:00 0
Size: 65492 kB
Rss: 47784 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 47784 kB
Swap: 17708 kB
Pss: 47784 kB
2aaad4000000-2aaad7ffb000 rwxp 2aaad4000000 00:00 0
Size: 65516 kB
Rss: 65516 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 65516 kB
Swap: 0 kB
Pss: 65516 kB
2aaad8000000-2aaadbe83000 rwxp 2aaad8000000 00:00 0
Size: 64012 kB
Rss: 64012 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 64012 kB
Swap: 0 kB
Pss: 64012 kB
2aaae8000000-2aaaebff7000 rwxp 2aaae8000000 00:00 0
Size: 65500 kB
Rss: 65492 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 65492 kB
Swap: 0 kB
Pss: 65492 kB
2aaaec000000-2aaaefffd000 rwxp 2aaaec000000 00:00 0
Size: 65524 kB
Rss: 59700 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 59700 kB
Swap: 0 kB
*I am wondering whether it is because I set /proc/sys/vm/swappiness to
zero, or if not, is it because of the resources.jar*? Does anyone know what
this jar is used for and have any idea about the cause of the memory usage
surge
Any help would be highly appreciated, thanks very much.
All the best,
Leon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20120312/e5abbd78/attachment.html
More information about the hotspot-gc-use
mailing list