OOM error
Y Srinivas Ramakrishna
Y.S.Ramakrishna at Sun.COM
Wed Nov 28 07:40:13 UTC 2007
In the case of the enclosed pmap (which you indicate is from
a JVM that has not encountered the C-heap exhaustion), as you might
expect there is space available in the C-heap (shown by the "[heap]"
annotated segment in yr pmap dump below).
So we'll need to look at the pmap from a failing process. You could one of
two things (or perhaps both):-
-XX:OnError="pmap %p"
will pmap the process just before exit with the error; otherwise you can do
-XX:OnError="gcore %p"
and then pmap the resulting core file.
That should show the C-heap exhausted and you'd be able to see what went
wrong. For additional information, since it appears that the process runs for a
while before running into this C-heap exhaustion, you might want to run
pmap on the process periodically and collect this pmap time-trace. You'd then
likely see what it was that was over time consuming address space and running
you out of C-heap.
Additionally (also in -XX:OnError) pfiles may provide additional information.
Hope that helps.
PS: Since this is technically not a GC issue, per se, I'd probably take this off
the GC list. I am accordingly cross-posting to the hotspot-runtime-dev list
where it might be a better fit and where there might be further suggestions.
-- ramki
----- Original Message -----
From: Anuj Lal <alal at hotwire.com>
Date: Tuesday, November 27, 2007 4:30 pm
Subject: RE: RE: OOM error
To: Y Srinivas Ramakrishna <Y.S.Ramakrishna at Sun.COM>
Cc: Neo Jia <neojia at gmail.com>, hotspot-gc-dev at openjdk.java.net
> Ramki
>
> This is pmap info when it is working correctly
> We had same issue on other server also. Its becoming p1 issue now. I"ll
> opem official sun case for this
>
> Anuj
>
>
>
> 20139: /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/bin/java
> -Xms2560m
> 00010000 64K r-x--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/bin/java
> 0002E000 16K rwx--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/bin/java
> 00032000 135328K rwx-- [ heap ]
> 24C00000 5904K r--s- dev:85,5 ino:169421
> 25380000 408K r-x--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libfontmana
> ger.so
> 253F4000 96K rwx--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libfontmana
> ger.so
> 25480000 24K rwx-R [ anon ]
> 25486000 488K rw--R [ anon ]
> 25580000 24K rwx-R [ anon ]
> 25586000 488K rw--R [ anon ]
> 25680000 24K ----R [ anon ]
> 25686000 488K rw--R [ anon ]
> 25780000 24K rwx-R [ anon ]
> 25786000 488K rw--R [ anon ]
> 25880000 24K rwx-R [ anon ]
> 25886000 488K rw--R [ anon ]
> 25980000 24K rwx-R [ anon ]
> 25986000 488K rw--R [ anon ]
> 25A80000 24K rwx-R [ anon ]
> 25A86000 488K rw--R [ anon ]
> 25B80000 24K rwx-R [ anon ]
> 25B86000 488K rw--R [ anon ]
> 25C80000 24K rwx-R [ anon ]
> 25C86000 488K rw--R [ anon ]
> 25D80000 24K ----R [ anon ]
> 25D86000 488K rw--R [ anon ]
> 25E80000 24K ----R [ anon ]
> 25E86000 488K rw--R [ anon ]
> 25F80000 24K ----R [ anon ]
> 25F86000 488K rw--R [ anon ]
> 26080000 24K ----R [ anon ]
> 26086000 488K rw--R [ anon ]
> 26180000 24K ----R [ anon ]
> 26186000 488K rw--R [ anon ]
> 26280000 24K ----R [ anon ]
> 26286000 488K rw--R [ anon ]
> 26380000 24K ----R [ anon ]
> 26386000 488K rw--R [ anon ]
> 26480000 24K ----R [ anon ]
> 26486000 488K rw--R [ anon ]
> 26580000 24K ----R [ anon ]
> 26586000 488K rw--R [ anon ]
> 26680000 24K ----R [ anon ]
> 26686000 488K rw--R [ anon ]
> 26780000 24K ----R [ anon ]
> 26786000 488K rw--R [ anon ]
> 26880000 24K ----R [ anon ]
> 26886000 488K rw--R [ anon ]
> 26980000 504K r--s- dev:85,5 ino:169364
> 26A80000 1424K r--s- dev:85,5 ino:169360
> 26C00000 24K ----R [ anon ]
> 26C06000 488K rw--R [ anon ]
> 26D00000 24K ----R [ anon ]
> 26D06000 488K rw--R [ anon ]
> 26E00000 24K ----R [ anon ]
> 26E06000 488K rw--R [ anon ]
> 26F00000 24K ----R [ anon ]
> 26F06000 488K rw--R [ anon ]
> 27000000 24K ----R [ anon ]
> 27006000 488K rw--R [ anon ]
> 27100000 1056K r-x--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libmlib_ima
> ge.so
> 27216000 8K rwx--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libmlib_ima
> ge.so
> 27280000 928K r-x--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libawt.so
> 27376000 40K rwx--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libawt.so
> 27380000 152K rwx--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libawt.so
> 27400000 4432K r--s- dev:85,5 ino:89133
> 27900000 24K ----R [ anon ]
> 27906000 488K rw--R [ anon ]
> 27A00000 24K ----R [ anon ]
> 27A06000 488K rw--R [ anon ]
> 27B00000 648K r--s- dev:85,5 ino:89132
> 27C00000 24K ----R [ anon ]
> 27C06000 488K rw--R [ anon ]
> 27D00000 24K ----R [ anon ]
> 27D06000 488K rw--R [ anon ]
> 27E00000 24K ----R [ anon ]
> 27E06000 488K rw--R [ anon ]
> 27F00000 24K ----R [ anon ]
> 27F06000 488K rw--R [ anon ]
> 28000000 24K ----R [ anon ]
> 28006000 488K rw--R [ anon ]
> 28100000 24K ----R [ anon ]
> 28106000 488K rw--R [ anon ]
> 28200000 24K ----R [ anon ]
> 28206000 488K rw--R [ anon ]
> 28300000 24K ----R [ anon ]
> 28306000 488K rw--R [ anon ]
> 28400000 24K ----R [ anon ]
> 28406000 488K rw--R [ anon ]
> 28500000 24K rwx-R [ anon ]
> 28506000 488K rw--R [ anon ]
> 28600000 24K ----R [ anon ]
> 28606000 488K rw--R [ anon ]
> 28700000 24K ----R [ anon ]
> 28706000 488K rw--R [ anon ]
> 28800000 24K ----R [ anon ]
> 28806000 488K rw--R [ anon ]
> 28900000 24K ----R [ anon ]
> 28906000 488K rw--R [ anon ]
> 28A00000 24K ----R [ anon ]
> 28A06000 488K rw--R [ anon ]
> 28B00000 24K ----R [ anon ]
> 28B06000 488K rw--R [ anon ]
> 28C00000 24K ----R [ anon ]
> 28C06000 488K rw--R [ anon ]
> 28D00000 24K ----R [ anon ]
> 28D06000 488K rw--R [ anon ]
> 28E00000 24K ----R [ anon ]
> 28E06000 488K rw--R [ anon ]
> 28F00000 24K ----R [ anon ]
> 28F06000 488K rw--R [ anon ]
> 29000000 24K ----R [ anon ]
> 29006000 488K rw--R [ anon ]
> 29100000 24K ----R [ anon ]
> 29106000 488K rw--R [ anon ]
> 29200000 24K ----R [ anon ]
> 29206000 488K rw--R [ anon ]
> 29300000 24K ----R [ anon ]
> 29306000 488K rw--R [ anon ]
> 29400000 24K ----R [ anon ]
> 29406000 488K rw--R [ anon ]
> 29500000 24K ----R [ anon ]
> 29506000 488K rw--R [ anon ]
> 29600000 24K ----R [ anon ]
> 29606000 488K rw--R [ anon ]
> 29700000 24K ----R [ anon ]
> 29706000 488K rw--R [ anon ]
> 29800000 24K ----R [ anon ]
> 29806000 488K rw--R [ anon ]
> 29900000 24K ----R [ anon ]
> 29906000 488K rw--R [ anon ]
> 29A00000 24K ----R [ anon ]
> 29A06000 488K rw--R [ anon ]
> 29B00000 24K ----R [ anon ]
> 29B06000 488K rw--R [ anon ]
> 29C00000 24K ----R [ anon ]
> 29C06000 488K rw--R [ anon ]
> 29D00000 24K ----R [ anon ]
> 29D06000 488K rw--R [ anon ]
> 29E00000 24K ----R [ anon ]
> 29E06000 488K rw--R [ anon ]
> 29F00000 24K ----R [ anon ]
> 29F06000 488K rw--R [ anon ]
> 2A000000 24K ----R [ anon ]
> 2A006000 488K rw--R [ anon ]
> 2A100000 24K ----R [ anon ]
> 2A106000 488K rw--R [ anon ]
> 2A200000 24K ----R [ anon ]
> 2A206000 488K rw--R [ anon ]
> 2A300000 24K ----R [ anon ]
> 2A306000 488K rw--R [ anon ]
> 2A400000 24K ----R [ anon ]
> 2A406000 488K rw--R [ anon ]
> 2A500000 24K ----R [ anon ]
> 2A506000 488K rw--R [ anon ]
> 2A600000 24K ----R [ anon ]
> 2A606000 488K rw--R [ anon ]
> 2A700000 24K ----R [ anon ]
> 2A706000 488K rw--R [ anon ]
> 2A800000 24K ----R [ anon ]
> 2A806000 488K rw--R [ anon ]
> 2A900000 24K ----R [ anon ]
> 2A906000 488K rw--R [ anon ]
> 2AA00000 24K ----R [ anon ]
> 2AA06000 488K rw--R [ anon ]
> 2AB00000 24K ----R [ anon ]
> 2AB06000 488K rw--R [ anon ]
> 2AC00000 24K ----R [ anon ]
> 2AC06000 488K rw--R [ anon ]
> 2AD00000 24K ----R [ anon ]
> 2AD06000 488K rw--R [ anon ]
> 2AE00000 24K ----R [ anon ]
> 2AE06000 488K rw--R [ anon ]
> 2AF00000 24K ----R [ anon ]
> 2AF06000 488K rw--R [ anon ]
> 2B000000 24K ----R [ anon ]
> 2B006000 488K rw--R [ anon ]
> 2B100000 24K ----R [ anon ]
> 2B106000 488K rw--R [ anon ]
> 2B200000 24K ----R [ anon ]
> 2B206000 488K rw--R [ anon ]
> 2B300000 24K ----R [ anon ]
> 2B306000 488K rw--R [ anon ]
> 2B400000 24K ----R [ anon ]
> 2B406000 488K rw--R [ anon ]
> 2B500000 24K ----R [ anon ]
> 2B506000 488K rw--R [ anon ]
> 2B600000 24K ----R [ anon ]
> 2B606000 488K rw--R [ anon ]
> 2B700000 24K ----R [ anon ]
> 2B706000 488K rw--R [ anon ]
> 2B800000 24K ----R [ anon ]
> 2B806000 488K rw--R [ anon ]
> 2B900000 24K ----R [ anon ]
> 2B906000 488K rw--R [ anon ]
> 2BA00000 24K ----R [ anon ]
> 2BA06000 488K rw--R [ anon ]
> 2BB00000 24K ----R [ anon ]
> 2BB06000 488K rw--R [ anon ]
> 2BC00000 24K ----R [ anon ]
> 2BC06000 488K rw--R [ anon ]
> 2BD00000 24K ----R [ anon ]
> 2BD06000 488K rw--R [ anon ]
> 2BE00000 24K ----R [ anon ]
> 2BE06000 488K rw--R [ anon ]
> 2BF00000 24K ----R [ anon ]
> 2BF06000 488K rw--R [ anon ]
> 2C000000 24K ----R [ anon ]
> 2C006000 488K rw--R [ anon ]
> 2C100000 24K ----R [ anon ]
> 2C106000 488K rw--R [ anon ]
> 2C200000 24K ----R [ anon ]
> 2C206000 488K rw--R [ anon ]
> 2C300000 24K ----R [ anon ]
> 2C306000 488K rw--R [ anon ]
> 2C400000 24K ----R [ anon ]
> 2C406000 488K rw--R [ anon ]
> 2C500000 24K ----R [ anon ]
> 2C506000 488K rw--R [ anon ]
> 2C600000 24K ----R [ anon ]
> 2C606000 488K rw--R [ anon ]
> 2C700000 24K ----R [ anon ]
> 2C706000 488K rw--R [ anon ]
> 2C800000 24K ----R [ anon ]
> 2C806000 488K rw--R [ anon ]
> 2C900000 24K ----R [ anon ]
> 2C906000 488K rw--R [ anon ]
> 2CA00000 24K ----R [ anon ]
> 2CA06000 488K rw--R [ anon ]
> 2CB00000 24K ----R [ anon ]
> 2CB06000 488K rw--R [ anon ]
> 2CC00000 24K ----R [ anon ]
> 2CC06000 488K rw--R [ anon ]
> 2CD00000 24K ----R [ anon ]
> 2CD06000 488K rw--R [ anon ]
> 2CE00000 24K ----R [ anon ]
> 2CE06000 488K rw--R [ anon ]
> 2CF00000 24K ----R [ anon ]
> 2CF06000 488K rw--R [ anon ]
> 2D000000 24K ----R [ anon ]
> 2D006000 488K rw--R [ anon ]
> 2D100000 24K ----R [ anon ]
> 2D106000 488K rw--R [ anon ]
> 2D200000 24K ----R [ anon ]
> 2D206000 488K rw--R [ anon ]
> 2D300000 24K ----R [ anon ]
> 2D306000 488K rw--R [ anon ]
> 2D400000 24K ----R [ anon ]
> 2D406000 488K rw--R [ anon ]
> 2D500000 24K ----R [ anon ]
> 2D506000 488K rw--R [ anon ]
> 2D600000 24K ----R [ anon ]
> 2D606000 488K rw--R [ anon ]
> 2D700000 24K ----R [ anon ]
> 2D706000 488K rw--R [ anon ]
> 2D800000 24K ----R [ anon ]
> 2D806000 488K rw--R [ anon ]
> 2D900000 24K ----R [ anon ]
> 2D906000 488K rw--R [ anon ]
> 2DA00000 24K ----R [ anon ]
> 2DA06000 488K rw--R [ anon ]
> 2DB00000 24K ----R [ anon ]
> 2DB06000 488K rw--R [ anon ]
> 2DC00000 24K ----R [ anon ]
> 2DC06000 488K rw--R [ anon ]
> 2DD00000 24K ----R [ anon ]
> 2DD06000 488K rw--R [ anon ]
> 2DE00000 24K ----R [ anon ]
> 2DE06000 488K rw--R [ anon ]
> 2DEB0000 176K r-x--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libjpeg.so
> 2DEEC000 8K rwx--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libjpeg.so
> 2DF00000 24K ----R [ anon ]
> 2DF06000 488K rw--R [ anon ]
> 2DFA0000 8K rwx-- [ anon ]
> 2DFB0000 96K r-x--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libdcpr.so
> 2DFD6000 80K rwx--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libdcpr.so
> 2E000000 24K ----R [ anon ]
> 2E006000 488K rw--R [ anon ]
> 2E090000 312K r--s- dev:85,5 ino:12797
> 2E0F0000 24K r--s- dev:85,5 ino:169374
> 2E100000 24K ----R [ anon ]
> 2E106000 488K rw--R [ anon ]
> 2E190000 32K r-x--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/headless/li
> bmawt.so
> 2E1A6000 8K rwx--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/headless/li
> bmawt.so
> 2E1B0000 8K rwx-- [ anon ]
> 2E1C0000 112K r--s- dev:85,5 ino:104786
> 2E1E0000 112K r--s- dev:85,5 ino:148909
> 2E200000 24K ----R [ anon ]
> 2E206000 488K rw--R [ anon ]
> 2E290000 24K r--s- dev:85,5 ino:104507
> 2E2A0000 32K r--s- dev:85,5 ino:90208
> 2E2B0000 64K r--s- dev:85,5 ino:103379
> 2E2D0000 32K r--s- dev:85,5 ino:90200
> 2E2E0000 56K r--s- dev:85,5 ino:90170
> 2E300000 24K ----R [ anon ]
> 2E306000 488K rw--R [ anon ]
> 2E390000 32K r--s- dev:85,5 ino:90131
> 2E3A0000 16K r--s- dev:85,5 ino:90152
> 2E3B0000 40K r--s- dev:85,5 ino:90192
> 2E3C0000 136K r-x-- /usr/lib/libelf.so.1
> 2E3F2000 8K rwx-- /usr/lib/libelf.so.1
> 2E400000 24K ----R [ anon ]
> 2E406000 488K rw--R [ anon ]
> 2E490000 48K r--s- dev:85,5 ino:90137
> 2E4A0000 248K r-x-- /usr/lib/libresolv.so.2
> 2E4EE000 16K rwx-- /usr/lib/libresolv.so.2
> 2E500000 1480K r--s- dev:85,5 ino:178161
> 2E680000 24K ----R [ anon ]
> 2E686000 488K rw--R [ anon ]
> 2E710000 56K r--s- dev:85,5 ino:90159
> 2E730000 16K rw--- [ anon ]
> 2E740000 40K r-x-- /usr/ucblib/libucb.so.1
> 2E75A000 8K rwx-- /usr/ucblib/libucb.so.1
> 2E760000 24K r-x--
> /opt/p4/3rdparty/bea/weblogic/9.2mp1/server/native/solaris/sparc/libmuxe
> r.so
> 2E774000 8K rwx--
> /opt/p4/3rdparty/bea/weblogic/9.2mp1/server/native/solaris/sparc/libmuxe
> r.so
> 2E780000 512K rw--R [ anon ]
> 2E810000 40K r--s- dev:85,5 ino:90183
> 2E820000 8K r-x--
> /opt/p4/3rdparty/bea/weblogic/9.2mp1/server/native/solaris/sparc/libwlfi
> leio2.so
> 2E830000 16K rwx--
> /opt/p4/3rdparty/bea/weblogic/9.2mp1/server/native/solaris/sparc/libwlfi
> leio2.so
> 2E840000 64K rw--- [ anon ]
> 2E860000 8K r-x--
> /opt/p4/3rdparty/bea/weblogic/9.2mp1/server/native/solaris/sparc/libstac
> kdump.so
> 2E870000 8K rwx--
> /opt/p4/3rdparty/bea/weblogic/9.2mp1/server/native/solaris/sparc/libstac
> kdump.so
> 2E880000 24K ----R [ anon ]
> 2E886000 488K rw--R [ anon ]
> 2E910000 48K r--s- dev:85,5 ino:90122
> 2E920000 8K r-x-- /usr/lib/libsendfile.so.1
> 2E932000 8K rwx-- /usr/lib/libsendfile.so.1
> 2E940000 8K r-x-- /usr/lib/libmd5.so.1
> 2E952000 8K rwx-- /usr/lib/libmd5.so.1
> 2E960000 40K r-x-- /usr/lib/libaio.so.1
> 2E97A000 8K rwx-- /usr/lib/libaio.so.1
> 2E980000 24K ----R [ anon ]
> 2E986000 488K rw--R [ anon ]
> 2EA10000 16K r--s- dev:85,5 ino:91540
> 2EA20000 24K r-x-- /usr/lib/librt.so.1
> 2EA36000 8K rwx-- /usr/lib/librt.so.1
> 2EA40000 200K r--s- dev:85,5 ino:178116
> 2EA80000 24K ----R [ anon ]
> 2EA86000 488K rw--R [ anon ]
> 2EB10000 32K rw--- [ anon ]
> 2EB20000 32K r-x--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libnio.so
> 2EB36000 8K rwx--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libnio.so
> 2EB40000 184K r--s- dev:85,5 ino:178351
> 2EB80000 2496K r--s- dev:85,5 ino:43427
> 2EE00000 784K r--s- dev:85,5 ino:8126
> 2EED0000 88K r--s- dev:85,5 ino:178142
> 2EEF0000 32K r--s- dev:85,5 ino:178349
> 2EF00000 64K r--s- dev:85,5 ino:19946
> 2EF20000 328K r--s- dev:85,5 ino:27067
> 2EF80000 776K r--s- dev:85,5 ino:39082
> 2F050000 64K r--s- dev:85,5 ino:16731
> 2F070000 376K r--s- dev:85,5 ino:7814
> 2F0E0000 288K r--s- dev:85,5 ino:7812
> 2F130000 248K r--s- dev:85,5 ino:43449
> 2F180000 1336K r--s- dev:85,5 ino:11522
> 2F2D0000 120K r--s- dev:85,5 ino:7816
> 2F300000 1496K r--s- dev:85,5 ino:7171
> 2F480000 880K r--s- dev:85,5 ino:16719
> 2F560000 56K r--s- dev:85,5 ino:8160
> 2F580000 64K r--s- dev:85,5 ino:7815
> 2F5A0000 336K r--s- dev:85,5 ino:16717
> 2F600000 1656K r--s- dev:85,5 ino:11690
> 2F7B0000 128K r--s- dev:85,5 ino:16720
> 2F7E0000 88K r--s- dev:85,5 ino:23479
> 2F800000 5344K r--s- dev:85,5 ino:25387
> 2FD40000 16K rw--- [ anon ]
> 2FD50000 120K r--s- dev:85,5 ino:19782
> 2FD80000 1152K r--s- dev:85,5 ino:7946
> 2FEB0000 24K r--s- dev:85,5 ino:43425
> 2FEC0000 208K r--s- dev:85,5 ino:7944
> 2FF00000 648K r--s- dev:85,5 ino:21820
> 2FFB0000 32K r--s- dev:85,5 ino:7813
> 2FFC0000 240K r--s- dev:85,5 ino:7945
> 30000000 4904K r--s- dev:85,5 ino:157426
> 304D0000 8K rwx-- [ anon ]
> 304E0000 24K r--s- dev:85,5 ino:43447
> 304F0000 120K r--s- dev:85,5 ino:16575
> 30520000 352K r--s- dev:85,5 ino:23507
> 30580000 1512K r--s- dev:85,5 ino:156707
> 30700000 616K r--s- dev:85,5 ino:8161
> 307A0000 312K r--s- dev:85,5 ino:12797
> 30800000 3968K r--s- dev:85,5 ino:9009
> 30BF0000 32K r--s- dev:85,5 ino:16718
> 30C00000 5040K r--s- dev:85,5 ino:9011
> 31100000 488K r--s- dev:85,5 ino:21821
> 31180000 480K r--s- dev:85,5 ino:44388
> 31200000 552K r--s- dev:85,5 ino:7744
> 31290000 8K r--s- dev:85,5 ino:169334
> 312A0000 352K r--s- dev:85,5 ino:26985
> 31300000 640K r--s- dev:85,5 ino:176761
> 313B0000 24K r--s- dev:85,5 ino:15965
> 313C0000 24K r--s- dev:85,5 ino:11502
> 313D0000 168K r--s- dev:85,5 ino:21694
> 31400000 6880K r--s- dev:85,5 ino:169093
> 31AC0000 16K r--s- dev:85,5 ino:15966
> 31AD0000 208K r--s- dev:85,5 ino:17946
> 31B10000 432K r--s- dev:85,5 ino:15781
> 31B80000 680K r--s- dev:85,5 ino:170149
> 31C30000 8K r--s- dev:85,5 ino:23497
> 31C40000 224K r--s- dev:85,5 ino:20121
> 31C80000 1000K r--s- dev:85,5 ino:169091
> 31D80000 976K r--s- dev:85,5 ino:169088
> 31E80000 2544K r--s- dev:85,5 ino:188396
> 32100000 512K r--s- dev:85,5 ino:170763
> 32190000 80K r--s- dev:85,5 ino:23502
> 321B0000 40K r--s- dev:85,5 ino:44387
> 321C0000 80K r--s- dev:85,5 ino:17995
> 321E0000 72K r--s- dev:85,5 ino:39073
> 32200000 1496K r--s- dev:85,5 ino:170766
> 32380000 776K r--s- dev:85,5 ino:188388
> 32450000 16K r--s- dev:85,5 ino:15778
> 32460000 112K r--s- dev:85,5 ino:7139
> 32480000 968K r--s- dev:85,5 ino:171312
> 32580000 752K r--s- dev:85,5 ino:176705
> 32650000 32K r--s- dev:85,5 ino:23508
> 32660000 72K r--s- dev:85,5 ino:13123
> 32680000 912K r--s- dev:85,5 ino:176805
> 32770000 32K r--s- dev:85,5 ino:20119
> 32780000 1512K r--s- dev:85,5 ino:178382
> 32900000 2616K r--s- dev:85,5 ino:169089
> 32BA0000 48K r--s- dev:85,5 ino:20243
> 32BB0000 256K r--s- dev:85,5 ino:170462
> 32C00000 52536K r--s- dev:85,5 ino:178597
> 35F60000 152K r--s- dev:85,5 ino:7752
> 35F90000 392K r--s- dev:85,5 ino:170432
> 36000000 1864K r--s- dev:85,5 ino:187108
> 361E0000 24K r--s- dev:85,5 ino:21822
> 361F0000 32K r--s- dev:85,5 ino:7745
> 36200000 1784K r--s- dev:85,5 ino:157391
> 363D0000 120K r--s- dev:85,5 ino:7190
> 36400000 6880K r--s- dev:85,5 ino:94624
> 36AC0000 200K r--s- dev:85,5 ino:188391
> 36B00000 648K r--s- dev:85,5 ino:178589
> 36BB0000 160K r--s- dev:85,5 ino:170190
> 36BE0000 112K r--s- dev:85,5 ino:170154
> 36C00000 6936K r--s- dev:85,5 ino:24574
> 372D0000 8K r--s- dev:85,5 ino:31008
> 372E0000 104K r--s- dev:85,5 ino:170159
> 37300000 2128K r--s- dev:85,5 ino:157398
> 37520000 96K r--s- dev:85,5 ino:170157
> 37540000 216K r--s- dev:85,5 ino:170483
> 37580000 2752K r--s- dev:85,5 ino:24575
> 37840000 32K r--s- dev:85,5 ino:7748
> 37850000 120K r--s- dev:85,5 ino:187721
> 37880000 3432K r--s- dev:85,5 ino:23916
> 37BE0000 8K r--s- dev:85,5 ino:7141
> 37BF0000 48K r--s- dev:85,5 ino:7800
> 37C00000 4096K rwx-- [ anon ]
> 38010000 8K r--s- dev:85,5 ino:9010
> 38020000 192K r--s- dev:85,5 ino:187853
> 38060000 88K r--s- dev:85,5 ino:188385
> 38080000 1768K r--s- dev:85,5 ino:23915
> 38240000 192K r--s- dev:85,5 ino:187665
> 38280000 1016K r--s- dev:85,5 ino:7170
> 38390000 40K r--s- dev:85,5 ino:176570
> 383A0000 312K r--s- dev:85,5 ino:188386
> 38400000 131072K rwx-- [ anon ]
> 40410000 424K r--s- dev:85,5 ino:187496
> 40480000 864K r--s- dev:85,5 ino:178588
> 40560000 72K r--s- dev:85,5 ino:178111
> 40580000 696K r--s- dev:85,5 ino:20117
> 40640000 8K r--s- dev:85,5 ino:170155
> 40650000 144K r--s- dev:85,5 ino:171315
> 40680000 1072K r--s- dev:85,5 ino:16891
> 407A0000 112K r--s- dev:85,5 ino:188378
> 407C0000 64K r--s- dev:85,5 ino:171316
> 407E0000 72K r--s- dev:85,5 ino:171306
> 40800000 73728K rwx-- [ anon ]
> 45000000 8K rwx-- [ anon ]
> 45010000 24K r--s- dev:85,5 ino:171297
> 45020000 24K r--s- dev:85,5 ino:171289
> 45030000 8K r--s- dev:85,5 ino:171305
> 45040000 8K r--s- dev:85,5 ino:171296
> 45050000 16K r--s- dev:85,5 ino:171288
> 45060000 8K r--s- dev:85,5 ino:171304
> 45070000 8K r--s- dev:85,5 ino:171295
> 45080000 168K r--s- dev:85,5 ino:171287
> 450B0000 8K r--s- dev:85,5 ino:171311
> 450C0000 40K r--s- dev:85,5 ino:171303
> 450D0000 8K r--s- dev:85,5 ino:171294
> 450E0000 40K r--s- dev:85,5 ino:171286
> 450F0000 8K r--s- dev:85,5 ino:171310
> 45100000 792K r--s- dev:85,5 ino:25468
> 451D0000 8K r--s- dev:85,5 ino:171293
> 451E0000 8K r--s- dev:85,5 ino:171285
> 451F0000 8K r--s- dev:85,5 ino:171309
> 45200000 24K ----R [ anon ]
> 45206000 488K rw--R [ anon ]
> 45290000 16K r--s- dev:85,5 ino:171301
> 452A0000 48K r--s- dev:85,5 ino:171292
> 452B0000 8K r--s- dev:85,5 ino:171284
> 452C0000 16K r--s- dev:85,5 ino:171308
> 452D0000 16K r--s- dev:85,5 ino:171300
> 452E0000 8K r--s- dev:85,5 ino:171291
> 452F0000 48K r--s- dev:85,5 ino:171307
> 45300000 24K ----R [ anon ]
> 45306000 488K rw--R [ anon ]
> 45390000 400K r--s- dev:85,5 ino:171302
> 45400000 4096K rwx-- [ anon ]
> 45800000 8K rwx-- [ anon ]
> 45810000 16K r--s- dev:85,5 ino:171283
> 45820000 72K r--s- dev:85,5 ino:171298
> 45840000 16K r--s- dev:85,5 ino:171290
> 45850000 8K r--s- dev:85,5 ino:171282
> 45860000 112K r--s- dev:85,5 ino:171271
> 45880000 472K r--s- dev:85,5 ino:24753
> 45900000 24K ----R [ anon ]
> 45906000 488K rw--R [ anon ]
> 45990000 136K r--s- dev:85,5 ino:171270
> 459C0000 64K r--s- dev:85,5 ino:171269
> 459E0000 24K r--s- dev:85,5 ino:176581
> 459F0000 16K r--s- dev:85,5 ino:176577
> 45A00000 24K ----R [ anon ]
> 45A06000 488K rw--R [ anon ]
> 45A90000 392K r--s- dev:85,5 ino:176527
> 45B00000 40K rwx-- [ anon ]
> 45B0A000 8K rwx-- [ anon ]
> 45B0C000 8K rwx-- [ anon ]
> 45B0E000 8K rwx-- [ anon ]
> 45B10000 8K rwx-- [ anon ]
> 45B12000 8K rwx-- [ anon ]
> 45B14000 8K rwx-- [ anon ]
> 45B16000 8K rwx-- [ anon ]
> 45B18000 8K rwx-- [ anon ]
> 45B1A000 8K rwx-- [ anon ]
> 45B1C000 8K rwx-- [ anon ]
> 45B1E000 8K rwx-- [ anon ]
> 45B20000 8K rwx-- [ anon ]
> 45B22000 8K rwx-- [ anon ]
> 45B24000 8K rwx-- [ anon ]
> 45B26000 8K rwx-- [ anon ]
> 45B28000 8K rwx-- [ anon ]
> 45B2A000 8K rwx-- [ anon ]
> 45B2C000 8K rwx-- [ anon ]
> 45B2E000 8K rwx-- [ anon ]
> 45B30000 8K rwx-- [ anon ]
> 45B32000 8K rwx-- [ anon ]
> 45B34000 8K rwx-- [ anon ]
> 45B36000 8K rwx-- [ anon ]
> 45B38000 8K rwx-- [ anon ]
> 45B3A000 152K rwx-- [ anon ]
> 45B60000 136K ----R [ anon ]
> 45B90000 256K r--s- dev:85,5 ino:176512
> 45BE0000 80K r--s- dev:85,5 ino:188399
> 45C00000 1024K rwx-- [ anon ]
> 45D00000 4096K rwx-- [ anon ]
> 46100000 32K rwx-- [ anon ]
> 46108000 8K rwx-- [ anon ]
> 4610A000 8K rwx-- [ anon ]
> 4610C000 8K rwx-- [ anon ]
> 4610E000 8K rwx-- [ anon ]
> 46110000 8K rwx-- [ anon ]
> 46112000 8K rwx-- [ anon ]
> 46114000 8K rwx-- [ anon ]
> 46116000 8K rwx-- [ anon ]
> 46118000 8K rwx-- [ anon ]
> 4611A000 8K rwx-- [ anon ]
> 4611C000 8K rwx-- [ anon ]
> 4611E000 8K rwx-- [ anon ]
> 46120000 8K rwx-- [ anon ]
> 46122000 8K rwx-- [ anon ]
> 46124000 8K rwx-- [ anon ]
> 46126000 8K rwx-- [ anon ]
> 46128000 8K rwx-- [ anon ]
> 4612A000 8K rwx-- [ anon ]
> 4612C000 8K rwx-- [ anon ]
> 4612E000 8K rwx-- [ anon ]
> 46130000 8K rwx-- [ anon ]
> 46132000 8K rwx-- [ anon ]
> 46134000 8K rwx-- [ anon ]
> 46136000 8K rwx-- [ anon ]
> 46138000 8K rwx-- [ anon ]
> 4613A000 152K rwx-- [ anon ]
> 46160000 128K ----R [ anon ]
> 46180000 8K rwx-- [ anon ]
> 46190000 392K r--s- dev:85,5 ino:177742
> 46200000 24K ----R [ anon ]
> 46206000 488K rw--R [ anon ]
> 46290000 8K r--s- dev:85,5 ino:176568
> 462A0000 8K r--s- dev:85,5 ino:187556
> 462B0000 40K r--s- dev:85,5 ino:178583
> 462C0000 40K r--s- dev:85,5 ino:178576
> 462D0000 16K r--s- dev:85,5 ino:169518
> 462E0000 8K r--s- dev:85,5 ino:168455
> 462F0000 40K r--s- dev:85,5 ino:168458
> 46300000 24K ----R [ anon ]
> 46306000 488K rw--R [ anon ]
> 46390000 16K r--s- dev:85,5 ino:168457
> 463A0000 24K r--s- dev:85,5 ino:168454
> 463B0000 8K r--s- dev:85,5 ino:168477
> 463C0000 64K r--s- dev:85,5 ino:24571
> 463E0000 96K r--s- dev:85,5 ino:24573
> 46400000 524288K rwx-- [ anon ]
> 66400000 2097152K rwx-- [ anon ]
> E6400000 16384K rwx-- [ anon ]
> E7400000 4096K rwx-- [ anon ]
> E7800000 4096K rwx-- [ anon ]
> E7C00000 4096K rwx-- [ anon ]
> E8000000 4096K rwx-- [ anon ]
> E8400000 4096K rwx-- [ anon ]
> E8800000 4096K rwx-- [ anon ]
> E8C00000 4096K rwx-- [ anon ]
> E9000000 4096K rwx-- [ anon ]
> E9400000 4096K rwx-- [ anon ]
> E9800000 4096K rwx-- [ anon ]
> E9C00000 4096K rwx-- [ anon ]
> EA000000 4096K rwx-- [ anon ]
> EA400000 4096K rwx-- [ anon ]
> EA800000 4096K rwx-- [ anon ]
> EAC00000 4096K rwx-- [ anon ]
> EB000000 4096K rwx-- [ anon ]
> EB400000 4096K rwx-- [ anon ]
> EB800000 4096K rwx-- [ anon ]
> EBC00000 4096K rwx-- [ anon ]
> EC000000 4096K rwx-- [ anon ]
> EC400000 4096K rwx-- [ anon ]
> EC800000 4096K rwx-- [ anon ]
> ECC00000 4096K rwx-- [ anon ]
> ED000000 4096K rwx-- [ anon ]
> ED400000 4096K rwx-- [ anon ]
> ED800000 77824K rwx-- [ anon ]
> F2400000 65536K ----R [ anon ]
> F6410000 8K r--s- dev:85,5 ino:168461
> F6420000 216K r--s- dev:85,5 ino:24576
> F6460000 8K r--s- dev:85,5 ino:24750
> F6470000 8K r--s- dev:85,5 ino:24747
> F6480000 336K r--s- dev:85,5 ino:23914
> F64E0000 88K r--s- dev:85,5 ino:23913
> F6500000 24K ----R [ anon ]
> F6506000 488K rw--R [ anon ]
> F6590000 32K r--s- dev:85,5 ino:24751
> F65A0000 88K r--s- dev:85,5 ino:9002
> F65C0000 200K r--s- dev:85,5 ino:7151
> F6600000 24K ----R [ anon ]
> F6606000 488K rw--R [ anon ]
> F6690000 16K r--s- dev:85,5 ino:7143
> F66A0000 72K r--s- dev:85,5 ino:7165
> F66C0000 16K r--s- dev:85,5 ino:7144
> F66D0000 80K r--s- dev:85,5 ino:7158
> F66F0000 16K r--s- dev:85,5 ino:7167
> F6700000 512K rw--R [ anon ]
> F6790000 416K r--s- dev:85,5 ino:7161
> F6800000 4096K rwx-- [ anon ]
> F6C00000 4096K rwx-- [ anon ]
> F7000000 4096K rwx-- [ anon ]
> F7400000 4096K rwx-- [ anon ]
> F7800000 4096K rwx-- [ anon ]
> F7C00000 4096K rwx-- [ anon ]
> F8000000 4096K rwx-- [ anon ]
> F8400000 4096K rwx-- [ anon ]
> F8800000 4096K rwx-- [ anon ]
> F8C00000 4096K rwx-- [ anon ]
> F9000000 4096K rwx-- [ anon ]
> F9400000 4096K rwx-- [ anon ]
> F9800000 16384K ----R [ anon ]
> FA810000 16K r--s- dev:85,5 ino:7159
> FA820000 368K r--s- dev:85,5 ino:211317
> FA880000 488K r--s- dev:85,5 ino:8285
> FA900000 512K rw--R [ anon ]
> FA990000 392K r--s- dev:85,5 ino:102341
> FAA00000 584K rwx-- [ anon ]
> FAAA0000 328K r--s- dev:85,5 ino:211325
> FAB00000 512K rw--R [ anon ]
> FAB90000 112K r--s- dev:85,5 ino:7179
> FABB0000 16K r--s- dev:85,5 ino:211316
> FABC0000 224K r--s- dev:85,5 ino:102332
> FAC00000 8640K r--s- dev:85,5 ino:25457
> FB480000 512K rw--R [ anon ]
> FB510000 64K r--s- dev:85,5 ino:8273
> FB530000 64K r--s- dev:85,5 ino:8276
> FB550000 168K r--s- dev:85,5 ino:8270
> FB580000 512K rw--R [ anon ]
> FB610000 48K r--s- dev:85,5 ino:7498
> FB620000 352K r--s- dev:85,5 ino:26726
> FB680000 24K rwx-- [ anon ]
> FB686000 8K rwx-- [ anon ]
> FB688000 8K rwx-- [ anon ]
> FB68A000 8K rwx-- [ anon ]
> FB68C000 8K rwx-- [ anon ]
> FB68E000 8K rwx-- [ anon ]
> FB690000 8K rwx-- [ anon ]
> FB692000 8K rwx-- [ anon ]
> FB694000 8K rwx-- [ anon ]
> FB696000 8K rwx-- [ anon ]
> FB698000 8K rwx-- [ anon ]
> FB69A000 8K rwx-- [ anon ]
> FB69C000 8K rwx-- [ anon ]
> FB69E000 8K rwx-- [ anon ]
> FB6A0000 8K rwx-- [ anon ]
> FB6A2000 8K rwx-- [ anon ]
> FB6A4000 8K rwx-- [ anon ]
> FB6A6000 8K rwx-- [ anon ]
> FB6A8000 8K rwx-- [ anon ]
> FB6AA000 8K rwx-- [ anon ]
> FB6AC000 8K rwx-- [ anon ]
> FB6AE000 8K rwx-- [ anon ]
> FB6B0000 8K rwx-- [ anon ]
> FB6B2000 8K rwx-- [ anon ]
> FB6B4000 8K rwx-- [ anon ]
> FB6B6000 8K rwx-- [ anon ]
> FB6B8000 8K rwx-- [ anon ]
> FB6BA000 8K rwx-- [ anon ]
> FB6BC000 8K rwx-- [ anon ]
> FB6BE000 8K rwx-- [ anon ]
> FB6C0000 8K rwx-- [ anon ]
> FB6C2000 8K rwx-- [ anon ]
> FB6C4000 8K rwx-- [ anon ]
> FB6C6000 8K rwx-- [ anon ]
> FB6C8000 8K rwx-- [ anon ]
> FB6CA000 8K rwx-- [ anon ]
> FB6CC000 8K rwx-- [ anon ]
> FB6CE000 8K rwx-- [ anon ]
> FB6D0000 8K rwx-- [ anon ]
> FB6D2000 8K rwx-- [ anon ]
> FB6D4000 8K rwx-- [ anon ]
> FB6D6000 8K rwx-- [ anon ]
> FB6D8000 8K rwx-- [ anon ]
> FB6DA000 8K rwx-- [ anon ]
> FB6DC000 8K rwx-- [ anon ]
> FB6DE000 8K rwx-- [ anon ]
> FB6E0000 8K rwx-- [ anon ]
> FB6E2000 8K rwx-- [ anon ]
> FB6E4000 8K rwx-- [ anon ]
> FB6E6000 8K rwx-- [ anon ]
> FB6E8000 8K rwx-- [ anon ]
> FB6EA000 8K rwx-- [ anon ]
> FB6EC000 8K rwx-- [ anon ]
> FB6EE000 8K rwx-- [ anon ]
> FB6F0000 8K rwx-- [ anon ]
> FB6F2000 8K rwx-- [ anon ]
> FB6F4000 8K rwx-- [ anon ]
> FB6F6000 8K rwx-- [ anon ]
> FB6F8000 8K rwx-- [ anon ]
> FB6FA000 8K rwx-- [ anon ]
> FB6FC000 8K rwx-- [ anon ]
> FB6FE000 8K rwx-- [ anon ]
> FB700000 8K rwx-- [ anon ]
> FB702000 8K rwx-- [ anon ]
> FB704000 8K rwx-- [ anon ]
> FB706000 8K rwx-- [ anon ]
> FB708000 8K rwx-- [ anon ]
> FB70A000 8K rwx-- [ anon ]
> FB70C000 8K rwx-- [ anon ]
> FB70E000 8K rwx-- [ anon ]
> FB710000 8K rwx-- [ anon ]
> FB712000 8K rwx-- [ anon ]
> FB714000 8K rwx-- [ anon ]
> FB716000 8K rwx-- [ anon ]
> FB718000 8K rwx-- [ anon ]
> FB71A000 8K rwx-- [ anon ]
> FB71C000 8K rwx-- [ anon ]
> FB71E000 8K rwx-- [ anon ]
> FB720000 8K rwx-- [ anon ]
> FB722000 8K rwx-- [ anon ]
> FB724000 8K rwx-- [ anon ]
> FB726000 8K rwx-- [ anon ]
> FB728000 8K rwx-- [ anon ]
> FB72A000 8K rwx-- [ anon ]
> FB72C000 8K rwx-- [ anon ]
> FB72E000 8K rwx-- [ anon ]
> FB730000 8K rwx-- [ anon ]
> FB732000 8K rwx-- [ anon ]
> FB734000 304K ----R [ anon ]
> FB790000 24K r--s- dev:85,5 ino:102340
> FB7A0000 48K r--s- dev:85,5 ino:8275
> FB7B0000 304K r--s- dev:85,5 ino:16889
> FB800000 39080K r--s- dev:85,5 ino:25497
> FDE30000 8K r--s- dev:85,5 ino:7155
> FDE40000 32K r--s- dev:85,5 ino:8274
> FDE50000 120K r--s- dev:85,5 ino:8269
> FDE80000 512K rw--R [ anon ]
> FDF10000 112K r--s- dev:85,5 ino:8271
> FDF30000 96K r--s- dev:85,5 ino:26727
> FDF50000 176K r--s- dev:85,5 ino:25470
> FDF80000 536K r--s- dev:85,5 ino:25490
> FE010000 88K r--s- dev:85,5 ino:26444
> FE030000 160K r--s- dev:85,5 ino:25469
> FE060000 24K r-x-- /usr/lib/nss_files.so.1
> FE076000 8K rwx-- /usr/lib/nss_files.so.1
> FE080000 504K r--s- dev:85,5 ino:24234
> FE110000 72K r-x--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libnet.so
> FE130000 8K rwx--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libnet.so
> FE140000 24K r-x--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libmanageme
> nt.so
> FE154000 8K rwx--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libmanageme
> nt.so
> FE160000 112K rw--- [ anon ]
> FE180000 1312K r--s- dev:85,5 ino:26635
> FE2D0000 16K r--s- dev:85,5 ino:20250
> FE2E0000 88K r--s- dev:85,5 ino:25489
> FE300000 1176K r--s- dev:85,5 ino:26633
> FE430000 8K rwx-- [ anon ]
> FE440000 192K r--s- dev:85,5 ino:26634
> FE480000 2528K r--s- dev:85,5 ino:26520
> FE700000 16K r--s- dev:85,5 ino:8280
> FE710000 192K r--s- dev:85,5 ino:26499
> FE750000 64K r-x--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libzip.so
> FE760000 8K rwx--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libzip.so
> FE770000 8K rwx-- [ anon ]
> FE780000 144K r-x--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libjava.so
> FE7B4000 8K rwx--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libjava.so
> FE7C0000 56K r-x--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libverify.s
> o
> FE7DE000 8K rwx--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/libverify.s
> o
> FE7F0000 8K r--s- dev:85,5 ino:25467
> FE800000 7984K r-x--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/server/libj
> vm.so
> FEFDC000 408K rwx--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/server/libj
> vm.so
> FF042000 56K rwx--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/server/libj
> vm.so
> FF060000 32K r-x--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/native_thre
> ads/libhpi.so
> FF078000 8K rwx--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/native_thre
> ads/libhpi.so
> FF07A000 8K rwx--
> /opt/p4/3rdparty/sun/java/sunos-sparc/1.5.0_13/jre/lib/sparc/native_thre
> ads/libhpi.so
> FF080000 568K r-x-- /usr/lib/libnsl.so.1
> FF11E000 40K rwx-- /usr/lib/libnsl.so.1
> FF128000 24K rwx-- /usr/lib/libnsl.so.1
> FF140000 32K rw-s- dev:0,2 ino:253268838
> FF150000 8K rwx-- [ anon ]
> FF160000 8K r---- [ anon ]
> FF170000 16K rw--- [ anon ]
> FF180000 16K r-x-- /usr/lib/libmp.so.2
> FF194000 8K rwx-- /usr/lib/libmp.so.2
> FF1A0000 8K rwx-- [ anon ]
> FF1B0000 224K r-x-- /usr/lib/libm.so.1
> FF1F6000 8K rwx-- /usr/lib/libm.so.1
> FF200000 48K r-x-- /usr/lib/libCrun.so.1
> FF21A000 8K rwx-- /usr/lib/libCrun.so.1
> FF21C000 16K rwx-- /usr/lib/libCrun.so.1
> FF230000 8K r-x-- /usr/lib/libsched.so.1
> FF242000 8K rwx-- /usr/lib/libsched.so.1
> FF250000 40K r-x-- /usr/lib/libsocket.so.1
> FF26A000 8K rwx-- /usr/lib/libsocket.so.1
> FF280000 688K r-x-- /usr/lib/libc.so.1
> FF33C000 24K rwx-- /usr/lib/libc.so.1
> FF342000 8K rwx-- /usr/lib/libc.so.1
> FF350000 8K rwxs- [ anon ]
> FF360000 16K rw--- [ anon ]
> FF370000 8K rwx-- [ anon ]
> FF380000 96K r-x-- /usr/lib/libthread.so.1
> FF3A8000 8K rwx-- /usr/lib/libthread.so.1
> FF3AA000 8K rwx-- /usr/lib/libthread.so.1
> FF3B0000 8K r-x-- /usr/platform/sun4u-us3/lib/libc_psr.so.1
> FF3C0000 192K r-x-- /usr/lib/ld.so.1
> FF3F0000 8K rwx-- /usr/lib/ld.so.1
> FF3F2000 8K rwx-- /usr/lib/ld.so.1
> FF3FA000 8K rwx-- /usr/lib/libdl.so.1
> FFB7C000 24K ----- [ anon ]
> FF400000 8192K rw--- [ stack ]
> total 3636576K
>
>
> -----Original Message-----
> From: Y Srinivas Ramakrishna [mailto:Y.S.Ramakrishna at Sun.COM]
> Sent: Tuesday, November 27, 2007 3:26 PM
> To: Anuj Lal
> Cc: Neo Jia; hotspot-gc-dev at openjdk.java.net
> Subject: Re: RE: OOM error
>
>
> pmap -rs will tell you what's taking up address space. It could be, for
> example,
> that you have lots of threads (so the space is gone to thread stacks,
> so
> you might
> try -Xss<...> or it could be that you have loaded lots of native
> libraries that have
> taken up address space, or it could be that you have lots of file
> descriptors or
> sockets or mmapped files or other native resources that are tying up
> your
> address space).
>
> (the reference to "swap space" in the message is usuallysomewhat
> misleading;
> i think there are plans to fix that message to be smarter / less
> misleading).
>
> -- ramki
>
> > We are on solaris 9.0 32 bit
> >
> > We aleady have "low" jvm heap. We don't want to go below this.
> >
> > I am pretty sure we are not running out of swap space
> >
> >
> >
> > This is what we have currently
> >
> > 8 processes: 46 sleeping, 2 on cpu
> >
> > CPU states: 95.2% idle, 3.9% user, 0.8% kernel, 0.0% iowait, 0.0%
> > swap
> >
> > Memory: 16G real, 12G free, 3510M swap in use, 18G swap free
> >
> >
> >
> > PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND
> >
> > 20139 weblogic 144 19 0 3389M 2362M cpu/2 87:19 5.16% java
> >
> >
> >
> > Anuj
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > -----Original Message-----
> > From: Y Srinivas Ramakrishna [mailto:Y.S.Ramakrishna at Sun.COM]
> > Sent: Tuesday, November 27, 2007 3:10 PM
> > To: Neo Jia
> > Cc: Anuj Lal; hotspot-gc-dev at openjdk.java.net
> > Subject: Re: OOM error
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Hi Anuj --
> >
> >
> >
> > > > Exception in thread "CompilerThread1" java.lang.OutOfMemoryError:
> > requested
> >
> > > > 32756 bytes for ChunkPool::allocate. Out of swap space?
> >
> >
> >
> > This is a request for 32 KB (not 32 MB as in the bug you quoted
> below).
> >
> >
> >
> > You should probably check pmap -rs on the process. You are almost
> > certainly out of
> >
> > C heap space (so, as you suggested below, reducing the Java heap might
> > help by
> >
> > freeing up your apparently scarce 32-bit virtual address space to the
> > c-heap).
> >
> > I note, though, that you have a rather modest (~2.6 GB?) heap, so on
> > Solaris
> >
> > for example, i'd expect plenty of free address space in the sbrk
> > segment. What
> >
> > platform are you on?
> >
> >
> >
> > -- ramki.
> >
> >
> >
> > > > We are already setting
> >
> > > >
> >
> > > > -XX:ReservedCodeCacheSize=64M
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > As per discussion in thread
> >
> > > >
> >
> > > > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5075468
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > I am sure we have enough heap as well as swap space available.
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > I am wondering above bug is fixed for 1.5.13?
> >
> > > >
> >
> > > > What is better option increasing -XX:ReservedCodeCacheSize or
> > decreasing
> >
> > > > heap size so that we have more non heap memory available?
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > Thanks
> >
> > > >
> >
> > > > ANuj
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > Jvm parameter
> >
> > > >
> >
> > > > with -Xms2560m -Xmx2560m -XX:MaxPermSize=256M
> >
> > > >
> >
> > > > -XX:+UseConcMarkSweepGC
> >
> > > >
> >
> > > > -XX:+UseParNewGC
> >
> > > >
> >
> > > > -XX:+CMSParallelRemarkEnabled
> >
> > > >
> >
> > > > -XX:+UseCMSCompactAtFullCollection
> >
> > > >
> >
> > > > -XX:CMSFullGCsBeforeCompac
> >
> > > >
> >
> > > > tion=0 -XX:+HandlePromotionFailure
> >
> > > >
> >
> > > > -XX:CMSInitiatingOccupancyFraction=68
> >
> > > >
> >
> > > > -XX:+UseCMSInitiatingOccupancyOnly -
> >
> > > >
> >
> > > > -XX:CMSMarkStackSize=32M
> >
> > > >
> >
> > > > XX:CodeCacheMinimumFreeSpace=2M
> >
> > > >
> >
> > > > -XX:ReservedCodeCacheSize=64M
> >
> > > >
> >
> > > > -XX:NewSize=512M -XX:MaxNewSize=51
> >
> > > >
> >
> > > > 2M -XX:MaxTenuringThreshold=6 -XX:SurvivorRatio=5
> > -XX:TargetSurvivorRatio=
> >
> > > >
> >
> > > > -XX:+CMSPermGenSweepingEnabled -XX:+CMSClassUnloadingEnabled
> >
> > > > -XX:CMSMaxAbortablePrecleanTime=1
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > wlstart:
> >
> > > >
> >
> > > > par new generation total 449408K, used 50959K [0x46400000,
> > 0x66400000,
> >
> > > > 0x66400000)
> >
> > > >
> >
> > > > eden space 374528K, 0% used [0x46400000, 0x46400000,
> 0x5d1c0000)
> >
> > > >
> >
> > > > from space 74880K, 68% used [0x61ae0000, 0x64ca3ed0,
> 0x66400000)
> >
> > > >
> >
> > > > to space 74880K, 0% used [0x5d1c0000, 0x5d1c0000,
> 0x61ae0000)
> >
> > > >
> >
> > > > concurrent mark-sweep generation total 2097152K, used 1071568K
> > [0x66400000,
> >
> > > > 0xe6400000, 0xe6400000)
> >
> > > >
> >
> > > > concurrent-mark-sweep perm gen total 211688K, used 127055K
> > [0xe6400000,
> >
> > > > 0xf32ba000, 0xf6400000)
> >
> > > >
> >
> > > > }
> >
> > > >
> >
> > > > , 0.5049826 secs]
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > Exception in thread "CompilerThread1" java.lang.OutOfMemoryError:
> > requested
> >
> > > > 32756 bytes for ChunkPool::allocate. Out of swap space?
> >
> > > >
> >
> > > > Java Result: 1
> >
> > >
> >
> > >
> >
> > >
> >
> > > --
> >
> > > I would remember that if researchers were not ambitious
> >
> > > probably today we haven't the technology we are using!
> >
>
More information about the hotspot-gc-dev
mailing list