From david.tavoularis at mycom-int.com Sat Jun 7 10:58:38 2008 From: david.tavoularis at mycom-int.com (David Tavoularis) Date: Sat, 07 Jun 2008 19:58:38 +0200 Subject: GCOverheadLimit does not work ? Message-ID: Hi, It seems that the mechanism GCOverheadLimit did not trigger, even if more than 98% of the total time is spent in garbage collection and less than 2% of the heap is recovered. The process spent 3 hours in this state, performing Full GC after Full GC. I am using Parallel Collector on Java6u4/Solaris9. Is it a bug or maybe I do not understand this feature ? "TimeStamp","GC duration","Duration between 2 FullGC" 57356.444 27.46 27.471 57383.915 27.14 27.137 57411.052 27.63 27.738 57438.79 33.72 33.724 57472.514 30.07 30.087 57502.601 27.84 27.842 57530.443 28.46 28.519 57558.962 29.03 29.032 57587.994 29.31 29.321 Full GC logs are available at http://people.via.ecp.fr/~tony/tmp/gc_200802282216.zip Thanks in advance -- David Tavoularis Command line : /usr/jdk/instances/jdk1.6.0/jre/bin/java -server -Xms3072m -Xmx3072m -XX:+UseParallelGC -XX:+AggressiveHeap -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -XX:+HeapDumpOnOutOfMemoryError ... Java version : "1.6.0_04" on Solaris 9 (Sparc) Java(TM) SE Runtime Environment (build 1.6.0_04-b12) Java HotSpot(TM) Server VM (build 10.0-b19, mixed mode) Extract of the GC logs : 57356.444: [Full GC [PSYoungGen: 262144K->260911K(317184K)] [PSOldGen: 1310719K->1310719K(1310720K)] 1572863K->1571631K(1627904K) [PSPermGen: 17242K->17241K(20480K)], 27.4570953 secs] [Times: user=27.46 sys=0.00, real=27.46 secs] Heap after GC invocations=836 (full 530): PSYoungGen total 317184K, used 260911K [0xcc000000, 0xfc000000, 0xfc000000) eden space 262144K, 99% used [0xcc000000,0xdbecbdd0,0xdc000000) from space 55040K, 0% used [0xf8a40000,0xf8a40000,0xfc000000) to space 262144K, 0% used [0xdc000000,0xdc000000,0xec000000) PSOldGen total 1310720K, used 1310719K [0x7c000000, 0xcc000000, 0xcc000000) object space 1310720K, 99% used [0x7c000000,0xcbfffff8,0xcc000000) PSPermGen total 20480K, used 17241K [0x78000000, 0x79400000, 0x7c000000) object space 20480K, 84% used [0x78000000,0x790d6618,0x79400000) } Total time for which application threads were stopped: 54.7390786 seconds Application time: 0.0001805 seconds Total time for which application threads were stopped: 0.0050949 seconds Application time: 0.0000762 seconds Total time for which application threads were stopped: 0.0015181 seconds Application time: 0.0006421 seconds {Heap before GC invocations=837 (full 531): PSYoungGen total 317184K, used 262144K [0xcc000000, 0xfc000000, 0xfc000000) eden space 262144K, 100% used [0xcc000000,0xdc000000,0xdc000000) from space 55040K, 0% used [0xf8a40000,0xf8a40000,0xfc000000) to space 262144K, 0% used [0xdc000000,0xdc000000,0xec000000) PSOldGen total 1310720K, used 1310719K [0x7c000000, 0xcc000000, 0xcc000000) object space 1310720K, 99% used [0x7c000000,0xcbfffff8,0xcc000000) PSPermGen total 20480K, used 17243K [0x78000000, 0x79400000, 0x7c000000) object space 20480K, 84% used [0x78000000,0x790d6da8,0x79400000) 57383.915: [Full GC [PSYoungGen: 262144K->262144K(317184K)] [PSOldGen: 1310719K->1310719K(1310720K)] 1572863K->1572863K(1627904K) [PSPermGen: 17243K->17243K(20480K)], 27.1362406 secs] [Times: user=27.14 sys=0.00, real=27.14 secs] Heap after GC invocations=837 (full 531): PSYoungGen total 317184K, used 262144K [0xcc000000, 0xfc000000, 0xfc000000) eden space 262144K, 100% used [0xcc000000,0xdc000000,0xdc000000) from space 55040K, 0% used [0xf8a40000,0xf8a40000,0xfc000000) to space 262144K, 0% used [0xdc000000,0xdc000000,0xec000000) PSOldGen total 1310720K, used 1310719K [0x7c000000, 0xcc000000, 0xcc000000) object space 1310720K, 99% used [0x7c000000,0xcbfffff8,0xcc000000) PSPermGen total 20480K, used 17243K [0x78000000, 0x79400000, 0x7c000000) object space 20480K, 84% used [0x78000000,0x790d6da8,0x79400000) } {Heap before GC invocations=838 (full 532): PSYoungGen total 317184K, used 262144K [0xcc000000, 0xfc000000, 0xfc000000) eden space 262144K, 100% used [0xcc000000,0xdc000000,0xdc000000) from space 55040K, 0% used [0xf8a40000,0xf8a40000,0xfc000000) to space 262144K, 0% used [0xdc000000,0xdc000000,0xec000000) PSOldGen total 1310720K, used 1310719K [0x7c000000, 0xcc000000, 0xcc000000) object space 1310720K, 99% used [0x7c000000,0xcbfffff8,0xcc000000) PSPermGen total 20480K, used 17243K [0x78000000, 0x79400000, 0x7c000000) object space 20480K, 84% used [0x78000000,0x790d6da8,0x79400000) 57411.052: [Full GC [PSYoungGen: 262144K->260919K(317184K)] [PSOldGen: 1310719K->1310719K(1310720K)] 1572863K->1571639K(1627904K) [PSPermGen: 17243K->17241K(20480K)], 27.6282006 secs] [Times: user=27.63 sys=0.00, real=27.63 secs] Heap after GC invocations=838 (full 532): PSYoungGen total 317184K, used 260919K [0xcc000000, 0xfc000000, 0xfc000000) eden space 262144K, 99% used [0xcc000000,0xdbecddc8,0xdc000000) from space 55040K, 0% used [0xf8a40000,0xf8a40000,0xfc000000) to space 262144K, 0% used [0xdc000000,0xdc000000,0xec000000) PSOldGen total 1310720K, used 1310719K [0x7c000000, 0xcc000000, 0xcc000000) object space 1310720K, 99% used [0x7c000000,0xcbfffff8,0xcc000000) PSPermGen total 20480K, used 17241K [0x78000000, 0x79400000, 0x7c000000) object space 20480K, 84% used [0x78000000,0x790d6658,0x79400000) } Total time for which application threads were stopped: 54.8192590 seconds Application time: 0.0001949 seconds Total time for which application threads were stopped: 0.0048801 seconds Application time: 0.0000836 seconds Total time for which application threads were stopped: 0.0011786 seconds Application time: 0.0007086 seconds {Heap before GC invocations=839 (full 533): PSYoungGen total 317184K, used 262144K [0xcc000000, 0xfc000000, 0xfc000000) eden space 262144K, 100% used [0xcc000000,0xdc000000,0xdc000000) from space 55040K, 0% used [0xf8a40000,0xf8a40000,0xfc000000) to space 262144K, 0% used [0xdc000000,0xdc000000,0xec000000) PSOldGen total 1310720K, used 1310719K [0x7c000000, 0xcc000000, 0xcc000000) object space 1310720K, 99% used [0x7c000000,0xcbfffff8,0xcc000000) PSPermGen total 20480K, used 17241K [0x78000000, 0x79400000, 0x7c000000) object space 20480K, 84% used [0x78000000,0x790d67f0,0x79400000) 57438.790: [Full GC [PSYoungGen: 262144K->262144K(317184K)] [PSOldGen: 1310719K->1310719K(1310720K)] 1572863K->1572863K(1627904K) [PSPermGen: 17241K->17241K(20480K)], 33.7237730 secs] [Times: user=33.29 sys=0.00, real=33.72 secs] Heap after GC invocations=839 (full 533): PSYoungGen total 317184K, used 262144K [0xcc000000, 0xfc000000, 0xfc000000) eden space 262144K, 100% used [0xcc000000,0xdc000000,0xdc000000) from space 55040K, 0% used [0xf8a40000,0xf8a40000,0xfc000000) to space 262144K, 0% used [0xdc000000,0xdc000000,0xec000000) PSOldGen total 1310720K, used 1310719K [0x7c000000, 0xcc000000, 0xcc000000) object space 1310720K, 99% used [0x7c000000,0xcbfffff8,0xcc000000) PSPermGen total 20480K, used 17241K [0x78000000, 0x79400000, 0x7c000000) object space 20480K, 84% used [0x78000000,0x790d67f0,0x79400000) } {Heap before GC invocations=840 (full 534): PSYoungGen total 317184K, used 262144K [0xcc000000, 0xfc000000, 0xfc000000) eden space 262144K, 100% used [0xcc000000,0xdc000000,0xdc000000) from space 55040K, 0% used [0xf8a40000,0xf8a40000,0xfc000000) to space 262144K, 0% used [0xdc000000,0xdc000000,0xec000000) PSOldGen total 1310720K, used 1310719K [0x7c000000, 0xcc000000, 0xcc000000) object space 1310720K, 99% used [0x7c000000,0xcbfffff8,0xcc000000) PSPermGen total 20480K, used 17241K [0x78000000, 0x79400000, 0x7c000000) object space 20480K, 84% used [0x78000000,0x790d67f0,0x79400000) 57472.514: [Full GC [PSYoungGen: 262144K->260951K(317184K)] [PSOldGen: 1310719K->1310719K(1310720K)] 1572863K->1571671K(1627904K) [PSPermGen: 17241K->17241K(20480K)], 30.0724968 secs] [Times: user=29.98 sys=0.01, real=30.07 secs] Heap after GC invocations=840 (full 534): PSYoungGen total 317184K, used 260951K [0xcc000000, 0xfc000000, 0xfc000000) eden space 262144K, 99% used [0xcc000000,0xdbed5e18,0xdc000000) from space 55040K, 0% used [0xf8a40000,0xf8a40000,0xfc000000) to space 262144K, 0% used [0xdc000000,0xdc000000,0xec000000) PSOldGen total 1310720K, used 1310719K [0x7c000000, 0xcc000000, 0xcc000000) object space 1310720K, 99% used [0x7c000000,0xcbfffff8,0xcc000000) PSPermGen total 20480K, used 17241K [0x78000000, 0x79400000, 0x7c000000) object space 20480K, 84% used [0x78000000,0x790d67f0,0x79400000) } Total time for which application threads were stopped: 63.8515774 seconds Application time: 0.0002396 seconds Total time for which application threads were stopped: 0.0055494 seconds Application time: 0.0000822 seconds Total time for which application threads were stopped: 0.0013594 seconds Application time: 0.0010679 seconds {Heap before GC invocations=841 (full 535): PSYoungGen total 317184K, used 262144K [0xcc000000, 0xfc000000, 0xfc000000) eden space 262144K, 100% used [0xcc000000,0xdc000000,0xdc000000) from space 55040K, 0% used [0xf8a40000,0xf8a40000,0xfc000000) to space 262144K, 0% used [0xdc000000,0xdc000000,0xec000000) PSOldGen total 1310720K, used 1310719K [0x7c000000, 0xcc000000, 0xcc000000) object space 1310720K, 99% used [0x7c000000,0xcbfffff8,0xcc000000) PSPermGen total 20480K, used 17242K [0x78000000, 0x79400000, 0x7c000000) object space 20480K, 84% used [0x78000000,0x790d6858,0x79400000) 57502.601: [Full GC [PSYoungGen: 262144K->262144K(317184K)] [PSOldGen: 1310719K->1310719K(1310720K)] 1572863K->1572863K(1627904K) [PSPermGen: 17242K->17242K(20480K)], 27.8416247 secs] [Times: user=27.84 sys=0.00, real=27.84 secs] Heap after GC invocations=841 (full 535): PSYoungGen total 317184K, used 262144K [0xcc000000, 0xfc000000, 0xfc000000) eden space 262144K, 100% used [0xcc000000,0xdc000000,0xdc000000) from space 55040K, 0% used [0xf8a40000,0xf8a40000,0xfc000000) to space 262144K, 0% used [0xdc000000,0xdc000000,0xec000000) PSOldGen total 1310720K, used 1310719K [0x7c000000, 0xcc000000, 0xcc000000) object space 1310720K, 99% used [0x7c000000,0xcbfffff8,0xcc000000) PSPermGen total 20480K, used 17242K [0x78000000, 0x79400000, 0x7c000000) object space 20480K, 84% used [0x78000000,0x790d6858,0x79400000) } {Heap before GC invocations=842 (full 536): PSYoungGen total 317184K, used 262144K [0xcc000000, 0xfc000000, 0xfc000000) eden space 262144K, 100% used [0xcc000000,0xdc000000,0xdc000000) from space 55040K, 0% used [0xf8a40000,0xf8a40000,0xfc000000) to space 262144K, 0% used [0xdc000000,0xdc000000,0xec000000) PSOldGen total 1310720K, used 1310719K [0x7c000000, 0xcc000000, 0xcc000000) object space 1310720K, 99% used [0x7c000000,0xcbfffff8,0xcc000000) PSPermGen total 20480K, used 17242K [0x78000000, 0x79400000, 0x7c000000) object space 20480K, 84% used [0x78000000,0x790d6858,0x79400000) 57530.443: [Full GC [PSYoungGen: 262144K->260976K(317184K)] [PSOldGen: 1310719K->1310719K(1310720K)] 1572863K->1571696K(1627904K) [PSPermGen: 17242K->17242K(20480K)], 28.4587198 secs] [Times: user=28.22 sys=0.00, real=28.46 secs] Heap after GC invocations=842 (full 536): PSYoungGen total 317184K, used 260976K [0xcc000000, 0xfc000000, 0xfc000000) eden space 262144K, 99% used [0xcc000000,0xdbedc060,0xdc000000) from space 55040K, 0% used [0xf8a40000,0xf8a40000,0xfc000000) to space 262144K, 0% used [0xdc000000,0xdc000000,0xec000000) PSOldGen total 1310720K, used 1310719K [0x7c000000, 0xcc000000, 0xcc000000) object space 1310720K, 99% used [0x7c000000,0xcbfffff8,0xcc000000) PSPermGen total 20480K, used 17242K [0x78000000, 0x79400000, 0x7c000000) object space 20480K, 84% used [0x78000000,0x790d6858,0x79400000) } Total time for which application threads were stopped: 56.3591418 seconds Application time: 0.0002362 seconds {Heap before GC invocations=843 (full 537): PSYoungGen total 317184K, used 262144K [0xcc000000, 0xfc000000, 0xfc000000) eden space 262144K, 100% used [0xcc000000,0xdc000000,0xdc000000) from space 55040K, 0% used [0xf8a40000,0xf8a40000,0xfc000000) to space 262144K, 0% used [0xdc000000,0xdc000000,0xec000000) PSOldGen total 1310720K, used 1310719K [0x7c000000, 0xcc000000, 0xcc000000) object space 1310720K, 99% used [0x7c000000,0xcbfffff8,0xcc000000) PSPermGen total 20480K, used 17242K [0x78000000, 0x79400000, 0x7c000000) object space 20480K, 84% used [0x78000000,0x790d68c8,0x79400000) 57558.962: [Full GC [PSYoungGen: 262144K->262144K(317184K)] [PSOldGen: 1310719K->1310719K(1310720K)] 1572863K->1572863K(1627904K) [PSPermGen: 17242K->17242K(20480K)], 29.0315209 secs] [Times: user=28.92 sys=0.00, real=29.03 secs] Heap after GC invocations=843 (full 537): PSYoungGen total 317184K, used 262144K [0xcc000000, 0xfc000000, 0xfc000000) eden space 262144K, 100% used [0xcc000000,0xdc000000,0xdc000000) from space 55040K, 0% used [0xf8a40000,0xf8a40000,0xfc000000) to space 262144K, 0% used [0xdc000000,0xdc000000,0xec000000) PSOldGen total 1310720K, used 1310719K [0x7c000000, 0xcc000000, 0xcc000000) object space 1310720K, 99% used [0x7c000000,0xcbfffff8,0xcc000000) PSPermGen total 20480K, used 17242K [0x78000000, 0x79400000, 0x7c000000) object space 20480K, 84% used [0x78000000,0x790d68c8,0x79400000) } {Heap before GC invocations=844 (full 538): PSYoungGen total 317184K, used 262144K [0xcc000000, 0xfc000000, 0xfc000000) eden space 262144K, 100% used [0xcc000000,0xdc000000,0xdc000000) from space 55040K, 0% used [0xf8a40000,0xf8a40000,0xfc000000) to space 262144K, 0% used [0xdc000000,0xdc000000,0xec000000) PSOldGen total 1310720K, used 1310719K [0x7c000000, 0xcc000000, 0xcc000000) object space 1310720K, 99% used [0x7c000000,0xcbfffff8,0xcc000000) PSPermGen total 20480K, used 17242K [0x78000000, 0x79400000, 0x7c000000) object space 20480K, 84% used [0x78000000,0x790d68c8,0x79400000) 57587.994: [Full GC [PSYoungGen: 262144K->260978K(317184K)] [PSOldGen: 1310719K->1310719K(1310720K)] 1572863K->1571698K(1627904K) [PSPermGen: 17242K->17242K(20480K)], 29.3142086 secs] [Times: user=29.29 sys=0.00, real=29.31 secs] Heap after GC invocations=844 (full 538): PSYoungGen total 317184K, used 260978K [0xcc000000, 0xfc000000, 0xfc000000) eden space 262144K, 99% used [0xcc000000,0xdbedc888,0xdc000000) from space 55040K, 0% used [0xf8a40000,0xf8a40000,0xfc000000) to space 262144K, 0% used [0xdc000000,0xdc000000,0xec000000) PSOldGen total 1310720K, used 1310719K [0x7c000000, 0xcc000000, 0xcc000000) object space 1310720K, 99% used [0x7c000000,0xcbfffff8,0xcc000000) PSPermGen total 20480K, used 17242K [0x78000000, 0x79400000, 0x7c000000) object space 20480K, 84% used [0x78000000,0x790d68c8,0x79400000) } Total time for which application threads were stopped: 58.3541271 seconds From Jon.Masamitsu at Sun.COM Mon Jun 9 09:08:27 2008 From: Jon.Masamitsu at Sun.COM (Jon Masamitsu) Date: Mon, 09 Jun 2008 09:08:27 -0700 Subject: GCOverheadLimit does not work ? In-Reply-To: References: Message-ID: <484D557B.1060201@Sun.COM> David, There is a bug somewhere here that relates to the GCOverheadLimit. I think the problem is that the survivor spaces are not being used and the survivor space is large enough so that the calculation of the amount of free space in the heap is > 2%. The survivor spaces are not being used because only full collections are being done later in the run and the survivor spaces are used for minor (young generation) collections. Though the description of the GCOverheadLimit refers to the amount of the heap collected, it really should just say that amount of free space in the heap. That's a bug in the specification for the parameter. I'm not sure that an out-of-memory should be thrown if there is 2% of the heap free - that's in keeping with trying to minimize the false positives resulting from this feature. Jon David Tavoularis wrote: > Hi, > > It seems that the mechanism GCOverheadLimit did not trigger, even if more than 98% of the total time is spent in garbage collection and less than 2% of the heap is recovered. The process spent 3 hours in this state, performing Full GC after Full GC. > > I am using Parallel Collector on Java6u4/Solaris9. Is it a bug or maybe I do not understand this feature ? > > "TimeStamp","GC duration","Duration between 2 FullGC" > 57356.444 27.46 27.471 > 57383.915 27.14 27.137 > 57411.052 27.63 27.738 > 57438.79 33.72 33.724 > 57472.514 30.07 30.087 > 57502.601 27.84 27.842 > 57530.443 28.46 28.519 > 57558.962 29.03 29.032 > 57587.994 29.31 29.321 > > Full GC logs are available at http://people.via.ecp.fr/~tony/tmp/gc_200802282216.zip > > Thanks in advance > From david.tavoularis at mycom-int.com Mon Jun 9 09:51:32 2008 From: david.tavoularis at mycom-int.com (David Tavoularis) Date: Mon, 09 Jun 2008 18:51:32 +0200 Subject: GCOverheadLimit does not work ? In-Reply-To: <484D557B.1060201@Sun.COM> References: <484D557B.1060201@Sun.COM> Message-ID: Hi Jon, I would like to understand the computation of the percentage for the following Full GC example : 57587.994: [Full GC [PSYoungGen: 262144K->260978K(317184K)] [PSOldGen: 1310719K->1310719K(1310720K)] 1572863K->1571698K(1627904K) [PSPermGen: 17242K->17242K(20480K)] Heap after GC : PSYoungGen total 317184K, used 260978K (eden=262144K, from=OK (out of 55040K)) PSOldGen total 1310720K, used 1310719K PSPermGen total 20480K, used 17242K So the percentage is computed like this : ((317184K-260978K)+(1310720K-1310719K)+(20480K-17242K))/(317184K+1310720K+20480K)=59445K/1648384K=3.6% > 2% The fix would be to compute it like this : ((262144K-260978K)+(1310720K-1310719K)+(20480K-17242K))/(317184K+1310720K+20480K)=4405K/1648384K=0.3% < 2% Is there a WA to this issue ? 1. Is is possible to change by property the percentage from 2% to 5% ? 2. Maybe I could reduce the young size from 256MB to 128MB, and it will automatically reduce (?) the survivor from 55040K to 27520K ? And the percentage would be in this case 1.9% and the OOM would be triggered. Thanks in advance for your explanations -- David On Mon, 09 Jun 2008 18:08:27 +0200, Jon Masamitsu wrote: > David, > > There is a bug somewhere here that relates to the GCOverheadLimit. > I think the problem is that the survivor spaces are not being > used and the survivor space is large enough so that the > calculation of the amount of free space in the heap is > 2%. > The survivor spaces are not being used because only full > collections are being done later in the run and the survivor > spaces are used for minor (young generation) collections. Though > the description of the GCOverheadLimit refers to the amount > of the heap collected, it really should just say that amount of > free space in the heap. That's a bug in the specification for > the parameter. I'm not sure that an out-of-memory should > be thrown if there is 2% of the heap free - that's in keeping > with trying to minimize the false positives resulting from this > feature. > > Jon > > David Tavoularis wrote: >> Hi, >> >> It seems that the mechanism GCOverheadLimit did not trigger, even if more than 98% of the total time is spent in garbage collection and less than 2% of the heap is recovered. The process spent 3 hours in this state, performing Full GC after Full GC. >> >> I am using Parallel Collector on Java6u4/Solaris9. Is it a bug or maybe I do not understand this feature ? >> >> "TimeStamp","GC duration","Duration between 2 FullGC" >> 57356.444 27.46 27.471 >> 57383.915 27.14 27.137 >> 57411.052 27.63 27.738 >> 57438.79 33.72 33.724 >> 57472.514 30.07 30.087 >> 57502.601 27.84 27.842 >> 57530.443 28.46 28.519 >> 57558.962 29.03 29.032 >> 57587.994 29.31 29.321 >> >> Full GC logs are available at http://people.via.ecp.fr/~tony/tmp/gc_200802282216.zip >> >> Thanks in advance >> > _______________________________________________ > hotspot-gc-use mailing list > hotspot-gc-use at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use > From Peter.Kessler at Sun.COM Mon Jun 9 11:28:45 2008 From: Peter.Kessler at Sun.COM (Peter B. Kessler) Date: Mon, 09 Jun 2008 11:28:45 -0700 Subject: GCOverheadLimit does not work ? In-Reply-To: References: <484D557B.1060201@Sun.COM> Message-ID: <484D765D.2000500@Sun.COM> There's product(uintx, GCHeapFreeLimit, 2, \ "Minimum percentage of free space after a full GC before an " \ "OutOfMemoryError is thrown (used with GCTimeLimit)") \ But I have to say that I've never gotten it to behave the way I thought it worked. ... peter David Tavoularis wrote: > Hi Jon, > > I would like to understand the computation of the percentage for the following Full GC example : > > 57587.994: [Full GC [PSYoungGen: 262144K->260978K(317184K)] [PSOldGen: 1310719K->1310719K(1310720K)] 1572863K->1571698K(1627904K) [PSPermGen: 17242K->17242K(20480K)] > Heap after GC : > PSYoungGen total 317184K, used 260978K (eden=262144K, from=OK (out of 55040K)) > PSOldGen total 1310720K, used 1310719K > PSPermGen total 20480K, used 17242K > > So the percentage is computed like this : ((317184K-260978K)+(1310720K-1310719K)+(20480K-17242K))/(317184K+1310720K+20480K)=59445K/1648384K=3.6% > 2% > > The fix would be to compute it like this : ((262144K-260978K)+(1310720K-1310719K)+(20480K-17242K))/(317184K+1310720K+20480K)=4405K/1648384K=0.3% < 2% > > > > Is there a WA to this issue ? > > 1. Is is possible to change by property the percentage from 2% to 5% ? > > 2. Maybe I could reduce the young size from 256MB to 128MB, and it will automatically reduce (?) the survivor from 55040K to 27520K ? And the percentage would be in this case 1.9% and the OOM would be triggered. > > > Thanks in advance for your explanations > From Jon.Masamitsu at Sun.COM Mon Jun 9 11:55:07 2008 From: Jon.Masamitsu at Sun.COM (Jon Masamitsu) Date: Mon, 09 Jun 2008 11:55:07 -0700 Subject: GCOverheadLimit does not work ? In-Reply-To: References: <484D557B.1060201@Sun.COM> Message-ID: <484D7C8B.6070301@Sun.COM> David Tavoularis wrote: > Hi Jon, > > I would like to understand the computation of the percentage for the following Full GC example : > > 57587.994: [Full GC [PSYoungGen: 262144K->260978K(317184K)] [PSOldGen: 1310719K->1310719K(1310720K)] 1572863K->1571698K(1627904K) [PSPermGen: 17242K->17242K(20480K)] > Heap after GC : > PSYoungGen total 317184K, used 260978K (eden=262144K, from=OK (out of 55040K)) > PSOldGen total 1310720K, used 1310719K > PSPermGen total 20480K, used 17242K > > So the percentage is computed like this : ((317184K-260978K)+(1310720K-1310719K)+(20480K-17242K))/(317184K+1310720K+20480K)=59445K/1648384K=3.6% > 2% > If you want a definitive answer, please take a look at the sources. As I recall it, the perm gen is not counted so it's more like ((317184K-260978K)+(1310720K-1310719K)) / (317184K+1310720K) = 3.5% > The fix would be to compute it like this : ((262144K-260978K)+(1310720K-1310719K)+(20480K-17242K))/(317184K+1310720K+20480K)=4405K/1648384K=0.3% < 2% > I don't think that is always the right answer. If the survivor space is being used (i.e., objects are copied to it and collected from it), then it should be counted. In a young gen collection I would expect it to be counted all the time but there might be exceptions (e.g., an always tenure policy). When it's a full collection if it was being used it should be counted. I would expect it to be used when there are young gen collections occurring between full collections. If the collections are back-to-back full collections and the survivor space is not being used it might make sense to not count it. Sorry, it's just not clear to me. Counting it sometimes and not counting it others might surprise some users. > > > Is there a WA to this issue ? WA? > > 1. Is is possible to change by property the percentage from 2% to 5% ? -XX:GCHeapFreeLimit=5 > > 2. Maybe I could reduce the young size from 256MB to 128MB, and it will automatically reduce (?) the survivor from 55040K to 27520K ? And the percentage would be in this case 1.9% and the OOM would be triggered. > That might work but what is your goal? This feature is to just alert unsuspecting users to a very bad GC situation. You already know you have a problem here, right? From aaisinzon at guidewire.com Thu Jun 19 11:30:33 2008 From: aaisinzon at guidewire.com (Alex Aisinzon) Date: Thu, 19 Jun 2008 11:30:33 -0700 Subject: JVM options to get full visibility on generation behavior and GC log analysis tools. Message-ID: <545E8962B7529546962672A564039F990DB06F18@exchange.guidewire.com> All I spend much time analyzing the behavior of the various generations. Their optimal sizing provides good performance improvements. I have several questions: * I add the following parameters to get good GC logs: "-XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCApplicationStoppedTime". I wonder if some additional options are useful. By example, the timestamps only track time since the JVM was started. To correlate an incident with application time stamps, it would be much more useful to have those time stamps use regular dates (2008-06-17 11:58:11,23 by example). * I have not found a good tool to analyze these logs and have to do that analysis by hand. HPJTune (HP) and GCViewer (TagTraum) are sometimes good enough but does not work when using more refined GC options like Parallel New Garbage Collection.. PrintGCStats is a last one that I have not found very useful either. Are there other tools available that someone could recommend? In comparison, the IBM JDK provides a single GC output format by using the verbose:gc option. IBM provides some very good tools to analyze those logs. Thanks in advance for helping tuning the Sun JDK as well as possible. This is for Sun JDK 1.5. Alex Aisinzon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20080619/9c376d02/attachment.html From rainer.jung at kippdata.de Thu Jun 19 13:19:00 2008 From: rainer.jung at kippdata.de (Rainer Jung) Date: Thu, 19 Jun 2008 22:19:00 +0200 Subject: JVM options to get full visibility on generation behavior and GC loganalysis tools. In-Reply-To: <545E8962B7529546962672A564039F990DB06F18@exchange.guidewire.com> References: <545E8962B7529546962672A564039F990DB06F18@exchange.guidewire.com> Message-ID: <485ABF34.5000303@kippdata.de> Alex Aisinzon schrieb: > I have several questions: > > * I add the following parameters to get good GC logs: > ?-XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC > -XX:+PrintGCApplicationConcurrentTime > -XX:+PrintGCApplicationStoppedTime". I wonder if some additional > options are useful. By example, the timestamps only track time > since the JVM was started. To correlate an incident with > application time stamps, it would be much more useful to have > those time stamps use regular dates (2008-06-17 11:58:11,23 by > example). Also useful: -XX:+PrintTenuringDistribution Timestamps: yes unfortunately until lately there were no absolute timestamps available, only relative to JVM start. In 1.6.0_04 the new flag -XX:+PrintGCDateStamps has been implemented. See http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6517301 and http://java.sun.com/javase/6/webnotes/ReleaseNotes.html (look for the bug id). > * I have not found a good tool to analyze these logs and have to do > that analysis by hand. HPJTune (HP) and GCViewer (TagTraum) are > sometimes good enough but does not work when using more refined GC > options like Parallel New Garbage Collection.. PrintGCStats is a > last one that I have not found very useful either. Are there other > tools available that someone could recommend? The same question form here: if someone did the work of writing a parser for those human readable but not really machine usable logs which would work with various flag combinations of verbosity, GC algorithms and JVM versions that would be really great. By the way: some things are unfortunately logically missing in the log files, like HeapAtGC at the end of a CMS. You can only get the status for the memory regions when the next minor GC runs, but not at the end of a CMS :( > In comparison, the IBM JDK provides a single GC output format by using > the verbose:gc option. IBM provides some very good tools to analyze > those logs. Regards, Rainer From Jon.Masamitsu at Sun.COM Mon Jun 23 07:00:39 2008 From: Jon.Masamitsu at Sun.COM (Jon Masamitsu) Date: Mon, 23 Jun 2008 07:00:39 -0700 Subject: JVM options to get full visibility on generation behavior and GC loganalysis tools. In-Reply-To: <485ABF34.5000303@kippdata.de> References: <545E8962B7529546962672A564039F990DB06F18@exchange.guidewire.com> <485ABF34.5000303@kippdata.de> Message-ID: <485FAC87.1070500@Sun.COM> You can find source for a tool called GChisto at https://gchisto.dev.java.net/ It's open source. It parses PrintGCDetails output but if you turn on flags which cause printing to be interspersed with the PrintGCDetails output, it may or may not work. Regarding PrintHeapAtGC output at the end of a CMS collection, it's not there probably as a matter of the format of the output. The PrintHeapAtGC includes the "top" of a generation where "top" is the next space in a generation to be allocated. CMS is a non moving collector (i.e., it maintains its free space in free lists) so there is in general no "top". Rainer Jung wrote: > Alex Aisinzon schrieb: >> I have several questions: >> >> * I add the following parameters to get good GC logs: >> ?-XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC >> -XX:+PrintGCApplicationConcurrentTime >> -XX:+PrintGCApplicationStoppedTime". I wonder if some additional >> options are useful. By example, the timestamps only track time >> since the JVM was started. To correlate an incident with >> application time stamps, it would be much more useful to have >> those time stamps use regular dates (2008-06-17 11:58:11,23 by >> example). > > Also useful: -XX:+PrintTenuringDistribution > Timestamps: yes unfortunately until lately there were no absolute > timestamps available, only relative to JVM start. In 1.6.0_04 the new > flag -XX:+PrintGCDateStamps has been implemented. See > > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6517301 > and > http://java.sun.com/javase/6/webnotes/ReleaseNotes.html (look for the > bug id). > >> * I have not found a good tool to analyze these logs and have to do >> that analysis by hand. HPJTune (HP) and GCViewer (TagTraum) are >> sometimes good enough but does not work when using more refined GC >> options like Parallel New Garbage Collection.. PrintGCStats is a >> last one that I have not found very useful either. Are there other >> tools available that someone could recommend? > > The same question form here: if someone did the work of writing a parser > for those human readable but not really machine usable logs which would > work with various flag combinations of verbosity, GC algorithms and JVM > versions that would be really great. > > By the way: some things are unfortunately logically missing in the log > files, like HeapAtGC at the end of a CMS. You can only get the status > for the memory regions when the next minor GC runs, but not at the end > of a CMS :( > >> In comparison, the IBM JDK provides a single GC output format by using >> the verbose:gc option. IBM provides some very good tools to analyze >> those logs. > > Regards, > > Rainer > > _______________________________________________ > hotspot-gc-use mailing list > hotspot-gc-use at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use