From Sujit.Das at cognizant.com Mon Aug 3 10:18:39 2009 From: Sujit.Das at cognizant.com (Sujit.Das at cognizant.com) Date: Mon, 3 Aug 2009 22:48:39 +0530 Subject: Direct promotion to old gen Message-ID: <19B27FD5AF2EAA49A66F787911CF519596D00C@CTSINCHNSXUU.cts.com> Hi All, I am trying to capture statistics to denote direct promotion from young gen to old gen. As per my understanding, minor collection will empty eden space and put surviving objects into 'To' survivor space. Objects that have survived more than Max Tenuring Threshold value will get promoted. Example: If max Tenuring Threshold is 4, then in age 4 size that gets printed in verbose GC output will be the size that should get promoted in old gen as part of next minor collection. Anything more than previous age4 size is result of direct promotion from eden space to old gen. Please confirm my understanding / approach. If above understanding is correct, I plan to get plot of (Promo from current minor collection - Age4 from previous minor collection) The values will help me to determine whether I need to tune my eden and survivor spaces. Your help much appreciated. Thanks, Sujit This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email or any action taken in reliance on this e-mail is strictly prohibited and may be unlawful. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20090803/6a9faec3/attachment.html From Y.S.Ramakrishna at Sun.COM Mon Aug 3 10:49:36 2009 From: Y.S.Ramakrishna at Sun.COM (Y.S.Ramakrishna at Sun.COM) Date: Mon, 03 Aug 2009 10:49:36 -0700 Subject: Direct promotion to old gen In-Reply-To: <19B27FD5AF2EAA49A66F787911CF519596D00C@CTSINCHNSXUU.cts.com> References: <19B27FD5AF2EAA49A66F787911CF519596D00C@CTSINCHNSXUU.cts.com> Message-ID: <4A772330.8020509@Sun.COM> Hi Sujit -- On 08/ 3/09 10:18 AM, Sujit.Das at cognizant.com wrote: > Hi All, > > I am trying to capture statistics to denote direct promotion from young > gen to old gen. > > As per my understanding, minor collection will empty eden space and put > surviving objects into 'To' survivor space. Objects that have survived > more than Max Tenuring Threshold value will get promoted. Example: If > max Tenuring Threshold is 4, then in age 4 size that gets printed in > verbose GC output will be the size that should get promoted in old gen > as part of next minor collection. Anything more than previous age4 size > is result of direct promotion from eden space to old gen. Your description is partially correct, in that you cannot have _more_ than the # in the previous age=4 cohort promoting into the old gen unless survivor spaces overflowed for some reason. But you may have fewer because some of them died before this GC. Thus, you cannot accurately get the "direct promotion size" (i.e. those below tenuring threshold age that got prematurely promoted to old gen) by simply subtracting the previous age=4 cohort from the total promoted size, for the simple reason that it is possible that only a small subset of the previous age=4 cohort survive a fifth GC (in fact for a well-tuned heap, you would expect that to be usually the case). Thus, more often than not, total promoted - (age=4 cohort) in yr example will be a negative number, in other words, there is a good chance you will be underestimating or at least measuring somewhat incorrectly what you were trying to measure. (I am wondering if heap profilers which know about the generational nature of the heap might be able to provide you with more accurate information on this, albeit at higher cost. Perhaps someone on the list would know?) PS: By the way, remember that there may also be direct allocation into the old generation (for example of very large objects) between 2 scavenges, so do not assume that the occupancy of the old generation before a scavenge is the same as that after the preceding scavenge (if you know what i mean). This is a mistake that people often make the first time they are looking at our GC logs. -- ramki > > Please confirm my understanding / approach. If above understanding is > correct, I plan to get plot of > (Promo from current minor collection - Age4 from previous minor collection) > > The values will help me to determine whether I need to tune my eden and > survivor spaces. > > Your help much appreciated. > > Thanks, > Sujit > This e-mail and any files transmitted with it are for the sole use of > the intended recipient(s) and may contain confidential and privileged > information. > If you are not the intended recipient, please contact the sender by > reply e-mail and destroy all copies of the original message. > Any unauthorized review, use, disclosure, dissemination, forwarding, > printing or copying of this email or any action taken in reliance on > this e-mail is strictly prohibited and may be unlawful. > > > ------------------------------------------------------------------------ > > _______________________________________________ > hotspot-gc-use mailing list > hotspot-gc-use at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use From Antonios.Printezis at sun.com Mon Aug 3 10:50:59 2009 From: Antonios.Printezis at sun.com (Tony Printezis) Date: Mon, 03 Aug 2009 13:50:59 -0400 Subject: Direct promotion to old gen In-Reply-To: <19B27FD5AF2EAA49A66F787911CF519596D00C@CTSINCHNSXUU.cts.com> References: <19B27FD5AF2EAA49A66F787911CF519596D00C@CTSINCHNSXUU.cts.com> Message-ID: <4A772383.20206@sun.com> Sujit, Sujit.Das at cognizant.com wrote: > Hi All, > > I am trying to capture statistics to denote direct promotion from > young gen to old gen. > > As per my understanding, minor collection will empty eden space and > put surviving objects into 'To' survivor space. Objects that have > survived more than Max Tenuring Threshold value will get promoted. That's correct. > Example: If max Tenuring Threshold is 4, then in age 4 size that gets > printed in verbose GC output will be the size that should get promoted > in old gen as part of next minor collection. I'm not quite sure what you're saying here. If MTT is 4, when objects reach age 4 they get promoted. You can print the per-age information with -XX:+PrintTenuringDistribution. > Anything more than previous age4 size is result of direct promotion > from eden space to old gen. It is possible that objects with a younger age than MTT (4, in this case, to be promoted). If, during GC, the survivor space overflows then we have to promote everything else (not only objects from the eden, but also objects from the from survivor space with younger ages). BTW, in our terminology, "direct" allocation is actually allocations that directly into the old gen and bypass the young gen alltogether (large objects typically). > Please confirm my understanding / approach. If above understanding is > correct, I plan to get plot of > (Promo from current minor collection - Age4 from previous minor > collection) > > The values will help me to determine whether I need to tune my eden > and survivor spaces. Please, also look at this talk I gave with Charlie Hunt at this year's JavaOne on how to tune the young gen: http://developers.sun.com/learning/javaoneonline/j1sessn.jsp?sessn=TS-4887&yr=2009&track=javase Tony > Your help much appreciated. > > Thanks, > Sujit > This e-mail and any files transmitted with it are for the sole use of > the intended recipient(s) and may contain confidential and privileged > information. > If you are not the intended recipient, please contact the sender by > reply e-mail and destroy all copies of the original message. > Any unauthorized review, use, disclosure, dissemination, forwarding, > printing or copying of this email or any action taken in reliance on > this e-mail is strictly prohibited and may be unlawful. > > ------------------------------------------------------------------------ > > _______________________________________________ > hotspot-gc-use mailing list > hotspot-gc-use at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use > -- --------------------------------------------------------------------- | Tony Printezis, Staff Engineer | Sun Microsystems Inc. | | | MS UBUR02-311 | | e-mail: tony.printezis at sun.com | 35 Network Drive | | office: +1 781 442 0998 (x20998) | Burlington, MA 01803-2756, USA | --------------------------------------------------------------------- e-mail client: Thunderbird (Linux) From Peter.Kessler at Sun.COM Mon Aug 3 11:38:32 2009 From: Peter.Kessler at Sun.COM (Peter B. Kessler) Date: Mon, 03 Aug 2009 11:38:32 -0700 Subject: Direct promotion to old gen In-Reply-To: <4A772383.20206@sun.com> References: <19B27FD5AF2EAA49A66F787911CF519596D00C@CTSINCHNSXUU.cts.com> <4A772383.20206@sun.com> Message-ID: <4A772EA8.5020609@Sun.COM> Tony Printezis wrote: > Sujit, > > Sujit.Das at cognizant.com wrote: >> Hi All, >> >> I am trying to capture statistics to denote direct promotion from >> young gen to old gen. >> >> .... >> Anything more than previous age4 size is result of direct promotion >> from eden space to old gen. > It is possible that objects with a younger age than MTT (4, in this > case, to be promoted). If, during GC, the survivor space overflows then > we have to promote everything else (not only objects from the eden, but > also objects from the from survivor space with younger ages). > .... A minor correction: I'm pretty sure that one object failing to fit in to-space doesn't imply that all further attempts will result in promotion. E.g., if you have one big object that fails to fit in the to-space, then it will be put in the old-space, but after that smaller objects may well fit in the to-space so we will put them there. That is, each object is considered individually: if the policy says we should put it in the old generation then we do, but otherwise we try to put it in the to-space. If it fits, we're happy, if it doesn't fit we put it in the old generation. The point of the age bits and the tenuring threshold policy is to advise us which relatively older objects should be promoted so that there's room for the relatively younger objects to fit in the to-space. We need a policy like that because we don't consider the objects in age order. ... peter From Antonios.Printezis at sun.com Mon Aug 3 11:41:05 2009 From: Antonios.Printezis at sun.com (Tony Printezis) Date: Mon, 03 Aug 2009 14:41:05 -0400 Subject: Direct promotion to old gen In-Reply-To: <4A772EA8.5020609@Sun.COM> References: <19B27FD5AF2EAA49A66F787911CF519596D00C@CTSINCHNSXUU.cts.com> <4A772383.20206@sun.com> <4A772EA8.5020609@Sun.COM> Message-ID: <4A772F41.7000904@sun.com> Peter B. Kessler wrote: > Tony Printezis wrote: >> Sujit, >> >> Sujit.Das at cognizant.com wrote: >>> Hi All, >>> >>> I am trying to capture statistics to denote direct promotion from >>> young gen to old gen. >>> >>> .... >>> Anything more than previous age4 size is result of direct promotion >>> from eden space to old gen. >> It is possible that objects with a younger age than MTT (4, in this >> case, to be promoted). If, during GC, the survivor space overflows >> then we have to promote everything else (not only objects from the >> eden, but also objects from the from survivor space with younger ages). >> .... > > A minor correction: I'm pretty sure that one object failing to fit in > to-space doesn't imply that all further attempts will result in > promotion. E.g., if you have one big object that fails to fit in the > to-space, then it will be put in the old-space, but after that smaller > objects may well fit in the to-space so we will put them there. That > is, each object is considered individually: if the policy says we > should put it in the old generation then we do, but otherwise we try > to put it in the to-space. If it fits, we're happy, if it doesn't fit > we put it in the old generation. Yes, this is absolutely correct Peter. Thanks. Tony > The point of the age bits and the tenuring threshold policy is to > advise us which relatively older objects should be promoted so that > there's room for the relatively younger objects to fit in the > to-space. We need a policy like that because we don't consider the > objects in age order. > > ... peter -- --------------------------------------------------------------------- | Tony Printezis, Staff Engineer | Sun Microsystems Inc. | | | MS UBUR02-311 | | e-mail: tony.printezis at sun.com | 35 Network Drive | | office: +1 781 442 0998 (x20998) | Burlington, MA 01803-2756, USA | --------------------------------------------------------------------- e-mail client: Thunderbird (Linux) From Sujit.Das at cognizant.com Mon Aug 3 12:01:51 2009 From: Sujit.Das at cognizant.com (Sujit.Das at cognizant.com) Date: Tue, 4 Aug 2009 00:31:51 +0530 Subject: Tenuring distribution Message-ID: <19B27FD5AF2EAA49A66F787911CF519596D00E@CTSINCHNSXUU.cts.com> Hi All, I am trying to look whether there is any scope in improving tenuring distribution by changing survivor space or Max Tenuring Threshold. I have read some Tenuring distribution examples. But following tenuring distribution output of my application confuses me and not sure whether the young generation is well-tuned or not. The following tenuring distribution O/P is result of consecutive minor collections. It increases from age 1 to 3 (sometimes upto age 4) and then drops to age 1. It then again increases with each minor collection. Please do let me know your thoughts. Desired survivor size 14188544 bytes, new threshold 4 (max 4) - age 1: 7040008 bytes, 7040008 total : 305088K->13787K(305088K), 0.0395292 secs] 906769K->626948K(3044288K)After GC: ... Desired survivor size 14188544 bytes, new threshold 4 (max 4) - age 1: 6448424 bytes, 6448424 total - age 2: 4461088 bytes, 10909512 total : 291163K->12426K(305088K), 0.0191044 secs] 904324K->625588K(3044288K)After GC: ... Desired survivor size 14188544 bytes, new threshold 3 (max 4) - age 1: 7150744 bytes, 7150744 total - age 2: 4394936 bytes, 11545680 total - age 3: 3582184 bytes, 15127864 total : 289802K->19070K(305088K), 0.0191912 secs] 902964K->632232K(3044288K)After GC: ... Desired survivor size 14188544 bytes, new threshold 1 (max 4) - age 1: 18196472 bytes, 18196472 total - age 2: 3800224 bytes, 21996696 total - age 3: 3340344 bytes, 25337040 total : 296446K->27712K(305088K), 0.0282980 secs] 909608K->648613K(3044288K)After GC: ... Desired survivor size 14188544 bytes, new threshold 4 (max 4) - age 1: 5757968 bytes, 5757968 total : 305088K->13117K(305088K), 0.0403760 secs] 925989K->645051K(3044288K)After GC: ... Desired survivor size 14188544 bytes, new threshold 4 (max 4) - age 1: 5984000 bytes, 5984000 total - age 2: 4197480 bytes, 10181480 total : 290493K->13133K(305088K), 0.0251218 secs] 922427K->645067K(3044288K)After GC: ... Desired survivor size 14188544 bytes, new threshold 3 (max 4) - age 1: 6765056 bytes, 6765056 total - age 2: 4159752 bytes, 10924808 total - age 3: 3534520 bytes, 14459328 total : 290509K->17608K(305088K), 0.0258073 secs] 922443K->649542K(3044288K)After GC: ... Desired survivor size 14188544 bytes, new threshold 1 (max 4) - age 1: 17919056 bytes, 17919056 total - age 2: 4852200 bytes, 22771256 total - age 3: 3152768 bytes, 25924024 total : 294984K->27712K(305088K), 0.0303612 secs] 926918K->667692K(3044288K)After GC: ... Desired survivor size 14188544 bytes, new threshold 4 (max 4) - age 1: 6579448 bytes, 6579448 total : 305088K->13396K(305088K), 0.0386087 secs] 945068K->664582K(3044288K)After GC: Thanks, Sujit This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email or any action taken in reliance on this e-mail is strictly prohibited and may be unlawful. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20090804/cfe48bfe/attachment.html From Y.S.Ramakrishna at Sun.COM Mon Aug 3 12:27:25 2009 From: Y.S.Ramakrishna at Sun.COM (Y.S.Ramakrishna at Sun.COM) Date: Mon, 03 Aug 2009 12:27:25 -0700 Subject: Tenuring distribution In-Reply-To: <19B27FD5AF2EAA49A66F787911CF519596D00E@CTSINCHNSXUU.cts.com> References: <19B27FD5AF2EAA49A66F787911CF519596D00E@CTSINCHNSXUU.cts.com> Message-ID: <4A773A1D.3020500@Sun.COM> On 08/ 3/09 12:01 PM, Sujit.Das at cognizant.com wrote: > Hi All, > > I am trying to look whether there is any scope in improving tenuring > distribution by changing survivor space or Max Tenuring Threshold. I > have read some Tenuring distribution examples. But following tenuring > distribution output of my application confuses me and not sure whether > the young generation is well-tuned or not. The following tenuring > distribution O/P is result of consecutive minor collections. It > increases from age 1 to 3 (sometimes upto age 4) and then drops to age > 1. It then again increases with each minor collection. Yes, as you can see, the TT drops to 3 and then to 1, and this happens because there is periodically a large number of young survivors (i.e. age 1) in the survivor space, so we pro-actively promote at the next scavenge to avoid the possibility of premature overflow. You could increase your survivor space size. (There is clearly some kind of periodic generation of a large volume of data in your application. You can see whether that correlates to any kind of load-pulsing, or convoying effects. Are you using a production load (which can be bursty) or a bursty test load or a uniform test load for these figures?) Once again, I think Tony & Charlie's talk slides will help in this kind of tuning. -- ramki > > Please do let me know your thoughts. > > Desired survivor size 14188544 bytes, new threshold 4 (max 4) > - age 1: 7040008 bytes, 7040008 total > : 305088K->13787K(305088K), 0.0395292 secs] > 906769K->626948K(3044288K)After GC: > ... > Desired survivor size 14188544 bytes, new threshold 4 (max 4) > - age 1: 6448424 bytes, 6448424 total > - age 2: 4461088 bytes, 10909512 total > : 291163K->12426K(305088K), 0.0191044 secs] > 904324K->625588K(3044288K)After GC: > ... > Desired survivor size 14188544 bytes, new threshold 3 (max 4) > - age 1: 7150744 bytes, 7150744 total > - age 2: 4394936 bytes, 11545680 total > - age 3: 3582184 bytes, 15127864 total > : 289802K->19070K(305088K), 0.0191912 secs] > 902964K->632232K(3044288K)After GC: > ... > Desired survivor size 14188544 bytes, new threshold 1 (max 4) > - age 1: 18196472 bytes, 18196472 total > - age 2: 3800224 bytes, 21996696 total > - age 3: 3340344 bytes, 25337040 total > : 296446K->27712K(305088K), 0.0282980 secs] > 909608K->648613K(3044288K)After GC: > ... > Desired survivor size 14188544 bytes, new threshold 4 (max 4) > - age 1: 5757968 bytes, 5757968 total > : 305088K->13117K(305088K), 0.0403760 secs] > 925989K->645051K(3044288K)After GC: > ... > Desired survivor size 14188544 bytes, new threshold 4 (max 4) > - age 1: 5984000 bytes, 5984000 total > - age 2: 4197480 bytes, 10181480 total > : 290493K->13133K(305088K), 0.0251218 secs] > 922427K->645067K(3044288K)After GC: > ... > Desired survivor size 14188544 bytes, new threshold 3 (max 4) > - age 1: 6765056 bytes, 6765056 total > - age 2: 4159752 bytes, 10924808 total > - age 3: 3534520 bytes, 14459328 total > : 290509K->17608K(305088K), 0.0258073 secs] > 922443K->649542K(3044288K)After GC: > ... > Desired survivor size 14188544 bytes, new threshold 1 (max 4) > - age 1: 17919056 bytes, 17919056 total > - age 2: 4852200 bytes, 22771256 total > - age 3: 3152768 bytes, 25924024 total > : 294984K->27712K(305088K), 0.0303612 secs] > 926918K->667692K(3044288K)After GC: > ... > Desired survivor size 14188544 bytes, new threshold 4 (max 4) > - age 1: 6579448 bytes, 6579448 total > : 305088K->13396K(305088K), 0.0386087 secs] > 945068K->664582K(3044288K)After GC: > > Thanks, > Sujit > This e-mail and any files transmitted with it are for the sole use of > the intended recipient(s) and may contain confidential and privileged > information. > If you are not the intended recipient, please contact the sender by > reply e-mail and destroy all copies of the original message. > Any unauthorized review, use, disclosure, dissemination, forwarding, > printing or copying of this email or any action taken in reliance on > this e-mail is strictly prohibited and may be unlawful. > > > ------------------------------------------------------------------------ > > _______________________________________________ > hotspot-gc-use mailing list > hotspot-gc-use at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use From Sujit.Das at cognizant.com Mon Aug 3 12:32:28 2009 From: Sujit.Das at cognizant.com (Sujit.Das at cognizant.com) Date: Tue, 4 Aug 2009 01:02:28 +0530 Subject: Tenuring distribution References: <19B27FD5AF2EAA49A66F787911CF519596D00E@CTSINCHNSXUU.cts.com> <4A773A1D.3020500@Sun.COM> Message-ID: <19B27FD5AF2EAA49A66F787911CF519596D00F@CTSINCHNSXUU.cts.com> The snippet below is for uniform test load in staging environment. Thanks, Sujit ________________________________ From: Y.S.Ramakrishna at Sun.COM [mailto:Y.S.Ramakrishna at Sun.COM] Sent: Mon 8/3/2009 2:27 PM To: Das, Sujit (Cognizant) Cc: hotspot-gc-use at openjdk.java.net Subject: Re: Tenuring distribution On 08/ 3/09 12:01 PM, Sujit.Das at cognizant.com wrote: > Hi All, > > I am trying to look whether there is any scope in improving tenuring > distribution by changing survivor space or Max Tenuring Threshold. I > have read some Tenuring distribution examples. But following tenuring > distribution output of my application confuses me and not sure whether > the young generation is well-tuned or not. The following tenuring > distribution O/P is result of consecutive minor collections. It > increases from age 1 to 3 (sometimes upto age 4) and then drops to age > 1. It then again increases with each minor collection. Yes, as you can see, the TT drops to 3 and then to 1, and this happens because there is periodically a large number of young survivors (i.e. age 1) in the survivor space, so we pro-actively promote at the next scavenge to avoid the possibility of premature overflow. You could increase your survivor space size. (There is clearly some kind of periodic generation of a large volume of data in your application. You can see whether that correlates to any kind of load-pulsing, or convoying effects. Are you using a production load (which can be bursty) or a bursty test load or a uniform test load for these figures?) Once again, I think Tony & Charlie's talk slides will help in this kind of tuning. -- ramki > > Please do let me know your thoughts. > > Desired survivor size 14188544 bytes, new threshold 4 (max 4) > - age 1: 7040008 bytes, 7040008 total > : 305088K->13787K(305088K), 0.0395292 secs] > 906769K->626948K(3044288K)After GC: > ... > Desired survivor size 14188544 bytes, new threshold 4 (max 4) > - age 1: 6448424 bytes, 6448424 total > - age 2: 4461088 bytes, 10909512 total > : 291163K->12426K(305088K), 0.0191044 secs] > 904324K->625588K(3044288K)After GC: > ... > Desired survivor size 14188544 bytes, new threshold 3 (max 4) > - age 1: 7150744 bytes, 7150744 total > - age 2: 4394936 bytes, 11545680 total > - age 3: 3582184 bytes, 15127864 total > : 289802K->19070K(305088K), 0.0191912 secs] > 902964K->632232K(3044288K)After GC: > ... > Desired survivor size 14188544 bytes, new threshold 1 (max 4) > - age 1: 18196472 bytes, 18196472 total > - age 2: 3800224 bytes, 21996696 total > - age 3: 3340344 bytes, 25337040 total > : 296446K->27712K(305088K), 0.0282980 secs] > 909608K->648613K(3044288K)After GC: > ... > Desired survivor size 14188544 bytes, new threshold 4 (max 4) > - age 1: 5757968 bytes, 5757968 total > : 305088K->13117K(305088K), 0.0403760 secs] > 925989K->645051K(3044288K)After GC: > ... > Desired survivor size 14188544 bytes, new threshold 4 (max 4) > - age 1: 5984000 bytes, 5984000 total > - age 2: 4197480 bytes, 10181480 total > : 290493K->13133K(305088K), 0.0251218 secs] > 922427K->645067K(3044288K)After GC: > ... > Desired survivor size 14188544 bytes, new threshold 3 (max 4) > - age 1: 6765056 bytes, 6765056 total > - age 2: 4159752 bytes, 10924808 total > - age 3: 3534520 bytes, 14459328 total > : 290509K->17608K(305088K), 0.0258073 secs] > 922443K->649542K(3044288K)After GC: > ... > Desired survivor size 14188544 bytes, new threshold 1 (max 4) > - age 1: 17919056 bytes, 17919056 total > - age 2: 4852200 bytes, 22771256 total > - age 3: 3152768 bytes, 25924024 total > : 294984K->27712K(305088K), 0.0303612 secs] > 926918K->667692K(3044288K)After GC: > ... > Desired survivor size 14188544 bytes, new threshold 4 (max 4) > - age 1: 6579448 bytes, 6579448 total > : 305088K->13396K(305088K), 0.0386087 secs] > 945068K->664582K(3044288K)After GC: > > Thanks, > Sujit > This e-mail and any files transmitted with it are for the sole use of > the intended recipient(s) and may contain confidential and privileged > information. > If you are not the intended recipient, please contact the sender by > reply e-mail and destroy all copies of the original message. > Any unauthorized review, use, disclosure, dissemination, forwarding, > printing or copying of this email or any action taken in reliance on > this e-mail is strictly prohibited and may be unlawful. > > > ------------------------------------------------------------------------ > > _______________________________________________ > hotspot-gc-use mailing list > hotspot-gc-use at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email or any action taken in reliance on this e-mail is strictly prohibited and may be unlawful. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20090804/fb453878/attachment.html From Y.S.Ramakrishna at Sun.COM Mon Aug 3 12:36:46 2009 From: Y.S.Ramakrishna at Sun.COM (Y.S.Ramakrishna at Sun.COM) Date: Mon, 03 Aug 2009 12:36:46 -0700 Subject: Direct promotion to old gen In-Reply-To: <4A772EA8.5020609@Sun.COM> References: <19B27FD5AF2EAA49A66F787911CF519596D00C@CTSINCHNSXUU.cts.com> <4A772383.20206@sun.com> <4A772EA8.5020609@Sun.COM> Message-ID: <4A773C4E.9060701@Sun.COM> On 08/ 3/09 11:38 AM, Peter B. Kessler wrote: ... > > A minor correction: I'm pretty sure that one object failing to fit in to-space doesn't imply that all further attempts will result in promotion. E.g., if you have one big object that fails to fit in the to-space, then it will be put in the old-space, but after that smaller objects may well fit in the to-space so we will put them there. That is, each object is considered individually: if the policy says we should put it in the old generation then we do, but otherwise we try to put it in the to-space. If it fits, we're happy, if it doesn't fit we put it in the old generation. > > The point of the age bits and the tenuring threshold policy is to advise us which relatively older objects should be promoted so that there's room for the relatively younger objects to fit in the to-space. We need a policy like that because we don't consider the objects in age order. Exactly; and furthermore, the "Survivor PLABs" used by our parallel scavengers can exacerbate that effect, especially in the presence of object size distributions with longer/fatter tails, where we can make the error of keeping an old large object (which might typically live a long time) in a survivor space at the expense of many smaller younger objects that will die soon. Such survivor space overflow is, as many of our users know, the bane of non-moving old gen collectors such as CMS, where they exacerbate fragmentation. -- ramki From Y.S.Ramakrishna at Sun.COM Mon Aug 3 13:00:42 2009 From: Y.S.Ramakrishna at Sun.COM (Y.S.Ramakrishna at Sun.COM) Date: Mon, 03 Aug 2009 13:00:42 -0700 Subject: Tenuring distribution In-Reply-To: <19B27FD5AF2EAA49A66F787911CF519596D00F@CTSINCHNSXUU.cts.com> References: <19B27FD5AF2EAA49A66F787911CF519596D00E@CTSINCHNSXUU.cts.com> <4A773A1D.3020500@Sun.COM> <19B27FD5AF2EAA49A66F787911CF519596D00F@CTSINCHNSXUU.cts.com> Message-ID: <4A7741EA.8010307@Sun.COM> On 08/ 3/09 12:32 PM, Sujit.Das at cognizant.com wrote: > The snippet below is for uniform test load in staging environment. If you want to age objects the full 4 scavenges before promotion, you will need to increase the survivor size to the sum of the maximum historical panel populations of objects of age 1 through 4.* (You will need to do this iteratively, until you reach a fixed point. Since we do not allow survivor space size to exceed that of Eden, there is a class of applications whose object lifetimes will never have a feasible solution to this objective of aging all objects for at least N scavenges before promoting.) [*} for the snippet below, the indicated survivor size would be maxpop(age1)+maxpop(age2)+maxpop(age3)+maxpop(age4) = = 18196472 + 4852200 + 3582184 + ... = at least 26630856 The iteration to a fixed point is needed because each increase will allow space for the "baby boom" to move through the survivor space. A rough (probably very loose) upper bound above might be 4*maxpop(age1) = 4*18196472 = 72785888. The optimal value would be something in between. But of course this loose upper bound assumes that all age 1 objects will live at least 4 scavenges, in which case it does _not_ make sense to uselessly copy them back-and-forth between the survivor spaces (especially if you expect them to survive the fifth scavenge and promote anyway). Typical survivor space sizing decisions might be made based on the relative costs of copying versus promotion (either in the short term, i.e. pause times, for low-pause collectors, or in the long term, i.e. end-to-end GC overhead, for throughput oriented collectors. Of course the long term effects of premature promotion in the case of concurrent collectors cannot be ignored, both for fragmentation effects in the case of non-moving old gen collectors, and for overwhelming concurrent GC threads with potentially more work than they have the capacity to handle.) As you can see there are multiple factors and costs, and you will need to iteratively tune to balance these according to your cost/utility function. As I write this, I realize that, if we do not do so already, pro-actively promoting objects that are very large (on the assumption, or perhaps based on profiling) might be a reasonable generational tenuring strategy. (I am sure someone must have written about this already elsewhere, and perhaps HotSpot already has support for it, i'd need to check our code.) -- ramki > > Desired survivor size 14188544 bytes, new threshold 4 (max 4) > > - age 1: 7040008 bytes, 7040008 total > > : 305088K->13787K(305088K), 0.0395292 secs] > > 906769K->626948K(3044288K)After GC: > > ... > > Desired survivor size 14188544 bytes, new threshold 4 (max 4) > > - age 1: 6448424 bytes, 6448424 total > > - age 2: 4461088 bytes, 10909512 total > > : 291163K->12426K(305088K), 0.0191044 secs] > > 904324K->625588K(3044288K)After GC: > > ... > > Desired survivor size 14188544 bytes, new threshold 3 (max 4) > > - age 1: 7150744 bytes, 7150744 total > > - age 2: 4394936 bytes, 11545680 total > > - age 3: 3582184 bytes, 15127864 total > > : 289802K->19070K(305088K), 0.0191912 secs] > > 902964K->632232K(3044288K)After GC: > > ... > > Desired survivor size 14188544 bytes, new threshold 1 (max 4) > > - age 1: 18196472 bytes, 18196472 total > > - age 2: 3800224 bytes, 21996696 total > > - age 3: 3340344 bytes, 25337040 total > > : 296446K->27712K(305088K), 0.0282980 secs] > > 909608K->648613K(3044288K)After GC: > > ... > > Desired survivor size 14188544 bytes, new threshold 4 (max 4) > > - age 1: 5757968 bytes, 5757968 total > > : 305088K->13117K(305088K), 0.0403760 secs] > > 925989K->645051K(3044288K)After GC: > > ... > > Desired survivor size 14188544 bytes, new threshold 4 (max 4) > > - age 1: 5984000 bytes, 5984000 total > > - age 2: 4197480 bytes, 10181480 total > > : 290493K->13133K(305088K), 0.0251218 secs] > > 922427K->645067K(3044288K)After GC: > > ... > > Desired survivor size 14188544 bytes, new threshold 3 (max 4) > > - age 1: 6765056 bytes, 6765056 total > > - age 2: 4159752 bytes, 10924808 total > > - age 3: 3534520 bytes, 14459328 total > > : 290509K->17608K(305088K), 0.0258073 secs] > > 922443K->649542K(3044288K)After GC: > > ... > > Desired survivor size 14188544 bytes, new threshold 1 (max 4) > > - age 1: 17919056 bytes, 17919056 total > > - age 2: 4852200 bytes, 22771256 total > > - age 3: 3152768 bytes, 25924024 total > > : 294984K->27712K(305088K), 0.0303612 secs] > > 926918K->667692K(3044288K)After GC: > > ... > > Desired survivor size 14188544 bytes, new threshold 4 (max 4) > > - age 1: 6579448 bytes, 6579448 total > > : 305088K->13396K(305088K), 0.0386087 secs] > > 945068K->664582K(3044288K)After GC: From shane.cox at gmail.com Wed Aug 12 11:55:04 2009 From: shane.cox at gmail.com (Shane Cox) Date: Wed, 12 Aug 2009 14:55:04 -0400 Subject: Strange gc log output In-Reply-To: References: Message-ID: I was wondering if anyone might know the cause of the following behaviors illustrated by the log excerpt below: 1) Huge jump in the timestamp printed to the GC log (150 days in this example .. though the process was started last night). 2) All of the GC timings are reported to be 0.0000000 seconds. 43329.690: [GC 43329.690: [ParNew: 37248K->5312K(37248K), 0.1000674 secs] 626205K->599580K(1698624K), 0.1006993 secs] 43332.695: [GC 43332.695: [ParNew: 37248K->5311K(37248K), 0.1070853 secs] 631516K->606736K(1698624K), 0.1079122 secs] 43340.565: [GC 43340.565: [ParNew: 37247K->5312K(37248K), 0.0807989 secs] 638672K->610581K(1698624K), 0.0811694 secs] 43348.221: [GC 43348.222: [ParNew: 37247K->5311K(37248K), 0.1515881 secs] 642517K->616038K(1698624K), 0.1529900 secs] 13060256.654: [GC 13060256.654: [ParNew: 37247K->5312K(37248K), 0.0000000 secs] 647974K->620519K(1698624K), 0.0000000 secs] 13060256.654: [GC 13060256.654: [ParNew: 37248K->5312K(37248K), 0.0000000 secs] 652455K->625737K(1698624K), 0.0000000 secs] 13060256.654: [GC 13060256.654: [ParNew: 37248K->5312K(37248K), 0.0000000 secs] 657673K->632234K(1698624K), 0.0000000 secs] 13060256.654: [GC 13060256.654: [ParNew: 37248K->5312K(37248K), 0.0000000 secs] 664170K->636084K(1698624K), 0.0000000 secs] I'm running the 64-bit JVM - version 1.6 update 7 on x86 Solaris. I tried to attach a zipped copy of the full gc log, but it exceeded the 100K limit. Any help would be appreciated. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20090812/a52e04d7/attachment.html From Y.S.Ramakrishna at Sun.COM Wed Aug 12 12:18:27 2009 From: Y.S.Ramakrishna at Sun.COM (Y. Srinivas Ramakrishna) Date: Wed, 12 Aug 2009 12:18:27 -0700 Subject: Strange gc log output In-Reply-To: References: Message-ID: <4A831583.6030302@sun.com> Your platform? There was discussion of this in the context of an old version of Solaris 10 on x64 recently and it turned out to be lack of a required kernel patch 137138-09. It is part of the the recommended patch cluster for Java SE on Solaris can be found here:- http://sunsolve.sun.com/show.do?target=patches/patch-access scroll down to the bottom and expand the "J2SE" patch cluster link. Download may require registration. Let us know whether it addresses yr issue or not (and the platform on which you observed the problem). -- ramki Shane Cox wrote: > I was wondering if anyone might know the cause of the following behaviors > illustrated by the log excerpt below: > 1) Huge jump in the timestamp printed to the GC log (150 days in this > example .. though the process was started last night). > 2) All of the GC timings are reported to be 0.0000000 seconds. > > > 43329.690: [GC 43329.690: [ParNew: 37248K->5312K(37248K), 0.1000674 secs] > 626205K->599580K(1698624K), 0.1006993 secs] > 43332.695: [GC 43332.695: [ParNew: 37248K->5311K(37248K), 0.1070853 secs] > 631516K->606736K(1698624K), 0.1079122 secs] > 43340.565: [GC 43340.565: [ParNew: 37247K->5312K(37248K), 0.0807989 secs] > 638672K->610581K(1698624K), 0.0811694 secs] > 43348.221: [GC 43348.222: [ParNew: 37247K->5311K(37248K), 0.1515881 secs] > 642517K->616038K(1698624K), 0.1529900 secs] > 13060256.654: [GC 13060256.654: [ParNew: 37247K->5312K(37248K), 0.0000000 > secs] 647974K->620519K(1698624K), 0.0000000 secs] > 13060256.654: [GC 13060256.654: [ParNew: 37248K->5312K(37248K), 0.0000000 > secs] 652455K->625737K(1698624K), 0.0000000 secs] > 13060256.654: [GC 13060256.654: [ParNew: 37248K->5312K(37248K), 0.0000000 > secs] 657673K->632234K(1698624K), 0.0000000 secs] > 13060256.654: [GC 13060256.654: [ParNew: 37248K->5312K(37248K), 0.0000000 > secs] 664170K->636084K(1698624K), 0.0000000 secs] > > I'm running the 64-bit JVM - version 1.6 update 7 on x86 Solaris. > > I tried to attach a zipped copy of the full gc log, but it exceeded the 100K > limit. > > Any help would be appreciated. > > > ------------------------------------------------------------------------ > > _______________________________________________ > hotspot-gc-use mailing list > hotspot-gc-use at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use > From shane.cox at gmail.com Fri Aug 14 14:53:31 2009 From: shane.cox at gmail.com (Shane Cox) Date: Fri, 14 Aug 2009 17:53:31 -0400 Subject: GC log parsing tools Message-ID: I was wondering if someone could recommend tools that parse GC log files and report statistics (gc counts, pause time, bytes promoted, etc.) I've used GC Portal in the past, but it doesn't seem to support all formats/syntax produced by 1.6 VMs. My objective is to compare GC logs from different benchmarks/tests to determine optimal gc settings. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20090814/ccaab51a/attachment.html From charlie.hunt at sun.com Sun Aug 16 19:16:07 2009 From: charlie.hunt at sun.com (charlie hunt) Date: Sun, 16 Aug 2009 21:16:07 -0500 Subject: GC log parsing tools In-Reply-To: References: Message-ID: <4A88BD67.70906@sun.com> I'd suggest looking at PrintGCStats and/or a graphical tool called GCHisto. PrintGCStats: http://java.sun.com/developer/technicalArticles/Programming/turbo/PrintGCStats.zip GCHisto: http://gchisto.dev.java.net/ * You'll need to build it from source. But, it's pretty trivial to build. hths, charlie ... Shane Cox wrote: > I was wondering if someone could recommend tools that parse GC log > files and report statistics (gc counts, pause time, bytes promoted, > etc.) I've used GC Portal in the past, but it doesn't seem to support > all formats/syntax produced by 1.6 VMs. My objective is to compare GC > logs from different benchmarks/tests to determine optimal gc settings. > > Thanks > ------------------------------------------------------------------------ > > _______________________________________________ > hotspot-gc-use mailing list > hotspot-gc-use at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use > From aaisinzon at guidewire.com Mon Aug 17 09:58:14 2009 From: aaisinzon at guidewire.com (Alex Aisinzon) Date: Mon, 17 Aug 2009 09:58:14 -0700 Subject: hotspot-gc-use Digest, Vol 20, Issue 6 In-Reply-To: References: Message-ID: Hi Shane The best tool I have found so far (and I have looked hard) is the IBM Support Assistant. That tool is like an IDE where you can download many plugins. One of them is a garbage collection analyzer. I have used successfully up to Sun JVM 1.5 (and IBM JVMs too, obviously). I would typically use the following options for the Sun JVM: "-XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCApplicationStoppedTime" I have noticed that this tool (along with others) would choke on the first couple of lines in the GC logs. I need to remove all lines before the first "useful entry" aka "{Heap before gc invocations=1:" Then, the tool would work great. Let me know if this helps Alex Aisinzon -----Original Message----- From: hotspot-gc-use-bounces at openjdk.java.net [mailto:hotspot-gc-use-bounces at openjdk.java.net] On Behalf Of hotspot-gc-use-request at openjdk.java.net Sent: Saturday, August 15, 2009 12:00 PM To: hotspot-gc-use at openjdk.java.net Subject: hotspot-gc-use Digest, Vol 20, Issue 6 Send hotspot-gc-use mailing list submissions to hotspot-gc-use at openjdk.java.net To subscribe or unsubscribe via the World Wide Web, visit http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use or, via email, send a message with subject or body 'help' to hotspot-gc-use-request at openjdk.java.net You can reach the person managing the list at hotspot-gc-use-owner at openjdk.java.net When replying, please edit your Subject line so it is more specific than "Re: Contents of hotspot-gc-use digest..." Today's Topics: 1. GC log parsing tools (Shane Cox) ---------------------------------------------------------------------- Message: 1 Date: Fri, 14 Aug 2009 17:53:31 -0400 From: Shane Cox Subject: GC log parsing tools To: hotspot-gc-use at openjdk.java.net Message-ID: Content-Type: text/plain; charset="iso-8859-1" I was wondering if someone could recommend tools that parse GC log files and report statistics (gc counts, pause time, bytes promoted, etc.) I've used GC Portal in the past, but it doesn't seem to support all formats/syntax produced by 1.6 VMs. My objective is to compare GC logs from different benchmarks/tests to determine optimal gc settings. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/200908 14/ccaab51a/attachment-0001.html ------------------------------ _______________________________________________ hotspot-gc-use mailing list hotspot-gc-use at openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use End of hotspot-gc-use Digest, Vol 20, Issue 6 ********************************************* From xguan at cse.unl.edu Mon Aug 31 22:39:54 2009 From: xguan at cse.unl.edu (Tony Guan) Date: Tue, 1 Sep 2009 00:39:54 -0500 Subject: capturing method entry/exit efficiently Message-ID: <2fcb552b0908312239s760942e2la7f1a995e6e2a591@mail.gmail.com> Dear all, My current research project with hotspot requires me to do something particular whenever a method(interpreted or compiled) is invoked. I need to know the thread and the method at the invocation time. What I am trying to do is to do some VM hacking based on the methods called. Question 1: Can I use BCI to achieve this? I am now able to capture the method_entry/exit events by writing a JVMTI agent, but it's not what I really need to do. By using JVMTI, performance is deteriorated a lot. And I am not sure if the compiled method can still be captured. (Though I know java1.5 has some JVMPI support in the compilation part, but not in java1.7. Am I right?). Question 2: I am trying find a way to enable the notify_method_enry/exit by partly simulating an JVMTI agent, that means that I modify several parts in the hotspot without actually use an external JVMTI agent. Is it feasible? (in terms of perfomance) Question 3: Is there some better way to capture the method_entry/exit event? Thanks for diluting the question marks in my mind! Tony (Xiaohua Guan)