From john.cuthbertson at oracle.com Wed Aug 1 21:11:37 2012 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Wed, 01 Aug 2012 14:11:37 -0700 Subject: RFR(S): 6818524: G1: use ergonomic resizing of PLABs Message-ID: <50199B89.1060703@oracle.com> Hi Everyone, Can I have a couple of volunteers review the changes for this CR? The webrev can be found at: http://cr.openjdk.java.net/~johnc/6818524/webrev.0/. These changes have been contributed by Brandon Mitchell (at Twitter). Summary: PLABStats instances are now employed in G1 to record information about old and young PLABs, and they are then used to adjust the PLAB sizes for the next GC. As a result the ResizePLAB flag is now being observed by G1. The previous behavior (and PLAB sizes) can be obtained by specifying -XX:-ResizePLAB. With these changes I see just over a net 1% performance gain, on our reference workload, when running with G1. Testing: GC test suite with and without ResizePLAB and PrintPLAB enabled to observe the changes in PLAB sizes; reference workload on multiple platforms; jprt. Thanks, JohnC From ysr1729 at gmail.com Thu Aug 2 18:24:40 2012 From: ysr1729 at gmail.com (Srinivas Ramakrishna) Date: Thu, 2 Aug 2012 11:24:40 -0700 Subject: RFR(S): 6818524: G1: use ergonomic resizing of PLABs In-Reply-To: <50199B89.1060703@oracle.com> References: <50199B89.1060703@oracle.com> Message-ID: Hi John, Brandon -- This looks good to me. One improvement that could be made (but which predates any of these changes) is that the ParGCAllocBuffer's min_size() and max_size() just return the lower and upper bounds set for TLAB's. Rather these should be set (perhaps statically if that is possible) by the containing heap or generation. The PLAB sizing calculation already does the requisite clipping, so if the min and max sizes were properly configured you would likely not need to do the clipping from above that you now have to do to keep G1's ergonomic PLAB sizes from becoming humongous. But that can be a separate reorg sice it will touch a bit more code. Otherwise this looks great! thanks. -- ramki On Wed, Aug 1, 2012 at 2:11 PM, John Cuthbertson < john.cuthbertson at oracle.com> wrote: > Hi Everyone, > > Can I have a couple of volunteers review the changes for this CR? The > webrev can be found at: http://cr.openjdk.java.net/~** > johnc/6818524/webrev.0/. > These changes have been contributed by Brandon Mitchell (at Twitter). > > Summary: > PLABStats instances are now employed in G1 to record information about old > and young PLABs, and they are then used to adjust the PLAB sizes for the > next GC. As a result the ResizePLAB flag is now being observed by G1. The > previous behavior (and PLAB sizes) can be obtained by specifying > -XX:-ResizePLAB. > > With these changes I see just over a net 1% performance gain, on our > reference workload, when running with G1. > > Testing: > GC test suite with and without ResizePLAB and PrintPLAB enabled to observe > the changes in PLAB sizes; reference workload on multiple platforms; jprt. > > Thanks, > > JohnC > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.coomes at oracle.com Fri Aug 3 03:49:06 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 03 Aug 2012 03:49:06 +0000 Subject: hg: hsx/hotspot-gc: Added tag jdk8-b50 for changeset 2fd67618b9a3 Message-ID: <20120803034906.AE07B47350@hg.openjdk.java.net> Changeset: 57c0aee73090 Author: katleman Date: 2012-08-02 15:32 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/rev/57c0aee73090 Added tag jdk8-b50 for changeset 2fd67618b9a3 ! .hgtags From john.coomes at oracle.com Fri Aug 3 03:49:12 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 03 Aug 2012 03:49:12 +0000 Subject: hg: hsx/hotspot-gc/corba: Added tag jdk8-b50 for changeset d20d9eb9f093 Message-ID: <20120803034915.E257347351@hg.openjdk.java.net> Changeset: 9b0f841ca9f7 Author: katleman Date: 2012-08-02 15:32 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/corba/rev/9b0f841ca9f7 Added tag jdk8-b50 for changeset d20d9eb9f093 ! .hgtags From john.coomes at oracle.com Fri Aug 3 03:49:22 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 03 Aug 2012 03:49:22 +0000 Subject: hg: hsx/hotspot-gc/jaxp: Added tag jdk8-b50 for changeset 2791ec55f66b Message-ID: <20120803034931.A0D9B47352@hg.openjdk.java.net> Changeset: dc1ea77ed9d9 Author: katleman Date: 2012-08-02 15:35 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jaxp/rev/dc1ea77ed9d9 Added tag jdk8-b50 for changeset 2791ec55f66b ! .hgtags From john.coomes at oracle.com Fri Aug 3 03:49:39 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 03 Aug 2012 03:49:39 +0000 Subject: hg: hsx/hotspot-gc/jaxws: Added tag jdk8-b50 for changeset bdab72e87b83 Message-ID: <20120803034945.379B047353@hg.openjdk.java.net> Changeset: 1a70b6333ebe Author: katleman Date: 2012-08-02 15:35 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jaxws/rev/1a70b6333ebe Added tag jdk8-b50 for changeset bdab72e87b83 ! .hgtags From john.coomes at oracle.com Fri Aug 3 03:49:54 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 03 Aug 2012 03:49:54 +0000 Subject: hg: hsx/hotspot-gc/jdk: Added tag jdk8-b50 for changeset e4bae5c53fca Message-ID: <20120803035046.A2E8D47354@hg.openjdk.java.net> Changeset: 79c9847d4a5f Author: katleman Date: 2012-08-02 15:36 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/79c9847d4a5f Added tag jdk8-b50 for changeset e4bae5c53fca ! .hgtags From john.coomes at oracle.com Fri Aug 3 03:52:27 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 03 Aug 2012 03:52:27 +0000 Subject: hg: hsx/hotspot-gc/langtools: Added tag jdk8-b50 for changeset b2d8a270f5f2 Message-ID: <20120803035236.07A2047355@hg.openjdk.java.net> Changeset: c4cd4cab2220 Author: katleman Date: 2012-08-02 15:37 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/langtools/rev/c4cd4cab2220 Added tag jdk8-b50 for changeset b2d8a270f5f2 ! .hgtags From alejandro.murillo at oracle.com Sat Aug 4 01:49:01 2012 From: alejandro.murillo at oracle.com (alejandro.murillo at oracle.com) Date: Sat, 04 Aug 2012 01:49:01 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 8 new changesets Message-ID: <20120804014927.0231A47377@hg.openjdk.java.net> Changeset: c01c8e05ec8c Author: katleman Date: 2012-08-02 15:33 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/c01c8e05ec8c Added tag jdk8-b50 for changeset 58f237a9e83a ! .hgtags Changeset: 3b01d0321dfa Author: zgu Date: 2012-07-30 10:25 -0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/3b01d0321dfa 7186778: MachO decoder implementation for MacOSX Summary: Implementation of decoder for Apple's MacOSX. The implementation is based on the patch provided by Kevin Walls. Reviewed-by: coleenp, kamg, kevinw ! src/os/bsd/vm/decoder_machO.cpp ! src/os/bsd/vm/decoder_machO.hpp ! src/os/bsd/vm/os_bsd.cpp ! src/os/windows/vm/decoder_windows.cpp ! src/os/windows/vm/decoder_windows.hpp ! src/share/vm/utilities/decoder.cpp ! src/share/vm/utilities/decoder.hpp ! src/share/vm/utilities/decoder_elf.hpp Changeset: 4bfef6df8881 Author: zgu Date: 2012-07-30 07:21 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/4bfef6df8881 Merge Changeset: 5e2dc722e70d Author: andrew Date: 2012-07-31 16:01 -0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/5e2dc722e70d 7186278: Build error after CR#6995781 / 7151532 with GCC 4.7.0 Summary: Templates need this object if not using template parameter in call Reviewed-by: coleenp, kamg, dholmes ! src/share/vm/utilities/hashtable.cpp ! src/share/vm/utilities/hashtable.hpp Changeset: e37a5219e297 Author: dcubed Date: 2012-07-31 18:37 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/e37a5219e297 Merge Changeset: 3b3ad1642970 Author: amurillo Date: 2012-08-03 13:13 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/3b3ad1642970 Merge Changeset: 663fc23da8d5 Author: amurillo Date: 2012-08-03 13:13 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/663fc23da8d5 Added tag hs24-b19 for changeset 3b3ad1642970 ! .hgtags Changeset: ef437ea56651 Author: amurillo Date: 2012-08-03 13:24 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/ef437ea56651 7189086: new hotspot build - hs24-b20 Reviewed-by: jcoomes ! make/hotspot_version From john.cuthbertson at oracle.com Mon Aug 6 18:10:08 2012 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Mon, 06 Aug 2012 11:10:08 -0700 Subject: RFR(S): 6818524: G1: use ergonomic resizing of PLABs In-Reply-To: References: <50199B89.1060703@oracle.com> Message-ID: <50200880.7070200@oracle.com> Hi Ramki, Thanks for the review. Using the containing heap or generation to bound the PLAB sizes is a good idea. I agree that it would be better to do the work under a separate CR - I'll submit an RFE. JohnC On 08/02/12 11:24, Srinivas Ramakrishna wrote: > Hi John, Brandon -- > > This looks good to me. > > One improvement that could be made (but which predates any of these > changes) is that the ParGCAllocBuffer's min_size() and max_size() > just return the lower and upper bounds set for TLAB's. Rather these > should be set (perhaps statically if that is possible) by the > containing heap or generation. > The PLAB sizing calculation already does the requisite clipping, so if > the min and max sizes were properly configured you would likely not > need to do the clipping from > above that you now have to do to keep G1's ergonomic PLAB sizes from > becoming humongous. > > But that can be a separate reorg sice it will touch a bit more code. > > Otherwise this looks great! > > thanks. > -- ramki > > On Wed, Aug 1, 2012 at 2:11 PM, John Cuthbertson > > wrote: > > Hi Everyone, > > Can I have a couple of volunteers review the changes for this CR? > The webrev can be found at: > http://cr.openjdk.java.net/~johnc/6818524/webrev.0/ > . These > changes have been contributed by Brandon Mitchell (at Twitter). > > Summary: > PLABStats instances are now employed in G1 to record information > about old and young PLABs, and they are then used to adjust the > PLAB sizes for the next GC. As a result the ResizePLAB flag is now > being observed by G1. The previous behavior (and PLAB sizes) can > be obtained by specifying -XX:-ResizePLAB. > > With these changes I see just over a net 1% performance gain, on > our reference workload, when running with G1. > > Testing: > GC test suite with and without ResizePLAB and PrintPLAB enabled to > observe the changes in PLAB sizes; reference workload on multiple > platforms; jprt. > > Thanks, > > JohnC > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.cuthbertson at oracle.com Mon Aug 6 22:08:00 2012 From: john.cuthbertson at oracle.com (john.cuthbertson at oracle.com) Date: Mon, 06 Aug 2012 22:08:00 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 6818524: G1: use ergonomic resizing of PLABs Message-ID: <20120806220803.10F77473C1@hg.openjdk.java.net> Changeset: aaf61e68b255 Author: johnc Date: 2012-08-06 12:20 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/aaf61e68b255 6818524: G1: use ergonomic resizing of PLABs Summary: Employ PLABStats instances to record information about survivor and old PLABs, and use the recorded stats to adjust the sizes of survivor and old PLABS. Reviewed-by: johnc, ysr Contributed-by: Brandon Mitchell ! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp - src/share/vm/gc_implementation/parNew/parGCAllocBuffer.cpp - src/share/vm/gc_implementation/parNew/parGCAllocBuffer.hpp ! src/share/vm/gc_implementation/parNew/parNewGeneration.cpp ! src/share/vm/gc_implementation/parNew/parNewGeneration.hpp + src/share/vm/gc_implementation/shared/parGCAllocBuffer.cpp + src/share/vm/gc_implementation/shared/parGCAllocBuffer.hpp ! src/share/vm/memory/tenuredGeneration.cpp ! src/share/vm/precompiled/precompiled.hpp From Michal.Frajt at partner.commerzbank.com Tue Aug 7 13:16:57 2012 From: Michal.Frajt at partner.commerzbank.com (Frajt, Michal) Date: Tue, 7 Aug 2012 15:16:57 +0200 Subject: CMSWaitDuration unstable behavior Message-ID: <1DDCA93502632C4DA22E9EE199CE907C4FD1647A@SE002593.cs.commerzbank.com> Hi all, We are using the incremental CMS collector for many years. We have a distributed application framework based on the subscribe-unsubscribe model where the data unsubscriptions are handled by the application layer just forgetting the strong reference to the distributed data. The underlying application framework layer is using weak references to trace the data requirement from the application layer. We keep the old generation processed permanently (incrementally) to get the week references released and reported within a short period of time (minutes). Unfortunately the incremental mode is missing the support for the CMSWaitDuration to place the initial mark phase right after the young space collection. With some new gen sizing optimization we went to a situation when the new gen is more or less big enough to keep the most of live objects with only a few promotions to the old gen. The incremental CMS is then started every minute in a random moment with pretty garbaged new gen. The initial mark takes 20-50 times more than a single new gen processing (40ms new gen, initial mark 1100ms). We decided to customize the OpenJDK 6 by adding the incremental mode CMSWaitDuration support. We took the same approach as the wait_on_cms_lock method does with the CGC_lock object. Unfortunately we realized that the CGC_lock mutex is additionally notified in some other situation than the young space collection finishing. The young space collection unrelated notifications are coming from the desynchronize method invocations. These unrelated notifications are causing the wait_on_cms_lock to return earlier than required. The initial mark phase is started before the young space collection even there is enough wait duration time specified to wait. We have fixed it by waiting again if the GenCollectedHeap::heap()->total_collections() counter is not changed after the CGC_long->wait method returns but not longer than the CMSWaitDuration in total. The initial mark is then always placed (if CMSWaitDuration is long enough) after the young space collection. Every initial mark phase takes no longer than 17ms (previously 1100ms). We tested the CMSWaitDuration behavior in the normal CMS mode. We specified the -XX:+UseCMSInitiatingOccupancyOnly and -XX:CMSInitiatingOccupancyFraction=10 to force the CMS running permanently (shouldConcurrentCollect should be returning true). The CMS initial-mark is many times started without waiting for the young space collection which makes the initial marking running 20-50 longer. We find this as unstable behavior of the CMSWaitDuration implementation related to the problem of the wait-notify signaling on the CGC_lock object. We disabled the explicit GC invocation (-XX:+DisableExplicitGC) to be sure there is no other reason to start the CMS initial mark phase before the young space collection. Is there any plan to get the CMSWaitDuration supported in the incremental mode and/or get it fixed in the normal mode? Thanks, Michal Frajt From kirk at kodewerk.com Wed Aug 8 11:56:02 2012 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Wed, 8 Aug 2012 13:56:02 +0200 Subject: RFR(S): 6818524: G1: use ergonomic resizing of PLABs In-Reply-To: <50200880.7070200@oracle.com> References: <50199B89.1060703@oracle.com> <50200880.7070200@oracle.com> Message-ID: <384C520D-25DE-4607-95A3-A3ABB9238302@kodewerk.com> Hi, I'm looking at this record. 919.673: [GC-- [PSYoungGen: 643582K->643582K(648704K)] 1905561K->2004508K(2014080K), 0.0319900 secs] This implies that there are +98947K bytes consumed during the collection. However, this is a STW collection implying that the source of the heap consumption is the garbage collector it's self which implies that objects are copied and then removed from young gen. If so, does this mean that there is a failure in that the VM have recognized that heap occupancy was @ 94.6% prior to the collection implying that is was very likely that tenured would be flooded by scavenge? Regards, Kirk From jon.masamitsu at oracle.com Wed Aug 8 15:11:07 2012 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 08 Aug 2012 08:11:07 -0700 Subject: CMSWaitDuration unstable behavior In-Reply-To: <1DDCA93502632C4DA22E9EE199CE907C4FD1647A@SE002593.cs.commerzbank.com> References: <1DDCA93502632C4DA22E9EE199CE907C4FD1647A@SE002593.cs.commerzbank.com> Message-ID: <5022818B.6070502@oracle.com> Michal, The engineer with the most experience on CMS left Oracle and I suspect this is not going to get fixed in the way you want. I've create CR 7189971 to capture your comments and it will be reviewed along with other RFE's for CMS but I would not be optimistic. Since you are customizing your own VM, did you consider explicitly invoking a young collection before the initial mark the way that it is done for the remark phase with the flag CMSScavengeBeforeRemark Jon On 8/7/2012 6:16 AM, Frajt, Michal wrote: > Hi all, > > We are using the incremental CMS collector for many years. We have a distributed application framework based on the subscribe-unsubscribe model where the data unsubscriptions are handled by the application layer just forgetting the strong reference to the distributed data. The underlying application framework layer is using weak references to trace the data requirement from the application layer. We keep the old generation processed permanently (incrementally) to get the week references released and reported within a short period of time (minutes). > > Unfortunately the incremental mode is missing the support for the CMSWaitDuration to place the initial mark phase right after the young space collection. With some new gen sizing optimization we went to a situation when the new gen is more or less big enough to keep the most of live objects with only a few promotions to the old gen. The incremental CMS is then started every minute in a random moment with pretty garbaged new gen. The initial mark takes 20-50 times more than a single new gen processing (40ms new gen, initial mark 1100ms). > > We decided to customize the OpenJDK 6 by adding the incremental mode CMSWaitDuration support. We took the same approach as the wait_on_cms_lock method does with the CGC_lock object. Unfortunately we realized that the CGC_lock mutex is additionally notified in some other situation than the young space collection finishing. The young space collection unrelated notifications are coming from the desynchronize method invocations. These unrelated notifications are causing the wait_on_cms_lock to return earlier than required. The initial mark phase is started before the young space collection even there is enough wait duration time specified to wait. We have fixed it by waiting again if the GenCollectedHeap::heap()->total_collections() counter is not changed after the CGC_long->wait method returns but not longer than the CMSWaitDuration in total. The initial mark is then always placed (if CMSWaitDuration is long enough) after the young space collection. Every initial mark phase takes no longer than 17ms (previously 1100ms). > > We tested the CMSWaitDuration behavior in the normal CMS mode. We specified the -XX:+UseCMSInitiatingOccupancyOnly and -XX:CMSInitiatingOccupancyFraction=10 to force the CMS running permanently (shouldConcurrentCollect should be returning true). The CMS initial-mark is many times started without waiting for the young space collection which makes the initial marking running 20-50 longer. We find this as unstable behavior of the CMSWaitDuration implementation related to the problem of the wait-notify signaling on the CGC_lock object. We disabled the explicit GC invocation (-XX:+DisableExplicitGC) to be sure there is no other reason to start the CMS initial mark phase before the young space collection. > > Is there any plan to get the CMSWaitDuration supported in the incremental mode and/or get it fixed in the normal mode? > > Thanks, > Michal Frajt > > From ysr1729 at gmail.com Wed Aug 8 18:56:53 2012 From: ysr1729 at gmail.com (Srinivas Ramakrishna) Date: Wed, 8 Aug 2012 11:56:53 -0700 Subject: CMSWaitDuration unstable behavior In-Reply-To: <5022818B.6070502@oracle.com> References: <1DDCA93502632C4DA22E9EE199CE907C4FD1647A@SE002593.cs.commerzbank.com> <5022818B.6070502@oracle.com> Message-ID: Hi Michal -- There's an RFE (lost in the mists of time) to piggyback initial marking work on a ParNew collection (and hence do it multi-threaded rather than single-threaded as is the case currently). But it never got implemented, unfortunately. That said, I understand your motivation is to reduce the duration of the initial mark pause in the face of using a large Eden space which is currently marked single-threaded. Unfortunately, CMSWaitDuration was never meant to control the scheduling of the initial mark pause in relation to the scavenge. Rather it was meant to be a maximum wait time for which the CMS collector would wait for a scavenge to occur -- if a scavenge did not occur within that time, CMS might decide to unequivocally take the action it might otherwise have taken immediately after the scavenge -- such as polling the old generation occupancy to decide if a new CMS cycle should start (hence initiating a new initial-mark pause), or if the "abortable preclean phase" should be exited in the absence of a scavenge occurring suitably soon. Thus, trying to retrofit CMSWaitDuration to meet your purpose of co-scheduling initial mark after scavenge is probably not the right thing to do. I think the easiest thing to do, as Jon suggested, is to have an explicit flag such as CMSScavengeBeforeInitialMark which would be analogous to the current role of CMSScavengeBeforeRemark. Here, ICMS wakes up at the normal time and takes control, but instead of doing an initial mark straightaay, it first initiates a parallel scavenge and follows that up with a single-threaded initial mark. Granted this will not cause an initial mark step to occur immediately after a "normal" scavenge as we really want, but rather cause an additional scavenge to happen just before an initial mark pause is scheduled in ICMS (exactly as is the case with the current CMSScavengeBeforeRemark where an extra scavenge occurs which is not very pleasant), but it would be far easier to implement without making any other changes in the system. The best solution of course is to implement the RFE to do the initial mark in parallel piggybacked on the scavenge and all your problems go away (ICMS may need a very minor adjustment for that). Anyone want to take a stab at parallelizing and piggybacking initial mark on scavenge? It would be a matter of extending the scavenge object and root scanning closures to new closures so as to not skip the references that point outside of young gen as is done for the normal parnew scanning closures, but to mark the appropriate bits in the CMS marking bit map. That's really theoretically all it will take. PS: Jon, if Michal takes the approach of CMSScavengeBeforeInitialMark, I'd say it would be useful to the broader community (not just ICMS users) if that were integrated into the main-line code, as it would be a via-media for CMS scaling in the absence of the piggybacking RFE which is really the best solution here. thanks! -- ramki On Wed, Aug 8, 2012 at 8:11 AM, Jon Masamitsu wrote: > Michal, > > The engineer with the most experience on CMS left Oracle > and I suspect this is not going to get fixed in the way you want. > > I've create CR 7189971 to capture your comments and it will be > reviewed along with other RFE's for CMS but I would not be > optimistic. > > Since you are customizing your own VM, did you consider > explicitly invoking a young collection before the initial mark > the way that it is done for the remark phase with the flag > > CMSScavengeBeforeRemark > > Jon > > > On 8/7/2012 6:16 AM, Frajt, Michal wrote: > >> Hi all, >> >> We are using the incremental CMS collector for many years. We have a >> distributed application framework based on the subscribe-unsubscribe model >> where the data unsubscriptions are handled by the application layer just >> forgetting the strong reference to the distributed data. The underlying >> application framework layer is using weak references to trace the data >> requirement from the application layer. We keep the old generation >> processed permanently (incrementally) to get the week references released >> and reported within a short period of time (minutes). >> >> Unfortunately the incremental mode is missing the support for the >> CMSWaitDuration to place the initial mark phase right after the young space >> collection. With some new gen sizing optimization we went to a situation >> when the new gen is more or less big enough to keep the most of live >> objects with only a few promotions to the old gen. The incremental CMS is >> then started every minute in a random moment with pretty garbaged new gen. >> The initial mark takes 20-50 times more than a single new gen processing >> (40ms new gen, initial mark 1100ms). >> >> We decided to customize the OpenJDK 6 by adding the incremental mode >> CMSWaitDuration support. We took the same approach as the wait_on_cms_lock >> method does with the CGC_lock object. Unfortunately we realized that the >> CGC_lock mutex is additionally notified in some other situation than the >> young space collection finishing. The young space collection unrelated >> notifications are coming from the desynchronize method invocations. These >> unrelated notifications are causing the wait_on_cms_lock to return earlier >> than required. The initial mark phase is started before the young space >> collection even there is enough wait duration time specified to wait. We >> have fixed it by waiting again if the GenCollectedHeap::heap()->**total_collections() >> counter is not changed after the CGC_long->wait method returns but not >> longer than the CMSWaitDuration in total. The initial mark is then always >> placed (if CMSWaitDuration is long enough) after the young space >> collection. Every initial mark phase takes no longer than 17ms (previously >> 1100ms). >> >> We tested the CMSWaitDuration behavior in the normal CMS mode. We >> specified the -XX:+**UseCMSInitiatingOccupancyOnly and -XX:** >> CMSInitiatingOccupancyFraction**=10 to force the CMS running permanently >> (shouldConcurrentCollect should be returning true). The CMS initial-mark is >> many times started without waiting for the young space collection which >> makes the initial marking running 20-50 longer. We find this as unstable >> behavior of the CMSWaitDuration implementation related to the problem of >> the wait-notify signaling on the CGC_lock object. We disabled the explicit >> GC invocation (-XX:+DisableExplicitGC) to be sure there is no other reason >> to start the CMS initial mark phase before the young space collection. >> >> Is there any plan to get the CMSWaitDuration supported in the incremental >> mode and/or get it fixed in the normal mode? >> >> Thanks, >> Michal Frajt >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ysr1729 at gmail.com Wed Aug 8 19:15:44 2012 From: ysr1729 at gmail.com (Srinivas Ramakrishna) Date: Wed, 8 Aug 2012 12:15:44 -0700 Subject: GC-- PSYoungGen Message-ID: [note: subject line corrected since this is unrelated to the thread in which it was posted] Hi Kirk -- This is called a promotion failure, and causes the scavenge to be unwound and followed up with a major collection of the whole heap. The "apparent" increase in heap usage is because some objects got copied into the old and survivor spaces, but the scavenge could not complete because it ran out of space to store further survivors from the young gen. The "unwinding" step does not claim back the space freed up by the previous copying step so looks like an increase in heap usage. I'd ignore the "after" size for such failed scavenges, which are denoted in the logs with the GC-- notation. The "--" indicates a failure of the scavenge mid-way. (Aside: I don't think the PrintGCStats script does that, so it would often show up as an unexplained increase in the heap size after the failed scavenge. This is a shortcoming of the script that can and should be fixed.) There are situations where the GC will not even attempt a scavenge based on recent promotion volume history and current space available in the old gen. This was not such a case -- in such cases, you will just see a full gc occur straightaway without a scavenge having been attempted -- in general the PSScavenge collector looks ahead one scavenge and will often invoke first a scavenge and immediately follow that up with a full gc cycle because it believes that the next scavenge would not succeed. If there is a sudden phase change or if the promotion volume is bursty, that look-ahead may sometimes be confounded and can both be either too conservative, causing a major cycle before it might be strictly necessary, or sometimes fail to cause a major cycle soon enough and run into a promotion failure, such as happened below for you. -- ramki On Wed, Aug 8, 2012 at 4:56 AM, Kirk Pepperdine wrote: > Hi, > > I'm looking at this record. > > 919.673: [GC-- [PSYoungGen: 643582K->643582K(648704K)] > 1905561K->2004508K(2014080K), 0.0319900 secs] > > This implies that there are +98947K bytes consumed during the collection. > However, this is a STW collection implying that the source of the heap > consumption is the garbage collector it's self which implies that objects > are copied and then removed from young gen. If so, does this mean that > there is a failure in that the VM have recognized that heap occupancy was @ > 94.6% prior to the collection implying that is was very likely that tenured > would be flooded by scavenge? > > Regards, > Kirk > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alejandro.murillo at oracle.com Wed Aug 8 19:39:31 2012 From: alejandro.murillo at oracle.com (alejandro.murillo at oracle.com) Date: Wed, 08 Aug 2012 19:39:31 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 7189729: jprt.properties should include release jdk7u8 Message-ID: <20120808193935.A04E747424@hg.openjdk.java.net> Changeset: eff5d59db7e1 Author: amurillo Date: 2012-08-07 09:53 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/eff5d59db7e1 7189729: jprt.properties should include release jdk7u8 Reviewed-by: jcoomes ! make/jprt.properties From kirk at kodewerk.com Wed Aug 8 19:41:20 2012 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Wed, 8 Aug 2012 21:41:20 +0200 Subject: GC-- PSYoungGen In-Reply-To: References: Message-ID: Hi Ramki, Thanks for the detailed explanation. It confirms my assumption that this is a phantom increase. You don't see many GC--'s showing up. My tooling ignores the change in volume but does count the occurrence and pause. As for this app, it's sort of a microbenchmark but unlike most benchmarks, the creation rate is pretty steady at the beginning but then starts falling off and might become bursty as the calculation progresses. Even so I was a bit surprised to see a PS attempted with heap occupancy > 90% -- Kirk On 2012-08-08, at 9:15 PM, Srinivas Ramakrishna wrote: > [note: subject line corrected since this is unrelated to the thread in which it was posted] > > Hi Kirk -- > > This is called a promotion failure, and causes the scavenge to be unwound and followed up with a major collection of the whole heap. > The "apparent" increase in heap usage is because some objects got copied into the old and survivor spaces, but the scavenge could not > complete because it ran out of space to store further survivors from the young gen. The "unwinding" step does not claim back the space > freed up by the previous copying step so looks like an increase in heap usage. I'd ignore the "after" size for such failed scavenges, which are > denoted in the logs with the GC-- notation. The "--" indicates a failure of the scavenge mid-way. (Aside: I don't think the PrintGCStats script does that, > so it would often show up as an unexplained increase in the heap size after the failed scavenge. This is a shortcoming of the script that > can and should be fixed.) > > There are situations where the GC will not even attempt a scavenge based on recent promotion volume history and current space > available in the old gen. This was not such a case -- in such cases, you will just see a full gc occur straightaway without a > scavenge having been attempted -- in general the PSScavenge collector looks ahead one scavenge and will often invoke first > a scavenge and immediately follow that up with a full gc cycle because it believes that the next scavenge would not succeed. > If there is a sudden phase change or if the promotion volume is bursty, that look-ahead may sometimes be confounded and can > both be either too conservative, causing a major cycle before it might be strictly necessary, or sometimes fail to cause a major cycle > soon enough and run into a promotion failure, such as happened below for you. > > -- ramki > > On Wed, Aug 8, 2012 at 4:56 AM, Kirk Pepperdine wrote: > Hi, > > I'm looking at this record. > > 919.673: [GC-- [PSYoungGen: 643582K->643582K(648704K)] 1905561K->2004508K(2014080K), 0.0319900 secs] > > This implies that there are +98947K bytes consumed during the collection. However, this is a STW collection implying that the source of the heap consumption is the garbage collector it's self which implies that objects are copied and then removed from young gen. If so, does this mean that there is a failure in that the VM have recognized that heap occupancy was @ 94.6% prior to the collection implying that is was very likely that tenured would be flooded by scavenge? > > Regards, > Kirk > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sitnikov.vladimir at gmail.com Wed Aug 8 20:51:17 2012 From: sitnikov.vladimir at gmail.com (Vladimir Sitnikov) Date: Thu, 9 Aug 2012 00:51:17 +0400 Subject: RFR(S): 6818524: G1: use ergonomic resizing of PLABs In-Reply-To: <384C520D-25DE-4607-95A3-A3ABB9238302@kodewerk.com> References: <50199B89.1060703@oracle.com> <50200880.7070200@oracle.com> <384C520D-25DE-4607-95A3-A3ABB9238302@kodewerk.com> Message-ID: 2012/8/8 Kirk Pepperdine > Hi, > > I'm looking at this record. > > 919.673: [GC-- [PSYoungGen: 643582K->643582K(648704K)] > 1905561K->2004508K(2014080K), 0.0319900 secs] > > Are you sure this is a G1 gc log? Those "GC--" stand for promotion failure case (VM have recognized that is was very likely that tenured would be flooded by scavenge). Please, refer to http://hg.openjdk.java.net/jdk6/jdk6/hotspot/file/a541ca8fa0e3/src/share/vm/gc_implementation/parallelScavenge/psScavenge.cpp 445 promotion_failure_occurred = promotion_failed(); 446 if (promotion_failure_occurred) { 447 clean_up_failed_promotion(); 448 if (PrintGC) { 449 gclog_or_tty->print("--"); 450 } 451 } -- Regards, Vladimir Sitnikov -------------- next part -------------- An HTML attachment was scrubbed... URL: From chunt at salesforce.com Wed Aug 8 21:24:10 2012 From: chunt at salesforce.com (Charlie Hunt) Date: Wed, 8 Aug 2012 14:24:10 -0700 Subject: GC-- PSYoungGen In-Reply-To: References: Message-ID: Hi Ramki, Very nice explanation! Thanks for documenting & sharing. :-) charlie .... On Aug 8, 2012, at 12:15 PM, Srinivas Ramakrishna wrote: [note: subject line corrected since this is unrelated to the thread in which it was posted] Hi Kirk -- This is called a promotion failure, and causes the scavenge to be unwound and followed up with a major collection of the whole heap. The "apparent" increase in heap usage is because some objects got copied into the old and survivor spaces, but the scavenge could not complete because it ran out of space to store further survivors from the young gen. The "unwinding" step does not claim back the space freed up by the previous copying step so looks like an increase in heap usage. I'd ignore the "after" size for such failed scavenges, which are denoted in the logs with the GC-- notation. The "--" indicates a failure of the scavenge mid-way. (Aside: I don't think the PrintGCStats script does that, so it would often show up as an unexplained increase in the heap size after the failed scavenge. This is a shortcoming of the script that can and should be fixed.) There are situations where the GC will not even attempt a scavenge based on recent promotion volume history and current space available in the old gen. This was not such a case -- in such cases, you will just see a full gc occur straightaway without a scavenge having been attempted -- in general the PSScavenge collector looks ahead one scavenge and will often invoke first a scavenge and immediately follow that up with a full gc cycle because it believes that the next scavenge would not succeed. If there is a sudden phase change or if the promotion volume is bursty, that look-ahead may sometimes be confounded and can both be either too conservative, causing a major cycle before it might be strictly necessary, or sometimes fail to cause a major cycle soon enough and run into a promotion failure, such as happened below for you. -- ramki On Wed, Aug 8, 2012 at 4:56 AM, Kirk Pepperdine > wrote: Hi, I'm looking at this record. 919.673: [GC-- [PSYoungGen: 643582K->643582K(648704K)] 1905561K->2004508K(2014080K), 0.0319900 secs] This implies that there are +98947K bytes consumed during the collection. However, this is a STW collection implying that the source of the heap consumption is the garbage collector it's self which implies that objects are copied and then removed from young gen. If so, does this mean that there is a failure in that the VM have recognized that heap occupancy was @ 94.6% prior to the collection implying that is was very likely that tenured would be flooded by scavenge? Regards, Kirk -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon.masamitsu at oracle.com Wed Aug 8 23:45:18 2012 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 08 Aug 2012 16:45:18 -0700 Subject: CMSWaitDuration unstable behavior In-Reply-To: References: <1DDCA93502632C4DA22E9EE199CE907C4FD1647A@SE002593.cs.commerzbank.com> <5022818B.6070502@oracle.com> Message-ID: <5022FA0E.7080706@oracle.com> On 8/8/2012 11:56 AM, Srinivas Ramakrishna wrote: > ... > > PS: Jon, if Michal takes the approach of CMSScavengeBeforeInitialMark, I'd > say it would be useful to the broader community (not > just ICMS users) if that were integrated into the main-line code, as it > would be a via-media for CMS scaling in the absence of the > piggybacking RFE which is really the best solution here. Agreed. Jon > thanks! > -- ramki > > On Wed, Aug 8, 2012 at 8:11 AM, Jon Masamitsuwrote: > >> Michal, >> >> The engineer with the most experience on CMS left Oracle >> and I suspect this is not going to get fixed in the way you want. >> >> I've create CR 7189971 to capture your comments and it will be >> reviewed along with other RFE's for CMS but I would not be >> optimistic. >> >> Since you are customizing your own VM, did you consider >> explicitly invoking a young collection before the initial mark >> the way that it is done for the remark phase with the flag >> >> CMSScavengeBeforeRemark >> >> Jon >> >> >> On 8/7/2012 6:16 AM, Frajt, Michal wrote: >> >>> Hi all, >>> >>> We are using the incremental CMS collector for many years. We have a >>> distributed application framework based on the subscribe-unsubscribe model >>> where the data unsubscriptions are handled by the application layer just >>> forgetting the strong reference to the distributed data. The underlying >>> application framework layer is using weak references to trace the data >>> requirement from the application layer. We keep the old generation >>> processed permanently (incrementally) to get the week references released >>> and reported within a short period of time (minutes). >>> >>> Unfortunately the incremental mode is missing the support for the >>> CMSWaitDuration to place the initial mark phase right after the young space >>> collection. With some new gen sizing optimization we went to a situation >>> when the new gen is more or less big enough to keep the most of live >>> objects with only a few promotions to the old gen. The incremental CMS is >>> then started every minute in a random moment with pretty garbaged new gen. >>> The initial mark takes 20-50 times more than a single new gen processing >>> (40ms new gen, initial mark 1100ms). >>> >>> We decided to customize the OpenJDK 6 by adding the incremental mode >>> CMSWaitDuration support. We took the same approach as the wait_on_cms_lock >>> method does with the CGC_lock object. Unfortunately we realized that the >>> CGC_lock mutex is additionally notified in some other situation than the >>> young space collection finishing. The young space collection unrelated >>> notifications are coming from the desynchronize method invocations. These >>> unrelated notifications are causing the wait_on_cms_lock to return earlier >>> than required. The initial mark phase is started before the young space >>> collection even there is enough wait duration time specified to wait. We >>> have fixed it by waiting again if the GenCollectedHeap::heap()->**total_collections() >>> counter is not changed after the CGC_long->wait method returns but not >>> longer than the CMSWaitDuration in total. The initial mark is then always >>> placed (if CMSWaitDuration is long enough) after the young space >>> collection. Every initial mark phase takes no longer than 17ms (previously >>> 1100ms). >>> >>> We tested the CMSWaitDuration behavior in the normal CMS mode. We >>> specified the -XX:+**UseCMSInitiatingOccupancyOnly and -XX:** >>> CMSInitiatingOccupancyFraction**=10 to force the CMS running permanently >>> (shouldConcurrentCollect should be returning true). The CMS initial-mark is >>> many times started without waiting for the young space collection which >>> makes the initial marking running 20-50 longer. We find this as unstable >>> behavior of the CMSWaitDuration implementation related to the problem of >>> the wait-notify signaling on the CGC_lock object. We disabled the explicit >>> GC invocation (-XX:+DisableExplicitGC) to be sure there is no other reason >>> to start the CMS initial mark phase before the young space collection. >>> >>> Is there any plan to get the CMSWaitDuration supported in the incremental >>> mode and/or get it fixed in the normal mode? >>> >>> Thanks, >>> Michal Frajt >>> >>> >>> From suenaga.yasumasa at lab.ntt.co.jp Thu Aug 9 01:38:11 2012 From: suenaga.yasumasa at lab.ntt.co.jp (Yasumasa Suenaga) Date: Thu, 09 Aug 2012 10:38:11 +0900 Subject: 6950794: Make the GC log file name parameterized In-Reply-To: <501093B7.7030300@lab.ntt.co.jp> References: <5007BDC8.2080905@lab.ntt.co.jp> <50081F1A.1030603@kippdata.de> <5008AD1F.3@lab.ntt.co.jp> <501093B7.7030300@lab.ntt.co.jp> Message-ID: <50231483.2060701@lab.ntt.co.jp> Hi, A few weeks ago, I posted a patch for implementing this RFE. @OpenJDK tweets my proposal :-) http://twitter.com/OpenJDK/status/232506237546295297 Advantages of this patch is that name of gc log can be defined by state of run-time. So we can define gc log name which depend on PID, start date of VM. And we can find the newest or the oldest log easily. For example, multiple VM is running on 1 OS and each VM writes gc log. Each VM is same setting and runs same application. If we set "-Xloggc:gc-%p.log", we can find log of specific VM easily. I'm sure that this patch helps us to operate gc logs. Please cooperate. Thanks, Yasumasa On 2012/07/26 9:47, Yasumasa Suenaga wrote: > Hi, > > I've made a patch for this RFE. > This patch allows for 4 parameters as following: > > %p - Java process' PID > %t - date stamp when log file is created (format: YYYY-MM-DD) > %n - if log rotation logic is enabled, then log segment id, otherwise "0" > %% - escaped '%' character in file name > > (These parameters are defined in 6950794) > > > Other features of this patch: > > * The oldest log is removed. If you set -XX:NumberOfGCLogFiles=5 and > gc.log.0 - 4 is exists, gc.log.0 is removed and gc.log.5 is created. > * Log segment id (%n) is unsigned int . If id is UINT_MAX (0xFFFFFFFF), > next id is zero. > * I expanded Arguments::copy_expand_pid() . This modification affects > -XX:OnError, -XX:ErrorFile, -XX:PerfDataSaveFile . These option also > can use parameters that this patch provides. > If this is not good, I can separate this feature from Arguments::copy_expand_pid() . > > > This patch works fine in my environment. > I would like to contribute this patch. > > Please cooperate. > > > Thanks, > > Yasumasa > > > On 2012/07/20 9:58, Yasumasa Suenaga wrote: >> Hi Rainer, >> >> Do you point the following as "negative consequences" ? : >> >> http://mail.openjdk.java.net/pipermail/hotspot-gc-use/2010-May/000613.html >> ---------------- >> * "f00.00000001 might have been detected as old and copied to the remote >> host and during the same time GC decides to now reuse it... That's why >> I personally find externally organized pruning better. Another thing I >> often miss is the ability to combine size and time based rotation." (Rainer) >> >> The proposal never reuses log files. We'll never overwrite anything. >> Instead, we'll delete the oldest files as we create new ones. If we tell >> the users to prune the older log files themselves, I know what the first >> bug filed against the new policy will be. :-) Regarding rotating based >> on both size and time: most people care about size so I think that's >> what we'll do. If you want more advanced management of the logs you'll >> have to set N to infinity (at least we'll need a way to say "never >> delete older files") so that HotSpot doesn't delete any files and you'll >> be able to copy them and delete them yourself. >> >> But, seriously, this is excellent feedback. You guys are doing more wild >> stuff with our logs than I had imagined. :-) >> ---------------- >> >> Tony says "The proposal never reuses log files. We'll never overwrite anything." >> However, seems to reuse the oldest log :-) >> >> hotspot/src/share/vm/utilities/ostream.cpp: >> void rotatingFileStream::rotate_log() >> >> >> We can find the oldest log with "stat" command and can check mtime of >> all logs on Linux. >> I think that "mtime" is updated every write(2) syscall . >> >> At least, status of this RFE is "3-Accepted" . >> So I believe that this RFE will be merge mainline someday :-) >> >> >> Thanks, >> >> Yasumasa >> >> On 2012/07/19 23:52, Rainer Jung wrote: >>> On 19.07.2012 09:56, Yasumasa Suenaga wrote: >>>> Hi, >>>> >>>> I use GC log rotation with -XX:+UseGCLogFileRotation . >>>> However, suffix of logfile is fixed ( .N : cyclic 0-(NumberOfGCLogFiles-1) ) . >>>> So I'm not easy to find the oldest log. >>>> (I have to check timestamp of file or GC event time.) >>> >>> See the discussion thread starting at >>> >>> http://mail.openjdk.java.net/pipermail/hotspot-gc-use/2010-May/000597.html >>> >>> including a reply of mine on the negative consequences of the numeric naming scheme and responses of Tony to the comments on his proposal. >>> >>> Regards, >>> >>> Rainer >>> >>>> I hope that this RFE is merged to JDK6/7/8. >>>> >>>> Someone is working on this RFE ? >>>> If none, I would like to contribute a patch. >>>> (Then, please someone become a sponsor of me :-) ) > From kirk at kodewerk.com Thu Aug 9 07:22:42 2012 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Thu, 9 Aug 2012 09:22:42 +0200 Subject: CMSWaitDuration unstable behavior In-Reply-To: <5022FA0E.7080706@oracle.com> References: <1DDCA93502632C4DA22E9EE199CE907C4FD1647A@SE002593.cs.commerzbank.com> <5022818B.6070502@oracle.com> <5022FA0E.7080706@oracle.com> Message-ID: Hi, +1 on Ramki's comment. IME, forcing a scavenge prior to the remark has always been beneficial and for the same underlying reasons I don't see why it wouldn't benefit the initial mark. Regards, Kirk On 2012-08-09, at 1:45 AM, Jon Masamitsu wrote: > > > On 8/8/2012 11:56 AM, Srinivas Ramakrishna wrote: >> ... >> >> PS: Jon, if Michal takes the approach of CMSScavengeBeforeInitialMark, I'd >> say it would be useful to the broader community (not >> just ICMS users) if that were integrated into the main-line code, as it >> would be a via-media for CMS scaling in the absence of the >> piggybacking RFE which is really the best solution here. > > Agreed. > > > Jon > >> thanks! >> -- ramki >> >> On Wed, Aug 8, 2012 at 8:11 AM, Jon Masamitsuwrote: >> >>> Michal, >>> >>> The engineer with the most experience on CMS left Oracle >>> and I suspect this is not going to get fixed in the way you want. >>> >>> I've create CR 7189971 to capture your comments and it will be >>> reviewed along with other RFE's for CMS but I would not be >>> optimistic. >>> >>> Since you are customizing your own VM, did you consider >>> explicitly invoking a young collection before the initial mark >>> the way that it is done for the remark phase with the flag >>> >>> CMSScavengeBeforeRemark >>> >>> Jon >>> >>> >>> On 8/7/2012 6:16 AM, Frajt, Michal wrote: >>> >>>> Hi all, >>>> >>>> We are using the incremental CMS collector for many years. We have a >>>> distributed application framework based on the subscribe-unsubscribe model >>>> where the data unsubscriptions are handled by the application layer just >>>> forgetting the strong reference to the distributed data. The underlying >>>> application framework layer is using weak references to trace the data >>>> requirement from the application layer. We keep the old generation >>>> processed permanently (incrementally) to get the week references released >>>> and reported within a short period of time (minutes). >>>> >>>> Unfortunately the incremental mode is missing the support for the >>>> CMSWaitDuration to place the initial mark phase right after the young space >>>> collection. With some new gen sizing optimization we went to a situation >>>> when the new gen is more or less big enough to keep the most of live >>>> objects with only a few promotions to the old gen. The incremental CMS is >>>> then started every minute in a random moment with pretty garbaged new gen. >>>> The initial mark takes 20-50 times more than a single new gen processing >>>> (40ms new gen, initial mark 1100ms). >>>> >>>> We decided to customize the OpenJDK 6 by adding the incremental mode >>>> CMSWaitDuration support. We took the same approach as the wait_on_cms_lock >>>> method does with the CGC_lock object. Unfortunately we realized that the >>>> CGC_lock mutex is additionally notified in some other situation than the >>>> young space collection finishing. The young space collection unrelated >>>> notifications are coming from the desynchronize method invocations. These >>>> unrelated notifications are causing the wait_on_cms_lock to return earlier >>>> than required. The initial mark phase is started before the young space >>>> collection even there is enough wait duration time specified to wait. We >>>> have fixed it by waiting again if the GenCollectedHeap::heap()->**total_collections() >>>> counter is not changed after the CGC_long->wait method returns but not >>>> longer than the CMSWaitDuration in total. The initial mark is then always >>>> placed (if CMSWaitDuration is long enough) after the young space >>>> collection. Every initial mark phase takes no longer than 17ms (previously >>>> 1100ms). >>>> >>>> We tested the CMSWaitDuration behavior in the normal CMS mode. We >>>> specified the -XX:+**UseCMSInitiatingOccupancyOnly and -XX:** >>>> CMSInitiatingOccupancyFraction**=10 to force the CMS running permanently >>>> (shouldConcurrentCollect should be returning true). The CMS initial-mark is >>>> many times started without waiting for the young space collection which >>>> makes the initial marking running 20-50 longer. We find this as unstable >>>> behavior of the CMSWaitDuration implementation related to the problem of >>>> the wait-notify signaling on the CGC_lock object. We disabled the explicit >>>> GC invocation (-XX:+DisableExplicitGC) to be sure there is no other reason >>>> to start the CMS initial mark phase before the young space collection. >>>> >>>> Is there any plan to get the CMSWaitDuration supported in the incremental >>>> mode and/or get it fixed in the normal mode? >>>> >>>> Thanks, >>>> Michal Frajt >>>> >>>> >>>> From john.cuthbertson at oracle.com Thu Aug 9 19:35:17 2012 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Thu, 09 Aug 2012 12:35:17 -0700 Subject: 6950794: Make the GC log file name parameterized In-Reply-To: <50231483.2060701@lab.ntt.co.jp> References: <5007BDC8.2080905@lab.ntt.co.jp> <50081F1A.1030603@kippdata.de> <5008AD1F.3@lab.ntt.co.jp> <501093B7.7030300@lab.ntt.co.jp> <50231483.2060701@lab.ntt.co.jp> Message-ID: <502410F5.2000802@oracle.com> Hi Yasumasa, Thanks for contributing the patch. There are a couple of reasons why no-one has volunteered to sponsor your patch yet. The first is that most of our team (and specifically the engineer who is looking at improving the whole GC logging) are out on vacation. They should start to come back later this month. Another reason is that changes to commonly used command line flags tend to be controversial and generate a fair amount of discussion and argument. So we also want to give the OpenJDK community a chance to try out your patch and comment and have some level of consensus before it is sponsored. Thanks, JohnC On 08/08/12 18:38, Yasumasa Suenaga wrote: > Hi, > > A few weeks ago, I posted a patch for implementing this RFE. > > @OpenJDK tweets my proposal :-) > http://twitter.com/OpenJDK/status/232506237546295297 > > > Advantages of this patch is that name of gc log can be defined by > state of run-time. So we can define gc log name which depend on > PID, start date of VM. And we can find the newest or the oldest > log easily. > > For example, multiple VM is running on 1 OS and each VM writes > gc log. Each VM is same setting and runs same application. > If we set "-Xloggc:gc-%p.log", we can find log of specific VM easily. > > > I'm sure that this patch helps us to operate gc logs. > Please cooperate. > > > Thanks, > > Yasumasa > > > On 2012/07/26 9:47, Yasumasa Suenaga wrote: >> Hi, >> >> I've made a patch for this RFE. >> This patch allows for 4 parameters as following: >> >> %p - Java process' PID >> %t - date stamp when log file is created (format: YYYY-MM-DD) >> %n - if log rotation logic is enabled, then log segment id, otherwise >> "0" >> %% - escaped '%' character in file name >> >> (These parameters are defined in 6950794) >> >> >> Other features of this patch: >> >> * The oldest log is removed. If you set -XX:NumberOfGCLogFiles=5 and >> gc.log.0 - 4 is exists, gc.log.0 is removed and gc.log.5 is created. >> * Log segment id (%n) is unsigned int . If id is UINT_MAX (0xFFFFFFFF), >> next id is zero. >> * I expanded Arguments::copy_expand_pid() . This modification affects >> -XX:OnError, -XX:ErrorFile, -XX:PerfDataSaveFile . These option also >> can use parameters that this patch provides. >> If this is not good, I can separate this feature from >> Arguments::copy_expand_pid() . >> >> >> This patch works fine in my environment. >> I would like to contribute this patch. >> >> Please cooperate. >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2012/07/20 9:58, Yasumasa Suenaga wrote: >>> Hi Rainer, >>> >>> Do you point the following as "negative consequences" ? : >>> >>> http://mail.openjdk.java.net/pipermail/hotspot-gc-use/2010-May/000613.html >>> >>> ---------------- >>> * "f00.00000001 might have been detected as old and copied to the >>> remote >>> host and during the same time GC decides to now reuse it... That's why >>> I personally find externally organized pruning better. Another thing I >>> often miss is the ability to combine size and time based rotation." >>> (Rainer) >>> >>> The proposal never reuses log files. We'll never overwrite anything. >>> Instead, we'll delete the oldest files as we create new ones. If we >>> tell >>> the users to prune the older log files themselves, I know what the >>> first >>> bug filed against the new policy will be. :-) Regarding rotating based >>> on both size and time: most people care about size so I think that's >>> what we'll do. If you want more advanced management of the logs you'll >>> have to set N to infinity (at least we'll need a way to say "never >>> delete older files") so that HotSpot doesn't delete any files and >>> you'll >>> be able to copy them and delete them yourself. >>> >>> But, seriously, this is excellent feedback. You guys are doing more >>> wild >>> stuff with our logs than I had imagined. :-) >>> ---------------- >>> >>> Tony says "The proposal never reuses log files. We'll never >>> overwrite anything." >>> However, seems to reuse the oldest log :-) >>> >>> hotspot/src/share/vm/utilities/ostream.cpp: >>> void rotatingFileStream::rotate_log() >>> >>> >>> We can find the oldest log with "stat" command and can check mtime of >>> all logs on Linux. >>> I think that "mtime" is updated every write(2) syscall . >>> >>> At least, status of this RFE is "3-Accepted" . >>> So I believe that this RFE will be merge mainline someday :-) >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> On 2012/07/19 23:52, Rainer Jung wrote: >>>> On 19.07.2012 09:56, Yasumasa Suenaga wrote: >>>>> Hi, >>>>> >>>>> I use GC log rotation with -XX:+UseGCLogFileRotation . >>>>> However, suffix of logfile is fixed ( .N : cyclic >>>>> 0-(NumberOfGCLogFiles-1) ) . >>>>> So I'm not easy to find the oldest log. >>>>> (I have to check timestamp of file or GC event time.) >>>> >>>> See the discussion thread starting at >>>> >>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-use/2010-May/000597.html >>>> >>>> >>>> including a reply of mine on the negative consequences of the >>>> numeric naming scheme and responses of Tony to the comments on his >>>> proposal. >>>> >>>> Regards, >>>> >>>> Rainer >>>> >>>>> I hope that this RFE is merged to JDK6/7/8. >>>>> >>>>> Someone is working on this RFE ? >>>>> If none, I would like to contribute a patch. >>>>> (Then, please someone become a sponsor of me :-) ) >> From suenaga.yasumasa at lab.ntt.co.jp Fri Aug 10 00:12:32 2012 From: suenaga.yasumasa at lab.ntt.co.jp (Yasumasa Suenaga) Date: Fri, 10 Aug 2012 09:12:32 +0900 Subject: 6950794: Make the GC log file name parameterized In-Reply-To: <502410F5.2000802@oracle.com> References: <5007BDC8.2080905@lab.ntt.co.jp> <50081F1A.1030603@kippdata.de> <5008AD1F.3@lab.ntt.co.jp> <501093B7.7030300@lab.ntt.co.jp> <50231483.2060701@lab.ntt.co.jp> <502410F5.2000802@oracle.com> Message-ID: <502451F0.7020309@lab.ntt.co.jp> Hi John, Thank you for replying. I've understood circumstances. I'd like to discuss about this RFE and more advantages of it. Thanks, Yasumasa On 2012/08/10 4:35, John Cuthbertson wrote: > Hi Yasumasa, > > Thanks for contributing the patch. > > There are a couple of reasons why no-one has volunteered to sponsor your patch yet. The first is that most of our team (and specifically the engineer who is looking at improving the whole GC logging) are out on vacation. They should start to come back later this month. Another reason is that changes to commonly used command line flags tend to be controversial and generate a fair amount of discussion and argument. So we also want to give the OpenJDK community a chance to try out your patch and comment and have some level of consensus before it is sponsored. > > Thanks, > > JohnC > > On 08/08/12 18:38, Yasumasa Suenaga wrote: >> Hi, >> >> A few weeks ago, I posted a patch for implementing this RFE. >> >> @OpenJDK tweets my proposal :-) >> http://twitter.com/OpenJDK/status/232506237546295297 >> >> >> Advantages of this patch is that name of gc log can be defined by >> state of run-time. So we can define gc log name which depend on >> PID, start date of VM. And we can find the newest or the oldest >> log easily. >> >> For example, multiple VM is running on 1 OS and each VM writes >> gc log. Each VM is same setting and runs same application. >> If we set "-Xloggc:gc-%p.log", we can find log of specific VM easily. >> >> >> I'm sure that this patch helps us to operate gc logs. >> Please cooperate. >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2012/07/26 9:47, Yasumasa Suenaga wrote: >>> Hi, >>> >>> I've made a patch for this RFE. >>> This patch allows for 4 parameters as following: >>> >>> %p - Java process' PID >>> %t - date stamp when log file is created (format: YYYY-MM-DD) >>> %n - if log rotation logic is enabled, then log segment id, otherwise "0" >>> %% - escaped '%' character in file name >>> >>> (These parameters are defined in 6950794) >>> >>> >>> Other features of this patch: >>> >>> * The oldest log is removed. If you set -XX:NumberOfGCLogFiles=5 and >>> gc.log.0 - 4 is exists, gc.log.0 is removed and gc.log.5 is created. >>> * Log segment id (%n) is unsigned int . If id is UINT_MAX (0xFFFFFFFF), >>> next id is zero. >>> * I expanded Arguments::copy_expand_pid() . This modification affects >>> -XX:OnError, -XX:ErrorFile, -XX:PerfDataSaveFile . These option also >>> can use parameters that this patch provides. >>> If this is not good, I can separate this feature from Arguments::copy_expand_pid() . >>> >>> >>> This patch works fine in my environment. >>> I would like to contribute this patch. >>> >>> Please cooperate. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2012/07/20 9:58, Yasumasa Suenaga wrote: >>>> Hi Rainer, >>>> >>>> Do you point the following as "negative consequences" ? : >>>> >>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-use/2010-May/000613.html >>>> ---------------- >>>> * "f00.00000001 might have been detected as old and copied to the remote >>>> host and during the same time GC decides to now reuse it... That's why >>>> I personally find externally organized pruning better. Another thing I >>>> often miss is the ability to combine size and time based rotation." (Rainer) >>>> >>>> The proposal never reuses log files. We'll never overwrite anything. >>>> Instead, we'll delete the oldest files as we create new ones. If we tell >>>> the users to prune the older log files themselves, I know what the first >>>> bug filed against the new policy will be. :-) Regarding rotating based >>>> on both size and time: most people care about size so I think that's >>>> what we'll do. If you want more advanced management of the logs you'll >>>> have to set N to infinity (at least we'll need a way to say "never >>>> delete older files") so that HotSpot doesn't delete any files and you'll >>>> be able to copy them and delete them yourself. >>>> >>>> But, seriously, this is excellent feedback. You guys are doing more wild >>>> stuff with our logs than I had imagined. :-) >>>> ---------------- >>>> >>>> Tony says "The proposal never reuses log files. We'll never overwrite anything." >>>> However, seems to reuse the oldest log :-) >>>> >>>> hotspot/src/share/vm/utilities/ostream.cpp: >>>> void rotatingFileStream::rotate_log() >>>> >>>> >>>> We can find the oldest log with "stat" command and can check mtime of >>>> all logs on Linux. >>>> I think that "mtime" is updated every write(2) syscall . >>>> >>>> At least, status of this RFE is "3-Accepted" . >>>> So I believe that this RFE will be merge mainline someday :-) >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> On 2012/07/19 23:52, Rainer Jung wrote: >>>>> On 19.07.2012 09:56, Yasumasa Suenaga wrote: >>>>>> Hi, >>>>>> >>>>>> I use GC log rotation with -XX:+UseGCLogFileRotation . >>>>>> However, suffix of logfile is fixed ( .N : cyclic 0-(NumberOfGCLogFiles-1) ) . >>>>>> So I'm not easy to find the oldest log. >>>>>> (I have to check timestamp of file or GC event time.) >>>>> >>>>> See the discussion thread starting at >>>>> >>>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-use/2010-May/000597.html >>>>> >>>>> including a reply of mine on the negative consequences of the numeric naming scheme and responses of Tony to the comments on his proposal. >>>>> >>>>> Regards, >>>>> >>>>> Rainer >>>>> >>>>>> I hope that this RFE is merged to JDK6/7/8. >>>>>> >>>>>> Someone is working on this RFE ? >>>>>> If none, I would like to contribute a patch. >>>>>> (Then, please someone become a sponsor of me :-) ) >>> From john.coomes at oracle.com Fri Aug 10 09:06:43 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 10 Aug 2012 09:06:43 +0000 Subject: hg: hsx/hotspot-gc: Added tag jdk8-b51 for changeset 57c0aee73090 Message-ID: <20120810090644.53CD147477@hg.openjdk.java.net> Changeset: 8d24def5ceb3 Author: katleman Date: 2012-08-09 18:00 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/rev/8d24def5ceb3 Added tag jdk8-b51 for changeset 57c0aee73090 ! .hgtags From john.coomes at oracle.com Fri Aug 10 09:06:50 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 10 Aug 2012 09:06:50 +0000 Subject: hg: hsx/hotspot-gc/corba: Added tag jdk8-b51 for changeset 9b0f841ca9f7 Message-ID: <20120810090654.123C147478@hg.openjdk.java.net> Changeset: 80689ff9cb49 Author: katleman Date: 2012-08-09 18:00 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/corba/rev/80689ff9cb49 Added tag jdk8-b51 for changeset 9b0f841ca9f7 ! .hgtags From john.coomes at oracle.com Fri Aug 10 09:07:01 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 10 Aug 2012 09:07:01 +0000 Subject: hg: hsx/hotspot-gc/jaxp: Added tag jdk8-b51 for changeset dc1ea77ed9d9 Message-ID: <20120810090710.88CBA47479@hg.openjdk.java.net> Changeset: bd3c00d57614 Author: katleman Date: 2012-08-09 18:01 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jaxp/rev/bd3c00d57614 Added tag jdk8-b51 for changeset dc1ea77ed9d9 ! .hgtags From john.coomes at oracle.com Fri Aug 10 09:07:19 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 10 Aug 2012 09:07:19 +0000 Subject: hg: hsx/hotspot-gc/jaxws: Added tag jdk8-b51 for changeset 1a70b6333ebe Message-ID: <20120810090725.EF4084747A@hg.openjdk.java.net> Changeset: f62bc618122e Author: katleman Date: 2012-08-09 18:01 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jaxws/rev/f62bc618122e Added tag jdk8-b51 for changeset 1a70b6333ebe ! .hgtags From john.coomes at oracle.com Fri Aug 10 09:08:25 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 10 Aug 2012 09:08:25 +0000 Subject: hg: hsx/hotspot-gc/jdk: 30 new changesets Message-ID: <20120810091528.3ACD74747B@hg.openjdk.java.net> Changeset: 28665fa73b4a Author: rupashka Date: 2012-07-19 19:09 +0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/28665fa73b4a 7124330: [macosx] javax.swing.JComboBox throws unexpected ClassCastException Reviewed-by: kizune ! src/macosx/classes/com/apple/laf/AquaComboBoxUI.java Changeset: b1c5e4a843f3 Author: leonidr Date: 2012-07-19 19:41 +0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/b1c5e4a843f3 7181027: [macosx] Unable to use headless mode Reviewed-by: anthony ! src/share/classes/java/awt/GraphicsEnvironment.java ! src/solaris/native/java/lang/java_props_md.c Changeset: f04d8dee2da9 Author: alexsch Date: 2012-07-23 17:41 +0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/f04d8dee2da9 7185512: The printout doesn't match image on screen. Reviewed-by: serb, bagiras ! src/windows/native/sun/windows/awt_TextArea.cpp ! src/windows/native/sun/windows/awt_TextComponent.cpp ! src/windows/native/sun/windows/awt_TextComponent.h Changeset: 8a5a71e853ed Author: alexsch Date: 2012-07-24 16:26 +0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/8a5a71e853ed 7129800: [macosx] Regression test OverrideRedirectWindowActivationTest fails due to timing issue Reviewed-by: rupashka + test/java/awt/Focus/OverrideRedirectWindowActivationTest/OverrideRedirectWindowActivationTest.java Changeset: 3502753a9d66 Author: rupashka Date: 2012-07-25 13:41 +0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/3502753a9d66 7167780: Hang javasoft.sqe.tests.api.javax.swing.Timer.Ctor2Tests Reviewed-by: alexsch ! src/share/classes/javax/swing/TimerQueue.java Changeset: 1a410846d85b Author: malenkov Date: 2012-07-25 19:14 +0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/1a410846d85b 4650871: Classes in sunw.* should be removed from workspace and rt.jar Reviewed-by: art, rupashka ! make/Makefile ! make/common/Release.gmk ! make/docs/CORE_PKGS.gmk - make/sunw/Makefile ! makefiles/CreateJars.gmk ! makefiles/docs/CORE_PKGS.gmk ! src/share/classes/sun/misc/MetaIndex.java - src/share/classes/sunw/io/Serializable.java - src/share/classes/sunw/util/EventListener.java - src/share/classes/sunw/util/EventObject.java Changeset: 80b1ecc79852 Author: denis Date: 2012-07-27 19:41 +0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/80b1ecc79852 7149068: java/awt/Window/Grab/GrabTest.java failed Reviewed-by: art, ant + test/java/awt/Window/Grab/GrabTest.java Changeset: 1579507a736f Author: lana Date: 2012-07-27 22:39 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/1579507a736f Merge - src/share/classes/sun/security/krb5/ServiceName.java - test/sun/security/krb5/ServiceNameClone.java - test/sun/security/mscapi/ShortRSAKey512.sh - test/sun/security/mscapi/ShortRSAKey768.sh Changeset: 1abb270d9038 Author: malenkov Date: 2012-07-30 13:35 +0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/1abb270d9038 7187618: PropertyDescriptor Performance Slow Reviewed-by: rupashka ! src/share/classes/com/sun/beans/TypeResolver.java ! src/share/classes/com/sun/beans/finder/MethodFinder.java ! src/share/classes/java/beans/IndexedPropertyDescriptor.java ! src/share/classes/java/beans/Introspector.java ! src/share/classes/java/beans/PropertyDescriptor.java + test/java/beans/Performance/Test7122740.java + test/java/beans/Performance/Test7184799.java Changeset: 896322c6f35f Author: alexsch Date: 2012-07-30 14:31 +0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/896322c6f35f 7184365: closed/java/awt/event/TextEvent/TextEventSequenceTest/TextEventSequenceTest fails Reviewed-by: serb, bagiras ! src/macosx/classes/sun/lwawt/LWTextAreaPeer.java ! src/macosx/classes/sun/lwawt/LWTextComponentPeer.java ! src/share/classes/java/awt/TextComponent.java ! src/solaris/classes/sun/awt/X11/XTextAreaPeer.java + test/java/awt/event/TextEvent/TextEventSequenceTest/TextEventSequenceTest.java Changeset: 773474da570b Author: khazra Date: 2012-07-18 15:19 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/773474da570b 7185051: Remove TestProviderLeak.java from the ProblemList Summary: Remove TestProviderLeak.java from jdk test problem list. Reviewed-by: khazra Contributed-by: dan.xu at oracle.com ! test/ProblemList.txt Changeset: 2c2e4ecc8f7e Author: chegar Date: 2012-07-19 18:19 +0100 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/2c2e4ecc8f7e 7081476: test/java/net/InetSocketAddress/B6469803.java failing intermittently Reviewed-by: chegar Contributed-by: Eric Wang ! test/ProblemList.txt ! test/java/net/DatagramPacket/ReuseBuf.java Changeset: 84cd98a5641c Author: sherman Date: 2012-07-19 21:23 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/84cd98a5641c 7130915: File.equals does not give expected results when path contains Non-English characters on Mac OS X Summary: to support Unicode nfd/nfc file path on Macos Reviewed-by: alanb ! make/java/nio/Makefile ! src/share/native/java/io/io_util.h ! src/solaris/classes/sun/nio/fs/BsdNativeDispatcher.java ! src/solaris/classes/sun/nio/fs/DefaultFileSystemProvider.java + src/solaris/classes/sun/nio/fs/MacOSXFileSystem.java + src/solaris/classes/sun/nio/fs/MacOSXFileSystemProvider.java + src/solaris/classes/sun/nio/fs/MacOSXNativeDispatcher.java ! src/solaris/classes/sun/nio/fs/UnixFileSystem.java ! src/solaris/classes/sun/nio/fs/UnixPath.java ! src/solaris/native/java/io/UnixFileSystem_md.c ! src/solaris/native/java/io/io_util_md.c ! src/solaris/native/java/io/io_util_md.h + src/solaris/native/sun/nio/fs/MacOSXNativeDispatcher.c + test/java/io/File/MacPathTest.java + test/java/io/File/MacPathTest.sh + test/java/nio/file/Path/MacPathTest.java + test/java/nio/file/Path/MacPathTest.sh Changeset: 5dc3f32c0d26 Author: weijun Date: 2012-07-21 19:56 +0800 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/5dc3f32c0d26 7180907: Jarsigner -verify fails if rsa file used sha-256 with authenticated attributes Reviewed-by: xuelei ! src/share/classes/com/sun/crypto/provider/OAEPParameters.java ! src/share/classes/sun/security/pkcs/PKCS7.java ! src/share/classes/sun/security/pkcs12/PKCS12KeyStore.java ! src/share/classes/sun/security/x509/AlgorithmId.java + test/sun/security/x509/AlgorithmId/NonStandardNames.java Changeset: 7e49c6f7507e Author: weijun Date: 2012-07-21 19:56 +0800 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/7e49c6f7507e 7178649: TEST BUG: BadKdc3.java needs improvement to ignore the unlikely but possible timeout Reviewed-by: xuelei ! test/sun/security/krb5/auto/BadKdc.java ! test/sun/security/krb5/auto/BadKdc1.java ! test/sun/security/krb5/auto/BadKdc2.java ! test/sun/security/krb5/auto/BadKdc3.java ! test/sun/security/krb5/auto/BadKdc4.java Changeset: 11d5ddc6a6d4 Author: alanb Date: 2012-07-22 20:32 +0100 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/11d5ddc6a6d4 6633549: (dc) Include-mode filtering of IPv6 sources does not block datagrams on Linux Reviewed-by: chegar ! src/solaris/native/sun/nio/ch/DatagramDispatcher.c ! src/solaris/native/sun/nio/ch/Net.c ! test/java/nio/channels/DatagramChannel/MulticastSendReceiveTests.java Changeset: f7731fc8c98a Author: weijun Date: 2012-07-24 09:20 +0800 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/f7731fc8c98a 7179796: GSSExceptionImpl outputs duplicate mech oid Reviewed-by: valeriep ! src/share/classes/sun/security/jgss/GSSCredentialImpl.java Changeset: e0e7cc711bda Author: xuelei Date: 2012-07-24 03:31 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/e0e7cc711bda 7185576: Need to consider the connection timeout at test/com/sun/jndi/ldap/InvalidLdapFilters.java Reviewed-by: vinnie ! test/com/sun/jndi/ldap/InvalidLdapFilters.java Changeset: a18f2806bef2 Author: sherman Date: 2012-07-24 12:17 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/a18f2806bef2 6653797: Reimplement JDK charset repository charsets.jar Summary: Migrated all jis based charsets to new implementation Reviewed-by: okutsu ! make/java/nio/FILES_java.gmk ! make/java/sun_nio/FILES_java.gmk ! make/sun/nio/cs/FILES_java.gmk ! make/tools/CharsetMapping/DoubleByte-X.java.template + make/tools/CharsetMapping/JIS_X_0201.c2b ! make/tools/CharsetMapping/JIS_X_0201.map + make/tools/CharsetMapping/JIS_X_0208.map + make/tools/CharsetMapping/JIS_X_0208_MS5022X.c2b + make/tools/CharsetMapping/JIS_X_0208_MS5022X.map + make/tools/CharsetMapping/JIS_X_0208_MS932.map + make/tools/CharsetMapping/JIS_X_0208_MS932.nr + make/tools/CharsetMapping/JIS_X_0208_Solaris.map + make/tools/CharsetMapping/JIS_X_0208_Solaris.nr + make/tools/CharsetMapping/JIS_X_0212.map + make/tools/CharsetMapping/JIS_X_0212_MS5022X.map + make/tools/CharsetMapping/JIS_X_0212_Solaris.map + make/tools/CharsetMapping/JIS_X_0212_Solaris.nr + make/tools/CharsetMapping/PCK.c2b + make/tools/CharsetMapping/PCK.map + make/tools/CharsetMapping/PCK.nr + make/tools/CharsetMapping/SJIS.c2b + make/tools/CharsetMapping/SJIS.map ! make/tools/CharsetMapping/dbcs ! make/tools/CharsetMapping/extsbcs ! make/tools/src/build/tools/charsetmapping/DBCS.java ! make/tools/src/build/tools/charsetmapping/SBCS.java ! src/share/classes/sun/nio/cs/SingleByte.java - src/share/classes/sun/nio/cs/SingleByteDecoder.java - src/share/classes/sun/nio/cs/SingleByteEncoder.java ! src/share/classes/sun/nio/cs/ext/DoubleByte.java - src/share/classes/sun/nio/cs/ext/DoubleByteDecoder.java ! src/share/classes/sun/nio/cs/ext/EUC_JP.java ! src/share/classes/sun/nio/cs/ext/EUC_JP_LINUX.java ! src/share/classes/sun/nio/cs/ext/EUC_JP_Open.java ! src/share/classes/sun/nio/cs/ext/IBM834.java ! src/share/classes/sun/nio/cs/ext/ISO2022_JP.java ! src/share/classes/sun/nio/cs/ext/ISO2022_JP_2.java - src/share/classes/sun/nio/cs/ext/JIS_X_0201.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_MS5022X_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_MS5022X_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_MS932_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_MS932_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_Solaris_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_Solaris_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_MS5022X_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_MS5022X_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_Solaris_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_Solaris_Encoder.java ! src/share/classes/sun/nio/cs/ext/MS50220.java ! src/share/classes/sun/nio/cs/ext/MS50221.java ! src/share/classes/sun/nio/cs/ext/MSISO2022JP.java - src/share/classes/sun/nio/cs/ext/PCK.java - src/share/classes/sun/nio/cs/ext/SJIS.java ! src/solaris/classes/sun/awt/motif/X11JIS0201.java ! src/solaris/classes/sun/awt/motif/X11JIS0208.java ! src/solaris/classes/sun/awt/motif/X11JIS0212.java ! test/java/nio/charset/Charset/NIOCharsetAvailabilityTest.java ! test/sun/nio/cs/OLD/DoubleByteDecoder.java ! test/sun/nio/cs/OLD/DoubleByteEncoder.java + test/sun/nio/cs/OLD/EUC_JP_LINUX_OLD.java + test/sun/nio/cs/OLD/EUC_JP_OLD.java + test/sun/nio/cs/OLD/EUC_JP_Open_OLD.java + test/sun/nio/cs/OLD/JIS_X_0201_OLD.java + test/sun/nio/cs/OLD/JIS_X_0208_Decoder.java + test/sun/nio/cs/OLD/JIS_X_0208_Encoder.java + test/sun/nio/cs/OLD/JIS_X_0208_OLD.java + test/sun/nio/cs/OLD/JIS_X_0208_Solaris_Decoder.java + test/sun/nio/cs/OLD/JIS_X_0208_Solaris_Encoder.java + test/sun/nio/cs/OLD/JIS_X_0212_Decoder.java + test/sun/nio/cs/OLD/JIS_X_0212_Encoder.java + test/sun/nio/cs/OLD/JIS_X_0212_OLD.java + test/sun/nio/cs/OLD/JIS_X_0212_Solaris_Decoder.java + test/sun/nio/cs/OLD/JIS_X_0212_Solaris_Encoder.java ! test/sun/nio/cs/OLD/MS932_OLD.java + test/sun/nio/cs/OLD/PCK_OLD.java + test/sun/nio/cs/OLD/SJIS_OLD.java + test/sun/nio/cs/OLD/SingleByteDecoder.java + test/sun/nio/cs/OLD/SingleByteEncoder.java ! test/sun/nio/cs/OLD/TestIBMDB.java ! test/sun/nio/cs/TestX11JIS0201.java Changeset: 74ceda3a98a0 Author: khazra Date: 2012-07-24 13:38 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/74ceda3a98a0 7184287: (prefs) BackingStoreException when calling flush on root node[macosx] Summary: Change implementation to enable user without administrative privileges to call flush Reviewed-by: alanb ! src/macosx/classes/java/util/prefs/MacOSXPreferences.java ! src/macosx/classes/java/util/prefs/MacOSXPreferencesFile.java ! test/ProblemList.txt Changeset: 42eac77355d2 Author: sherman Date: 2012-07-25 12:47 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/42eac77355d2 7186829: test/sun/nio/cs/OLD/JIS_X_0201_OLD.java failed in jdk8 TL nightly build Summary: fixed the test case Reviewed-by: alanb ! test/sun/nio/cs/OLD/JIS_X_0201_OLD.java Changeset: f267302093d4 Author: weijun Date: 2012-07-26 20:38 +0800 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/f267302093d4 7187051: ShortRSAKeynnn.sh tests should do cleanup before start test Reviewed-by: xuelei ! test/sun/security/mscapi/ShortRSAKey1024.sh Changeset: 35fec642fd32 Author: coffeys Date: 2012-07-26 22:00 +0100 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/35fec642fd32 7179879: SSLSocket connect times out instead of throwing socket closed exception Reviewed-by: xuelei, chegar ! src/share/classes/sun/security/ssl/SSLSocketImpl.java Changeset: 018e555a7a07 Author: dmocek Date: 2012-07-27 16:53 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/018e555a7a07 7186111: fix bugs in java/rmi/activation/ActivationSystem/unregisterGroup/UnregisterGroup Reviewed-by: smarks, jgish ! test/java/rmi/activation/ActivationSystem/unregisterGroup/UnregisterGroup.java ! test/java/rmi/activation/ActivationSystem/unregisterGroup/group.security.policy ! test/java/rmi/activation/ActivationSystem/unregisterGroup/rmid.security.policy ! test/java/rmi/activation/ActivationSystem/unregisterGroup/security.policy Changeset: a093f6247b52 Author: lana Date: 2012-07-27 22:26 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/a093f6247b52 Merge Changeset: 78644d4a9a32 Author: dxu Date: 2012-07-30 04:57 +0100 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/78644d4a9a32 7185340: TEST_BUG: java/nio/channels/AsynchronousSocketChannel/Leaky.java failing intermittently [win] Reviewed-by: alanb ! test/java/nio/channels/AsynchronousSocketChannel/Leaky.java Changeset: 38263aa28324 Author: michaelm Date: 2012-07-30 22:32 +0100 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/38263aa28324 7120665: Change Java SE spec so that external networking not required Reviewed-by: alanb ! src/share/classes/java/net/NetworkInterface.java ! src/share/classes/java/net/package.html Changeset: b08697af1c56 Author: lana Date: 2012-07-31 18:38 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/b08697af1c56 Merge - src/share/classes/sun/nio/cs/SingleByteDecoder.java - src/share/classes/sun/nio/cs/SingleByteEncoder.java - src/share/classes/sun/nio/cs/ext/DoubleByteDecoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0201.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_MS5022X_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_MS5022X_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_MS932_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_MS932_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_Solaris_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_Solaris_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_MS5022X_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_MS5022X_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_Solaris_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_Solaris_Encoder.java - src/share/classes/sun/nio/cs/ext/PCK.java - src/share/classes/sun/nio/cs/ext/SJIS.java Changeset: e865efbc7105 Author: lana Date: 2012-08-06 15:53 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/e865efbc7105 Merge - make/sunw/Makefile - src/share/classes/sun/nio/cs/SingleByteDecoder.java - src/share/classes/sun/nio/cs/SingleByteEncoder.java - src/share/classes/sun/nio/cs/ext/DoubleByteDecoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0201.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_MS5022X_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_MS5022X_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_MS932_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_MS932_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_Solaris_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_Solaris_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_MS5022X_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_MS5022X_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_Solaris_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_Solaris_Encoder.java - src/share/classes/sun/nio/cs/ext/PCK.java - src/share/classes/sun/nio/cs/ext/SJIS.java - src/share/classes/sunw/io/Serializable.java - src/share/classes/sunw/util/EventListener.java - src/share/classes/sunw/util/EventObject.java Changeset: b3b0d75cb117 Author: katleman Date: 2012-08-09 18:02 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/b3b0d75cb117 Added tag jdk8-b51 for changeset e865efbc7105 ! .hgtags From john.coomes at oracle.com Fri Aug 10 09:17:58 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 10 Aug 2012 09:17:58 +0000 Subject: hg: hsx/hotspot-gc/langtools: Added tag jdk8-b51 for changeset c4cd4cab2220 Message-ID: <20120810091808.E0EF94747C@hg.openjdk.java.net> Changeset: 23032c78b2d1 Author: katleman Date: 2012-08-09 18:02 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/langtools/rev/23032c78b2d1 Added tag jdk8-b51 for changeset c4cd4cab2220 ! .hgtags From suenaga.yasumasa at lab.ntt.co.jp Tue Aug 14 09:20:19 2012 From: suenaga.yasumasa at lab.ntt.co.jp (Yasumasa Suenaga) Date: Tue, 14 Aug 2012 18:20:19 +0900 Subject: 7090324: gclog rotation via external tool In-Reply-To: <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> References: <4E6D5DEC.8090109@oss.ntt.co.jp> <4E6DDB8F.4020301@oracle.com><3568943A-E6A4-4287-8319-8B0993F070F9@oracle.com><4E7A85AE.10507@oss.ntt.co.jp> <4E7B19EC.5070700@oracle.com> <4E7B2227.9070406@kippdata.de> <4E805629.7030703@oss.ntt.co.jp> <4E846158.3020606@oracle.com> <5017B761.5070203@lab.ntt.co.jp> <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> Message-ID: <502A1853.70508@lab.ntt.co.jp> Hi Staffan, May I ask you 2 questions about this JEP? 1. One of goals of this JEP is defined as following: "File rotation of log files by size (similar to what is available for GC logs today)" My patch realizes log rotation by external trigger. 7090324 is included in this JEP? (Is 7090324 included in "Logging can be configured dynamically at runtime via jcmd or MBeans" ? ) 2. I've posted a patch for "6950794: Make the GC log file name parameterized" . http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2012-July/004758.html Could you include this RFE in this JEP? If we use log rotation, I think that name of logs is very important for log management. Thanks, Yasumasa On 2012/08/14 16:50, Staffan Larsen wrote: > Hi Yasumasa, > > This work should be done in the context of the Unified Logging JEP [1]. Unfortunately, that work has not yet started due to resource constraints. Comments on the proposal are welcome. > > [1] http://openjdk.java.net/jeps/158 > > /Staffan > > On 31 jul 2012, at 12:45, Yasumasa Suenaga > wrote: > >> Hi, >> >> I've posted a patch for gc log rotation via jinfo tool last year. >> Then I received an email that essence of my patch will be implemented as >> "subcommands" of jcmd. >> >> Now, jcmd is provided by OpenJDK. However, current jcmd does not seem to >> implement function of gclog rotation. >> >> Will this RFE be implemented? >> >> I need to rotate gclog which is synchronized with signal, cron, etc... on Linux. >> So, if this RFE is not implemented for a while, I would like to contribute >> patch for jcmd. >> >> >> Regards, >> >> Yasumasa >> >> >> On 2011/09/29 21:15, James Melvin wrote: >>> Hi Yasumasa, >>> >>> Thanks very much for your OpenJDK contribution! As part of the effort to >>> port JRockit features to HotSpot, we plan to introduce a consolidated >>> commandline serviceability tool (jcmd) to potentially replace many of >>> the existing tools in the bin directory, such as jmap, jstack, jinfo and >>> others. Over the next few update releases, we plan to add several jcmd >>> *subcommands* instead to accomplish specific tasks or affect the running >>> JVM instance. We'd like to use the essence of your contribution in one >>> of the jcmd subcommands (instead of extending jinfo) to ask the JVM to >>> begin rotating GC logs. We hope you find this attractive and hope you >>> will help review and perfect the change. >>> >>> Thanks, >>> >>> Jim Melvin >>> Sr. Engineering Manager >>> Oracle >>> >>> >>> >>> On 9/26/11 6:38 AM, Yasumasa Suenaga wrote: >>>> (I've changed subject of this email to new RFE.) >>>> >>>> This RFE is enhancement of current gclog implementation. >>>> So, I'd like to discuss about rotating gclog. >>>> >>>> My customers need gclog rotation which is invoked by external trigger. >>>> So I've requested this RFE and made a patch. >>>> >>>> >>>> In many case on Linux, logfile is rotated by signal (e.g. SIGHUP) . >>>> The aim of this RFE is to synchronize gclog and the other logs. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> (2011/09/22 20:55), Rainer Jung wrote: >>>>> On 22.09.2011 13:20, Dmitry Samersoff wrote: >>>>>> >>>>>> Yasumasa, >>>>>> >>>>>> On 2011-09-22 04:47, Yasumasa Suenaga wrote: >>>>>>> If we can think Java on Linux and Solaris only, syslog is best >>>>>>> solution. >>>>>>> However, Windows usually doesn't have syslog. >>>>>>> >>>>>>> So, I think that gclog is needed for logging GC stats with platform >>>>>>> independent in realtime. >>>>>> >>>>>> Windows has it's own logging API as reach as syslog is or ever better >>>>>> as well as numerous syslog implementations. >>>>>> >>>>>> Native windows logging API was completely redesigned for Windows 2008 >>>>>> server and now it allows for developers to send a structured events from >>>>>> theirs application. >>>>> >>>>> AFAIK log rotation for loggc is already implemented though not >>>>> necessarily yet released. The change discussed here is about supporting >>>>> an externally generated rotation trigger, e.g. via a signal, instead of >>>>> only rotating by size or time via a startup parameter. >>>>> >>>>> If you want support for syslog or Windows APIs included, it would be >>>>> best to start a new discussion. >>>>> >>>>> A GC log for an application under load might easily produce a block of >>>>> about 1.5 KB size every few seconds. I seriously doubt, that the need >>>>> for log file rotation can be argued away by referring to syslog or >>>>> Windows log API as the correct solution. >>>>> >>>>> The messages are not really line formatted, the format can vary a lot >>>>> (depending on the excat XX switches), the traffic can be quite high and >>>>> AFAIK the JVM writes it synchronously, so there must be absolutely no >>>>> risk in writing it out with very little latency. In addition, for >>>>> analysis, you wouldn't want to look at each event individually, but >>>>> instead process the whole file through a script or tool, which should >>>>> not depend on the output specifics of a platform specific log apparatus. >>>>> >>>>> Regards, >>>>> >>>>> Rainer >>>>> >>>> >> > From Dmitry.Samersoff at oracle.com Tue Aug 14 10:17:20 2012 From: Dmitry.Samersoff at oracle.com (Dmitry Samersoff) Date: Tue, 14 Aug 2012 14:17:20 +0400 Subject: JEP 158: Unified JVM logging In-Reply-To: References: <4E6D5DEC.8090109@oss.ntt.co.jp> <4E6DDB8F.4020301@oracle.com><3568943A-E6A4-4287-8319-8B0993F070F9@oracle.com><4E7A85AE.10507@oss.ntt.co.jp> <4E7B19EC.5070700@oracle.com> <4E7B2227.9070406@kippdata.de> <4E805629.7030703@oss.ntt.co.jp> <4E846158.3020606@oracle.com> <5017B761.5070203@lab.ntt.co.jp> <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> <502A1853.70508@lab.ntt.co.jp> Message-ID: <502A25B0.20004@oracle.com> Kirk, > However I do have very serious concerns about this JEP in that it > doesn't fix the problems that exist in current logging frameworks, > it only mimics them. > http://openjdk.java.net/jeps/158 Any comments is much appreciated. Personally, I think that log rotation is out of scope and responsibility of JVM. -Dmitry On 2012-08-14 13:26, Kirk Pepperdine wrote: > Hi Yasumasa, > > I'm not sure that log file rotation is a part of this JEP. > However I do have very serious concerns about this JEP in that it doesn't fix the problems that exist in current logging frameworks, it only mimics them. > > Regards, > Kirk > > On 2012-08-14, at 11:20 AM, Yasumasa Suenaga wrote: > >> Hi Staffan, >> >> May I ask you 2 questions about this JEP? >> >> 1. One of goals of this JEP is defined as following: >> "File rotation of log files by size (similar to what is available for GC logs today)" >> My patch realizes log rotation by external trigger. >> 7090324 is included in this JEP? >> (Is 7090324 included in "Logging can be configured dynamically at runtime via jcmd or MBeans" ? ) >> >> 2. I've posted a patch for "6950794: Make the GC log file name parameterized" . >> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2012-July/004758.html >> Could you include this RFE in this JEP? >> If we use log rotation, I think that name of logs is very important for log management. >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2012/08/14 16:50, Staffan Larsen wrote: >>> Hi Yasumasa, >>> >>> This work should be done in the context of the Unified Logging JEP [1]. Unfortunately, that work has not yet started due to resource constraints. Comments on the proposal are welcome. >>> >>> [1] http://openjdk.java.net/jeps/158 >>> >>> /Staffan >>> >>> On 31 jul 2012, at 12:45, Yasumasa Suenaga > wrote: >>> >>>> Hi, >>>> >>>> I've posted a patch for gc log rotation via jinfo tool last year. >>>> Then I received an email that essence of my patch will be implemented as >>>> "subcommands" of jcmd. >>>> >>>> Now, jcmd is provided by OpenJDK. However, current jcmd does not seem to >>>> implement function of gclog rotation. >>>> >>>> Will this RFE be implemented? >>>> >>>> I need to rotate gclog which is synchronized with signal, cron, etc... on Linux. >>>> So, if this RFE is not implemented for a while, I would like to contribute >>>> patch for jcmd. >>>> >>>> >>>> Regards, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2011/09/29 21:15, James Melvin wrote: >>>>> Hi Yasumasa, >>>>> >>>>> Thanks very much for your OpenJDK contribution! As part of the effort to >>>>> port JRockit features to HotSpot, we plan to introduce a consolidated >>>>> commandline serviceability tool (jcmd) to potentially replace many of >>>>> the existing tools in the bin directory, such as jmap, jstack, jinfo and >>>>> others. Over the next few update releases, we plan to add several jcmd >>>>> *subcommands* instead to accomplish specific tasks or affect the running >>>>> JVM instance. We'd like to use the essence of your contribution in one >>>>> of the jcmd subcommands (instead of extending jinfo) to ask the JVM to >>>>> begin rotating GC logs. We hope you find this attractive and hope you >>>>> will help review and perfect the change. >>>>> >>>>> Thanks, >>>>> >>>>> Jim Melvin >>>>> Sr. Engineering Manager >>>>> Oracle >>>>> >>>>> >>>>> >>>>> On 9/26/11 6:38 AM, Yasumasa Suenaga wrote: >>>>>> (I've changed subject of this email to new RFE.) >>>>>> >>>>>> This RFE is enhancement of current gclog implementation. >>>>>> So, I'd like to discuss about rotating gclog. >>>>>> >>>>>> My customers need gclog rotation which is invoked by external trigger. >>>>>> So I've requested this RFE and made a patch. >>>>>> >>>>>> >>>>>> In many case on Linux, logfile is rotated by signal (e.g. SIGHUP) . >>>>>> The aim of this RFE is to synchronize gclog and the other logs. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> (2011/09/22 20:55), Rainer Jung wrote: >>>>>>> On 22.09.2011 13:20, Dmitry Samersoff wrote: >>>>>>>> >>>>>>>> Yasumasa, >>>>>>>> >>>>>>>> On 2011-09-22 04:47, Yasumasa Suenaga wrote: >>>>>>>>> If we can think Java on Linux and Solaris only, syslog is best >>>>>>>>> solution. >>>>>>>>> However, Windows usually doesn't have syslog. >>>>>>>>> >>>>>>>>> So, I think that gclog is needed for logging GC stats with platform >>>>>>>>> independent in realtime. >>>>>>>> >>>>>>>> Windows has it's own logging API as reach as syslog is or ever better >>>>>>>> as well as numerous syslog implementations. >>>>>>>> >>>>>>>> Native windows logging API was completely redesigned for Windows 2008 >>>>>>>> server and now it allows for developers to send a structured events from >>>>>>>> theirs application. >>>>>>> >>>>>>> AFAIK log rotation for loggc is already implemented though not >>>>>>> necessarily yet released. The change discussed here is about supporting >>>>>>> an externally generated rotation trigger, e.g. via a signal, instead of >>>>>>> only rotating by size or time via a startup parameter. >>>>>>> >>>>>>> If you want support for syslog or Windows APIs included, it would be >>>>>>> best to start a new discussion. >>>>>>> >>>>>>> A GC log for an application under load might easily produce a block of >>>>>>> about 1.5 KB size every few seconds. I seriously doubt, that the need >>>>>>> for log file rotation can be argued away by referring to syslog or >>>>>>> Windows log API as the correct solution. >>>>>>> >>>>>>> The messages are not really line formatted, the format can vary a lot >>>>>>> (depending on the excat XX switches), the traffic can be quite high and >>>>>>> AFAIK the JVM writes it synchronously, so there must be absolutely no >>>>>>> risk in writing it out with very little latency. In addition, for >>>>>>> analysis, you wouldn't want to look at each event individually, but >>>>>>> instead process the whole file through a script or tool, which should >>>>>>> not depend on the output specifics of a platform specific log apparatus. >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Rainer >>>>>>> >>>>>> >>>> >>> >> > -- Dmitry Samersoff Java Hotspot development team, SPB04 * There will come soft rains ... From kirk at kodewerk.com Tue Aug 14 09:26:53 2012 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Tue, 14 Aug 2012 11:26:53 +0200 Subject: 7090324: gclog rotation via external tool In-Reply-To: <502A1853.70508@lab.ntt.co.jp> References: <4E6D5DEC.8090109@oss.ntt.co.jp> <4E6DDB8F.4020301@oracle.com><3568943A-E6A4-4287-8319-8B0993F070F9@oracle.com><4E7A85AE.10507@oss.ntt.co.jp> <4E7B19EC.5070700@oracle.com> <4E7B2227.9070406@kippdata.de> <4E805629.7030703@oss.ntt.co.jp> <4E846158.3020606@oracle.com> <5017B761.5070203@lab.ntt.co.jp> <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> <502A1853.70508@lab.ntt.co.jp> Message-ID: Hi Yasumasa, I'm not sure that log file rotation is a part of this JEP. However I do have very serious concerns about this JEP in that it doesn't fix the problems that exist in current logging frameworks, it only mimics them. Regards, Kirk On 2012-08-14, at 11:20 AM, Yasumasa Suenaga wrote: > Hi Staffan, > > May I ask you 2 questions about this JEP? > > 1. One of goals of this JEP is defined as following: > "File rotation of log files by size (similar to what is available for GC logs today)" > My patch realizes log rotation by external trigger. > 7090324 is included in this JEP? > (Is 7090324 included in "Logging can be configured dynamically at runtime via jcmd or MBeans" ? ) > > 2. I've posted a patch for "6950794: Make the GC log file name parameterized" . > http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2012-July/004758.html > Could you include this RFE in this JEP? > If we use log rotation, I think that name of logs is very important for log management. > > > Thanks, > > Yasumasa > > > On 2012/08/14 16:50, Staffan Larsen wrote: >> Hi Yasumasa, >> >> This work should be done in the context of the Unified Logging JEP [1]. Unfortunately, that work has not yet started due to resource constraints. Comments on the proposal are welcome. >> >> [1] http://openjdk.java.net/jeps/158 >> >> /Staffan >> >> On 31 jul 2012, at 12:45, Yasumasa Suenaga > wrote: >> >>> Hi, >>> >>> I've posted a patch for gc log rotation via jinfo tool last year. >>> Then I received an email that essence of my patch will be implemented as >>> "subcommands" of jcmd. >>> >>> Now, jcmd is provided by OpenJDK. However, current jcmd does not seem to >>> implement function of gclog rotation. >>> >>> Will this RFE be implemented? >>> >>> I need to rotate gclog which is synchronized with signal, cron, etc... on Linux. >>> So, if this RFE is not implemented for a while, I would like to contribute >>> patch for jcmd. >>> >>> >>> Regards, >>> >>> Yasumasa >>> >>> >>> On 2011/09/29 21:15, James Melvin wrote: >>>> Hi Yasumasa, >>>> >>>> Thanks very much for your OpenJDK contribution! As part of the effort to >>>> port JRockit features to HotSpot, we plan to introduce a consolidated >>>> commandline serviceability tool (jcmd) to potentially replace many of >>>> the existing tools in the bin directory, such as jmap, jstack, jinfo and >>>> others. Over the next few update releases, we plan to add several jcmd >>>> *subcommands* instead to accomplish specific tasks or affect the running >>>> JVM instance. We'd like to use the essence of your contribution in one >>>> of the jcmd subcommands (instead of extending jinfo) to ask the JVM to >>>> begin rotating GC logs. We hope you find this attractive and hope you >>>> will help review and perfect the change. >>>> >>>> Thanks, >>>> >>>> Jim Melvin >>>> Sr. Engineering Manager >>>> Oracle >>>> >>>> >>>> >>>> On 9/26/11 6:38 AM, Yasumasa Suenaga wrote: >>>>> (I've changed subject of this email to new RFE.) >>>>> >>>>> This RFE is enhancement of current gclog implementation. >>>>> So, I'd like to discuss about rotating gclog. >>>>> >>>>> My customers need gclog rotation which is invoked by external trigger. >>>>> So I've requested this RFE and made a patch. >>>>> >>>>> >>>>> In many case on Linux, logfile is rotated by signal (e.g. SIGHUP) . >>>>> The aim of this RFE is to synchronize gclog and the other logs. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> (2011/09/22 20:55), Rainer Jung wrote: >>>>>> On 22.09.2011 13:20, Dmitry Samersoff wrote: >>>>>>> >>>>>>> Yasumasa, >>>>>>> >>>>>>> On 2011-09-22 04:47, Yasumasa Suenaga wrote: >>>>>>>> If we can think Java on Linux and Solaris only, syslog is best >>>>>>>> solution. >>>>>>>> However, Windows usually doesn't have syslog. >>>>>>>> >>>>>>>> So, I think that gclog is needed for logging GC stats with platform >>>>>>>> independent in realtime. >>>>>>> >>>>>>> Windows has it's own logging API as reach as syslog is or ever better >>>>>>> as well as numerous syslog implementations. >>>>>>> >>>>>>> Native windows logging API was completely redesigned for Windows 2008 >>>>>>> server and now it allows for developers to send a structured events from >>>>>>> theirs application. >>>>>> >>>>>> AFAIK log rotation for loggc is already implemented though not >>>>>> necessarily yet released. The change discussed here is about supporting >>>>>> an externally generated rotation trigger, e.g. via a signal, instead of >>>>>> only rotating by size or time via a startup parameter. >>>>>> >>>>>> If you want support for syslog or Windows APIs included, it would be >>>>>> best to start a new discussion. >>>>>> >>>>>> A GC log for an application under load might easily produce a block of >>>>>> about 1.5 KB size every few seconds. I seriously doubt, that the need >>>>>> for log file rotation can be argued away by referring to syslog or >>>>>> Windows log API as the correct solution. >>>>>> >>>>>> The messages are not really line formatted, the format can vary a lot >>>>>> (depending on the excat XX switches), the traffic can be quite high and >>>>>> AFAIK the JVM writes it synchronously, so there must be absolutely no >>>>>> risk in writing it out with very little latency. In addition, for >>>>>> analysis, you wouldn't want to look at each event individually, but >>>>>> instead process the whole file through a script or tool, which should >>>>>> not depend on the output specifics of a platform specific log apparatus. >>>>>> >>>>>> Regards, >>>>>> >>>>>> Rainer >>>>>> >>>>> >>> >> > From suenaga.yasumasa at lab.ntt.co.jp Tue Aug 14 10:56:37 2012 From: suenaga.yasumasa at lab.ntt.co.jp (Yasumasa Suenaga) Date: Tue, 14 Aug 2012 19:56:37 +0900 Subject: JEP 158: Unified JVM logging In-Reply-To: <502A25B0.20004@oracle.com> References: <4E6D5DEC.8090109@oss.ntt.co.jp> <4E6DDB8F.4020301@oracle.com><3568943A-E6A4-4287-8319-8B0993F070F9@oracle.com><4E7A85AE.10507@oss.ntt.co.jp> <4E7B19EC.5070700@oracle.com> <4E7B2227.9070406@kippdata.de> <4E805629.7030703@oss.ntt.co.jp> <4E846158.3020606@oracle.com> <5017B761.5070203@lab.ntt.co.jp> <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> <502A1853.70508@lab.ntt.co.jp> <502A25B0.20004@oracle.com> Message-ID: <502A2EE5.4050705@lab.ntt.co.jp> Hi, Other softwares (e.g. syslog, apache) are implemented log rotation support. When these software receives SIGHUP, they close current log and reopen it. I think that JVM should be supported same level of log management at least. Current implementation, JVM open logs at start, and never close it. So we can't have timing of moving and archiving logs. JVM have to run on Windows. However, Windows is not implemented signal trigger. (BREAK is reserved for Thread-Dump) Thus I think that JVM should have function of log rotation in multi-platform. Thanks, Yasumasa On 2012/08/14 19:17, Dmitry Samersoff wrote: > Kirk, > >> However I do have very serious concerns about this JEP in that it >> doesn't fix the problems that exist in current logging frameworks, >> it only mimics them. > >> http://openjdk.java.net/jeps/158 > > Any comments is much appreciated. > > Personally, I think that log rotation is out of scope and responsibility > of JVM. > > -Dmitry > > > On 2012-08-14 13:26, Kirk Pepperdine wrote: >> Hi Yasumasa, >> >> I'm not sure that log file rotation is a part of this JEP. >> However I do have very serious concerns about this JEP in that it doesn't fix the problems that exist in current logging frameworks, it only mimics them. >> >> Regards, >> Kirk >> >> On 2012-08-14, at 11:20 AM, Yasumasa Suenaga wrote: >> >>> Hi Staffan, >>> >>> May I ask you 2 questions about this JEP? >>> >>> 1. One of goals of this JEP is defined as following: >>> "File rotation of log files by size (similar to what is available for GC logs today)" >>> My patch realizes log rotation by external trigger. >>> 7090324 is included in this JEP? >>> (Is 7090324 included in "Logging can be configured dynamically at runtime via jcmd or MBeans" ? ) >>> >>> 2. I've posted a patch for "6950794: Make the GC log file name parameterized" . >>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2012-July/004758.html >>> Could you include this RFE in this JEP? >>> If we use log rotation, I think that name of logs is very important for log management. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2012/08/14 16:50, Staffan Larsen wrote: >>>> Hi Yasumasa, >>>> >>>> This work should be done in the context of the Unified Logging JEP [1]. Unfortunately, that work has not yet started due to resource constraints. Comments on the proposal are welcome. >>>> >>>> [1] http://openjdk.java.net/jeps/158 >>>> >>>> /Staffan >>>> >>>> On 31 jul 2012, at 12:45, Yasumasa Suenaga> wrote: >>>> >>>>> Hi, >>>>> >>>>> I've posted a patch for gc log rotation via jinfo tool last year. >>>>> Then I received an email that essence of my patch will be implemented as >>>>> "subcommands" of jcmd. >>>>> >>>>> Now, jcmd is provided by OpenJDK. However, current jcmd does not seem to >>>>> implement function of gclog rotation. >>>>> >>>>> Will this RFE be implemented? >>>>> >>>>> I need to rotate gclog which is synchronized with signal, cron, etc... on Linux. >>>>> So, if this RFE is not implemented for a while, I would like to contribute >>>>> patch for jcmd. >>>>> >>>>> >>>>> Regards, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2011/09/29 21:15, James Melvin wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> Thanks very much for your OpenJDK contribution! As part of the effort to >>>>>> port JRockit features to HotSpot, we plan to introduce a consolidated >>>>>> commandline serviceability tool (jcmd) to potentially replace many of >>>>>> the existing tools in the bin directory, such as jmap, jstack, jinfo and >>>>>> others. Over the next few update releases, we plan to add several jcmd >>>>>> *subcommands* instead to accomplish specific tasks or affect the running >>>>>> JVM instance. We'd like to use the essence of your contribution in one >>>>>> of the jcmd subcommands (instead of extending jinfo) to ask the JVM to >>>>>> begin rotating GC logs. We hope you find this attractive and hope you >>>>>> will help review and perfect the change. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Jim Melvin >>>>>> Sr. Engineering Manager >>>>>> Oracle >>>>>> >>>>>> >>>>>> >>>>>> On 9/26/11 6:38 AM, Yasumasa Suenaga wrote: >>>>>>> (I've changed subject of this email to new RFE.) >>>>>>> >>>>>>> This RFE is enhancement of current gclog implementation. >>>>>>> So, I'd like to discuss about rotating gclog. >>>>>>> >>>>>>> My customers need gclog rotation which is invoked by external trigger. >>>>>>> So I've requested this RFE and made a patch. >>>>>>> >>>>>>> >>>>>>> In many case on Linux, logfile is rotated by signal (e.g. SIGHUP) . >>>>>>> The aim of this RFE is to synchronize gclog and the other logs. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> (2011/09/22 20:55), Rainer Jung wrote: >>>>>>>> On 22.09.2011 13:20, Dmitry Samersoff wrote: >>>>>>>>> >>>>>>>>> Yasumasa, >>>>>>>>> >>>>>>>>> On 2011-09-22 04:47, Yasumasa Suenaga wrote: >>>>>>>>>> If we can think Java on Linux and Solaris only, syslog is best >>>>>>>>>> solution. >>>>>>>>>> However, Windows usually doesn't have syslog. >>>>>>>>>> >>>>>>>>>> So, I think that gclog is needed for logging GC stats with platform >>>>>>>>>> independent in realtime. >>>>>>>>> >>>>>>>>> Windows has it's own logging API as reach as syslog is or ever better >>>>>>>>> as well as numerous syslog implementations. >>>>>>>>> >>>>>>>>> Native windows logging API was completely redesigned for Windows 2008 >>>>>>>>> server and now it allows for developers to send a structured events from >>>>>>>>> theirs application. >>>>>>>> >>>>>>>> AFAIK log rotation for loggc is already implemented though not >>>>>>>> necessarily yet released. The change discussed here is about supporting >>>>>>>> an externally generated rotation trigger, e.g. via a signal, instead of >>>>>>>> only rotating by size or time via a startup parameter. >>>>>>>> >>>>>>>> If you want support for syslog or Windows APIs included, it would be >>>>>>>> best to start a new discussion. >>>>>>>> >>>>>>>> A GC log for an application under load might easily produce a block of >>>>>>>> about 1.5 KB size every few seconds. I seriously doubt, that the need >>>>>>>> for log file rotation can be argued away by referring to syslog or >>>>>>>> Windows log API as the correct solution. >>>>>>>> >>>>>>>> The messages are not really line formatted, the format can vary a lot >>>>>>>> (depending on the excat XX switches), the traffic can be quite high and >>>>>>>> AFAIK the JVM writes it synchronously, so there must be absolutely no >>>>>>>> risk in writing it out with very little latency. In addition, for >>>>>>>> analysis, you wouldn't want to look at each event individually, but >>>>>>>> instead process the whole file through a script or tool, which should >>>>>>>> not depend on the output specifics of a platform specific log apparatus. >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Rainer >>>>>>>> >>>>>>> >>>>> >>>> >>> >> > > From staffan.larsen at oracle.com Tue Aug 14 07:50:03 2012 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Tue, 14 Aug 2012 09:50:03 +0200 Subject: 7090324: gclog rotation via external tool In-Reply-To: <5017B761.5070203@lab.ntt.co.jp> References: <4E6D5DEC.8090109@oss.ntt.co.jp> <4E6DDB8F.4020301@oracle.com><3568943A-E6A4-4287-8319-8B0993F070F9@oracle.com><4E7A85AE.10507@oss.ntt.co.jp> <4E7B19EC.5070700@oracle.com> <4E7B2227.9070406@kippdata.de> <4E805629.7030703@oss.ntt.co.jp> <4E846158.3020606@oracle.com> <5017B761.5070203@lab.ntt.co.jp> Message-ID: <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> Hi Yasumasa, This work should be done in the context of the Unified Logging JEP [1]. Unfortunately, that work has not yet started due to resource constraints. Comments on the proposal are welcome. [1] http://openjdk.java.net/jeps/158 /Staffan On 31 jul 2012, at 12:45, Yasumasa Suenaga wrote: > Hi, > > I've posted a patch for gc log rotation via jinfo tool last year. > Then I received an email that essence of my patch will be implemented as > "subcommands" of jcmd. > > Now, jcmd is provided by OpenJDK. However, current jcmd does not seem to > implement function of gclog rotation. > > Will this RFE be implemented? > > I need to rotate gclog which is synchronized with signal, cron, etc... on Linux. > So, if this RFE is not implemented for a while, I would like to contribute > patch for jcmd. > > > Regards, > > Yasumasa > > > On 2011/09/29 21:15, James Melvin wrote: >> Hi Yasumasa, >> >> Thanks very much for your OpenJDK contribution! As part of the effort to >> port JRockit features to HotSpot, we plan to introduce a consolidated >> commandline serviceability tool (jcmd) to potentially replace many of >> the existing tools in the bin directory, such as jmap, jstack, jinfo and >> others. Over the next few update releases, we plan to add several jcmd >> *subcommands* instead to accomplish specific tasks or affect the running >> JVM instance. We'd like to use the essence of your contribution in one >> of the jcmd subcommands (instead of extending jinfo) to ask the JVM to >> begin rotating GC logs. We hope you find this attractive and hope you >> will help review and perfect the change. >> >> Thanks, >> >> Jim Melvin >> Sr. Engineering Manager >> Oracle >> >> >> >> On 9/26/11 6:38 AM, Yasumasa Suenaga wrote: >>> (I've changed subject of this email to new RFE.) >>> >>> This RFE is enhancement of current gclog implementation. >>> So, I'd like to discuss about rotating gclog. >>> >>> My customers need gclog rotation which is invoked by external trigger. >>> So I've requested this RFE and made a patch. >>> >>> >>> In many case on Linux, logfile is rotated by signal (e.g. SIGHUP) . >>> The aim of this RFE is to synchronize gclog and the other logs. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> (2011/09/22 20:55), Rainer Jung wrote: >>>> On 22.09.2011 13:20, Dmitry Samersoff wrote: >>>>> >>>>> Yasumasa, >>>>> >>>>> On 2011-09-22 04:47, Yasumasa Suenaga wrote: >>>>>> If we can think Java on Linux and Solaris only, syslog is best >>>>>> solution. >>>>>> However, Windows usually doesn't have syslog. >>>>>> >>>>>> So, I think that gclog is needed for logging GC stats with platform >>>>>> independent in realtime. >>>>> >>>>> Windows has it's own logging API as reach as syslog is or ever better >>>>> as well as numerous syslog implementations. >>>>> >>>>> Native windows logging API was completely redesigned for Windows 2008 >>>>> server and now it allows for developers to send a structured events from >>>>> theirs application. >>>> >>>> AFAIK log rotation for loggc is already implemented though not >>>> necessarily yet released. The change discussed here is about supporting >>>> an externally generated rotation trigger, e.g. via a signal, instead of >>>> only rotating by size or time via a startup parameter. >>>> >>>> If you want support for syslog or Windows APIs included, it would be >>>> best to start a new discussion. >>>> >>>> A GC log for an application under load might easily produce a block of >>>> about 1.5 KB size every few seconds. I seriously doubt, that the need >>>> for log file rotation can be argued away by referring to syslog or >>>> Windows log API as the correct solution. >>>> >>>> The messages are not really line formatted, the format can vary a lot >>>> (depending on the excat XX switches), the traffic can be quite high and >>>> AFAIK the JVM writes it synchronously, so there must be absolutely no >>>> risk in writing it out with very little latency. In addition, for >>>> analysis, you wouldn't want to look at each event individually, but >>>> instead process the whole file through a script or tool, which should >>>> not depend on the output specifics of a platform specific log apparatus. >>>> >>>> Regards, >>>> >>>> Rainer >>>> >>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kirk at kodewerk.com Wed Aug 15 08:44:15 2012 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Wed, 15 Aug 2012 10:44:15 +0200 Subject: JEP 158: Unified JVM logging In-Reply-To: <502A25B0.20004@oracle.com> References: <4E6D5DEC.8090109@oss.ntt.co.jp> <4E6DDB8F.4020301@oracle.com><3568943A-E6A4-4287-8319-8B0993F070F9@oracle.com><4E7A85AE.10507@oss.ntt.co.jp> <4E7B19EC.5070700@oracle.com> <4E7B2227.9070406@kippdata.de> <4E805629.7030703@oss.ntt.co.jp> <4E846158.3020606@oracle.com> <5017B761.5070203@lab.ntt.co.jp> <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> <502A1853.70508@lab.ntt.co.jp> <502A25B0.20004@oracle.com> Message-ID: Hi Dmitry, Lets start with this. Common logging command-line options for all components Logging is performed at different levels: error, warning, info, debug, trace Logging messages are in human-readable plain text IMHO, these current set of goals are based on the assumptions that all component is hierarchical in nature and that they will log the same way without the possibility of binary dumps. I would argue these assumptions over-reach and preclude certain types of logging that take place in the JVM today. The current system of command-line (experimental) options follow a format that is quite consistent. The options provide semantic meaning as to what will be logged. This semantic meaning is lost when we reduce everything to the level of error, warning, info, debug and trace. It's sets up arbitrary categories in which developers will be left to decide what level a particular activity should be logged at. To make this real I'll use the -XX:+PrintTenuringDistribution flag as an example. When I set that flag it's very clear what's to be logged. My question is, should this be logged at the info level? How about debug? Maybe trace? And if I set those levels, what else comes along for the ride? IOWs, if I attempt to keep the same model of GC logging, we'd need different levels. If we need different levels then we're about to violate the first goal. What this example suggests is that gc logging would be better served by using categories (or subjects if we can take something from the pub/sub metaphor) or tags or labels or what ever you want to call them. Using tags, I can create a system of levels using error, warning, info, debug, and trace if that fits how I'm setting up logging. But from levels it's difficult to implement things using categories. This is a case where the one of the non-goals should be, we shouldn't prevent people from structuring their logs as they see best fits the component they are instrumenting. The non-goals that should be reconsidered is to force human-readability. I would argue that this is again over-reaching in that properly structured log files should almost *never* be read by a human. In the cases why force an unnecessary requirement. I say this because I've run into a number of applications where logging was the primary bottleneck. In those cases there were two primary problems, one was bulk and the other was threading (lock contention which I shan't address here). Now while trying to solve the problem of bulkiness and lock contention might be considered a premature optimization at this stage and I certainty wouldn't disagree, I would not like to see designs or specifications that would naturally preclude some very obvious solutions. One of the solutions to bulk is to use a more compact format be some form of binary. That option has been taken away for what I would call an ideological requirement. There is no technical reason why I shouldn't be able to log in binary other than connivence. In addition, you don't know what else is happening on the system (disk??) you're logging to so it behoves you to be as light as possible. I'll leave it at this for now. -- Kirk On 2012-08-14, at 12:17 PM, Dmitry Samersoff wrote: > Kirk, > >> However I do have very serious concerns about this JEP in that it >> doesn't fix the problems that exist in current logging frameworks, >> it only mimics them. > >> http://openjdk.java.net/jeps/158 > > Any comments is much appreciated. > > Personally, I think that log rotation is out of scope and responsibility > of JVM. > > -Dmitry > > > On 2012-08-14 13:26, Kirk Pepperdine wrote: >> Hi Yasumasa, >> >> I'm not sure that log file rotation is a part of this JEP. >> However I do have very serious concerns about this JEP in that it doesn't fix the problems that exist in current logging frameworks, it only mimics them. >> >> Regards, >> Kirk >> >> On 2012-08-14, at 11:20 AM, Yasumasa Suenaga wrote: >> >>> Hi Staffan, >>> >>> May I ask you 2 questions about this JEP? >>> >>> 1. One of goals of this JEP is defined as following: >>> "File rotation of log files by size (similar to what is available for GC logs today)" >>> My patch realizes log rotation by external trigger. >>> 7090324 is included in this JEP? >>> (Is 7090324 included in "Logging can be configured dynamically at runtime via jcmd or MBeans" ? ) >>> >>> 2. I've posted a patch for "6950794: Make the GC log file name parameterized" . >>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2012-July/004758.html >>> Could you include this RFE in this JEP? >>> If we use log rotation, I think that name of logs is very important for log management. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2012/08/14 16:50, Staffan Larsen wrote: >>>> Hi Yasumasa, >>>> >>>> This work should be done in the context of the Unified Logging JEP [1]. Unfortunately, that work has not yet started due to resource constraints. Comments on the proposal are welcome. >>>> >>>> [1] http://openjdk.java.net/jeps/158 >>>> >>>> /Staffan >>>> >>>> On 31 jul 2012, at 12:45, Yasumasa Suenaga > wrote: >>>> >>>>> Hi, >>>>> >>>>> I've posted a patch for gc log rotation via jinfo tool last year. >>>>> Then I received an email that essence of my patch will be implemented as >>>>> "subcommands" of jcmd. >>>>> >>>>> Now, jcmd is provided by OpenJDK. However, current jcmd does not seem to >>>>> implement function of gclog rotation. >>>>> >>>>> Will this RFE be implemented? >>>>> >>>>> I need to rotate gclog which is synchronized with signal, cron, etc... on Linux. >>>>> So, if this RFE is not implemented for a while, I would like to contribute >>>>> patch for jcmd. >>>>> >>>>> >>>>> Regards, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2011/09/29 21:15, James Melvin wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> Thanks very much for your OpenJDK contribution! As part of the effort to >>>>>> port JRockit features to HotSpot, we plan to introduce a consolidated >>>>>> commandline serviceability tool (jcmd) to potentially replace many of >>>>>> the existing tools in the bin directory, such as jmap, jstack, jinfo and >>>>>> others. Over the next few update releases, we plan to add several jcmd >>>>>> *subcommands* instead to accomplish specific tasks or affect the running >>>>>> JVM instance. We'd like to use the essence of your contribution in one >>>>>> of the jcmd subcommands (instead of extending jinfo) to ask the JVM to >>>>>> begin rotating GC logs. We hope you find this attractive and hope you >>>>>> will help review and perfect the change. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Jim Melvin >>>>>> Sr. Engineering Manager >>>>>> Oracle >>>>>> >>>>>> >>>>>> >>>>>> On 9/26/11 6:38 AM, Yasumasa Suenaga wrote: >>>>>>> (I've changed subject of this email to new RFE.) >>>>>>> >>>>>>> This RFE is enhancement of current gclog implementation. >>>>>>> So, I'd like to discuss about rotating gclog. >>>>>>> >>>>>>> My customers need gclog rotation which is invoked by external trigger. >>>>>>> So I've requested this RFE and made a patch. >>>>>>> >>>>>>> >>>>>>> In many case on Linux, logfile is rotated by signal (e.g. SIGHUP) . >>>>>>> The aim of this RFE is to synchronize gclog and the other logs. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> (2011/09/22 20:55), Rainer Jung wrote: >>>>>>>> On 22.09.2011 13:20, Dmitry Samersoff wrote: >>>>>>>>> >>>>>>>>> Yasumasa, >>>>>>>>> >>>>>>>>> On 2011-09-22 04:47, Yasumasa Suenaga wrote: >>>>>>>>>> If we can think Java on Linux and Solaris only, syslog is best >>>>>>>>>> solution. >>>>>>>>>> However, Windows usually doesn't have syslog. >>>>>>>>>> >>>>>>>>>> So, I think that gclog is needed for logging GC stats with platform >>>>>>>>>> independent in realtime. >>>>>>>>> >>>>>>>>> Windows has it's own logging API as reach as syslog is or ever better >>>>>>>>> as well as numerous syslog implementations. >>>>>>>>> >>>>>>>>> Native windows logging API was completely redesigned for Windows 2008 >>>>>>>>> server and now it allows for developers to send a structured events from >>>>>>>>> theirs application. >>>>>>>> >>>>>>>> AFAIK log rotation for loggc is already implemented though not >>>>>>>> necessarily yet released. The change discussed here is about supporting >>>>>>>> an externally generated rotation trigger, e.g. via a signal, instead of >>>>>>>> only rotating by size or time via a startup parameter. >>>>>>>> >>>>>>>> If you want support for syslog or Windows APIs included, it would be >>>>>>>> best to start a new discussion. >>>>>>>> >>>>>>>> A GC log for an application under load might easily produce a block of >>>>>>>> about 1.5 KB size every few seconds. I seriously doubt, that the need >>>>>>>> for log file rotation can be argued away by referring to syslog or >>>>>>>> Windows log API as the correct solution. >>>>>>>> >>>>>>>> The messages are not really line formatted, the format can vary a lot >>>>>>>> (depending on the excat XX switches), the traffic can be quite high and >>>>>>>> AFAIK the JVM writes it synchronously, so there must be absolutely no >>>>>>>> risk in writing it out with very little latency. In addition, for >>>>>>>> analysis, you wouldn't want to look at each event individually, but >>>>>>>> instead process the whole file through a script or tool, which should >>>>>>>> not depend on the output specifics of a platform specific log apparatus. >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Rainer >>>>>>>> >>>>>>> >>>>> >>>> >>> >> > > > -- > Dmitry Samersoff > Java Hotspot development team, SPB04 > * There will come soft rains ... > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martijnverburg at gmail.com Wed Aug 15 08:51:59 2012 From: martijnverburg at gmail.com (Martijn Verburg) Date: Wed, 15 Aug 2012 09:51:59 +0100 Subject: JEP 158: Unified JVM logging In-Reply-To: References: <4E6D5DEC.8090109@oss.ntt.co.jp> <4E6DDB8F.4020301@oracle.com> <3568943A-E6A4-4287-8319-8B0993F070F9@oracle.com> <4E7A85AE.10507@oss.ntt.co.jp> <4E7B19EC.5070700@oracle.com> <4E7B2227.9070406@kippdata.de> <4E805629.7030703@oss.ntt.co.jp> <4E846158.3020606@oracle.com> <5017B761.5070203@lab.ntt.co.jp> <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> <502A1853.70508@lab.ntt.co.jp> <502A25B0.20004@oracle.com> Message-ID: I'll only add to this that an option to provide a Human Readable format out of the box would be desirable, but not as a default. On 15 August 2012 09:44, Kirk Pepperdine wrote: > Hi Dmitry, > > Lets start with this. > > Common logging command-line options for all components > Logging is performed at different levels: error, warning, info, debug, trace > Logging messages are in human-readable plain text > > > IMHO, these current set of goals are based on the assumptions that all > component is hierarchical in nature and that they will log the same way > without the possibility of binary dumps. I would argue these assumptions > over-reach and preclude certain types of logging that take place in the JVM > today. > > The current system of command-line (experimental) options follow a format > that is quite consistent. The options provide semantic meaning as to what > will be logged. This semantic meaning is lost when we reduce everything to > the level of error, warning, info, debug and trace. It's sets up arbitrary > categories in which developers will be left to decide what level a > particular activity should be logged at. To make this real I'll use the > -XX:+PrintTenuringDistribution flag as an example. When I set that flag it's > very clear what's to be logged. My question is, should this be logged at the > info level? How about debug? Maybe trace? And if I set those levels, what > else comes along for the ride? IOWs, if I attempt to keep the same model of > GC logging, we'd need different levels. If we need different levels then > we're about to violate the first goal. > > What this example suggests is that gc logging would be better served by > using categories (or subjects if we can take something from the pub/sub > metaphor) or tags or labels or what ever you want to call them. Using tags, > I can create a system of levels using error, warning, info, debug, and trace > if that fits how I'm setting up logging. But from levels it's difficult to > implement things using categories. This is a case where the one of the > non-goals should be, we shouldn't prevent people from structuring their logs > as they see best fits the component they are instrumenting. > > The non-goals that should be reconsidered is to force human-readability. I > would argue that this is again over-reaching in that properly structured log > files should almost *never* be read by a human. In the cases why force an > unnecessary requirement. I say this because I've run into a number of > applications where logging was the primary bottleneck. In those cases there > were two primary problems, one was bulk and the other was threading (lock > contention which I shan't address here). Now while trying to solve the > problem of bulkiness and lock contention might be considered a premature > optimization at this stage and I certainty wouldn't disagree, I would not > like to see designs or specifications that would naturally preclude some > very obvious solutions. One of the solutions to bulk is to use a more > compact format be some form of binary. That option has been taken away for > what I would call an ideological requirement. There is no technical reason > why I shouldn't be able to log in binary other than connivence. In addition, > you don't know what else is happening on the system (disk??) you're logging > to so it behoves you to be as light as possible. > > I'll leave it at this for now. > > -- Kirk > > On 2012-08-14, at 12:17 PM, Dmitry Samersoff > wrote: > > Kirk, > > However I do have very serious concerns about this JEP in that it > doesn't fix the problems that exist in current logging frameworks, > it only mimics them. > > > http://openjdk.java.net/jeps/158 > > > Any comments is much appreciated. > > Personally, I think that log rotation is out of scope and responsibility > of JVM. > > -Dmitry > > > On 2012-08-14 13:26, Kirk Pepperdine wrote: > > Hi Yasumasa, > > I'm not sure that log file rotation is a part of this JEP. > However I do have very serious concerns about this JEP in that it doesn't > fix the problems that exist in current logging frameworks, it only mimics > them. > > Regards, > Kirk > > On 2012-08-14, at 11:20 AM, Yasumasa Suenaga > wrote: > > Hi Staffan, > > May I ask you 2 questions about this JEP? > > 1. One of goals of this JEP is defined as following: > "File rotation of log files by size (similar to what is available for GC > logs today)" > My patch realizes log rotation by external trigger. > 7090324 is included in this JEP? > (Is 7090324 included in "Logging can be configured dynamically at runtime > via jcmd or MBeans" ? ) > > 2. I've posted a patch for "6950794: Make the GC log file name > parameterized" . > > http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2012-July/004758.html > Could you include this RFE in this JEP? > If we use log rotation, I think that name of logs is very important for > log management. > > > Thanks, > > Yasumasa > > > On 2012/08/14 16:50, Staffan Larsen wrote: > > Hi Yasumasa, > > This work should be done in the context of the Unified Logging JEP [1]. > Unfortunately, that work has not yet started due to resource constraints. > Comments on the proposal are welcome. > > [1] http://openjdk.java.net/jeps/158 > > /Staffan > > On 31 jul 2012, at 12:45, Yasumasa Suenaga > wrote: > > Hi, > > I've posted a patch for gc log rotation via jinfo tool last year. > Then I received an email that essence of my patch will be implemented as > "subcommands" of jcmd. > > Now, jcmd is provided by OpenJDK. However, current jcmd does not seem to > implement function of gclog rotation. > > Will this RFE be implemented? > > I need to rotate gclog which is synchronized with signal, cron, etc... on > Linux. > So, if this RFE is not implemented for a while, I would like to contribute > patch for jcmd. > > > Regards, > > Yasumasa > > > On 2011/09/29 21:15, James Melvin wrote: > > Hi Yasumasa, > > Thanks very much for your OpenJDK contribution! As part of the effort to > port JRockit features to HotSpot, we plan to introduce a consolidated > commandline serviceability tool (jcmd) to potentially replace many of > the existing tools in the bin directory, such as jmap, jstack, jinfo and > others. Over the next few update releases, we plan to add several jcmd > *subcommands* instead to accomplish specific tasks or affect the running > JVM instance. We'd like to use the essence of your contribution in one > of the jcmd subcommands (instead of extending jinfo) to ask the JVM to > begin rotating GC logs. We hope you find this attractive and hope you > will help review and perfect the change. > > Thanks, > > Jim Melvin > Sr. Engineering Manager > Oracle > > > > On 9/26/11 6:38 AM, Yasumasa Suenaga wrote: > > (I've changed subject of this email to new RFE.) > > This RFE is enhancement of current gclog implementation. > So, I'd like to discuss about rotating gclog. > > My customers need gclog rotation which is invoked by external trigger. > So I've requested this RFE and made a patch. > > > In many case on Linux, logfile is rotated by signal (e.g. SIGHUP) . > The aim of this RFE is to synchronize gclog and the other logs. > > > Thanks, > > Yasumasa > > (2011/09/22 20:55), Rainer Jung wrote: > > On 22.09.2011 13:20, Dmitry Samersoff wrote: > > > Yasumasa, > > On 2011-09-22 04:47, Yasumasa Suenaga wrote: > > If we can think Java on Linux and Solaris only, syslog is best > solution. > However, Windows usually doesn't have syslog. > > So, I think that gclog is needed for logging GC stats with platform > independent in realtime. > > > Windows has it's own logging API as reach as syslog is or ever better > as well as numerous syslog implementations. > > Native windows logging API was completely redesigned for Windows 2008 > server and now it allows for developers to send a structured events from > theirs application. > > > AFAIK log rotation for loggc is already implemented though not > necessarily yet released. The change discussed here is about supporting > an externally generated rotation trigger, e.g. via a signal, instead of > only rotating by size or time via a startup parameter. > > If you want support for syslog or Windows APIs included, it would be > best to start a new discussion. > > A GC log for an application under load might easily produce a block of > about 1.5 KB size every few seconds. I seriously doubt, that the need > for log file rotation can be argued away by referring to syslog or > Windows log API as the correct solution. > > The messages are not really line formatted, the format can vary a lot > (depending on the excat XX switches), the traffic can be quite high and > AFAIK the JVM writes it synchronously, so there must be absolutely no > risk in writing it out with very little latency. In addition, for > analysis, you wouldn't want to look at each event individually, but > instead process the whole file through a script or tool, which should > not depend on the output specifics of a platform specific log apparatus. > > Regards, > > Rainer > > > > > > > > > -- > Dmitry Samersoff > Java Hotspot development team, SPB04 > * There will come soft rains ... > > > From kirk at kodewerk.com Wed Aug 15 08:56:07 2012 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Wed, 15 Aug 2012 10:56:07 +0200 Subject: JEP 158: Unified JVM logging In-Reply-To: <502A25B0.20004@oracle.com> References: <4E6D5DEC.8090109@oss.ntt.co.jp> <4E6DDB8F.4020301@oracle.com><3568943A-E6A4-4287-8319-8B0993F070F9@oracle.com><4E7A85AE.10507@oss.ntt.co.jp> <4E7B19EC.5070700@oracle.com> <4E7B2227.9070406@kippdata.de> <4E805629.7030703@oss.ntt.co.jp> <4E846158.3020606@oracle.com> <5017B761.5070203@lab.ntt.co.jp> <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> <502A1853.70508@lab.ntt.co.jp> <502A25B0.20004@oracle.com> Message-ID: <6976D1D8-DD7B-43D8-B03D-E0D3E05BD771@kodewerk.com> Hi Dmitry, I should have added that Neal Ford has an excellent talk on taxonomy systems and tags vs. hierarchies. It includes a bit on the 5 kingdoms that all organisms are split into. Yet many organisms show characteristics from more than one kingdom while some others show characteristics of none. Problem is, the current hierarchical system forces categorization in a way that doesn't work where as a tag system would easily be able to cope. I'm trying to find a link to the talk. Regards, Kirk On 2012-08-14, at 12:17 PM, Dmitry Samersoff wrote: > Kirk, > >> However I do have very serious concerns about this JEP in that it >> doesn't fix the problems that exist in current logging frameworks, >> it only mimics them. > >> http://openjdk.java.net/jeps/158 > > Any comments is much appreciated. > > Personally, I think that log rotation is out of scope and responsibility > of JVM. > > -Dmitry > > > On 2012-08-14 13:26, Kirk Pepperdine wrote: >> Hi Yasumasa, >> >> I'm not sure that log file rotation is a part of this JEP. >> However I do have very serious concerns about this JEP in that it doesn't fix the problems that exist in current logging frameworks, it only mimics them. >> >> Regards, >> Kirk >> >> On 2012-08-14, at 11:20 AM, Yasumasa Suenaga wrote: >> >>> Hi Staffan, >>> >>> May I ask you 2 questions about this JEP? >>> >>> 1. One of goals of this JEP is defined as following: >>> "File rotation of log files by size (similar to what is available for GC logs today)" >>> My patch realizes log rotation by external trigger. >>> 7090324 is included in this JEP? >>> (Is 7090324 included in "Logging can be configured dynamically at runtime via jcmd or MBeans" ? ) >>> >>> 2. I've posted a patch for "6950794: Make the GC log file name parameterized" . >>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2012-July/004758.html >>> Could you include this RFE in this JEP? >>> If we use log rotation, I think that name of logs is very important for log management. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2012/08/14 16:50, Staffan Larsen wrote: >>>> Hi Yasumasa, >>>> >>>> This work should be done in the context of the Unified Logging JEP [1]. Unfortunately, that work has not yet started due to resource constraints. Comments on the proposal are welcome. >>>> >>>> [1] http://openjdk.java.net/jeps/158 >>>> >>>> /Staffan >>>> >>>> On 31 jul 2012, at 12:45, Yasumasa Suenaga > wrote: >>>> >>>>> Hi, >>>>> >>>>> I've posted a patch for gc log rotation via jinfo tool last year. >>>>> Then I received an email that essence of my patch will be implemented as >>>>> "subcommands" of jcmd. >>>>> >>>>> Now, jcmd is provided by OpenJDK. However, current jcmd does not seem to >>>>> implement function of gclog rotation. >>>>> >>>>> Will this RFE be implemented? >>>>> >>>>> I need to rotate gclog which is synchronized with signal, cron, etc... on Linux. >>>>> So, if this RFE is not implemented for a while, I would like to contribute >>>>> patch for jcmd. >>>>> >>>>> >>>>> Regards, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2011/09/29 21:15, James Melvin wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> Thanks very much for your OpenJDK contribution! As part of the effort to >>>>>> port JRockit features to HotSpot, we plan to introduce a consolidated >>>>>> commandline serviceability tool (jcmd) to potentially replace many of >>>>>> the existing tools in the bin directory, such as jmap, jstack, jinfo and >>>>>> others. Over the next few update releases, we plan to add several jcmd >>>>>> *subcommands* instead to accomplish specific tasks or affect the running >>>>>> JVM instance. We'd like to use the essence of your contribution in one >>>>>> of the jcmd subcommands (instead of extending jinfo) to ask the JVM to >>>>>> begin rotating GC logs. We hope you find this attractive and hope you >>>>>> will help review and perfect the change. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Jim Melvin >>>>>> Sr. Engineering Manager >>>>>> Oracle >>>>>> >>>>>> >>>>>> >>>>>> On 9/26/11 6:38 AM, Yasumasa Suenaga wrote: >>>>>>> (I've changed subject of this email to new RFE.) >>>>>>> >>>>>>> This RFE is enhancement of current gclog implementation. >>>>>>> So, I'd like to discuss about rotating gclog. >>>>>>> >>>>>>> My customers need gclog rotation which is invoked by external trigger. >>>>>>> So I've requested this RFE and made a patch. >>>>>>> >>>>>>> >>>>>>> In many case on Linux, logfile is rotated by signal (e.g. SIGHUP) . >>>>>>> The aim of this RFE is to synchronize gclog and the other logs. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> (2011/09/22 20:55), Rainer Jung wrote: >>>>>>>> On 22.09.2011 13:20, Dmitry Samersoff wrote: >>>>>>>>> >>>>>>>>> Yasumasa, >>>>>>>>> >>>>>>>>> On 2011-09-22 04:47, Yasumasa Suenaga wrote: >>>>>>>>>> If we can think Java on Linux and Solaris only, syslog is best >>>>>>>>>> solution. >>>>>>>>>> However, Windows usually doesn't have syslog. >>>>>>>>>> >>>>>>>>>> So, I think that gclog is needed for logging GC stats with platform >>>>>>>>>> independent in realtime. >>>>>>>>> >>>>>>>>> Windows has it's own logging API as reach as syslog is or ever better >>>>>>>>> as well as numerous syslog implementations. >>>>>>>>> >>>>>>>>> Native windows logging API was completely redesigned for Windows 2008 >>>>>>>>> server and now it allows for developers to send a structured events from >>>>>>>>> theirs application. >>>>>>>> >>>>>>>> AFAIK log rotation for loggc is already implemented though not >>>>>>>> necessarily yet released. The change discussed here is about supporting >>>>>>>> an externally generated rotation trigger, e.g. via a signal, instead of >>>>>>>> only rotating by size or time via a startup parameter. >>>>>>>> >>>>>>>> If you want support for syslog or Windows APIs included, it would be >>>>>>>> best to start a new discussion. >>>>>>>> >>>>>>>> A GC log for an application under load might easily produce a block of >>>>>>>> about 1.5 KB size every few seconds. I seriously doubt, that the need >>>>>>>> for log file rotation can be argued away by referring to syslog or >>>>>>>> Windows log API as the correct solution. >>>>>>>> >>>>>>>> The messages are not really line formatted, the format can vary a lot >>>>>>>> (depending on the excat XX switches), the traffic can be quite high and >>>>>>>> AFAIK the JVM writes it synchronously, so there must be absolutely no >>>>>>>> risk in writing it out with very little latency. In addition, for >>>>>>>> analysis, you wouldn't want to look at each event individually, but >>>>>>>> instead process the whole file through a script or tool, which should >>>>>>>> not depend on the output specifics of a platform specific log apparatus. >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Rainer >>>>>>>> >>>>>>> >>>>> >>>> >>> >> > > > -- > Dmitry Samersoff > Java Hotspot development team, SPB04 > * There will come soft rains ... > > From kirk at kodewerk.com Wed Aug 15 09:25:22 2012 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Wed, 15 Aug 2012 11:25:22 +0200 Subject: JEP 158: Unified JVM logging In-Reply-To: <502A25B0.20004@oracle.com> References: <4E6D5DEC.8090109@oss.ntt.co.jp> <4E6DDB8F.4020301@oracle.com><3568943A-E6A4-4287-8319-8B0993F070F9@oracle.com><4E7A85AE.10507@oss.ntt.co.jp> <4E7B19EC.5070700@oracle.com> <4E7B2227.9070406@kippdata.de> <4E805629.7030703@oss.ntt.co.jp> <4E846158.3020606@oracle.com> <5017B761.5070203@lab.ntt.co.jp> <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> <502A1853.70508@lab.ntt.co.jp> <502A25B0.20004@oracle.com> Message-ID: <883AE36C-A267-43EE-956F-DC8B7C52EBA3@kodewerk.com> Hi Dmitry, Neal Ford talk is call "Abstraction Distraction".. it's downloadable and you can see it here. http://vimeo.com/44235657 -- Kirk On 2012-08-14, at 12:17 PM, Dmitry Samersoff wrote: > Kirk, > >> However I do have very serious concerns about this JEP in that it >> doesn't fix the problems that exist in current logging frameworks, >> it only mimics them. > >> http://openjdk.java.net/jeps/158 > > Any comments is much appreciated. > > Personally, I think that log rotation is out of scope and responsibility > of JVM. > > -Dmitry > > > On 2012-08-14 13:26, Kirk Pepperdine wrote: >> Hi Yasumasa, >> >> I'm not sure that log file rotation is a part of this JEP. >> However I do have very serious concerns about this JEP in that it doesn't fix the problems that exist in current logging frameworks, it only mimics them. >> >> Regards, >> Kirk >> >> On 2012-08-14, at 11:20 AM, Yasumasa Suenaga wrote: >> >>> Hi Staffan, >>> >>> May I ask you 2 questions about this JEP? >>> >>> 1. One of goals of this JEP is defined as following: >>> "File rotation of log files by size (similar to what is available for GC logs today)" >>> My patch realizes log rotation by external trigger. >>> 7090324 is included in this JEP? >>> (Is 7090324 included in "Logging can be configured dynamically at runtime via jcmd or MBeans" ? ) >>> >>> 2. I've posted a patch for "6950794: Make the GC log file name parameterized" . >>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2012-July/004758.html >>> Could you include this RFE in this JEP? >>> If we use log rotation, I think that name of logs is very important for log management. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2012/08/14 16:50, Staffan Larsen wrote: >>>> Hi Yasumasa, >>>> >>>> This work should be done in the context of the Unified Logging JEP [1]. Unfortunately, that work has not yet started due to resource constraints. Comments on the proposal are welcome. >>>> >>>> [1] http://openjdk.java.net/jeps/158 >>>> >>>> /Staffan >>>> >>>> On 31 jul 2012, at 12:45, Yasumasa Suenaga > wrote: >>>> >>>>> Hi, >>>>> >>>>> I've posted a patch for gc log rotation via jinfo tool last year. >>>>> Then I received an email that essence of my patch will be implemented as >>>>> "subcommands" of jcmd. >>>>> >>>>> Now, jcmd is provided by OpenJDK. However, current jcmd does not seem to >>>>> implement function of gclog rotation. >>>>> >>>>> Will this RFE be implemented? >>>>> >>>>> I need to rotate gclog which is synchronized with signal, cron, etc... on Linux. >>>>> So, if this RFE is not implemented for a while, I would like to contribute >>>>> patch for jcmd. >>>>> >>>>> >>>>> Regards, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2011/09/29 21:15, James Melvin wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> Thanks very much for your OpenJDK contribution! As part of the effort to >>>>>> port JRockit features to HotSpot, we plan to introduce a consolidated >>>>>> commandline serviceability tool (jcmd) to potentially replace many of >>>>>> the existing tools in the bin directory, such as jmap, jstack, jinfo and >>>>>> others. Over the next few update releases, we plan to add several jcmd >>>>>> *subcommands* instead to accomplish specific tasks or affect the running >>>>>> JVM instance. We'd like to use the essence of your contribution in one >>>>>> of the jcmd subcommands (instead of extending jinfo) to ask the JVM to >>>>>> begin rotating GC logs. We hope you find this attractive and hope you >>>>>> will help review and perfect the change. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Jim Melvin >>>>>> Sr. Engineering Manager >>>>>> Oracle >>>>>> >>>>>> >>>>>> >>>>>> On 9/26/11 6:38 AM, Yasumasa Suenaga wrote: >>>>>>> (I've changed subject of this email to new RFE.) >>>>>>> >>>>>>> This RFE is enhancement of current gclog implementation. >>>>>>> So, I'd like to discuss about rotating gclog. >>>>>>> >>>>>>> My customers need gclog rotation which is invoked by external trigger. >>>>>>> So I've requested this RFE and made a patch. >>>>>>> >>>>>>> >>>>>>> In many case on Linux, logfile is rotated by signal (e.g. SIGHUP) . >>>>>>> The aim of this RFE is to synchronize gclog and the other logs. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> (2011/09/22 20:55), Rainer Jung wrote: >>>>>>>> On 22.09.2011 13:20, Dmitry Samersoff wrote: >>>>>>>>> >>>>>>>>> Yasumasa, >>>>>>>>> >>>>>>>>> On 2011-09-22 04:47, Yasumasa Suenaga wrote: >>>>>>>>>> If we can think Java on Linux and Solaris only, syslog is best >>>>>>>>>> solution. >>>>>>>>>> However, Windows usually doesn't have syslog. >>>>>>>>>> >>>>>>>>>> So, I think that gclog is needed for logging GC stats with platform >>>>>>>>>> independent in realtime. >>>>>>>>> >>>>>>>>> Windows has it's own logging API as reach as syslog is or ever better >>>>>>>>> as well as numerous syslog implementations. >>>>>>>>> >>>>>>>>> Native windows logging API was completely redesigned for Windows 2008 >>>>>>>>> server and now it allows for developers to send a structured events from >>>>>>>>> theirs application. >>>>>>>> >>>>>>>> AFAIK log rotation for loggc is already implemented though not >>>>>>>> necessarily yet released. The change discussed here is about supporting >>>>>>>> an externally generated rotation trigger, e.g. via a signal, instead of >>>>>>>> only rotating by size or time via a startup parameter. >>>>>>>> >>>>>>>> If you want support for syslog or Windows APIs included, it would be >>>>>>>> best to start a new discussion. >>>>>>>> >>>>>>>> A GC log for an application under load might easily produce a block of >>>>>>>> about 1.5 KB size every few seconds. I seriously doubt, that the need >>>>>>>> for log file rotation can be argued away by referring to syslog or >>>>>>>> Windows log API as the correct solution. >>>>>>>> >>>>>>>> The messages are not really line formatted, the format can vary a lot >>>>>>>> (depending on the excat XX switches), the traffic can be quite high and >>>>>>>> AFAIK the JVM writes it synchronously, so there must be absolutely no >>>>>>>> risk in writing it out with very little latency. In addition, for >>>>>>>> analysis, you wouldn't want to look at each event individually, but >>>>>>>> instead process the whole file through a script or tool, which should >>>>>>>> not depend on the output specifics of a platform specific log apparatus. >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Rainer >>>>>>>> >>>>>>> >>>>> >>>> >>> >> > > > -- > Dmitry Samersoff > Java Hotspot development team, SPB04 > * There will come soft rains ... > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Dmitry.Samersoff at oracle.com Wed Aug 15 09:29:49 2012 From: Dmitry.Samersoff at oracle.com (Dmitry Samersoff) Date: Wed, 15 Aug 2012 13:29:49 +0400 Subject: JEP 158: Unified JVM logging In-Reply-To: <883AE36C-A267-43EE-956F-DC8B7C52EBA3@kodewerk.com> References: <4E6D5DEC.8090109@oss.ntt.co.jp> <4E6DDB8F.4020301@oracle.com><3568943A-E6A4-4287-8319-8B0993F070F9@oracle.com><4E7A85AE.10507@oss.ntt.co.jp> <4E7B19EC.5070700@oracle.com> <4E7B2227.9070406@kippdata.de> <4E805629.7030703@oss.ntt.co.jp> <4E846158.3020606@oracle.com> <5017B761.5070203@lab.ntt.co.jp> <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> <502A1853.70508@lab.ntt.co.jp> <502A25B0.20004@oracle.com> <883AE36C-A267-43EE-956F-DC8B7C52EBA3@kodewerk.com> Message-ID: <502B6C0D.3040802@oracle.com> Kirk, Thank you. I'm downloading it. PS: Tags vs hierarchy discussion is as old as taxonomy it self. -Dmitry On 2012-08-15 13:25, Kirk Pepperdine wrote: > Hi Dmitry, > > Neal Ford talk is call "Abstraction Distraction".. it's downloadable and > you can see it here. http://vimeo.com/44235657 > > -- Kirk > > On 2012-08-14, at 12:17 PM, Dmitry Samersoff > > wrote: > >> Kirk, >> >>> However I do have very serious concerns about this JEP in that it >>> doesn't fix the problems that exist in current logging frameworks, >>> it only mimics them. >> >>> http://openjdk.java.net/jeps/158 >> >> Any comments is much appreciated. >> >> Personally, I think that log rotation is out of scope and responsibility >> of JVM. >> >> -Dmitry >> >> >> On 2012-08-14 13:26, Kirk Pepperdine wrote: >>> Hi Yasumasa, >>> >>> I'm not sure that log file rotation is a part of this JEP. >>> However I do have very serious concerns about this JEP in that it >>> doesn't fix the problems that exist in current logging frameworks, it >>> only mimics them. >>> >>> Regards, >>> Kirk >>> >>> On 2012-08-14, at 11:20 AM, Yasumasa Suenaga >>> >> > wrote: >>> >>>> Hi Staffan, >>>> >>>> May I ask you 2 questions about this JEP? >>>> >>>> 1. One of goals of this JEP is defined as following: >>>> "File rotation of log files by size (similar to what is >>>> available for GC logs today)" >>>> My patch realizes log rotation by external trigger. >>>> 7090324 is included in this JEP? >>>> (Is 7090324 included in "Logging can be configured dynamically at >>>> runtime via jcmd or MBeans" ? ) >>>> >>>> 2. I've posted a patch for "6950794: Make the GC log file name >>>> parameterized" . >>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2012-July/004758.html >>>> Could you include this RFE in this JEP? >>>> If we use log rotation, I think that name of logs is very >>>> important for log management. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2012/08/14 16:50, Staffan Larsen wrote: >>>>> Hi Yasumasa, >>>>> >>>>> This work should be done in the context of the Unified Logging JEP >>>>> [1]. Unfortunately, that work has not yet started due to resource >>>>> constraints. Comments on the proposal are welcome. >>>>> >>>>> [1] http://openjdk.java.net/jeps/158 >>>>> >>>>> /Staffan >>>>> >>>>> On 31 jul 2012, at 12:45, Yasumasa Suenaga >>>>> >>>> >>>>> > wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I've posted a patch for gc log rotation via jinfo tool last year. >>>>>> Then I received an email that essence of my patch will be >>>>>> implemented as >>>>>> "subcommands" of jcmd. >>>>>> >>>>>> Now, jcmd is provided by OpenJDK. However, current jcmd does not >>>>>> seem to >>>>>> implement function of gclog rotation. >>>>>> >>>>>> Will this RFE be implemented? >>>>>> >>>>>> I need to rotate gclog which is synchronized with signal, cron, >>>>>> etc... on Linux. >>>>>> So, if this RFE is not implemented for a while, I would like to >>>>>> contribute >>>>>> patch for jcmd. >>>>>> >>>>>> >>>>>> Regards, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2011/09/29 21:15, James Melvin wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> Thanks very much for your OpenJDK contribution! As part of the >>>>>>> effort to >>>>>>> port JRockit features to HotSpot, we plan to introduce a consolidated >>>>>>> commandline serviceability tool (jcmd) to potentially replace many of >>>>>>> the existing tools in the bin directory, such as jmap, jstack, >>>>>>> jinfo and >>>>>>> others. Over the next few update releases, we plan to add several >>>>>>> jcmd >>>>>>> *subcommands* instead to accomplish specific tasks or affect the >>>>>>> running >>>>>>> JVM instance. We'd like to use the essence of your contribution >>>>>>> in one >>>>>>> of the jcmd subcommands (instead of extending jinfo) to ask the >>>>>>> JVM to >>>>>>> begin rotating GC logs. We hope you find this attractive and hope you >>>>>>> will help review and perfect the change. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Jim Melvin >>>>>>> Sr. Engineering Manager >>>>>>> Oracle >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 9/26/11 6:38 AM, Yasumasa Suenaga wrote: >>>>>>>> (I've changed subject of this email to new RFE.) >>>>>>>> >>>>>>>> This RFE is enhancement of current gclog implementation. >>>>>>>> So, I'd like to discuss about rotating gclog. >>>>>>>> >>>>>>>> My customers need gclog rotation which is invoked by external >>>>>>>> trigger. >>>>>>>> So I've requested this RFE and made a patch. >>>>>>>> >>>>>>>> >>>>>>>> In many case on Linux, logfile is rotated by signal (e.g. SIGHUP) . >>>>>>>> The aim of this RFE is to synchronize gclog and the other logs. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> (2011/09/22 20:55), Rainer Jung wrote: >>>>>>>>> On 22.09.2011 13:20, Dmitry Samersoff wrote: >>>>>>>>>> >>>>>>>>>> Yasumasa, >>>>>>>>>> >>>>>>>>>> On 2011-09-22 04:47, Yasumasa Suenaga wrote: >>>>>>>>>>> If we can think Java on Linux and Solaris only, syslog is best >>>>>>>>>>> solution. >>>>>>>>>>> However, Windows usually doesn't have syslog. >>>>>>>>>>> >>>>>>>>>>> So, I think that gclog is needed for logging GC stats with >>>>>>>>>>> platform >>>>>>>>>>> independent in realtime. >>>>>>>>>> >>>>>>>>>> Windows has it's own logging API as reach as syslog is or ever >>>>>>>>>> better >>>>>>>>>> as well as numerous syslog implementations. >>>>>>>>>> >>>>>>>>>> Native windows logging API was completely redesigned for >>>>>>>>>> Windows 2008 >>>>>>>>>> server and now it allows for developers to send a structured >>>>>>>>>> events from >>>>>>>>>> theirs application. >>>>>>>>> >>>>>>>>> AFAIK log rotation for loggc is already implemented though not >>>>>>>>> necessarily yet released. The change discussed here is about >>>>>>>>> supporting >>>>>>>>> an externally generated rotation trigger, e.g. via a signal, >>>>>>>>> instead of >>>>>>>>> only rotating by size or time via a startup parameter. >>>>>>>>> >>>>>>>>> If you want support for syslog or Windows APIs included, it >>>>>>>>> would be >>>>>>>>> best to start a new discussion. >>>>>>>>> >>>>>>>>> A GC log for an application under load might easily produce a >>>>>>>>> block of >>>>>>>>> about 1.5 KB size every few seconds. I seriously doubt, that >>>>>>>>> the need >>>>>>>>> for log file rotation can be argued away by referring to syslog or >>>>>>>>> Windows log API as the correct solution. >>>>>>>>> >>>>>>>>> The messages are not really line formatted, the format can vary >>>>>>>>> a lot >>>>>>>>> (depending on the excat XX switches), the traffic can be quite >>>>>>>>> high and >>>>>>>>> AFAIK the JVM writes it synchronously, so there must be >>>>>>>>> absolutely no >>>>>>>>> risk in writing it out with very little latency. In addition, for >>>>>>>>> analysis, you wouldn't want to look at each event individually, but >>>>>>>>> instead process the whole file through a script or tool, which >>>>>>>>> should >>>>>>>>> not depend on the output specifics of a platform specific log >>>>>>>>> apparatus. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Rainer >>>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >>> >> >> >> -- >> Dmitry Samersoff >> Java Hotspot development team, SPB04 >> * There will come soft rains ... >> >> > -- Dmitry Samersoff Java Hotspot development team, SPB04 * There will come soft rains ... From kirk at kodewerk.com Wed Aug 15 10:09:28 2012 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Wed, 15 Aug 2012 12:09:28 +0200 Subject: JEP 158: Unified JVM logging In-Reply-To: <502B6C0D.3040802@oracle.com> References: <4E6D5DEC.8090109@oss.ntt.co.jp> <4E6DDB8F.4020301@oracle.com><3568943A-E6A4-4287-8319-8B0993F070F9@oracle.com><4E7A85AE.10507@oss.ntt.co.jp> <4E7B19EC.5070700@oracle.com> <4E7B2227.9070406@kippdata.de> <4E805629.7030703@oss.ntt.co.jp> <4E846158.3020606@oracle.com> <5017B761.5070203@lab.ntt.co.jp> <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> <502A1853.70508@lab.ntt.co.jp> <502A25B0.20004@oracle.com> <883AE36C-A267-43EE-956F-DC8B7C52EBA3@kodewerk.com> <502B6C0D.3040802@oracle.com> Message-ID: <27886498-DC82-4022-8AA8-88D57E60366A@kodewerk.com> On 2012-08-15, at 11:29 AM, Dmitry Samersoff wrote: > Kirk, > > Thank you. I'm downloading it. > > PS: > Tags vs hierarchy discussion is as old as taxonomy it self. which is why I'm surprised we're here ;-) Kirk From kirk at kodewerk.com Wed Aug 15 12:01:53 2012 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Wed, 15 Aug 2012 14:01:53 +0200 Subject: concurrent mode failure during concurrent sweep phase Message-ID: <4673403A-6D16-4917-86F5-670C5E329A0A@kodewerk.com> Hi all, : 81092K->81092K(81856K), 0.3419045 secs]5274.145: [CMS5281.056: [CMS-concurrent-sweep: 11.066/12.602 secs] In all of the GC logs I've got, this one doesn't show up very often. Now that I'm looking at it I'm asking myself... does it really make sense to declare a CMF when one is so close to the end of the CMS cycle? This really literally just small numbers of instructions away from the CMS-reset.... Wouldn't it make more sense to delay the ParNew that is hoisting survivors into CMS tenured vs triggering a full gc with it's (most likely longer) STW pause? Or is this a signal that the failure is most likely due to high fragmentation? Regards, Kirk From Dmitry.Samersoff at oracle.com Wed Aug 15 13:28:01 2012 From: Dmitry.Samersoff at oracle.com (Dmitry Samersoff) Date: Wed, 15 Aug 2012 17:28:01 +0400 Subject: JEP 158: Unified JVM logging In-Reply-To: <6976D1D8-DD7B-43D8-B03D-E0D3E05BD771@kodewerk.com> References: <4E6D5DEC.8090109@oss.ntt.co.jp> <4E6DDB8F.4020301@oracle.com><3568943A-E6A4-4287-8319-8B0993F070F9@oracle.com><4E7A85AE.10507@oss.ntt.co.jp> <4E7B19EC.5070700@oracle.com> <4E7B2227.9070406@kippdata.de> <4E805629.7030703@oss.ntt.co.jp> <4E846158.3020606@oracle.com> <5017B761.5070203@lab.ntt.co.jp> <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> <502A1853.70508@lab.ntt.co.jp> <502A25B0.20004@oracle.com> <6976D1D8-DD7B-43D8-B03D-E0D3E05BD771@kodewerk.com> Message-ID: <502BA3E1.5000702@oracle.com> Kirk, On 2012-08-15 12:56, Kirk Pepperdine wrote: > I should have added that Neal Ford has an excellent talk on taxonomy systems and tags vs. hierarchies. > It includes a bit on the 5 kingdoms that all organisms are split into. Yet many organisms show characteristics > from more than one kingdom while some others show characteristics of none. Problem is, > the current hierarchical system forces categorization in a way that doesn't work where > as a tag system would easily be able to cope. I'm trying to find a link to the talk. Really good talk. Stepping aside from logging problem - hierarchy require additional efforts from a people who builds hierarchy, but tags put the same efforts to people who uses it. If I'm constructing new butterfly and check description of Insecta, Lepidoptera I should get a list of the most important characteristics of all butterflies. It's responsibility of persons building hierarchy to create this list. I trust them and believe that if I give all these characteristics to my being it become a butterfly. But with tags I should build the list of characteristics by my self, e.g. start from insects with tags "four wings", and then refine the mix of Lepidoptera and Hymenoptera using other tags to distinguish between butterfly and bees. So tags give you more power but require much more efforts in regular case. -Dmitry > > Regards, > Kirk > > On 2012-08-14, at 12:17 PM, Dmitry Samersoff wrote: > >> Kirk, >> >>> However I do have very serious concerns about this JEP in that it >>> doesn't fix the problems that exist in current logging frameworks, >>> it only mimics them. >> >>> http://openjdk.java.net/jeps/158 >> >> Any comments is much appreciated. >> >> Personally, I think that log rotation is out of scope and responsibility >> of JVM. >> >> -Dmitry >> >> >> On 2012-08-14 13:26, Kirk Pepperdine wrote: >>> Hi Yasumasa, >>> >>> I'm not sure that log file rotation is a part of this JEP. >>> However I do have very serious concerns about this JEP in that it doesn't fix the problems that exist in current logging frameworks, it only mimics them. >>> >>> Regards, >>> Kirk >>> >>> On 2012-08-14, at 11:20 AM, Yasumasa Suenaga wrote: >>> >>>> Hi Staffan, >>>> >>>> May I ask you 2 questions about this JEP? >>>> >>>> 1. One of goals of this JEP is defined as following: >>>> "File rotation of log files by size (similar to what is available for GC logs today)" >>>> My patch realizes log rotation by external trigger. >>>> 7090324 is included in this JEP? >>>> (Is 7090324 included in "Logging can be configured dynamically at runtime via jcmd or MBeans" ? ) >>>> >>>> 2. I've posted a patch for "6950794: Make the GC log file name parameterized" . >>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2012-July/004758.html >>>> Could you include this RFE in this JEP? >>>> If we use log rotation, I think that name of logs is very important for log management. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2012/08/14 16:50, Staffan Larsen wrote: >>>>> Hi Yasumasa, >>>>> >>>>> This work should be done in the context of the Unified Logging JEP [1]. Unfortunately, that work has not yet started due to resource constraints. Comments on the proposal are welcome. >>>>> >>>>> [1] http://openjdk.java.net/jeps/158 >>>>> >>>>> /Staffan >>>>> >>>>> On 31 jul 2012, at 12:45, Yasumasa Suenaga > wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I've posted a patch for gc log rotation via jinfo tool last year. >>>>>> Then I received an email that essence of my patch will be implemented as >>>>>> "subcommands" of jcmd. >>>>>> >>>>>> Now, jcmd is provided by OpenJDK. However, current jcmd does not seem to >>>>>> implement function of gclog rotation. >>>>>> >>>>>> Will this RFE be implemented? >>>>>> >>>>>> I need to rotate gclog which is synchronized with signal, cron, etc... on Linux. >>>>>> So, if this RFE is not implemented for a while, I would like to contribute >>>>>> patch for jcmd. >>>>>> >>>>>> >>>>>> Regards, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2011/09/29 21:15, James Melvin wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> Thanks very much for your OpenJDK contribution! As part of the effort to >>>>>>> port JRockit features to HotSpot, we plan to introduce a consolidated >>>>>>> commandline serviceability tool (jcmd) to potentially replace many of >>>>>>> the existing tools in the bin directory, such as jmap, jstack, jinfo and >>>>>>> others. Over the next few update releases, we plan to add several jcmd >>>>>>> *subcommands* instead to accomplish specific tasks or affect the running >>>>>>> JVM instance. We'd like to use the essence of your contribution in one >>>>>>> of the jcmd subcommands (instead of extending jinfo) to ask the JVM to >>>>>>> begin rotating GC logs. We hope you find this attractive and hope you >>>>>>> will help review and perfect the change. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Jim Melvin >>>>>>> Sr. Engineering Manager >>>>>>> Oracle >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 9/26/11 6:38 AM, Yasumasa Suenaga wrote: >>>>>>>> (I've changed subject of this email to new RFE.) >>>>>>>> >>>>>>>> This RFE is enhancement of current gclog implementation. >>>>>>>> So, I'd like to discuss about rotating gclog. >>>>>>>> >>>>>>>> My customers need gclog rotation which is invoked by external trigger. >>>>>>>> So I've requested this RFE and made a patch. >>>>>>>> >>>>>>>> >>>>>>>> In many case on Linux, logfile is rotated by signal (e.g. SIGHUP) . >>>>>>>> The aim of this RFE is to synchronize gclog and the other logs. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> (2011/09/22 20:55), Rainer Jung wrote: >>>>>>>>> On 22.09.2011 13:20, Dmitry Samersoff wrote: >>>>>>>>>> >>>>>>>>>> Yasumasa, >>>>>>>>>> >>>>>>>>>> On 2011-09-22 04:47, Yasumasa Suenaga wrote: >>>>>>>>>>> If we can think Java on Linux and Solaris only, syslog is best >>>>>>>>>>> solution. >>>>>>>>>>> However, Windows usually doesn't have syslog. >>>>>>>>>>> >>>>>>>>>>> So, I think that gclog is needed for logging GC stats with platform >>>>>>>>>>> independent in realtime. >>>>>>>>>> >>>>>>>>>> Windows has it's own logging API as reach as syslog is or ever better >>>>>>>>>> as well as numerous syslog implementations. >>>>>>>>>> >>>>>>>>>> Native windows logging API was completely redesigned for Windows 2008 >>>>>>>>>> server and now it allows for developers to send a structured events from >>>>>>>>>> theirs application. >>>>>>>>> >>>>>>>>> AFAIK log rotation for loggc is already implemented though not >>>>>>>>> necessarily yet released. The change discussed here is about supporting >>>>>>>>> an externally generated rotation trigger, e.g. via a signal, instead of >>>>>>>>> only rotating by size or time via a startup parameter. >>>>>>>>> >>>>>>>>> If you want support for syslog or Windows APIs included, it would be >>>>>>>>> best to start a new discussion. >>>>>>>>> >>>>>>>>> A GC log for an application under load might easily produce a block of >>>>>>>>> about 1.5 KB size every few seconds. I seriously doubt, that the need >>>>>>>>> for log file rotation can be argued away by referring to syslog or >>>>>>>>> Windows log API as the correct solution. >>>>>>>>> >>>>>>>>> The messages are not really line formatted, the format can vary a lot >>>>>>>>> (depending on the excat XX switches), the traffic can be quite high and >>>>>>>>> AFAIK the JVM writes it synchronously, so there must be absolutely no >>>>>>>>> risk in writing it out with very little latency. In addition, for >>>>>>>>> analysis, you wouldn't want to look at each event individually, but >>>>>>>>> instead process the whole file through a script or tool, which should >>>>>>>>> not depend on the output specifics of a platform specific log apparatus. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Rainer >>>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >>> >> >> >> -- >> Dmitry Samersoff >> Java Hotspot development team, SPB04 >> * There will come soft rains ... >> >> > -- Dmitry Samersoff Java Hotspot development team, SPB04 * There will come soft rains ... From kirk at kodewerk.com Wed Aug 15 13:49:13 2012 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Wed, 15 Aug 2012 15:49:13 +0200 Subject: JEP 158: Unified JVM logging In-Reply-To: <502BA3E1.5000702@oracle.com> References: <4E6D5DEC.8090109@oss.ntt.co.jp> <4E6DDB8F.4020301@oracle.com><3568943A-E6A4-4287-8319-8B0993F070F9@oracle.com><4E7A85AE.10507@oss.ntt.co.jp> <4E7B19EC.5070700@oracle.com> <4E7B2227.9070406@kippdata.de> <4E805629.7030703@oss.ntt.co.jp> <4E846158.3020606@oracle.com> <5017B761.5070203@lab.ntt.co.jp> <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> <502A1853.70508@lab.ntt.co.jp> <502A25B0.20004@oracle.com> <6976D1D8-DD7B-43D8-B03D-E0D3E05BD771@kodewerk.com> <502BA3E1.5000702@oracle.com> Message-ID: <2896294D-7C54-4CDC-A7C8-46E3B000F206@kodewerk.com> On 2012-08-15, at 3:28 PM, Dmitry Samersoff wrote: > Kirk, > > On 2012-08-15 12:56, Kirk Pepperdine wrote: >> I should have added that Neal Ford has an excellent talk on taxonomy systems and tags vs. hierarchies. >> It includes a bit on the 5 kingdoms that all organisms are split into. Yet many organisms show characteristics >> from more than one kingdom while some others show characteristics of none. Problem is, >> the current hierarchical system forces categorization in a way that doesn't work where >> as a tag system would easily be able to cope. I'm trying to find a link to the talk. > > Really good talk. > > Stepping aside from logging problem - > > hierarchy require additional efforts from a people who builds hierarchy, > but tags put the same efforts to people who uses it. > > If I'm constructing new butterfly and check description of Insecta, > Lepidoptera I should get a list of the most important characteristics of > all butterflies. It's responsibility of persons building hierarchy to > create this list. I trust them and believe that if I give all these > characteristics to my being it become a butterfly. I think the point of Neal's talk is this type of taxonomy is better suited to tags. As you mentioned, a butterfly has wings.. but so do bees.. and birds.. and bats. So maybe not such a good idea to use wings a differentiator in a hierarchy. But as a tag it works out very well. A taxonomy for butterflies would simply include all the relevant tags. > > But with tags I should build the list of characteristics by my self, > e.g. start from insects with tags "four wings", and then refine the mix > of Lepidoptera and Hymenoptera using other tags to distinguish between > butterfly and bees. > > So tags give you more power but require much more efforts in regular case. I guess what I've also implicitly said is that the current GC logging system is essentially tag based. My feeling is that tags could be cleaned up which means I could probably get everything that I might want in about a dozen flags or less. I don't really want to get into an implementation discussion so lets just call the following a bad implementation and go from there. java -log:gc,pauseTime,tenuring,stoppedTime,occupancyConfiguredSize,tag5,tag6 my.main This seems about the same amount of effort as it to log today which isn't insignificant but it's also not a big burden that it's worth the loss of flexibility that is currently enjoyed. And again, tags don't preclude someone defining hierarchies of tags if that makes sense to they way their component works. -- Kirk From Dmitry.Samersoff at oracle.com Wed Aug 15 14:19:47 2012 From: Dmitry.Samersoff at oracle.com (Dmitry Samersoff) Date: Wed, 15 Aug 2012 18:19:47 +0400 Subject: JEP 158: Unified JVM logging In-Reply-To: References: <4E6D5DEC.8090109@oss.ntt.co.jp> <4E6DDB8F.4020301@oracle.com><3568943A-E6A4-4287-8319-8B0993F070F9@oracle.com><4E7A85AE.10507@oss.ntt.co.jp> <4E7B19EC.5070700@oracle.com> <4E7B2227.9070406@kippdata.de> <4E805629.7030703@oss.ntt.co.jp> <4E846158.3020606@oracle.com> <5017B761.5070203@lab.ntt.co.jp> <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> <502A1853.70508@lab.ntt.co.jp> <502A25B0.20004@oracle.com> Message-ID: <502BB003.8020701@oracle.com> Kirk, Please see below. ** Staffan, please correct me if I misunderstood something. On 2012-08-15 12:44, Kirk Pepperdine wrote: > Hi Dmitry, > > Lets start with this. > > * Common logging command-line options for all components > * Logging is performed at different levels: error, warning, info, > debug, trace > * Logging messages are in human-readable plain text > > > IMHO, these current set of goals are based on the assumptions that all > component is hierarchical in nature and that they will log the same way > without the possibility of binary dumps. I would argue these assumptions > over-reach and preclude certain types of logging that take place in the > JVM today. Components should be independent. With plain tag systems we for sure end up with the same tags in different areas and messed log in result. Treat categories as namespaces rather than true hierarchy. > The current system of command-line (experimental) options follow a > format that is quite consistent. The options provide semantic meaning as > to what will be logged. This semantic meaning is lost when we reduce > everything to the level of error, warning, info, debug and trace. I guess Levels are independent to categories. i.e. I think Log:info,gc,younggen should be possible. Levels - warning, info etc primary intended to indicate production system impact caused by logging. debug should never used in production. info is ok for production but has some performance impact warning has no performance impact etc. > It's > sets up arbitrary categories in which developers will be left to decide > what level a particular activity should be logged at. To make this real > I'll use the -XX:+PrintTenuringDistribution flag as an example. When I > set that flag it's very clear what's to be logged. My question is, > should this be logged at the info level? How about debug? Maybe trace? > And if I set those levels, what else comes along for the ride? IOWs, if > I attempt to keep the same model of GC logging, we'd need different > levels. If we need different levels then we're about to violate the > first goal. > > What this example suggests is that gc logging would be better served by > using categories (or subjects if we can take something from the pub/sub > metaphor) or tags or labels or what ever you want to call them. Using > tags, I can create a system of levels using error, warning, info, debug, > and trace if that fits how I'm setting up logging. But from levels it's > difficult to implement things using categories. This is a case where the > one of the non-goals should be, we shouldn't prevent people from > structuring their logs as they see best fits the component they are > instrumenting. > > The non-goals that should be reconsidered is to force human-readability. > I would argue that this is again over-reaching in that properly > structured log files should almost *never* be read by a human. Human readable means that human can use grep to check for (e.g) error presence. It doesn't mean that people would load logs to theirs iPADs and read with family at the evening. Lots of big customers uses dedicated log servers and often this log servers don't have java. (e.g. in the past I used to use OpenBSD or FreeBSD for all infrastructure tasks). We shouldn't require all admins have special decoding tool installed on all machines they works with. > In thee > cases why force an unnecessary requirement. I say this because I've run > into a number of applications where logging was the primary bottleneck. > In those cases there were two primary problems, one was bulk and the > other was threading (lock contention which I shan't address here). I'm no ready to talk about treading. But IMHO, bulk should be beaten on other levels: From separate log servers to good piece of software to analyze raw logs and store important counters only. Also we should reduce *needs* of logging. > Now > while trying to solve the problem of bulkiness and lock contention might > be considered a premature optimization at this stage and I certainty > wouldn't disagree, I would not like to see designs or specifications > that would naturally preclude some very obvious solutions. One of the > solutions to bulk is to use a more compact format be some form of > binary. That option has been taken away for what I would call an > ideological requirement. There is no technical reason why I shouldn't be > able to log in binary other than connivence. In addition, you don't know > what else is happening on the system (disk??) you're logging to so it > behoves you to be as light as possible. -Dmitry > > I'll leave it at this for now. > > -- Kirk > > On 2012-08-14, at 12:17 PM, Dmitry Samersoff > > wrote: > >> Kirk, >> >>> However I do have very serious concerns about this JEP in that it >>> doesn't fix the problems that exist in current logging frameworks, >>> it only mimics them. >> >>> http://openjdk.java.net/jeps/158 >> >> Any comments is much appreciated. >> >> Personally, I think that log rotation is out of scope and responsibility >> of JVM. >> >> -Dmitry >> >> >> On 2012-08-14 13:26, Kirk Pepperdine wrote: >>> Hi Yasumasa, >>> >>> I'm not sure that log file rotation is a part of this JEP. >>> However I do have very serious concerns about this JEP in that it >>> doesn't fix the problems that exist in current logging frameworks, it >>> only mimics them. >>> >>> Regards, >>> Kirk >>> >>> On 2012-08-14, at 11:20 AM, Yasumasa Suenaga >>> >> > wrote: >>> >>>> Hi Staffan, >>>> >>>> May I ask you 2 questions about this JEP? >>>> >>>> 1. One of goals of this JEP is defined as following: >>>> "File rotation of log files by size (similar to what is >>>> available for GC logs today)" >>>> My patch realizes log rotation by external trigger. >>>> 7090324 is included in this JEP? >>>> (Is 7090324 included in "Logging can be configured dynamically at >>>> runtime via jcmd or MBeans" ? ) >>>> >>>> 2. I've posted a patch for "6950794: Make the GC log file name >>>> parameterized" . >>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2012-July/004758.html >>>> Could you include this RFE in this JEP? >>>> If we use log rotation, I think that name of logs is very >>>> important for log management. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2012/08/14 16:50, Staffan Larsen wrote: >>>>> Hi Yasumasa, >>>>> >>>>> This work should be done in the context of the Unified Logging JEP >>>>> [1]. Unfortunately, that work has not yet started due to resource >>>>> constraints. Comments on the proposal are welcome. >>>>> >>>>> [1] http://openjdk.java.net/jeps/158 >>>>> >>>>> /Staffan >>>>> >>>>> On 31 jul 2012, at 12:45, Yasumasa Suenaga >>>>> >>>> >>>>> > wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I've posted a patch for gc log rotation via jinfo tool last year. >>>>>> Then I received an email that essence of my patch will be >>>>>> implemented as >>>>>> "subcommands" of jcmd. >>>>>> >>>>>> Now, jcmd is provided by OpenJDK. However, current jcmd does not >>>>>> seem to >>>>>> implement function of gclog rotation. >>>>>> >>>>>> Will this RFE be implemented? >>>>>> >>>>>> I need to rotate gclog which is synchronized with signal, cron, >>>>>> etc... on Linux. >>>>>> So, if this RFE is not implemented for a while, I would like to >>>>>> contribute >>>>>> patch for jcmd. >>>>>> >>>>>> >>>>>> Regards, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2011/09/29 21:15, James Melvin wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> Thanks very much for your OpenJDK contribution! As part of the >>>>>>> effort to >>>>>>> port JRockit features to HotSpot, we plan to introduce a consolidated >>>>>>> commandline serviceability tool (jcmd) to potentially replace many of >>>>>>> the existing tools in the bin directory, such as jmap, jstack, >>>>>>> jinfo and >>>>>>> others. Over the next few update releases, we plan to add several >>>>>>> jcmd >>>>>>> *subcommands* instead to accomplish specific tasks or affect the >>>>>>> running >>>>>>> JVM instance. We'd like to use the essence of your contribution >>>>>>> in one >>>>>>> of the jcmd subcommands (instead of extending jinfo) to ask the >>>>>>> JVM to >>>>>>> begin rotating GC logs. We hope you find this attractive and hope you >>>>>>> will help review and perfect the change. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Jim Melvin >>>>>>> Sr. Engineering Manager >>>>>>> Oracle >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 9/26/11 6:38 AM, Yasumasa Suenaga wrote: >>>>>>>> (I've changed subject of this email to new RFE.) >>>>>>>> >>>>>>>> This RFE is enhancement of current gclog implementation. >>>>>>>> So, I'd like to discuss about rotating gclog. >>>>>>>> >>>>>>>> My customers need gclog rotation which is invoked by external >>>>>>>> trigger. >>>>>>>> So I've requested this RFE and made a patch. >>>>>>>> >>>>>>>> >>>>>>>> In many case on Linux, logfile is rotated by signal (e.g. SIGHUP) . >>>>>>>> The aim of this RFE is to synchronize gclog and the other logs. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> (2011/09/22 20:55), Rainer Jung wrote: >>>>>>>>> On 22.09.2011 13:20, Dmitry Samersoff wrote: >>>>>>>>>> >>>>>>>>>> Yasumasa, >>>>>>>>>> >>>>>>>>>> On 2011-09-22 04:47, Yasumasa Suenaga wrote: >>>>>>>>>>> If we can think Java on Linux and Solaris only, syslog is best >>>>>>>>>>> solution. >>>>>>>>>>> However, Windows usually doesn't have syslog. >>>>>>>>>>> >>>>>>>>>>> So, I think that gclog is needed for logging GC stats with >>>>>>>>>>> platform >>>>>>>>>>> independent in realtime. >>>>>>>>>> >>>>>>>>>> Windows has it's own logging API as reach as syslog is or ever >>>>>>>>>> better >>>>>>>>>> as well as numerous syslog implementations. >>>>>>>>>> >>>>>>>>>> Native windows logging API was completely redesigned for >>>>>>>>>> Windows 2008 >>>>>>>>>> server and now it allows for developers to send a structured >>>>>>>>>> events from >>>>>>>>>> theirs application. >>>>>>>>> >>>>>>>>> AFAIK log rotation for loggc is already implemented though not >>>>>>>>> necessarily yet released. The change discussed here is about >>>>>>>>> supporting >>>>>>>>> an externally generated rotation trigger, e.g. via a signal, >>>>>>>>> instead of >>>>>>>>> only rotating by size or time via a startup parameter. >>>>>>>>> >>>>>>>>> If you want support for syslog or Windows APIs included, it >>>>>>>>> would be >>>>>>>>> best to start a new discussion. >>>>>>>>> >>>>>>>>> A GC log for an application under load might easily produce a >>>>>>>>> block of >>>>>>>>> about 1.5 KB size every few seconds. I seriously doubt, that >>>>>>>>> the need >>>>>>>>> for log file rotation can be argued away by referring to syslog or >>>>>>>>> Windows log API as the correct solution. >>>>>>>>> >>>>>>>>> The messages are not really line formatted, the format can vary >>>>>>>>> a lot >>>>>>>>> (depending on the excat XX switches), the traffic can be quite >>>>>>>>> high and >>>>>>>>> AFAIK the JVM writes it synchronously, so there must be >>>>>>>>> absolutely no >>>>>>>>> risk in writing it out with very little latency. In addition, for >>>>>>>>> analysis, you wouldn't want to look at each event individually, but >>>>>>>>> instead process the whole file through a script or tool, which >>>>>>>>> should >>>>>>>>> not depend on the output specifics of a platform specific log >>>>>>>>> apparatus. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Rainer >>>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >>> >> >> >> -- >> Dmitry Samersoff >> Java Hotspot development team, SPB04 >> * There will come soft rains ... >> >> > -- Dmitry Samersoff Java Hotspot development team, SPB04 * There will come soft rains ... From jon.masamitsu at oracle.com Wed Aug 15 14:46:40 2012 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 15 Aug 2012 07:46:40 -0700 Subject: concurrent mode failure during concurrent sweep phase In-Reply-To: <4673403A-6D16-4917-86F5-670C5E329A0A@kodewerk.com> References: <4673403A-6D16-4917-86F5-670C5E329A0A@kodewerk.com> Message-ID: <502BB650.4020105@oracle.com> Kirk, In this case the sweep has completed, right? So no additional free space is going to be found. Even if we delayed the young gen collection, it would still fail. In general there is a way to get CMS to hold off the young gen collection by completing the concurrent collection in a STW phase. I can't swear that it still works but the idea was that if we were sweeping (I think), we should move the collection from a concurrent collection to a STW collection and finish the sweep during a STW. I've only seen it worsen the situation (i.e., the sweeping is so slow that it is much faster to do the full collection). I think that code should be ripped out. Jon On 8/15/2012 5:01 AM, Kirk Pepperdine wrote: > Hi all, > > : 81092K->81092K(81856K), 0.3419045 secs]5274.145: [CMS5281.056: [CMS-concurrent-sweep: 11.066/12.602 secs] > > In all of the GC logs I've got, this one doesn't show up very often. Now that I'm looking at it I'm asking myself... does it really make sense to declare a CMF when one is so close to the end of the CMS cycle? This really literally just small numbers of instructions away from the CMS-reset.... Wouldn't it make more sense to delay the ParNew that is hoisting survivors into CMS tenured vs triggering a full gc with it's (most likely longer) STW pause? Or is this a signal that the failure is most likely due to high fragmentation? > > Regards, > Kirk From kirk at kodewerk.com Wed Aug 15 14:57:04 2012 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Wed, 15 Aug 2012 16:57:04 +0200 Subject: concurrent mode failure during concurrent sweep phase In-Reply-To: <502BB650.4020105@oracle.com> References: <4673403A-6D16-4917-86F5-670C5E329A0A@kodewerk.com> <502BB650.4020105@oracle.com> Message-ID: <17272F1B-C6A1-4155-B68C-B3AE776F4CF4@kodewerk.com> Hi Jon, Thanks for the response, I don't believe the sweep has completed as that phase is pre-empted by the concurrent mode failure. If the promotion fails outside of a CMS phase then it seems to go directly to a Full GC without registering a CMF. I guess one of the assumptions that I was under was that any newly freed space wouldn't be visible until after the CMS-reset. Regards, Kirk On 2012-08-15, at 4:46 PM, Jon Masamitsu wrote: > Kirk, > > In this case the sweep has completed, right? So no additional free space is > going to be found. Even if we delayed the young gen collection, it would still > fail. > > In general there is a way to get CMS to hold off the young gen collection > by completing the concurrent collection in a STW phase. I can't swear > that it still works but the idea was that if we were sweeping (I think), we should > move the collection from a concurrent collection to a STW collection > and finish the sweep during a STW. I've only seen it worsen the > situation (i.e., the sweeping is so slow that it is much faster to do the > full collection). I think that code should be ripped out. > > Jon > > > On 8/15/2012 5:01 AM, Kirk Pepperdine wrote: >> Hi all, >> >> : 81092K->81092K(81856K), 0.3419045 secs]5274.145: [CMS5281.056: [CMS-concurrent-sweep: 11.066/12.602 secs] >> >> In all of the GC logs I've got, this one doesn't show up very often. Now that I'm looking at it I'm asking myself... does it really make sense to declare a CMF when one is so close to the end of the CMS cycle? This really literally just small numbers of instructions away from the CMS-reset.... Wouldn't it make more sense to delay the ParNew that is hoisting survivors into CMS tenured vs triggering a full gc with it's (most likely longer) STW pause? Or is this a signal that the failure is most likely due to high fragmentation? >> >> Regards, >> Kirk From jon.masamitsu at oracle.com Wed Aug 15 21:20:18 2012 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 15 Aug 2012 14:20:18 -0700 Subject: concurrent mode failure during concurrent sweep phase In-Reply-To: <17272F1B-C6A1-4155-B68C-B3AE776F4CF4@kodewerk.com> References: <4673403A-6D16-4917-86F5-670C5E329A0A@kodewerk.com> <502BB650.4020105@oracle.com> <17272F1B-C6A1-4155-B68C-B3AE776F4CF4@kodewerk.com> Message-ID: <502C1292.10708@oracle.com> Kirk, You're probably right about the sweep not completing. A CMS sweep walks over the old gen looking for dead objects and adds them to the free lists as they are found. That newly added space is available for allocation as soon as it is on the free lists. Jon On 8/15/2012 7:57 AM, Kirk Pepperdine wrote: > Hi Jon, > > Thanks for the response, I don't believe the sweep has completed as that phase is pre-empted by the concurrent mode failure. If the promotion fails outside of a CMS phase then it seems to go directly to a Full GC without registering a CMF. > > I guess one of the assumptions that I was under was that any newly freed space wouldn't be visible until after the CMS-reset. > > Regards, > Kirk > > On 2012-08-15, at 4:46 PM, Jon Masamitsu wrote: > >> Kirk, >> >> In this case the sweep has completed, right? So no additional free space is >> going to be found. Even if we delayed the young gen collection, it would still >> fail. >> >> In general there is a way to get CMS to hold off the young gen collection >> by completing the concurrent collection in a STW phase. I can't swear >> that it still works but the idea was that if we were sweeping (I think), we should >> move the collection from a concurrent collection to a STW collection >> and finish the sweep during a STW. I've only seen it worsen the >> situation (i.e., the sweeping is so slow that it is much faster to do the >> full collection). I think that code should be ripped out. >> >> Jon >> >> >> On 8/15/2012 5:01 AM, Kirk Pepperdine wrote: >>> Hi all, >>> >>> : 81092K->81092K(81856K), 0.3419045 secs]5274.145: [CMS5281.056: [CMS-concurrent-sweep: 11.066/12.602 secs] >>> >>> In all of the GC logs I've got, this one doesn't show up very often. Now that I'm looking at it I'm asking myself... does it really make sense to declare a CMF when one is so close to the end of the CMS cycle? This really literally just small numbers of instructions away from the CMS-reset.... Wouldn't it make more sense to delay the ParNew that is hoisting survivors into CMS tenured vs triggering a full gc with it's (most likely longer) STW pause? Or is this a signal that the failure is most likely due to high fragmentation? >>> >>> Regards, >>> Kirk From bengt.rutisson at oracle.com Thu Aug 16 10:48:09 2012 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 16 Aug 2012 12:48:09 +0200 Subject: RFR(S): 7041879: G1: introduce stress testing parameter to cause frequent evacuation failures In-Reply-To: <50085D41.90004@oracle.com> References: <50085D41.90004@oracle.com> Message-ID: <502CCFE9.2090101@oracle.com> Hi John, Overall I think this looks fine. It's definitely a good stress option to have. Thanks for fixing this! Functionally it looks good to me, but I have some style questions: * Why do we add the new methods as inline methods in the g1CollectedHeap.inline.hpp file? These are only debug methods and they are only used from g1CollectedHeap.cpp. I would prefer to place the implementations in g1CollectedHeap.cpp and let the compiler decide whether to inline or not. * In the constructor for G1CollectedHeap I think we could call reset_evacuation_should_fail() instead of these lines: #ifndef PRODUCT _evacuation_failure_alot_for_current_gc = false; _evacuation_failure_alot_count = 0; _evacuation_failure_alot_gc_number = 0; #endif // !PRODUCT * G1CollectedHeap::evacuation_should_fail() Why do we need evacuation_should_fail(volatile size_t* count)? It is always called with _evacuation_failure_alot_count as parameter. How about using that directly instead of passing it as a parameter? Also, the nesting level is getting three levels deep. We could flatten that by bailing out early if we are not triggering evacuation failures. So, I think I would prefer evacuation_should_fail() to look something like: bool G1CollectedHeap::evacuation_should_fail() { if (!G1EvacuationFailureALot || !_evacuation_failure_alot_for_current_gc) { return false; } // Access to _evacuation_failure_alot_count is not atomic; the value does not have to be exact. if (++_evacuation_failure_alot_count >= G1EvacuationFailureALotCount) { _evacuation_failure_alot_count = 0; return true; } return false; } * G1CollectedHeap::evacuation_failure_alot_for_gc_type() and G1CollectedHeap::set_evacuation_failure_alot_for_current_gc() I am not sure that the method evacuation_failure_alot_for_gc_type() actually makes the code easier to understand. I think I would prefer to inline that logic in set_evacuation_failure_alot_for_current_gc(). Something like: void G1CollectedHeap::set_evacuation_failure_alot_for_current_gc() { if (G1EvacuationFailureALot) { // Note we can't assert that _evacuation_failure_alot_for_current_gc // is clear here. It may have been set during a previous GC but that GC // did not copy enough objects (i.e. G1EvacuationFailureALotCount) to // trigger an evacuation failure and clear the flags and and counts. _evacuation_failure_alot_for_current_gc = false; // Check if we have gone over the interval. const size_t gc_num = total_collections(); const size_t elapsed_gcs = gc_num - _evacuation_failure_alot_gc_number; if (elapsed_gcs >= G1EvacuationFailureALotInterval) { const bool gcs_are_young = g1_policy()->gcs_are_young(); const bool during_im = g1_policy()->during_initial_mark_pause(); const bool during_marking = mark_in_progress(); if (gcs_are_young) { _evacuation_failure_alot_for_current_gc = G1EvacuationFailureALotDuringYoungGC || (during_marking && G1EvacuationFailureALotDuringConcMark) || (during_im && G1EvacuationFailureALotDuringInitialMark); } else { _evacuation_failure_alot_for_current_gc = G1EvacuationFailureALotDuringMixedGC; } } } * If we keep any changes in g1CollectedHeap.inline.hpp it should get its copyright year updated Thanks, Bengt On 2012-07-19 21:17, John Cuthbertson wrote: > Hi Everyone, > > Can I have couple of volunteers to review the changes for this CR? The > webrev can be found at: > http://cr.openjdk.java.net/~johnc/7041879/webrev.0/ > > Summary: > The changes adds mechanism, which is only available in non-product > builds, for forcing evacuation failures. The mechanism is analogous to > the PromotionFailureALot functionality in the other collectors. It is > enabled with the G1EvacuationFailureALot flag and is controlled using > sub-flags: G1EvacuationFailureALotCount and > G1EvacuationFailureALotInterval control the frequency and number of > evacuation failures, and G1EvacuationFailureALotDuringYoungGC, > G1EvacuationFailureALotDuringMixedGC, > G1EvacuationFailureALotDuringInitialMark, and G1EvacuationFailureALot > control during type of GC we should/can trigger evacuation failures. > > An earlier variant of this patch was previously employed to stress > test Tony's marking changes. > > Testing: > Dacapo/xalan to test the various flags and settings. GC test suite and > jprt with G1EvacuationFailureALot and heap verification enabled (the > sub-flags for G1EvacuationFailureALot were left at their default values). > > Thanks, > > JohnC From john.cuthbertson at oracle.com Thu Aug 16 23:09:51 2012 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Thu, 16 Aug 2012 16:09:51 -0700 Subject: RFR(S): 7192128: G1: Extend fix for 6948537 to G1's BOT Message-ID: <502D7DBF.50804@oracle.com> Hi Everyone, Can I have a couple of volunteers to review the fix for this CR? The webrev can be found at: http://cr.openjdk.java.net/~johnc/7192128/webrev.0/ Summary: A while back, Ramki discovered an issue on Niagara systems where concurrent readers of the block offset table could see spurious zero entries as a result of the implementation of memset using the SPARC BIS instruction. At the time it was thought that G1 was not affected by this issue. During testing of the perm-gen removal changes, the development engineers started to see assertion failures and crashes in G1's block offset table. The assertions were about erroneous contents of the offset array. As a result the values used in the assertion were printed and the value being displayed should not have failed the assertion: > --- a/src/share/vm/gc_implementation/g1/g1BlockOffsetTable.cpp > +++ b/src/share/vm/gc_implementation/g1/g1BlockOffsetTable.cpp > @@ -546,7 +546,10 @@ > assert(_array->offset_array(j) > 0 && > _array->offset_array(j) <= > (u_char) (N_words+BlockOffsetArray::N_powers-1), > - "offset array should have been set"); > + err_msg("offset array should have been set " > + SIZE_FORMAT " not > 0 OR " SIZE_FORMAT " not <= " > + SIZE_FORMAT, _array->offset_array(j), > _array->offset_array(j), > + (N_words+BlockOffsetArray::N_powers-1))); > } > #endif > } # To suppress the following error report, specify this argument # after -XX: or in .hotspotrc: SuppressErrorAt=/g1BlockOffsetTable.cpp:552 # # A fatal error has been detected by the Java Runtime Environment: # # Internal Error (/tmp/jprt/P1/173804.cphillim/s/src/share/vm/gc_implementation/g1/g1BlockOffsetTable.cpp:552), pid=5210, tid=27 # assert(_array->offset_array(j) > 0 && _array->offset_array(j) <= (u_char) (N_words+BlockOffsetArray::N_powers-1)) failed: offset array should have been set 65 not > 0 OR 65 not <= 77 # So we had a value which failed the assertion check that, when it was re-read for the error message, had a value which should have passed the assertion check. It seems that G1 is not immune to the problem seen in 6948537 and I believe that the recent PLAB resizing change has increased the likelihood of hitting the issue as it can increase the amount of concurrent refinement of BlockOffsetTable entries. Testing: jprt runs with the perm-gen removal changes, command line tests on sparc and x86 systems, GCOld (which was the failing test) on SPARC systems. Thanks, JohnC From john.coomes at oracle.com Fri Aug 17 04:04:34 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 17 Aug 2012 04:04:34 +0000 Subject: hg: hsx/hotspot-gc: Added tag jdk8-b52 for changeset 8d24def5ceb3 Message-ID: <20120817040434.47F534758D@hg.openjdk.java.net> Changeset: febd7ff52800 Author: katleman Date: 2012-08-16 11:43 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/rev/febd7ff52800 Added tag jdk8-b52 for changeset 8d24def5ceb3 ! .hgtags From john.coomes at oracle.com Fri Aug 17 04:04:41 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 17 Aug 2012 04:04:41 +0000 Subject: hg: hsx/hotspot-gc/corba: Added tag jdk8-b52 for changeset 80689ff9cb49 Message-ID: <20120817040443.D7CAA4758E@hg.openjdk.java.net> Changeset: 63aeb7a2472f Author: katleman Date: 2012-08-16 11:43 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/corba/rev/63aeb7a2472f Added tag jdk8-b52 for changeset 80689ff9cb49 ! .hgtags From john.coomes at oracle.com Fri Aug 17 04:04:49 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 17 Aug 2012 04:04:49 +0000 Subject: hg: hsx/hotspot-gc/jaxp: Added tag jdk8-b52 for changeset bd3c00d57614 Message-ID: <20120817040457.01DA54758F@hg.openjdk.java.net> Changeset: 2c566f25c39f Author: katleman Date: 2012-08-16 11:43 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jaxp/rev/2c566f25c39f Added tag jdk8-b52 for changeset bd3c00d57614 ! .hgtags From john.coomes at oracle.com Fri Aug 17 04:05:02 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 17 Aug 2012 04:05:02 +0000 Subject: hg: hsx/hotspot-gc/jaxws: Added tag jdk8-b52 for changeset f62bc618122e Message-ID: <20120817040508.57E0847590@hg.openjdk.java.net> Changeset: 8a35fd644d3c Author: katleman Date: 2012-08-16 11:43 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jaxws/rev/8a35fd644d3c Added tag jdk8-b52 for changeset f62bc618122e ! .hgtags From john.coomes at oracle.com Fri Aug 17 04:06:03 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 17 Aug 2012 04:06:03 +0000 Subject: hg: hsx/hotspot-gc/jdk: 29 new changesets Message-ID: <20120817041130.ADB0147591@hg.openjdk.java.net> Changeset: 21c590fdc8cb Author: mullan Date: 2012-08-01 11:06 -0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/21c590fdc8cb 7179715: OCSP revocation checking fails if the signer certificate is identified using the key ID Reviewed-by: vinnie ! src/share/classes/sun/security/provider/certpath/OCSPResponse.java Changeset: 9a5a3741bac9 Author: mullan Date: 2012-08-01 11:08 -0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/9a5a3741bac9 Merge - makefiles/LegacyMakefiles.gmk - makefiles/OldImages.gmk - makefiles/com/sun/crypto/provider/Makefile - makefiles/common/Classes.gmk - makefiles/common/Cscope.gmk - makefiles/common/Defs-embedded.gmk - makefiles/common/Defs-linux.gmk - makefiles/common/Defs-macosx.gmk - makefiles/common/Defs-solaris.gmk - makefiles/common/Defs-windows.gmk - makefiles/common/Defs.gmk - makefiles/common/Demo.gmk - makefiles/common/Library.gmk - makefiles/common/Mapfile-vers.gmk - makefiles/common/Modules.gmk - makefiles/common/Program.gmk - makefiles/common/Release-embedded.gmk - makefiles/common/Release-macosx.gmk - makefiles/common/Release.gmk - makefiles/common/Rules.gmk - makefiles/common/Subdirs.gmk - makefiles/common/internal/Defs-corba.gmk - makefiles/common/internal/Defs-jaxp.gmk - makefiles/common/internal/Defs-jaxws.gmk - makefiles/common/internal/Defs-langtools.gmk - makefiles/common/internal/ImportComponents.gmk - makefiles/common/internal/NativeCompileRules.gmk - makefiles/common/internal/Resources.gmk - makefiles/common/shared/Compiler-gcc.gmk - makefiles/common/shared/Compiler-llvm.gmk - makefiles/common/shared/Compiler-msvc.gmk - makefiles/common/shared/Compiler-sun.gmk - makefiles/common/shared/Defs-control.gmk - makefiles/common/shared/Defs-java.gmk - makefiles/common/shared/Defs-javadoc.gmk - makefiles/common/shared/Defs-linux.gmk - makefiles/common/shared/Defs-macosx.gmk - makefiles/common/shared/Defs-solaris.gmk - makefiles/common/shared/Defs-versions.gmk - makefiles/common/shared/Defs-windows.gmk - makefiles/common/shared/Defs.gmk - makefiles/common/shared/Platform.gmk - makefiles/common/shared/PrivateDefs.gmk-example - makefiles/common/shared/Sanity-Settings.gmk - makefiles/java/Makefile - makefiles/java/invoke/Makefile - makefiles/java/redist/Makefile - makefiles/java/redist/sajdi/Makefile - makefiles/javax/crypto/Defs-jce.gmk - makefiles/javax/crypto/Makefile - makefiles/javax/crypto/policy/limited/LIMITED - makefiles/javax/crypto/policy/limited/default_local.policy - makefiles/javax/crypto/policy/limited/exempt_local.policy - makefiles/javax/crypto/policy/unlimited/UNLIMITED - makefiles/javax/crypto/policy/unlimited/default_US_export.policy - makefiles/javax/crypto/policy/unlimited/default_local.policy - makefiles/mkdemo/Makefile - makefiles/mkdemo/jni/Makefile - makefiles/mkdemo/jni/Poller/Makefile - makefiles/mkdemo/jvmti/Makefile - makefiles/mkdemo/jvmti/README.txt - makefiles/mkdemo/jvmti/hprof/Makefile - makefiles/mkdemo/jvmti/mapfile-vers - makefiles/mkdemo/management/README.txt - makefiles/sun/jkernel/Makefile - makefiles/sun/security/ec/Makefile - makefiles/sun/security/pkcs11/FILES_c.gmk - makefiles/sun/security/pkcs11/Makefile - makefiles/sun/security/pkcs11/mapfile-vers - src/share/classes/sun/nio/cs/SingleByteDecoder.java - src/share/classes/sun/nio/cs/SingleByteEncoder.java - src/share/classes/sun/nio/cs/ext/DoubleByteDecoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0201.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_MS5022X_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_MS5022X_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_MS932_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_MS932_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_Solaris_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0208_Solaris_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_MS5022X_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_MS5022X_Encoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_Solaris_Decoder.java - src/share/classes/sun/nio/cs/ext/JIS_X_0212_Solaris_Encoder.java - src/share/classes/sun/nio/cs/ext/PCK.java - src/share/classes/sun/nio/cs/ext/SJIS.java - src/share/classes/sun/security/krb5/ServiceName.java - test/sun/security/krb5/ServiceNameClone.java - test/sun/security/mscapi/ShortRSAKey512.sh - test/sun/security/mscapi/ShortRSAKey768.sh Changeset: 184da100cf45 Author: jgish Date: 2012-07-27 16:17 -0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/184da100cf45 6914123: (str) Missing synchronization in java.lang.String#contentEquals(CharSequence) Summary: Change contentEquals( CharSequence cs ) to do synchronization if cs is a StringBuffer Reviewed-by: mduigou Contributed-by: Jim Gish ! src/share/classes/java/lang/String.java Changeset: 75bda37d0337 Author: omajid Date: 2012-08-01 22:13 +0100 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/75bda37d0337 6844255: Potential stack corruption in GetJavaProperties Summary: Use dynamically allocated buffers for temp and encoding. Reviewed-by: alanb, andrew ! src/solaris/native/java/lang/java_props_md.c Changeset: b0bfa441d70f Author: mullan Date: 2012-08-02 10:40 -0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/b0bfa441d70f 7026347: Certificate and X509CRL should have verify(PublicKey key, Provider sigProvider) Reviewed-by: mullan, xuelei, weijun Contributed-by: jason.uh at oracle.com ! src/share/classes/java/security/cert/Certificate.java ! src/share/classes/java/security/cert/X509CRL.java ! src/share/classes/java/security/cert/X509Certificate.java ! src/share/classes/sun/security/x509/X509CRLImpl.java ! src/share/classes/sun/security/x509/X509CertImpl.java + test/sun/security/x509/X509CRLImpl/Verify.java + test/sun/security/x509/X509CertImpl/Verify.java Changeset: 4e8bafdcefda Author: mullan Date: 2012-08-02 10:42 -0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/4e8bafdcefda Merge Changeset: 8a82e5f9c47f Author: dmocek Date: 2012-08-02 18:12 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/8a82e5f9c47f 7187876: ClassCastException in TCPTransport.executeAcceptLoop Reviewed-by: dholmes, smarks ! src/share/classes/sun/rmi/transport/tcp/TCPTransport.java Changeset: 1468b0af0d06 Author: sherman Date: 2012-08-03 13:40 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/1468b0af0d06 7188852: Move implementation of De/Inflater.getBytesRead/Writtten() to java from native Summary: re-implemented getBytesRead/Writtten() at java level Reviewed-by: andrew, alanb ! make/java/zip/mapfile-vers ! src/share/classes/java/util/zip/Deflater.java ! src/share/classes/java/util/zip/Inflater.java ! src/share/native/java/util/zip/Deflater.c ! src/share/native/java/util/zip/Inflater.c ! src/share/native/java/util/zip/zlib-1.2.5/compress.c ! src/share/native/java/util/zip/zlib-1.2.5/inflate.c ! src/share/native/java/util/zip/zlib-1.2.5/patches/ChangeLog_java ! src/share/native/java/util/zip/zlib-1.2.5/zlib.h + test/java/util/zip/TotalInOut.java Changeset: 3521fcad4b5f Author: ksrini Date: 2012-07-31 06:10 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/3521fcad4b5f 7188114: (launcher) need an alternate command line parser for Windows Reviewed-by: darcy, dholmes, jjh Contributed-by: akhil.arora at oracle.com + src/windows/bin/cmdtoargs.c Changeset: 2dd41a2dfe54 Author: ksrini Date: 2012-07-31 06:14 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/2dd41a2dfe54 7146424: Wildcard expansion for single entry classpath Reviewed-by: dholmes, darcy, jjh, sherman ! make/common/Program.gmk ! make/java/jli/Makefile ! make/java/jli/mapfile-vers ! src/share/bin/java.c ! src/share/bin/java.h ! src/share/bin/jli_util.c ! src/share/bin/jli_util.h ! src/share/bin/main.c ! src/share/bin/wildcard.c ! src/share/classes/sun/launcher/LauncherHelper.java ! src/share/classes/sun/launcher/resources/launcher.properties - src/solaris/bin/java_md.c ! src/solaris/bin/java_md_common.c ! src/windows/bin/java_md.c ! test/tools/launcher/Arrrghs.java ! test/tools/launcher/TestHelper.java ! test/tools/launcher/ToolsOpts.java Changeset: e0ef14d89741 Author: alanb Date: 2012-08-07 12:47 +0100 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/e0ef14d89741 7076644: TEST_BUG: test/java/io/File/Basic.java fails with cygwin Reviewed-by: alanb Contributed-by: Eric Wang ! test/ProblemList.txt ! test/java/io/File/basic.sh Changeset: b0d6552ba301 Author: lana Date: 2012-08-07 20:23 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/b0d6552ba301 Merge - make/sunw/Makefile - src/share/classes/sunw/io/Serializable.java - src/share/classes/sunw/util/EventListener.java - src/share/classes/sunw/util/EventObject.java ! src/solaris/native/java/lang/java_props_md.c Changeset: d87e86aaf2b3 Author: andrew Date: 2012-08-08 12:37 +0100 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/d87e86aaf2b3 7189533: GetJavaProperties should free temporary file if subsequent allocations fails Summary: Add missing calls to free Reviewed-by: alanb, dholmes, sherman ! src/solaris/native/java/lang/java_props_md.c Changeset: a50e92a980a5 Author: alanb Date: 2012-08-08 15:31 +0100 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/a50e92a980a5 7189886: (aio) Add test coverage for AsynchronousChannelGroup.withThreadPool Reviewed-by: alanb Contributed-by: amy.lu at oracle.com ! test/java/nio/channels/AsynchronousChannelGroup/AsExecutor.java ! test/java/nio/channels/AsynchronousChannelGroup/Basic.java ! test/java/nio/channels/AsynchronousChannelGroup/Restart.java Changeset: a44671e0b6d7 Author: ksrini Date: 2012-08-08 09:29 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/a44671e0b6d7 7189944: (launcher) test/tools/launcher/Arrrrghs.java needs a couple of minor fixes Reviewed-by: darcy, jgish ! test/tools/launcher/Arrrghs.java ! test/tools/launcher/TestHelper.java Changeset: bf85c3ab2637 Author: dsamersoff Date: 2012-08-09 14:52 +0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/bf85c3ab2637 7183753: [TEST] Some colon in the diff for this test Summary: Reference output file contains extra colon Reviewed-by: sspitsyn, mgronlun ! test/sun/tools/jcmd/help_help.out Changeset: da8649489aff Author: lana Date: 2012-08-10 10:15 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/da8649489aff Merge Changeset: 05e5ce861a58 Author: jrose Date: 2012-07-12 00:10 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/05e5ce861a58 7153157: ClassValue.get does not return if computeValue calls remove Summary: Track intermediate states more precisely, according to spec. Reviewed-by: twisti, forax ! src/share/classes/java/lang/ClassValue.java Changeset: beeb1d5ecd9e Author: jrose Date: 2012-07-12 00:11 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/beeb1d5ecd9e 7129034: VM crash with a field setter method with a filterArguments Summary: add null checks before unsafe calls that take a variable base reference; update unit tests Reviewed-by: kvn, twisti ! src/share/classes/java/lang/invoke/MethodHandleImpl.java ! test/java/lang/invoke/MethodHandlesTest.java Changeset: 556141c6326c Author: jrose Date: 2012-07-12 00:12 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/556141c6326c 7087658: MethodHandles.Lookup.findVirtual is confused by interface methods that are multiply inherited Reviewed-by: twisti ! src/share/classes/java/lang/invoke/MemberName.java Changeset: 78f1f4e4e9c7 Author: jrose Date: 2012-07-12 00:12 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/78f1f4e4e9c7 7127687: MethodType leaks memory due to interning Summary: Replace internTable with a weak-reference version. Reviewed-by: sundar, forax, brutisso Contributed-by: james.laskey at oracle.com ! src/share/classes/java/lang/invoke/MethodType.java Changeset: 050116960e99 Author: twisti Date: 2012-07-24 10:47 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/050116960e99 7023639: JSR 292 method handle invocation needs a fast path for compiled code 6984705: JSR 292 method handle creation should not go through JNI Summary: remove assembly code for JDK 7 chained method handles Reviewed-by: jrose, twisti, mhaupt, forax Contributed-by: John Rose , Christian Thalinger , Michael Haupt - src/share/classes/java/lang/invoke/AdapterMethodHandle.java ! src/share/classes/java/lang/invoke/BoundMethodHandle.java ! src/share/classes/java/lang/invoke/CallSite.java - src/share/classes/java/lang/invoke/CountingMethodHandle.java ! src/share/classes/java/lang/invoke/DirectMethodHandle.java + src/share/classes/java/lang/invoke/DontInline.java + src/share/classes/java/lang/invoke/ForceInline.java + src/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java ! src/share/classes/java/lang/invoke/Invokers.java + src/share/classes/java/lang/invoke/LambdaForm.java ! src/share/classes/java/lang/invoke/MemberName.java ! src/share/classes/java/lang/invoke/MethodHandle.java ! src/share/classes/java/lang/invoke/MethodHandleImpl.java + src/share/classes/java/lang/invoke/MethodHandleInfo.java ! src/share/classes/java/lang/invoke/MethodHandleNatives.java ! src/share/classes/java/lang/invoke/MethodHandleStatics.java ! src/share/classes/java/lang/invoke/MethodHandles.java ! src/share/classes/java/lang/invoke/MethodType.java ! src/share/classes/java/lang/invoke/MethodTypeForm.java + src/share/classes/java/lang/invoke/SimpleMethodHandle.java ! src/share/classes/java/lang/invoke/package-info.java ! src/share/classes/sun/invoke/util/ValueConversions.java ! src/share/classes/sun/invoke/util/VerifyAccess.java ! src/share/classes/sun/invoke/util/VerifyType.java ! src/share/classes/sun/invoke/util/Wrapper.java ! src/share/classes/sun/misc/Unsafe.java + test/java/lang/invoke/7157574/Test7157574.java ! test/java/lang/invoke/InvokeGenericTest.java ! test/java/lang/invoke/JavaDocExamplesTest.java + test/java/lang/invoke/MaxTest.java ! test/java/lang/invoke/MethodHandlesTest.java + test/java/lang/invoke/PrivateInvokeTest.java ! test/java/lang/invoke/ThrowExceptionsTest.java + test/java/lang/invoke/remote/RemoteExample.java ! test/sun/invoke/util/ValueConversionsTest.java Changeset: 64e24cc8e009 Author: twisti Date: 2012-08-07 14:31 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/64e24cc8e009 7188911: nightly failures after JSR 292 lazy method handle update (round 2) Reviewed-by: kvn, jrose ! src/share/classes/java/lang/invoke/BoundMethodHandle.java ! src/share/classes/java/lang/invoke/MemberName.java Changeset: e1d063685dc8 Author: twisti Date: 2012-08-09 15:03 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/e1d063685dc8 7190416: JSR 292: typo in InvokerBytecodeGenerator.getConstantPoolSize Reviewed-by: jrose ! src/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java Changeset: 865c411ebcae Author: twisti Date: 2012-08-10 16:03 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/865c411ebcae Merge - src/share/classes/java/lang/invoke/AdapterMethodHandle.java - src/share/classes/java/lang/invoke/CountingMethodHandle.java Changeset: 93ddd9560751 Author: youdwei Date: 2012-08-13 10:45 +0800 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/93ddd9560751 7189611: Venezuela current Currency should be Bs.F. Reviewed-by: okutsu ! src/share/classes/sun/util/resources/CurrencyNames_es_VE.properties ! test/sun/text/resources/LocaleData ! test/sun/text/resources/LocaleDataTest.java Changeset: 90a9acfde9e6 Author: mfang Date: 2012-08-13 16:26 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/90a9acfde9e6 Merge Changeset: e8569a473cee Author: katleman Date: 2012-08-15 18:04 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/e8569a473cee Merge Changeset: 2c6933c5106b Author: katleman Date: 2012-08-16 11:43 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/2c6933c5106b Added tag jdk8-b52 for changeset e8569a473cee ! .hgtags From john.coomes at oracle.com Fri Aug 17 04:13:09 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 17 Aug 2012 04:13:09 +0000 Subject: hg: hsx/hotspot-gc/langtools: 7 new changesets Message-ID: <20120817041329.0F02847592@hg.openjdk.java.net> Changeset: cddc2c894cc6 Author: mcimadamore Date: 2012-08-02 18:22 +0100 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/langtools/rev/cddc2c894cc6 7175911: Simplify error reporting API in Check.CheckContext interface Summary: Make error messages generated during Check.checkType more uniform and more scalable Reviewed-by: jjg, dlsmith ! src/share/classes/com/sun/tools/javac/comp/Attr.java ! src/share/classes/com/sun/tools/javac/comp/Check.java ! src/share/classes/com/sun/tools/javac/comp/Infer.java ! src/share/classes/com/sun/tools/javac/comp/Resolve.java ! src/share/classes/com/sun/tools/javac/resources/compiler.properties ! test/tools/javac/6840059/T6840059.out ! test/tools/javac/6979683/TestCast6979683_BAD34.java.errlog ! test/tools/javac/6979683/TestCast6979683_BAD35.java.errlog ! test/tools/javac/6979683/TestCast6979683_BAD36.java.errlog ! test/tools/javac/6979683/TestCast6979683_BAD37.java.errlog ! test/tools/javac/6979683/TestCast6979683_BAD38.java.errlog ! test/tools/javac/6979683/TestCast6979683_BAD39.java.errlog ! test/tools/javac/7132880/T7132880.out ! test/tools/javac/Diagnostics/6722234/T6722234a_1.out ! test/tools/javac/Diagnostics/6722234/T6722234a_2.out ! test/tools/javac/Diagnostics/6722234/T6722234c.out ! test/tools/javac/Diagnostics/6722234/T6722234d_1.out ! test/tools/javac/Diagnostics/6722234/T6722234d_2.out ! test/tools/javac/Diagnostics/6799605/T6799605.out ! test/tools/javac/Diagnostics/6862608/T6862608a.out ! test/tools/javac/Diagnostics/6862608/T6862608b.out ! test/tools/javac/OverrideChecks/6400189/T6400189a.out ! test/tools/javac/OverrideChecks/6400189/T6400189b.out ! test/tools/javac/StringsInSwitch/BadlyTypedLabel1.out ! test/tools/javac/StringsInSwitch/BadlyTypedLabel2.out ! test/tools/javac/T6326754.out ! test/tools/javac/TryWithResources/TwrOnNonResource.out ! test/tools/javac/cast/6270087/T6270087neg.out ! test/tools/javac/cast/6557182/T6557182.out ! test/tools/javac/cast/6665356/T6665356.out ! test/tools/javac/cast/6795580/T6795580.out ! test/tools/javac/cast/6932571/T6932571neg.out ! test/tools/javac/cast/7005095/T7005095neg.out ! test/tools/javac/cast/7005671/T7005671.out ! test/tools/javac/diags/examples/CantApplyDiamond1.java ! test/tools/javac/diags/examples/IncompatibleTypes1.java ! test/tools/javac/diags/examples/InconvertibleTypes.java ! test/tools/javac/diags/examples/InferNoConformingAssignment.java ! test/tools/javac/diags/examples/InferVarargsArgumentMismatch.java ! test/tools/javac/diags/examples/InferredDoNotConformToLower.java ! test/tools/javac/diags/examples/KindnameConstructor.java ! test/tools/javac/diags/examples/NoUniqueMaximalInstance.java ! test/tools/javac/diags/examples/NotApplicableMethodFound.java ! test/tools/javac/diags/examples/PossibleLossPrecision.java ! test/tools/javac/diags/examples/ResourceNotApplicableToType.java ! test/tools/javac/diags/examples/VarargsArgumentMismatch.java ! test/tools/javac/diags/examples/VerboseResolveMulti1.java ! test/tools/javac/diags/examples/WhereFreshTvar.java ! test/tools/javac/diags/examples/WhereIntersection.java ! test/tools/javac/diags/examples/WhereTypeVar.java ! test/tools/javac/generics/6207386/T6207386.out ! test/tools/javac/generics/diamond/neg/Neg05.out ! test/tools/javac/generics/diamond/neg/Neg06.out ! test/tools/javac/generics/diamond/neg/Neg10.out ! test/tools/javac/generics/inference/6315770/T6315770.out ! test/tools/javac/generics/inference/6611449/T6611449.out ! test/tools/javac/generics/inference/6638712/T6638712a.out ! test/tools/javac/generics/inference/6638712/T6638712b.out ! test/tools/javac/generics/inference/6638712/T6638712c.out ! test/tools/javac/generics/inference/6638712/T6638712e.out ! test/tools/javac/generics/inference/6650759/T6650759m.out ! test/tools/javac/generics/inference/6838943/T6838943.out ! test/tools/javac/generics/inference/7086586/T7086586.out ! test/tools/javac/generics/inference/7154127/T7154127.out ! test/tools/javac/generics/rawOverride/7062745/T7062745neg.out ! test/tools/javac/generics/wildcards/6886247/T6886247_2.out ! test/tools/javac/multicatch/Neg06.out ! test/tools/javac/multicatch/Neg07.out ! test/tools/javac/types/CastObjectToPrimitiveTest.out ! test/tools/javac/varargs/6313164/T6313164.out ! test/tools/javac/varargs/7097436/T7097436.out Changeset: e5cf1569d3a4 Author: mcimadamore Date: 2012-08-02 18:23 +0100 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/langtools/rev/e5cf1569d3a4 7175538: Integrate efectively final check with DA/DU analysis Summary: Allow generalized effectively-final analysis for all local variables Reviewed-by: jjg, dlsmith ! src/share/classes/com/sun/tools/javac/code/Source.java ! src/share/classes/com/sun/tools/javac/comp/Attr.java ! src/share/classes/com/sun/tools/javac/comp/Flow.java ! src/share/classes/com/sun/tools/javac/parser/JavacParser.java ! src/share/classes/com/sun/tools/javac/resources/compiler.properties + test/tools/javac/diags/examples/CantRefNonEffectivelyFinalVar.java + test/tools/javac/lambda/EffectivelyFinalTest.java + test/tools/javac/lambda/EffectivelyFinalTest01.out + test/tools/javac/lambda/EffectivelyFinalTest02.out Changeset: 2d75e7c952b8 Author: mcimadamore Date: 2012-08-02 18:24 +0100 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/langtools/rev/2d75e7c952b8 7187104: Inference cleanup: remove redundant exception classes in Infer.java Summary: Remove unused exception classes in Infer.java Reviewed-by: jjg ! src/share/classes/com/sun/tools/javac/comp/Infer.java Changeset: cfa70d7ac944 Author: lana Date: 2012-08-07 20:24 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/langtools/rev/cfa70d7ac944 Merge Changeset: f071cd32d297 Author: sundar Date: 2012-08-08 22:17 +0530 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/langtools/rev/f071cd32d297 7178324: Crash when compiling for(i : x) try(AutoCloseable x = ...) {} Reviewed-by: darcy, jjg ! src/share/classes/com/sun/tools/javac/tree/TreeInfo.java + test/tools/javac/TryWithResources/T7178324.java Changeset: 1d2db0e5eabc Author: lana Date: 2012-08-10 10:14 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/langtools/rev/1d2db0e5eabc Merge Changeset: d3d0b9cd76e0 Author: katleman Date: 2012-08-16 11:43 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/langtools/rev/d3d0b9cd76e0 Added tag jdk8-b52 for changeset 1d2db0e5eabc ! .hgtags From bengt.rutisson at oracle.com Fri Aug 17 06:18:59 2012 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 17 Aug 2012 08:18:59 +0200 Subject: RFR(S): 7192128: G1: Extend fix for 6948537 to G1's BOT In-Reply-To: <502D7DBF.50804@oracle.com> References: <502D7DBF.50804@oracle.com> Message-ID: <502DE253.9030002@oracle.com> Hi John, Great that you found this issue and fixed it so quickly! I think the change looks good. An optional change that you can do if you would like to is: The G1CollectedHeap constructor now has this code: #ifdef SPARC // Issue a stern warning, but allow use for experimentation and debugging. if (VM_Version::is_sun4v() && UseMemSetInBOT) { assert(!FLAG_IS_DEFAULT(UseMemSetInBOT), "Error"); warning("Experimental flag -XX:+UseMemSetInBOT is known to cause instability" " on sun4v; please understand that you are using at your own risk!"); } #endif Which is duplicated code from the CMSCollector constructor. Could we move it to a common place? Maybe even include it in vm_version_sparc.cpp -> VM_Version::initialize() where we have the other special treatment of UseMemSetInBOT for CMS and G1? Something like: if (is_niagara()) { ... if (UseMemSetInBOT && (UseConcMarkSweepGC || UseG1GC)) { if (FLAG_IS_DEFAULT(UseMemSetInBOT) { FLAG_SET_DEFAULT(UseMemSetInBOT, false); } else { warning("Experimental flag -XX:+UseMemSetInBOT is known to cause instability" " on sun4v; please understand that you are using at your own risk!"); } } ... } It would also have been nice to be able have a wrapper method for the 4 places where we now have this type of code: if (UseMemSetInBOT) { memset(&_offset_array[index_for(left)], offset, num_cards); } else { size_t i = index_for(left); const size_t end = i + num_cards; for (; i < end; i++) { _offset_array[i] = offset; } } But I'm fine with leaving it as it is for now. Since I know this change is kind of urgent and my comments are optional I'm fine with you pushing the change as it is. Cheers, Bengt On 2012-08-17 01:09, John Cuthbertson wrote: > Hi Everyone, > > Can I have a couple of volunteers to review the fix for this CR? The > webrev can be found at: > http://cr.openjdk.java.net/~johnc/7192128/webrev.0/ > > Summary: > A while back, Ramki discovered an issue on Niagara systems where > concurrent readers of the block offset table could see spurious zero > entries as a result of the implementation of memset using the SPARC > BIS instruction. At the time it was thought that G1 was not affected > by this issue. > > During testing of the perm-gen removal changes, the development > engineers started to see assertion failures and crashes in G1's block > offset table. The assertions were about erroneous contents of the > offset array. As a result the values used in the assertion were > printed and the value being displayed should not have failed the > assertion: > >> --- a/src/share/vm/gc_implementation/g1/g1BlockOffsetTable.cpp >> +++ b/src/share/vm/gc_implementation/g1/g1BlockOffsetTable.cpp >> @@ -546,7 +546,10 @@ >> assert(_array->offset_array(j) > 0 && >> _array->offset_array(j) <= >> (u_char) (N_words+BlockOffsetArray::N_powers-1), >> - "offset array should have been set"); >> + err_msg("offset array should have been set " >> + SIZE_FORMAT " not > 0 OR " SIZE_FORMAT " not <= " >> + SIZE_FORMAT, _array->offset_array(j), >> _array->offset_array(j), >> + (N_words+BlockOffsetArray::N_powers-1))); >> } >> #endif >> } > > # To suppress the following error report, specify this argument > # after -XX: or in .hotspotrc: > SuppressErrorAt=/g1BlockOffsetTable.cpp:552 > # > # A fatal error has been detected by the Java Runtime Environment: > # > # Internal Error > (/tmp/jprt/P1/173804.cphillim/s/src/share/vm/gc_implementation/g1/g1BlockOffsetTable.cpp:552), > pid=5210, tid=27 > # assert(_array->offset_array(j) > 0 && _array->offset_array(j) <= > (u_char) (N_words+BlockOffsetArray::N_powers-1)) failed: offset array > should have been set 65 not > 0 OR 65 not <= 77 > # > > So we had a value which failed the assertion check that, when it was > re-read for the error message, had a value which should have passed > the assertion check. > > It seems that G1 is not immune to the problem seen in 6948537 and I > believe that the recent PLAB resizing change has increased the > likelihood of hitting the issue as it can increase the amount of > concurrent refinement of BlockOffsetTable entries. > > Testing: jprt runs with the perm-gen removal changes, command line > tests on sparc and x86 systems, GCOld (which was the failing test) on > SPARC systems. > > Thanks, > > JohnC From bengt.rutisson at oracle.com Fri Aug 17 08:37:45 2012 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 17 Aug 2012 10:37:45 +0200 Subject: RFR(S): 7185699: G1: Prediction model discrepancies In-Reply-To: <5012D243.9000608@oracle.com> References: <500EE63F.2020001@oracle.com> <5012D243.9000608@oracle.com> Message-ID: <502E02D9.8020608@oracle.com> Hi John, This looks good to me. Nice cleanup! Two very minor white space issues: g1CollectorPolicy.cpp, line 1859 in the new code. There seem to be one indentation level too much. g1CollectorPolicy.cpp, line 1931 in the new code. Should be a space after the "," in the paramter list "(hr,gcs_are_young())" Ship it! Bengt On 2012-07-27 19:39, John Cuthbertson wrote: > Hi Everyone, > > A new webrev incorporating feedback from Thomas can be found at: > http://cr.openjdk.java.net/~johnc/7185699/webrev.1/ > > Thanks, > > JohnC > > On 07/24/12 11:15, John Cuthbertson wrote: >> Hi Everyone, >> >> Can I have a couple of volunteers to review the changes for this CR? >> The webrev can be found at: >> http://cr.openjdk.java.net/~johnc/7185699/webrev.0/. >> >> Background: >> While I was going through the code for the current mixed GC policy >> and the code that adds non-young regions to the collection set, I >> found a couple of minor bugs associated with the prediction code. The >> first was the calculation of the number of unprocessed dirty card at >> the start of the GC - this was using routines that return a number of >> bytes rather than the number of entries. As a result we were vastly >> overestimating the base pause time. The second was in the code that >> calculates the GC efficiency of a non-young region. The GC efficiency >> of a region is calculated at the end of marking. The code, however, >> was using the wrong routine to predict the number of cards for a >> given RSet length. It was using the data collected for young GCs when >> it should have been using the data collected for mixed GCs. >> >> The other changes are minor cleanups. These help to slightly increase >> the amount of inlining with the Solaris Studio compiler and save an >> iteration over the collection set. >> >> Testing: >> Dacapo2006 with Ergonomic output enabled; Dacapo2006 with >> PrintLivenessInfo enabled (before and after); GC test suite with >> verification; jprt. >> >> Thanks, >> >> JohnC > From jon.masamitsu at oracle.com Fri Aug 17 14:01:54 2012 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Fri, 17 Aug 2012 07:01:54 -0700 Subject: RFR(S): 7192128: G1: Extend fix for 6948537 to G1's BOT In-Reply-To: <502D7DBF.50804@oracle.com> References: <502D7DBF.50804@oracle.com> Message-ID: <502E4ED2.3060409@oracle.com> John, Changes look good. Thanks for the fix. Jon On 8/16/2012 4:09 PM, John Cuthbertson wrote: > Hi Everyone, > > Can I have a couple of volunteers to review the fix for this CR? The > webrev can be found at: > http://cr.openjdk.java.net/~johnc/7192128/webrev.0/ > > Summary: > A while back, Ramki discovered an issue on Niagara systems where > concurrent readers of the block offset table could see spurious zero > entries as a result of the implementation of memset using the SPARC > BIS instruction. At the time it was thought that G1 was not affected > by this issue. > > During testing of the perm-gen removal changes, the development > engineers started to see assertion failures and crashes in G1's block > offset table. The assertions were about erroneous contents of the > offset array. As a result the values used in the assertion were > printed and the value being displayed should not have failed the > assertion: > >> --- a/src/share/vm/gc_implementation/g1/g1BlockOffsetTable.cpp >> +++ b/src/share/vm/gc_implementation/g1/g1BlockOffsetTable.cpp >> @@ -546,7 +546,10 @@ >> assert(_array->offset_array(j) > 0 && >> _array->offset_array(j) <= >> (u_char) (N_words+BlockOffsetArray::N_powers-1), >> - "offset array should have been set"); >> + err_msg("offset array should have been set " >> + SIZE_FORMAT " not > 0 OR " SIZE_FORMAT " not <= " >> + SIZE_FORMAT, _array->offset_array(j), >> _array->offset_array(j), >> + (N_words+BlockOffsetArray::N_powers-1))); >> } >> #endif >> } > > # To suppress the following error report, specify this argument > # after -XX: or in .hotspotrc: > SuppressErrorAt=/g1BlockOffsetTable.cpp:552 > # > # A fatal error has been detected by the Java Runtime Environment: > # > # Internal Error > (/tmp/jprt/P1/173804.cphillim/s/src/share/vm/gc_implementation/g1/g1BlockOffsetTable.cpp:552), > pid=5210, tid=27 > # assert(_array->offset_array(j) > 0 && _array->offset_array(j) <= > (u_char) (N_words+BlockOffsetArray::N_powers-1)) failed: offset array > should have been set 65 not > 0 OR 65 not <= 77 > # > > So we had a value which failed the assertion check that, when it was > re-read for the error message, had a value which should have passed > the assertion check. > > It seems that G1 is not immune to the problem seen in 6948537 and I > believe that the recent PLAB resizing change has increased the > likelihood of hitting the issue as it can increase the amount of > concurrent refinement of BlockOffsetTable entries. > > Testing: jprt runs with the perm-gen removal changes, command line > tests on sparc and x86 systems, GCOld (which was the failing test) on > SPARC systems. > > Thanks, > > JohnC From john.cuthbertson at oracle.com Fri Aug 17 19:02:51 2012 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Fri, 17 Aug 2012 12:02:51 -0700 Subject: RFR(S): 7192128: G1: Extend fix for 6948537 to G1's BOT In-Reply-To: <502DE253.9030002@oracle.com> References: <502D7DBF.50804@oracle.com> <502DE253.9030002@oracle.com> Message-ID: <502E955B.1050400@oracle.com> Hi Bengt, Thanks for reviewing the code so quickly. On 08/16/12 23:18, Bengt Rutisson wrote: > > Hi John, > > Great that you found this issue and fixed it so quickly! > > I think the change looks good. > > An optional change that you can do if you would like to is: > > The G1CollectedHeap constructor now has this code: > > #ifdef SPARC > // Issue a stern warning, but allow use for experimentation and > debugging. > if (VM_Version::is_sun4v() && UseMemSetInBOT) { > assert(!FLAG_IS_DEFAULT(UseMemSetInBOT), "Error"); > warning("Experimental flag -XX:+UseMemSetInBOT is known to cause > instability" > " on sun4v; please understand that you are using at your > own risk!"); > } > #endif > > Which is duplicated code from the CMSCollector constructor. Could we > move it to a common place? Maybe even include it in > vm_version_sparc.cpp -> VM_Version::initialize() where we have the > other special treatment of UseMemSetInBOT for CMS and G1? Something like: > > if (is_niagara()) { > ... > if (UseMemSetInBOT && (UseConcMarkSweepGC || UseG1GC)) { > if (FLAG_IS_DEFAULT(UseMemSetInBOT) { > FLAG_SET_DEFAULT(UseMemSetInBOT, false); > } else { > warning("Experimental flag -XX:+UseMemSetInBOT is known to > cause instability" > " on sun4v; please understand that you are using at > your own risk!"); > } > } > ... > } > This is a great idea. Done. > It would also have been nice to be able have a wrapper method for the > 4 places where we now have this type of code: > > if (UseMemSetInBOT) { > memset(&_offset_array[index_for(left)], offset, num_cards); > } else { > size_t i = index_for(left); > const size_t end = i + num_cards; > for (; i < end; i++) { > _offset_array[i] = offset; > } > } > > But I'm fine with leaving it as it is for now. > > I would prefer to leave it as it is for now. We do have a CR to reconcile G1's BOT with the that of the other collectors - I think that is when we can introduce the wrapper (and it would be back down to only two instances). I'll add a comment to that CR. JohnC From john.cuthbertson at oracle.com Fri Aug 17 19:03:39 2012 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Fri, 17 Aug 2012 12:03:39 -0700 Subject: RFR(S): 7192128: G1: Extend fix for 6948537 to G1's BOT In-Reply-To: <502E4ED2.3060409@oracle.com> References: <502D7DBF.50804@oracle.com> <502E4ED2.3060409@oracle.com> Message-ID: <502E958B.2060205@oracle.com> Hi Jon, Thanks for the review. JohnC On 08/17/12 07:01, Jon Masamitsu wrote: > John, > > Changes look good. Thanks for the fix. > > Jon > > On 8/16/2012 4:09 PM, John Cuthbertson wrote: >> Hi Everyone, >> >> Can I have a couple of volunteers to review the fix for this CR? The >> webrev can be found at: >> http://cr.openjdk.java.net/~johnc/7192128/webrev.0/ >> >> Summary: >> A while back, Ramki discovered an issue on Niagara systems where >> concurrent readers of the block offset table could see spurious zero >> entries as a result of the implementation of memset using the SPARC >> BIS instruction. At the time it was thought that G1 was not affected >> by this issue. >> >> During testing of the perm-gen removal changes, the development >> engineers started to see assertion failures and crashes in G1's block >> offset table. The assertions were about erroneous contents of the >> offset array. As a result the values used in the assertion were >> printed and the value being displayed should not have failed the >> assertion: >> >>> --- a/src/share/vm/gc_implementation/g1/g1BlockOffsetTable.cpp >>> +++ b/src/share/vm/gc_implementation/g1/g1BlockOffsetTable.cpp >>> @@ -546,7 +546,10 @@ >>> assert(_array->offset_array(j) > 0 && >>> _array->offset_array(j) <= >>> (u_char) (N_words+BlockOffsetArray::N_powers-1), >>> - "offset array should have been set"); >>> + err_msg("offset array should have been set " >>> + SIZE_FORMAT " not > 0 OR " SIZE_FORMAT " not <= " >>> + SIZE_FORMAT, _array->offset_array(j), >>> _array->offset_array(j), >>> + (N_words+BlockOffsetArray::N_powers-1))); >>> } >>> #endif >>> } >> >> # To suppress the following error report, specify this argument >> # after -XX: or in .hotspotrc: >> SuppressErrorAt=/g1BlockOffsetTable.cpp:552 >> # >> # A fatal error has been detected by the Java Runtime Environment: >> # >> # Internal Error >> (/tmp/jprt/P1/173804.cphillim/s/src/share/vm/gc_implementation/g1/g1BlockOffsetTable.cpp:552), >> pid=5210, tid=27 >> # assert(_array->offset_array(j) > 0 && _array->offset_array(j) <= >> (u_char) (N_words+BlockOffsetArray::N_powers-1)) failed: offset array >> should have been set 65 not > 0 OR 65 not <= 77 >> # >> >> So we had a value which failed the assertion check that, when it was >> re-read for the error message, had a value which should have passed >> the assertion check. >> >> It seems that G1 is not immune to the problem seen in 6948537 and I >> believe that the recent PLAB resizing change has increased the >> likelihood of hitting the issue as it can increase the amount of >> concurrent refinement of BlockOffsetTable entries. >> >> Testing: jprt runs with the perm-gen removal changes, command line >> tests on sparc and x86 systems, GCOld (which was the failing test) on >> SPARC systems. >> >> Thanks, >> >> JohnC From john.cuthbertson at oracle.com Fri Aug 17 19:22:40 2012 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Fri, 17 Aug 2012 12:22:40 -0700 Subject: RFR(S): 7185699: G1: Prediction model discrepancies In-Reply-To: <502E02D9.8020608@oracle.com> References: <500EE63F.2020001@oracle.com> <5012D243.9000608@oracle.com> <502E02D9.8020608@oracle.com> Message-ID: <502E9A00.4040504@oracle.com> Hi Bengt, Thanks for the review. JohnC On 08/17/12 01:37, Bengt Rutisson wrote: > > Hi John, > > This looks good to me. Nice cleanup! > > Two very minor white space issues: > > g1CollectorPolicy.cpp, line 1859 in the new code. There seem to be one > indentation level too much. > > g1CollectorPolicy.cpp, line 1931 in the new code. Should be a space > after the "," in the paramter list "(hr,gcs_are_young())" > > Ship it! > Bengt > > On 2012-07-27 19:39, John Cuthbertson wrote: >> Hi Everyone, >> >> A new webrev incorporating feedback from Thomas can be found at: >> http://cr.openjdk.java.net/~johnc/7185699/webrev.1/ >> >> Thanks, >> >> JohnC >> >> On 07/24/12 11:15, John Cuthbertson wrote: >>> Hi Everyone, >>> >>> Can I have a couple of volunteers to review the changes for this CR? >>> The webrev can be found at: >>> http://cr.openjdk.java.net/~johnc/7185699/webrev.0/. >>> >>> Background: >>> While I was going through the code for the current mixed GC policy >>> and the code that adds non-young regions to the collection set, I >>> found a couple of minor bugs associated with the prediction code. >>> The first was the calculation of the number of unprocessed dirty >>> card at the start of the GC - this was using routines that return a >>> number of bytes rather than the number of entries. As a result we >>> were vastly overestimating the base pause time. The second was in >>> the code that calculates the GC efficiency of a non-young region. >>> The GC efficiency of a region is calculated at the end of marking. >>> The code, however, was using the wrong routine to predict the number >>> of cards for a given RSet length. It was using the data collected >>> for young GCs when it should have been using the data collected for >>> mixed GCs. >>> >>> The other changes are minor cleanups. These help to slightly >>> increase the amount of inlining with the Solaris Studio compiler and >>> save an iteration over the collection set. >>> >>> Testing: >>> Dacapo2006 with Ergonomic output enabled; Dacapo2006 with >>> PrintLivenessInfo enabled (before and after); GC test suite with >>> verification; jprt. >>> >>> Thanks, >>> >>> JohnC >> > From kirk at kodewerk.com Fri Aug 17 20:32:45 2012 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Fri, 17 Aug 2012 22:32:45 +0200 Subject: JEP 158: Unified JVM logging In-Reply-To: <2FF50E25-8A2C-4837-8C46-44E8C5CD6180@oracle.com> References: <4E6D5DEC.8090109@oss.ntt.co.jp> <4E6DDB8F.4020301@oracle.com><3568943A-E6A4-4287-8319-8B0993F070F9@oracle.com><4E7A85AE.10507@oss.ntt.co.jp> <4E7B19EC.5070700@oracle.com> <4E7B2227.9070406@kippdata.de> <4E805629.7030703@oss.ntt.co.jp> <4E846158.3020606@oracle.com> <5017B761.5070203@lab.ntt.co.jp> <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> <502A1853.70508@lab.ntt.co.jp> <502A25B0.20004@oracle.com> <502BB003.8020701@oracle.com> <2FF50E25-8A2C-4837-8C46-44E8C5CD6180@oracle.com> Message-ID: <325CA5D7-13B4-4A72-ADA2-C7BFAA9F1353@kodewerk.com> Hi Staffan, Thanks for the response. You call it levels, I call is at hierarchy. I'm happy to change to the work levels. On 2012-08-17, at 10:20 PM, Staffan Larsen wrote: > > On 15 aug 2012, at 16:19, Dmitry Samersoff wrote: > >> On 2012-08-15 12:44, Kirk Pepperdine wrote: >> >>> The current system of command-line (experimental) options follow a >>> format that is quite consistent. The options provide semantic meaning as >>> to what will be logged. This semantic meaning is lost when we reduce >>> everything to the level of error, warning, info, debug and trace. >> >> I guess Levels are independent to categories. i.e. I think >> >> Log:info,gc,younggen should be possible > > The suggestion in the JEP is that each category/component has several levels, so in that sense they are not independent. To enable logging you would specify a list of {component, level} tuples, like: > > gc:info, younggen:trace > > But to make it easier to type in the most common case the 'info' level is considered 'default' if nothing else is specified. So the above would be equivalent to saying just > > gc, younggen:trace Right, and this is at the heart of one of my objections is that the predefined levels used by everyone lack semantic meaning where as being able to define tags or groups of tags would allow one to define levels but not force one to use levels. There certainly is a case for levels and I would not want them precluded but I do not believe it is desirable for all components to hierarchically reduce log messages. If I had a better idea of the schedule for the JEP I'd have a better understanding of when I might be able to pitch in rather then just play seagull. Regards, Kirk From john.cuthbertson at oracle.com Fri Aug 17 21:10:50 2012 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Fri, 17 Aug 2012 14:10:50 -0700 Subject: RFR(S): 7192128: G1: Extend fix for 6948537 to G1's BOT In-Reply-To: <502E955B.1050400@oracle.com> References: <502D7DBF.50804@oracle.com> <502DE253.9030002@oracle.com> <502E955B.1050400@oracle.com> Message-ID: <502EB35A.2080106@oracle.com> Hi Bengt, I believe I'll have to leave the warnings being emitted from g1CollectedHeap.cpp and concurrentMarkSweepGeneration.cpp. Moving the warning into VM_Version::initialize() causes the warning to come out twice: dr-evil{jcuthber}:293> ./gamma -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC -XX:+UseMemSetInBOT -version Using java runtime at: /net/jre.us.oracle.com/p/v42/jdk/7/fcs/b147/binaries/solaris-sparcv9/jre Java HotSpot(TM) 64-Bit Server VM warning: Experimental flag -XX:+UseMemSetInBOT is known to cause instability on sun4v; please understand that you are using at your own risk! Java HotSpot(TM) 64-Bit Server VM warning: Experimental flag -XX:+UseMemSetInBOT is known to cause instability on sun4v; please understand that you are using at your own risk! java version "1.7.0" Java(TM) SE Runtime Environment (build 1.7.0-b147) Java HotSpot(TM) 64-Bit Server VM (build 24.0-b20-internal-jvmg, mixed mode) It appears we are calling VM_Version::initialize() twice (for whatever reason). Once from Arguments::check_vm_args_consistency(): // Check the consistency of vm_init_args bool Arguments::check_vm_args_consistency() { // Method for adding checks for flag consistency. // The intent is to warn the user of all possible conflicts, // before returning an error. // Note: Needs platform-dependent factoring. bool status = true; #if ( (defined(COMPILER2) && defined(SPARC))) // NOTE: The call to VM_Version_init depends on the fact that VM_Version_init // on sparc doesn't require generation of a stub as is the case on, e.g., // x86. Normally, VM_Version_init must be called from init_globals in // init.cpp, which is called by the initial java thread *after* arguments // have been parsed. VM_Version_init gets called twice on sparc. extern void VM_Version_init(); VM_Version_init(); if (!VM_Version::has_v9()) { jio_fprintf(defaultStream::error_stream(), "V8 Machine detected, Server requires V9\n"); status = false; } #endif /* COMPILER2 && SPARC */ So I'll go back to the original code. JohnC On 08/17/12 12:02, John Cuthbertson wrote: > Hi Bengt, > > Thanks for reviewing the code so quickly. > > > On 08/16/12 23:18, Bengt Rutisson wrote: >> >> Hi John, >> >> Great that you found this issue and fixed it so quickly! >> >> I think the change looks good. >> >> An optional change that you can do if you would like to is: >> >> The G1CollectedHeap constructor now has this code: >> >> #ifdef SPARC >> // Issue a stern warning, but allow use for experimentation and >> debugging. >> if (VM_Version::is_sun4v() && UseMemSetInBOT) { >> assert(!FLAG_IS_DEFAULT(UseMemSetInBOT), "Error"); >> warning("Experimental flag -XX:+UseMemSetInBOT is known to cause >> instability" >> " on sun4v; please understand that you are using at your >> own risk!"); >> } >> #endif >> >> Which is duplicated code from the CMSCollector constructor. Could we >> move it to a common place? Maybe even include it in >> vm_version_sparc.cpp -> VM_Version::initialize() where we have the >> other special treatment of UseMemSetInBOT for CMS and G1? Something >> like: >> >> if (is_niagara()) { >> ... >> if (UseMemSetInBOT && (UseConcMarkSweepGC || UseG1GC)) { >> if (FLAG_IS_DEFAULT(UseMemSetInBOT) { >> FLAG_SET_DEFAULT(UseMemSetInBOT, false); >> } else { >> warning("Experimental flag -XX:+UseMemSetInBOT is known to >> cause instability" >> " on sun4v; please understand that you are using at >> your own risk!"); >> } >> } >> ... >> } >> > > This is a great idea. Done. > >> It would also have been nice to be able have a wrapper method for the >> 4 places where we now have this type of code: >> >> if (UseMemSetInBOT) { >> memset(&_offset_array[index_for(left)], offset, num_cards); >> } else { >> size_t i = index_for(left); >> const size_t end = i + num_cards; >> for (; i < end; i++) { >> _offset_array[i] = offset; >> } >> } >> >> But I'm fine with leaving it as it is for now. >> >> > I would prefer to leave it as it is for now. We do have a CR to > reconcile G1's BOT with the that of the other collectors - I think > that is when we can introduce the wrapper (and it would be back down > to only two instances). I'll add a comment to that CR. > > JohnC > From staffan.larsen at oracle.com Fri Aug 17 19:44:26 2012 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 17 Aug 2012 21:44:26 +0200 Subject: JEP 158 In-Reply-To: <40E80414-7CC9-40B1-B801-59AC33C821D2@kodewerk.com> References: <20120618213859.738A913EE@eggemoggin.niobe.net> <25BA359D-58E7-48F3-9575-E555A31FE3C2@kodewerk.com> <4FE10530.5030901@oracle.com> <1BC2C076-7066-406A-87B2-A6F4618B7ECC@kodewerk.com> <4FE17C77.2050800@oracle.com> <40E80414-7CC9-40B1-B801-59AC33C821D2@kodewerk.com> Message-ID: <77BD6C9D-9939-4F76-9C1B-8A731DD92FCB@oracle.com> I don't think the JEP has been updated, only published. I have not had time to do any changes to it. /Staffan On 10 jul 2012, at 17:16, Kirk Pepperdine wrote: > Hi Staffan, > > I've noticed that the specification has been updated but still contains the hierarchical level system. Is it your intention to maintain level in the specification? > > Regards, > Kirk > > On 2012-06-20, at 12:50 PM, Staffan Larsen wrote: > >> Hi, >> >> Thanks for the feedback! >> >> It is going to be hard to find the right balance between a system that provides great power and one that is simple to use and to implement. In general, I will lean more towards making it simple, but I want to make sure we cover the major use cases as well. >> >> It looks like there are three possible ways to select log messages being discussed here. I'd like to discuss how to solve your problem for each one. >> >> 1) Selection based on component/level. (This is what is described in the JEP.) >> In this case we can have any number of components. So we can have gc and tenuring as different component. They will each have a level associated with them. However there is no relationship between the gc and the tenuring component, so to enable both you need to say -Xverbose:gc,tenuring. To only enable tenuring you can say -Xverbose:tenuring. To enable all gc messages, except tenuring you say -Xverbose:gc. >> >> 2) Selection based on component/level where the component is a hierarchy. >> In this case the different steps in the hierarchy can have different levels associated with them, but if there is no explicit level associated, the parent level will be used. So to enable both tenuring and gc you would say -Xverbose:gc. To enable only gc you would say -Xverbose:gc,gc.tenuring=none (I made up the 'none' level). To enable only tenuring you would say -Xverbose:gc.tenuring. >> >> 3) Selection based on tags. >> In this case log messages are associated with a set of tags instead of a component/level tuple. You select messages by specifying tags you want to see. I imagine that in this case the tenuring messages would be tagged with both the gc and the tenuring tags, but that there would be other messages tagged with gc only. To enable gc and tenuring you would say -Xverbose:gc,tenuring. To enable all gc messages you say -Xverbose:gc. There is no way to see only gc messages without seeing the tenuring messages. >> >> Is this a fair description of the different ways? I'm especially interested in the last one - I'm not sure I captured your idea correctly there. >> >> Thanks, >> /Staffan >> >> >> On 20 jun 2012, at 09:32, Jesper Wilhelmsson wrote: >> >>> Hi Kirk, >>> >>> I'm CC'ing serviceability on this since this is really their JEP and discussions around it should go on the serviceability list, even though it seems you are mainly interested in GC logging at this point. >>> >>> I understand what you want and I see that the logging level won't help us get there. I don't agree that the logging we have today can't fit nicely into a hierarchical scheme though, it just needs to be more fine grained to achieve what you want. >>> >>> We can be pretty generous with modules and in principal have one module for each verbose flag that exists today. Personally I don't think that is a good idea, but it certainly is possible. I would rather like to propose a different solution. >>> >>> How about we have a gc module that can be filtered based on different sub modules. The sub modules could be fairly close to the existing verbose flags that exists today if that turns out to be a good way to divide information. It could look like this >>> >>> -Xverbose:gc=info,gc.tenuring=debug >>> >>> to set regular gc verbose to info level (I would say that is close to PrintGC) and turn on more detailed logging for tenuring. Or >>> >>> -Xverbose:gc.tenuring >>> >>> that could be equal to what that flag prints today. Let's see what the serviceability team thinks since they are the ones who will actually implement this in the end. >>> >>> Another solution that I don't really like but guess is easier to implement is to add the current verbose flag to the actual message so that the logs can be filtered based on that. But this will clutter the messages and we would still have the problem to decide on which level things should get logged. >>> /Jesper >>> >>> >>> On 2012-06-20 07:28, Kirk Pepperdine wrote: >>>> >>>> Hi Jesper, >>>> >>>> I did read the spec and I do like the ability to specify the "component" that you'd like to log information from. So I feel that is a great improvement over the (broken) pattern established in every major logging Java framework. I'm going to stick to GC logging just because I've spent so much time puzzling over them and adjusting my parser to deal with all the changes that have continuously crept into them. While 'm certainly not going to argue for keeping the current GC logging framework what I will say is that it's not all bad in that the flags that have been provided to me are almost always semantically meaningful in that they tell me what I'm going to see. In this spirit I'd like to see a category like TenuringDetails for example. Is this information INFO, DEBUG, or TRACE? hard to say but it's clearly TenuringDetails and so this is a subcategory that I'd like to define and it's clearly not a subcategory that you'd want to define a generalize logging framework. And it is here that this specification over-reaches. It tries to define logging categories that are not only are devoid of meaning, they assume a hierarchical structure to them. Going back to GC logging I would argue that while there is some hierarchy in there, most of the messages don't nicely fit into this imposed hierarchical developer centric list of categories. >>>> >>>> I think we could easily both agree that it would be ridiculous for me to ask that you add PrintTunuring to the list of levels yet that is exactly what I want. So I guess what I'm asking is, what would the spec look like if you removed the log levels from it and allowed me to define my own or to not even use levels at all. >>>> >>>> Regards, >>>> Kirk >>>> >>>> On 2012-06-20, at 1:03 AM, Jesper Wilhelmsson wrote: >>>> >>>>> Hi Kirk, >>>>> >>>>> To select what should be logged there should be logging modules. A module could be for example class loading, gc, jit compiler etc. The logging level is just a way to control how much logging you want. Setting gc=info would give you some basic gc logging while gc=debug would give you more detailed info. >>>>> >>>>> A typical command line could look like >>>>> >>>>> -Xverbose:gc=debug,finalizer=info,compiler,alloc,cookies=trace >>>>> /Jesper >>>>> >>>>> >>>>> On 2012-06-19 23:44, Kirk Pepperdine wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> I see the logging framework JEP finally was published. This is great news. >>>>>> >>>>>> I'd like to comment on this quality >>>>>> >>>>>> "Logging is performed at different levels: error, warning, info, debug, trace" >>>>>> >>>>>> If we accept the problems associated with level based logging, these name work for generic frameworks such as Log4J and JDK logging. However, the names are meaningless in that they carry no semantic context with what would be logged. The nice thing about the current set of flags is they convey the information that will be printed. >>>>>> >>>>>> On the question of log levels. I was hoping that we would have learned from the follies of using level based logging instead of a digital or tag based system. IOWs a on or off different aspects without having to eat the whole elephant of records that some developer arbitrarily decided should be dumped at a particular log level. One can level tags.. but you can't get tags or digital behaviour from levels. >>>>>> >>>>>> Kind regards, >>>>>> Kirk Pepperdine >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From staffan.larsen at oracle.com Fri Aug 17 20:08:32 2012 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 17 Aug 2012 22:08:32 +0200 Subject: JEP 158: Unified JVM logging In-Reply-To: References: <4E6D5DEC.8090109@oss.ntt.co.jp> <4E6DDB8F.4020301@oracle.com><3568943A-E6A4-4287-8319-8B0993F070F9@oracle.com><4E7A85AE.10507@oss.ntt.co.jp> <4E7B19EC.5070700@oracle.com> <4E7B2227.9070406@kippdata.de> <4E805629.7030703@oss.ntt.co.jp> <4E846158.3020606@oracle.com> <5017B761.5070203@lab.ntt.co.jp> <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> <502A1853.70508@lab.ntt.co.jp> <502A25B0.20004@oracle.com> Message-ID: Hi, I would agree that there are great merits to tag-based systems instead of hierarchical systems. However, the JEP does not have a hierarchical system in mind. Components, as the JEP refers to them, are not hierarchical, but they are disjunct. Disjunct in the sense that a particular log output in the source code can only belong to one component - this would be a difference from a purely tag based system. For each component in the proposal, there is the possibility of logging messages of different severities; these are the 'levels' (maybe a bad and confusing terminology?). I think I do understand your use-case and I would be happy if we could find a workable system that solves this use-case. I gave it a shot in [1] - my suggestion #3. This suggested implementation had some serious problems in the way you could select log messages. I would love to see a better proposal for how this could be implemented. Logging information in binary certainly reduces bulk and can also make it much easier to parse and visualize the data. The JRockit Flight Recorder technology has a binary format for storing information (it is also a self-describing format which includes meta-data to facilitate parsing) which is optimized for storage speed and size. As you probably know, we are working on a port of JFR to HotSpot. Thanks, /Staffan [1] http://mail.openjdk.java.net/pipermail/serviceability-dev/2012-June/006369.html On 15 aug 2012, at 10:44, Kirk Pepperdine wrote: > Hi Dmitry, > > Lets start with this. > > Common logging command-line options for all components > Logging is performed at different levels: error, warning, info, debug, trace > Logging messages are in human-readable plain text > > IMHO, these current set of goals are based on the assumptions that all component is hierarchical in nature and that they will log the same way without the possibility of binary dumps. I would argue these assumptions over-reach and preclude certain types of logging that take place in the JVM today. > > The current system of command-line (experimental) options follow a format that is quite consistent. The options provide semantic meaning as to what will be logged. This semantic meaning is lost when we reduce everything to the level of error, warning, info, debug and trace. It's sets up arbitrary categories in which developers will be left to decide what level a particular activity should be logged at. To make this real I'll use the -XX:+PrintTenuringDistribution flag as an example. When I set that flag it's very clear what's to be logged. My question is, should this be logged at the info level? How about debug? Maybe trace? And if I set those levels, what else comes along for the ride? IOWs, if I attempt to keep the same model of GC logging, we'd need different levels. If we need different levels then we're about to violate the first goal. > > What this example suggests is that gc logging would be better served by using categories (or subjects if we can take something from the pub/sub metaphor) or tags or labels or what ever you want to call them. Using tags, I can create a system of levels using error, warning, info, debug, and trace if that fits how I'm setting up logging. But from levels it's difficult to implement things using categories. This is a case where the one of the non-goals should be, we shouldn't prevent people from structuring their logs as they see best fits the component they are instrumenting. > > The non-goals that should be reconsidered is to force human-readability. I would argue that this is again over-reaching in that properly structured log files should almost *never* be read by a human. In the cases why force an unnecessary requirement. I say this because I've run into a number of applications where logging was the primary bottleneck. In those cases there were two primary problems, one was bulk and the other was threading (lock contention which I shan't address here). Now while trying to solve the problem of bulkiness and lock contention might be considered a premature optimization at this stage and I certainty wouldn't disagree, I would not like to see designs or specifications that would naturally preclude some very obvious solutions. One of the solutions to bulk is to use a more compact format be some form of binary. That option has been taken away for what I would call an ideological requirement. There is no technical reason why I shouldn't be able to log in binary other than connivence. In addition, you don't know what else is happening on the system (disk??) you're logging to so it behoves you to be as light as possible. > > I'll leave it at this for now. > > -- Kirk > > On 2012-08-14, at 12:17 PM, Dmitry Samersoff wrote: > >> Kirk, >> >>> However I do have very serious concerns about this JEP in that it >>> doesn't fix the problems that exist in current logging frameworks, >>> it only mimics them. >> >>> http://openjdk.java.net/jeps/158 >> >> Any comments is much appreciated. >> >> Personally, I think that log rotation is out of scope and responsibility >> of JVM. >> >> -Dmitry >> >> >> On 2012-08-14 13:26, Kirk Pepperdine wrote: >>> Hi Yasumasa, >>> >>> I'm not sure that log file rotation is a part of this JEP. >>> However I do have very serious concerns about this JEP in that it doesn't fix the problems that exist in current logging frameworks, it only mimics them. >>> >>> Regards, >>> Kirk >>> >>> On 2012-08-14, at 11:20 AM, Yasumasa Suenaga wrote: >>> >>>> Hi Staffan, >>>> >>>> May I ask you 2 questions about this JEP? >>>> >>>> 1. One of goals of this JEP is defined as following: >>>> "File rotation of log files by size (similar to what is available for GC logs today)" >>>> My patch realizes log rotation by external trigger. >>>> 7090324 is included in this JEP? >>>> (Is 7090324 included in "Logging can be configured dynamically at runtime via jcmd or MBeans" ? ) >>>> >>>> 2. I've posted a patch for "6950794: Make the GC log file name parameterized" . >>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2012-July/004758.html >>>> Could you include this RFE in this JEP? >>>> If we use log rotation, I think that name of logs is very important for log management. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2012/08/14 16:50, Staffan Larsen wrote: >>>>> Hi Yasumasa, >>>>> >>>>> This work should be done in the context of the Unified Logging JEP [1]. Unfortunately, that work has not yet started due to resource constraints. Comments on the proposal are welcome. >>>>> >>>>> [1] http://openjdk.java.net/jeps/158 >>>>> >>>>> /Staffan >>>>> >>>>> On 31 jul 2012, at 12:45, Yasumasa Suenaga > wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I've posted a patch for gc log rotation via jinfo tool last year. >>>>>> Then I received an email that essence of my patch will be implemented as >>>>>> "subcommands" of jcmd. >>>>>> >>>>>> Now, jcmd is provided by OpenJDK. However, current jcmd does not seem to >>>>>> implement function of gclog rotation. >>>>>> >>>>>> Will this RFE be implemented? >>>>>> >>>>>> I need to rotate gclog which is synchronized with signal, cron, etc... on Linux. >>>>>> So, if this RFE is not implemented for a while, I would like to contribute >>>>>> patch for jcmd. >>>>>> >>>>>> >>>>>> Regards, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2011/09/29 21:15, James Melvin wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> Thanks very much for your OpenJDK contribution! As part of the effort to >>>>>>> port JRockit features to HotSpot, we plan to introduce a consolidated >>>>>>> commandline serviceability tool (jcmd) to potentially replace many of >>>>>>> the existing tools in the bin directory, such as jmap, jstack, jinfo and >>>>>>> others. Over the next few update releases, we plan to add several jcmd >>>>>>> *subcommands* instead to accomplish specific tasks or affect the running >>>>>>> JVM instance. We'd like to use the essence of your contribution in one >>>>>>> of the jcmd subcommands (instead of extending jinfo) to ask the JVM to >>>>>>> begin rotating GC logs. We hope you find this attractive and hope you >>>>>>> will help review and perfect the change. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Jim Melvin >>>>>>> Sr. Engineering Manager >>>>>>> Oracle >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 9/26/11 6:38 AM, Yasumasa Suenaga wrote: >>>>>>>> (I've changed subject of this email to new RFE.) >>>>>>>> >>>>>>>> This RFE is enhancement of current gclog implementation. >>>>>>>> So, I'd like to discuss about rotating gclog. >>>>>>>> >>>>>>>> My customers need gclog rotation which is invoked by external trigger. >>>>>>>> So I've requested this RFE and made a patch. >>>>>>>> >>>>>>>> >>>>>>>> In many case on Linux, logfile is rotated by signal (e.g. SIGHUP) . >>>>>>>> The aim of this RFE is to synchronize gclog and the other logs. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> (2011/09/22 20:55), Rainer Jung wrote: >>>>>>>>> On 22.09.2011 13:20, Dmitry Samersoff wrote: >>>>>>>>>> >>>>>>>>>> Yasumasa, >>>>>>>>>> >>>>>>>>>> On 2011-09-22 04:47, Yasumasa Suenaga wrote: >>>>>>>>>>> If we can think Java on Linux and Solaris only, syslog is best >>>>>>>>>>> solution. >>>>>>>>>>> However, Windows usually doesn't have syslog. >>>>>>>>>>> >>>>>>>>>>> So, I think that gclog is needed for logging GC stats with platform >>>>>>>>>>> independent in realtime. >>>>>>>>>> >>>>>>>>>> Windows has it's own logging API as reach as syslog is or ever better >>>>>>>>>> as well as numerous syslog implementations. >>>>>>>>>> >>>>>>>>>> Native windows logging API was completely redesigned for Windows 2008 >>>>>>>>>> server and now it allows for developers to send a structured events from >>>>>>>>>> theirs application. >>>>>>>>> >>>>>>>>> AFAIK log rotation for loggc is already implemented though not >>>>>>>>> necessarily yet released. The change discussed here is about supporting >>>>>>>>> an externally generated rotation trigger, e.g. via a signal, instead of >>>>>>>>> only rotating by size or time via a startup parameter. >>>>>>>>> >>>>>>>>> If you want support for syslog or Windows APIs included, it would be >>>>>>>>> best to start a new discussion. >>>>>>>>> >>>>>>>>> A GC log for an application under load might easily produce a block of >>>>>>>>> about 1.5 KB size every few seconds. I seriously doubt, that the need >>>>>>>>> for log file rotation can be argued away by referring to syslog or >>>>>>>>> Windows log API as the correct solution. >>>>>>>>> >>>>>>>>> The messages are not really line formatted, the format can vary a lot >>>>>>>>> (depending on the excat XX switches), the traffic can be quite high and >>>>>>>>> AFAIK the JVM writes it synchronously, so there must be absolutely no >>>>>>>>> risk in writing it out with very little latency. In addition, for >>>>>>>>> analysis, you wouldn't want to look at each event individually, but >>>>>>>>> instead process the whole file through a script or tool, which should >>>>>>>>> not depend on the output specifics of a platform specific log apparatus. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Rainer >>>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >>> >> >> >> -- >> Dmitry Samersoff >> Java Hotspot development team, SPB04 >> * There will come soft rains ... >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From staffan.larsen at oracle.com Fri Aug 17 20:12:38 2012 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 17 Aug 2012 22:12:38 +0200 Subject: 7090324: gclog rotation via external tool In-Reply-To: <502A1853.70508@lab.ntt.co.jp> References: <4E6D5DEC.8090109@oss.ntt.co.jp> <4E6DDB8F.4020301@oracle.com><3568943A-E6A4-4287-8319-8B0993F070F9@oracle.com><4E7A85AE.10507@oss.ntt.co.jp> <4E7B19EC.5070700@oracle.com> <4E7B2227.9070406@kippdata.de> <4E805629.7030703@oss.ntt.co.jp> <4E846158.3020606@oracle.com> <5017B761.5070203@lab.ntt.co.jp> <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> <502A1853.70508@lab.ntt.co.jp> Message-ID: <7C2E0A4F-79DF-4D75-808B-A08619B8B303@oracle.com> Hi Yasumasa, As the JEP stands today, neither of these features are a part of the JEP, but they could be (the JEP is only a proposal). What I would like to see, though is that log rotation and log file naming works the same across all of the JVM. I don't want to have one implementation for the GC and one for the compilers, for example. This 'unification' is partly what the JEP is about. So for this reason, implementing these features should be done in the context of the JEP. Thanks, /Staffan On 14 aug 2012, at 11:20, Yasumasa Suenaga wrote: > Hi Staffan, > > May I ask you 2 questions about this JEP? > > 1. One of goals of this JEP is defined as following: > "File rotation of log files by size (similar to what is available for GC logs today)" > My patch realizes log rotation by external trigger. > 7090324 is included in this JEP? > (Is 7090324 included in "Logging can be configured dynamically at runtime via jcmd or MBeans" ? ) > > 2. I've posted a patch for "6950794: Make the GC log file name parameterized" . > http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2012-July/004758.html > Could you include this RFE in this JEP? > If we use log rotation, I think that name of logs is very important for log management. > > > Thanks, > > Yasumasa > > > On 2012/08/14 16:50, Staffan Larsen wrote: >> Hi Yasumasa, >> >> This work should be done in the context of the Unified Logging JEP [1]. Unfortunately, that work has not yet started due to resource constraints. Comments on the proposal are welcome. >> >> [1] http://openjdk.java.net/jeps/158 >> >> /Staffan >> >> On 31 jul 2012, at 12:45, Yasumasa Suenaga > wrote: >> >>> Hi, >>> >>> I've posted a patch for gc log rotation via jinfo tool last year. >>> Then I received an email that essence of my patch will be implemented as >>> "subcommands" of jcmd. >>> >>> Now, jcmd is provided by OpenJDK. However, current jcmd does not seem to >>> implement function of gclog rotation. >>> >>> Will this RFE be implemented? >>> >>> I need to rotate gclog which is synchronized with signal, cron, etc... on Linux. >>> So, if this RFE is not implemented for a while, I would like to contribute >>> patch for jcmd. >>> >>> >>> Regards, >>> >>> Yasumasa >>> >>> >>> On 2011/09/29 21:15, James Melvin wrote: >>>> Hi Yasumasa, >>>> >>>> Thanks very much for your OpenJDK contribution! As part of the effort to >>>> port JRockit features to HotSpot, we plan to introduce a consolidated >>>> commandline serviceability tool (jcmd) to potentially replace many of >>>> the existing tools in the bin directory, such as jmap, jstack, jinfo and >>>> others. Over the next few update releases, we plan to add several jcmd >>>> *subcommands* instead to accomplish specific tasks or affect the running >>>> JVM instance. We'd like to use the essence of your contribution in one >>>> of the jcmd subcommands (instead of extending jinfo) to ask the JVM to >>>> begin rotating GC logs. We hope you find this attractive and hope you >>>> will help review and perfect the change. >>>> >>>> Thanks, >>>> >>>> Jim Melvin >>>> Sr. Engineering Manager >>>> Oracle >>>> >>>> >>>> >>>> On 9/26/11 6:38 AM, Yasumasa Suenaga wrote: >>>>> (I've changed subject of this email to new RFE.) >>>>> >>>>> This RFE is enhancement of current gclog implementation. >>>>> So, I'd like to discuss about rotating gclog. >>>>> >>>>> My customers need gclog rotation which is invoked by external trigger. >>>>> So I've requested this RFE and made a patch. >>>>> >>>>> >>>>> In many case on Linux, logfile is rotated by signal (e.g. SIGHUP) . >>>>> The aim of this RFE is to synchronize gclog and the other logs. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> (2011/09/22 20:55), Rainer Jung wrote: >>>>>> On 22.09.2011 13:20, Dmitry Samersoff wrote: >>>>>>> >>>>>>> Yasumasa, >>>>>>> >>>>>>> On 2011-09-22 04:47, Yasumasa Suenaga wrote: >>>>>>>> If we can think Java on Linux and Solaris only, syslog is best >>>>>>>> solution. >>>>>>>> However, Windows usually doesn't have syslog. >>>>>>>> >>>>>>>> So, I think that gclog is needed for logging GC stats with platform >>>>>>>> independent in realtime. >>>>>>> >>>>>>> Windows has it's own logging API as reach as syslog is or ever better >>>>>>> as well as numerous syslog implementations. >>>>>>> >>>>>>> Native windows logging API was completely redesigned for Windows 2008 >>>>>>> server and now it allows for developers to send a structured events from >>>>>>> theirs application. >>>>>> >>>>>> AFAIK log rotation for loggc is already implemented though not >>>>>> necessarily yet released. The change discussed here is about supporting >>>>>> an externally generated rotation trigger, e.g. via a signal, instead of >>>>>> only rotating by size or time via a startup parameter. >>>>>> >>>>>> If you want support for syslog or Windows APIs included, it would be >>>>>> best to start a new discussion. >>>>>> >>>>>> A GC log for an application under load might easily produce a block of >>>>>> about 1.5 KB size every few seconds. I seriously doubt, that the need >>>>>> for log file rotation can be argued away by referring to syslog or >>>>>> Windows log API as the correct solution. >>>>>> >>>>>> The messages are not really line formatted, the format can vary a lot >>>>>> (depending on the excat XX switches), the traffic can be quite high and >>>>>> AFAIK the JVM writes it synchronously, so there must be absolutely no >>>>>> risk in writing it out with very little latency. In addition, for >>>>>> analysis, you wouldn't want to look at each event individually, but >>>>>> instead process the whole file through a script or tool, which should >>>>>> not depend on the output specifics of a platform specific log apparatus. >>>>>> >>>>>> Regards, >>>>>> >>>>>> Rainer >>>>>> >>>>> >>> >> > From staffan.larsen at oracle.com Fri Aug 17 20:20:00 2012 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 17 Aug 2012 22:20:00 +0200 Subject: JEP 158: Unified JVM logging In-Reply-To: <502BB003.8020701@oracle.com> References: <4E6D5DEC.8090109@oss.ntt.co.jp> <4E6DDB8F.4020301@oracle.com><3568943A-E6A4-4287-8319-8B0993F070F9@oracle.com><4E7A85AE.10507@oss.ntt.co.jp> <4E7B19EC.5070700@oracle.com> <4E7B2227.9070406@kippdata.de> <4E805629.7030703@oss.ntt.co.jp> <4E846158.3020606@oracle.com> <5017B761.5070203@lab.ntt.co.jp> <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> <502A1853.70508@lab.ntt.co.jp> <502A25B0.20004@oracle.com> <502BB003.8020701@oracle.com> Message-ID: <2FF50E25-8A2C-4837-8C46-44E8C5CD6180@oracle.com> On 15 aug 2012, at 16:19, Dmitry Samersoff wrote: > On 2012-08-15 12:44, Kirk Pepperdine wrote: > >> The current system of command-line (experimental) options follow a >> format that is quite consistent. The options provide semantic meaning as >> to what will be logged. This semantic meaning is lost when we reduce >> everything to the level of error, warning, info, debug and trace. > > I guess Levels are independent to categories. i.e. I think > > Log:info,gc,younggen should be possible The suggestion in the JEP is that each category/component has several levels, so in that sense they are not independent. To enable logging you would specify a list of {component, level} tuples, like: gc:info, younggen:trace But to make it easier to type in the most common case the 'info' level is considered 'default' if nothing else is specified. So the above would be equivalent to saying just gc, younggen:trace Thanks, /Staffan From alejandro.murillo at oracle.com Sat Aug 18 00:28:21 2012 From: alejandro.murillo at oracle.com (alejandro.murillo at oracle.com) Date: Sat, 18 Aug 2012 00:28:21 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 13 new changesets Message-ID: <20120818002852.DEB3D475B8@hg.openjdk.java.net> Changeset: abc951e44e1b Author: katleman Date: 2012-08-09 18:00 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/abc951e44e1b Added tag jdk8-b51 for changeset 663fc23da8d5 ! .hgtags Changeset: 1d7922586cf6 Author: twisti Date: 2012-07-24 10:51 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/1d7922586cf6 7023639: JSR 292 method handle invocation needs a fast path for compiled code 6984705: JSR 292 method handle creation should not go through JNI Summary: remove assembly code for JDK 7 chained method handles Reviewed-by: jrose, twisti, kvn, mhaupt Contributed-by: John Rose , Christian Thalinger , Michael Haupt ! agent/src/share/classes/sun/jvm/hotspot/code/CodeBlob.java ! agent/src/share/classes/sun/jvm/hotspot/code/CodeCache.java - agent/src/share/classes/sun/jvm/hotspot/code/RicochetBlob.java ! agent/src/share/classes/sun/jvm/hotspot/runtime/Frame.java ! agent/src/share/classes/sun/jvm/hotspot/runtime/VM.java ! agent/src/share/classes/sun/jvm/hotspot/runtime/sparc/SPARCFrame.java - agent/src/share/classes/sun/jvm/hotspot/runtime/sparc/SPARCRicochetFrame.java ! agent/src/share/classes/sun/jvm/hotspot/runtime/x86/X86Frame.java - agent/src/share/classes/sun/jvm/hotspot/runtime/x86/X86RicochetFrame.java ! make/solaris/makefiles/fastdebug.make ! make/solaris/makefiles/optimized.make ! make/solaris/makefiles/product.make ! src/cpu/sparc/vm/assembler_sparc.cpp ! src/cpu/sparc/vm/assembler_sparc.hpp ! src/cpu/sparc/vm/c1_LIRAssembler_sparc.cpp ! src/cpu/sparc/vm/cppInterpreter_sparc.cpp ! src/cpu/sparc/vm/frame_sparc.cpp ! src/cpu/sparc/vm/interp_masm_sparc.cpp ! src/cpu/sparc/vm/interpreterGenerator_sparc.hpp ! src/cpu/sparc/vm/interpreter_sparc.cpp ! src/cpu/sparc/vm/methodHandles_sparc.cpp ! src/cpu/sparc/vm/methodHandles_sparc.hpp ! src/cpu/sparc/vm/sharedRuntime_sparc.cpp ! src/cpu/sparc/vm/stubGenerator_sparc.cpp ! src/cpu/sparc/vm/templateInterpreter_sparc.cpp ! src/cpu/sparc/vm/templateTable_sparc.cpp ! src/cpu/sparc/vm/templateTable_sparc.hpp ! src/cpu/sparc/vm/vtableStubs_sparc.cpp ! src/cpu/x86/vm/assembler_x86.cpp ! src/cpu/x86/vm/assembler_x86.hpp ! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp ! src/cpu/x86/vm/cppInterpreter_x86.cpp ! src/cpu/x86/vm/frame_x86.cpp ! src/cpu/x86/vm/interp_masm_x86_32.cpp ! src/cpu/x86/vm/interp_masm_x86_64.cpp ! src/cpu/x86/vm/interpreterGenerator_x86.hpp ! src/cpu/x86/vm/interpreter_x86_32.cpp ! src/cpu/x86/vm/interpreter_x86_64.cpp ! src/cpu/x86/vm/methodHandles_x86.cpp ! src/cpu/x86/vm/methodHandles_x86.hpp ! src/cpu/x86/vm/sharedRuntime_x86_32.cpp ! src/cpu/x86/vm/sharedRuntime_x86_64.cpp ! src/cpu/x86/vm/stubGenerator_x86_32.cpp ! src/cpu/x86/vm/stubGenerator_x86_64.cpp ! src/cpu/x86/vm/templateInterpreter_x86_32.cpp ! src/cpu/x86/vm/templateInterpreter_x86_64.cpp ! src/cpu/x86/vm/templateTable_x86_32.cpp ! src/cpu/x86/vm/templateTable_x86_32.hpp ! src/cpu/x86/vm/templateTable_x86_64.cpp ! src/cpu/x86/vm/templateTable_x86_64.hpp ! src/cpu/x86/vm/vtableStubs_x86_32.cpp ! src/cpu/x86/vm/vtableStubs_x86_64.cpp ! src/cpu/zero/vm/cppInterpreter_zero.cpp ! src/cpu/zero/vm/interpreterGenerator_zero.hpp ! src/cpu/zero/vm/interpreter_zero.cpp ! src/share/vm/adlc/output_h.cpp ! src/share/vm/asm/assembler.cpp ! src/share/vm/asm/assembler.hpp ! src/share/vm/asm/register.hpp ! src/share/vm/c1/c1_Canonicalizer.cpp ! src/share/vm/c1/c1_Canonicalizer.hpp ! src/share/vm/c1/c1_Compilation.cpp ! src/share/vm/c1/c1_FrameMap.cpp ! src/share/vm/c1/c1_FrameMap.hpp ! src/share/vm/c1/c1_GraphBuilder.cpp ! src/share/vm/c1/c1_GraphBuilder.hpp ! src/share/vm/c1/c1_Instruction.cpp ! src/share/vm/c1/c1_Instruction.hpp ! src/share/vm/c1/c1_InstructionPrinter.cpp ! src/share/vm/c1/c1_InstructionPrinter.hpp ! src/share/vm/c1/c1_LIR.hpp ! src/share/vm/c1/c1_LIRGenerator.cpp ! src/share/vm/c1/c1_LIRGenerator.hpp ! src/share/vm/c1/c1_Optimizer.cpp ! src/share/vm/c1/c1_ValueMap.hpp ! src/share/vm/c1/c1_ValueStack.cpp ! src/share/vm/c1/c1_ValueStack.hpp ! src/share/vm/c1/c1_ValueType.cpp ! src/share/vm/c1/c1_ValueType.hpp ! src/share/vm/ci/bcEscapeAnalyzer.cpp ! src/share/vm/ci/ciClassList.hpp ! src/share/vm/ci/ciEnv.cpp + src/share/vm/ci/ciMemberName.cpp + src/share/vm/ci/ciMemberName.hpp ! src/share/vm/ci/ciMethod.cpp ! src/share/vm/ci/ciMethod.hpp ! src/share/vm/ci/ciMethodHandle.cpp ! src/share/vm/ci/ciMethodHandle.hpp ! src/share/vm/ci/ciObject.hpp ! src/share/vm/ci/ciObjectFactory.cpp ! src/share/vm/ci/ciSignature.hpp ! src/share/vm/ci/ciStreams.cpp ! src/share/vm/ci/ciStreams.hpp ! src/share/vm/ci/ciSymbol.cpp ! src/share/vm/ci/ciSymbol.hpp ! src/share/vm/ci/ciTypeFlow.cpp ! src/share/vm/classfile/classFileParser.cpp ! src/share/vm/classfile/classFileParser.hpp ! src/share/vm/classfile/javaClasses.cpp ! src/share/vm/classfile/javaClasses.hpp ! src/share/vm/classfile/symbolTable.hpp ! src/share/vm/classfile/systemDictionary.cpp ! src/share/vm/classfile/systemDictionary.hpp ! src/share/vm/classfile/vmSymbols.cpp ! src/share/vm/classfile/vmSymbols.hpp ! src/share/vm/code/codeBlob.cpp ! src/share/vm/code/codeBlob.hpp ! src/share/vm/code/codeCache.cpp ! src/share/vm/code/debugInfoRec.cpp ! src/share/vm/code/nmethod.cpp ! src/share/vm/code/vtableStubs.hpp ! src/share/vm/compiler/compileBroker.cpp ! src/share/vm/compiler/compileBroker.hpp ! src/share/vm/interpreter/abstractInterpreter.hpp ! src/share/vm/interpreter/bytecode.cpp ! src/share/vm/interpreter/bytecode.hpp ! src/share/vm/interpreter/bytecodeInterpreter.cpp ! src/share/vm/interpreter/bytecodes.cpp ! src/share/vm/interpreter/bytecodes.hpp ! src/share/vm/interpreter/interpreter.cpp ! src/share/vm/interpreter/interpreterRuntime.cpp ! src/share/vm/interpreter/interpreterRuntime.hpp ! src/share/vm/interpreter/linkResolver.cpp ! src/share/vm/interpreter/linkResolver.hpp ! src/share/vm/interpreter/rewriter.cpp ! src/share/vm/interpreter/rewriter.hpp ! src/share/vm/interpreter/templateInterpreter.cpp ! src/share/vm/interpreter/templateTable.cpp ! src/share/vm/interpreter/templateTable.hpp ! src/share/vm/oops/constantPoolOop.cpp ! src/share/vm/oops/constantPoolOop.hpp ! src/share/vm/oops/cpCacheOop.cpp ! src/share/vm/oops/cpCacheOop.hpp ! src/share/vm/oops/generateOopMap.cpp ! src/share/vm/oops/instanceKlass.cpp ! src/share/vm/oops/methodKlass.cpp ! src/share/vm/oops/methodOop.cpp ! src/share/vm/oops/methodOop.hpp ! src/share/vm/oops/symbol.cpp ! src/share/vm/opto/bytecodeInfo.cpp ! src/share/vm/opto/callGenerator.cpp ! src/share/vm/opto/callGenerator.hpp ! src/share/vm/opto/callnode.cpp ! src/share/vm/opto/callnode.hpp ! src/share/vm/opto/doCall.cpp ! src/share/vm/opto/escape.cpp ! src/share/vm/opto/escape.hpp ! src/share/vm/opto/graphKit.cpp ! src/share/vm/opto/graphKit.hpp ! src/share/vm/opto/library_call.cpp ! src/share/vm/opto/matcher.cpp ! src/share/vm/opto/node.hpp ! src/share/vm/opto/parse.hpp ! src/share/vm/opto/parse1.cpp ! src/share/vm/opto/phaseX.hpp ! src/share/vm/prims/jvmtiTagMap.cpp - src/share/vm/prims/methodHandleWalk.cpp - src/share/vm/prims/methodHandleWalk.hpp ! src/share/vm/prims/methodHandles.cpp ! src/share/vm/prims/methodHandles.hpp ! src/share/vm/prims/unsafe.cpp ! src/share/vm/runtime/arguments.cpp ! src/share/vm/runtime/deoptimization.cpp ! src/share/vm/runtime/fieldDescriptor.hpp ! src/share/vm/runtime/fprofiler.cpp ! src/share/vm/runtime/frame.cpp ! src/share/vm/runtime/frame.hpp ! src/share/vm/runtime/globals.hpp ! src/share/vm/runtime/os.cpp ! src/share/vm/runtime/reflection.cpp ! src/share/vm/runtime/sharedRuntime.cpp ! src/share/vm/runtime/sharedRuntime.hpp ! src/share/vm/runtime/signature.hpp ! src/share/vm/runtime/stubRoutines.cpp ! src/share/vm/runtime/stubRoutines.hpp ! src/share/vm/runtime/vframe.cpp ! src/share/vm/runtime/vframeArray.cpp ! src/share/vm/runtime/vmStructs.cpp ! src/share/vm/services/heapDumper.cpp ! src/share/vm/utilities/accessFlags.hpp ! src/share/vm/utilities/exceptions.hpp Changeset: 977007096840 Author: twisti Date: 2012-07-27 16:14 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/977007096840 7187290: nightly failures after JSR 292 lazy method handle update Reviewed-by: kvn, twisti ! src/share/vm/c1/c1_GraphBuilder.cpp ! src/share/vm/classfile/javaClasses.cpp ! src/share/vm/oops/methodOop.cpp ! src/share/vm/opto/doCall.cpp Changeset: 6c5b7a6becc8 Author: kvn Date: 2012-07-30 09:49 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/6c5b7a6becc8 7187454: stack overflow in C2 compiler thread on Solaris x86 Summary: Added new FormatBufferResource class to use thread's resource area for error message buffer. Reviewed-by: twisti ! make/solaris/makefiles/fastdebug.make ! src/share/vm/opto/callGenerator.cpp ! src/share/vm/opto/chaitin.cpp ! src/share/vm/opto/compile.cpp ! src/share/vm/opto/doCall.cpp ! src/share/vm/opto/escape.cpp ! src/share/vm/opto/idealGraphPrinter.cpp ! src/share/vm/opto/node.hpp ! src/share/vm/opto/parse1.cpp ! src/share/vm/utilities/debug.cpp ! src/share/vm/utilities/debug.hpp Changeset: 8cb110fd7627 Author: kvn Date: 2012-08-01 14:10 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/8cb110fd7627 7188227: VM should recognize M-series SPARC Summary: Check kstat data for SPARC-M. Reviewed-by: roland ! src/cpu/sparc/vm/vm_version_sparc.hpp ! src/os_cpu/solaris_sparc/vm/vm_version_solaris_sparc.cpp Changeset: b72784e722ff Author: twisti Date: 2012-08-01 14:44 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/b72784e722ff 7188276: JSR 292: assert(ct == T_OBJECT) failed: rt=T_OBJECT, ct=13 Reviewed-by: kvn, jrose ! src/share/vm/opto/doCall.cpp Changeset: 93c71eb28866 Author: twisti Date: 2012-08-07 14:32 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/93c71eb28866 7188911: nightly failures after JSR 292 lazy method handle update (round 2) Reviewed-by: kvn, jrose ! src/share/vm/classfile/verifier.cpp ! src/share/vm/classfile/vmSymbols.hpp ! src/share/vm/interpreter/linkResolver.cpp ! src/share/vm/oops/methodOop.hpp ! src/share/vm/prims/jvm.cpp ! src/share/vm/prims/methodHandles.cpp ! src/share/vm/prims/nativeLookup.cpp ! src/share/vm/runtime/sharedRuntime.cpp ! src/share/vm/runtime/sharedRuntime.hpp ! src/share/vm/utilities/exceptions.cpp ! src/share/vm/utilities/exceptions.hpp Changeset: ee7edf31f688 Author: twisti Date: 2012-08-10 15:50 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/ee7edf31f688 Merge - agent/src/share/classes/sun/jvm/hotspot/code/RicochetBlob.java - agent/src/share/classes/sun/jvm/hotspot/runtime/sparc/SPARCRicochetFrame.java - agent/src/share/classes/sun/jvm/hotspot/runtime/x86/X86RicochetFrame.java ! make/solaris/makefiles/fastdebug.make ! src/share/vm/opto/phaseX.hpp - src/share/vm/prims/methodHandleWalk.cpp - src/share/vm/prims/methodHandleWalk.hpp Changeset: 4c8f2a12e757 Author: twisti Date: 2012-08-10 17:50 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/4c8f2a12e757 Merge Changeset: 6d0436885201 Author: amurillo Date: 2012-08-10 23:07 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/6d0436885201 Added tag hs24-b20 for changeset 4c8f2a12e757 ! .hgtags Changeset: 6898d85cf0bb Author: amurillo Date: 2012-08-10 23:19 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/6898d85cf0bb 7190772: new hotspot build - hs24-b21 Reviewed-by: jcoomes ! make/hotspot_version Changeset: d5ec46c7da5c Author: amurillo Date: 2012-08-15 16:49 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/d5ec46c7da5c 7191765: make jdk8 the default jprt release for hs24 Reviewed-by: jcoomes ! make/jprt.properties Changeset: 3958f0acde31 Author: amurillo Date: 2012-08-17 15:41 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/3958f0acde31 Merge ! make/jprt.properties - src/share/vm/gc_implementation/parNew/parGCAllocBuffer.cpp - src/share/vm/gc_implementation/parNew/parGCAllocBuffer.hpp From bengt.rutisson at oracle.com Mon Aug 20 08:20:55 2012 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 20 Aug 2012 10:20:55 +0200 Subject: RFR(S): 7192128: G1: Extend fix for 6948537 to G1's BOT In-Reply-To: <502EB35A.2080106@oracle.com> References: <502D7DBF.50804@oracle.com> <502DE253.9030002@oracle.com> <502E955B.1050400@oracle.com> <502EB35A.2080106@oracle.com> Message-ID: <5031F367.3060608@oracle.com> Hi John, "VM_Version_init gets called twice on sparc." I didn't dig any deeper into that, but it does sound like something that should be cleaned up. But not as part of your changes. Going back to your original code is fine. I'm also fine with leaving the other duplicated code until we remove the G1 BOT implementation. So, again: Ship it! :) Bengt On 2012-08-17 23:10, John Cuthbertson wrote: > Hi Bengt, > > I believe I'll have to leave the warnings being emitted from > g1CollectedHeap.cpp and concurrentMarkSweepGeneration.cpp. Moving the > warning into VM_Version::initialize() causes the warning to come out > twice: > > dr-evil{jcuthber}:293> ./gamma -XX:+UnlockExperimentalVMOptions > -XX:+UseG1GC -XX:+UseMemSetInBOT -version > Using java runtime at: > /net/jre.us.oracle.com/p/v42/jdk/7/fcs/b147/binaries/solaris-sparcv9/jre > Java HotSpot(TM) 64-Bit Server VM warning: Experimental flag > -XX:+UseMemSetInBOT is known to cause instability on sun4v; please > understand that you are using at your own risk! > Java HotSpot(TM) 64-Bit Server VM warning: Experimental flag > -XX:+UseMemSetInBOT is known to cause instability on sun4v; please > understand that you are using at your own risk! > java version "1.7.0" > Java(TM) SE Runtime Environment (build 1.7.0-b147) > Java HotSpot(TM) 64-Bit Server VM (build 24.0-b20-internal-jvmg, mixed > mode) > > It appears we are calling VM_Version::initialize() twice (for whatever > reason). Once from Arguments::check_vm_args_consistency(): > > // Check the consistency of vm_init_args > bool Arguments::check_vm_args_consistency() { > // Method for adding checks for flag consistency. > // The intent is to warn the user of all possible conflicts, > // before returning an error. > // Note: Needs platform-dependent factoring. > bool status = true; > > #if ( (defined(COMPILER2) && defined(SPARC))) > // NOTE: The call to VM_Version_init depends on the fact that > VM_Version_init > // on sparc doesn't require generation of a stub as is the case on, > e.g., > // x86. Normally, VM_Version_init must be called from init_globals in > // init.cpp, which is called by the initial java thread *after* > arguments > // have been parsed. VM_Version_init gets called twice on sparc. > extern void VM_Version_init(); > VM_Version_init(); > if (!VM_Version::has_v9()) { > jio_fprintf(defaultStream::error_stream(), > "V8 Machine detected, Server requires V9\n"); > status = false; > } > #endif /* COMPILER2 && SPARC */ > > So I'll go back to the original code. > > JohnC > > On 08/17/12 12:02, John Cuthbertson wrote: >> Hi Bengt, >> >> Thanks for reviewing the code so quickly. >> >> >> On 08/16/12 23:18, Bengt Rutisson wrote: >>> >>> Hi John, >>> >>> Great that you found this issue and fixed it so quickly! >>> >>> I think the change looks good. >>> >>> An optional change that you can do if you would like to is: >>> >>> The G1CollectedHeap constructor now has this code: >>> >>> #ifdef SPARC >>> // Issue a stern warning, but allow use for experimentation and >>> debugging. >>> if (VM_Version::is_sun4v() && UseMemSetInBOT) { >>> assert(!FLAG_IS_DEFAULT(UseMemSetInBOT), "Error"); >>> warning("Experimental flag -XX:+UseMemSetInBOT is known to cause >>> instability" >>> " on sun4v; please understand that you are using at your >>> own risk!"); >>> } >>> #endif >>> >>> Which is duplicated code from the CMSCollector constructor. Could we >>> move it to a common place? Maybe even include it in >>> vm_version_sparc.cpp -> VM_Version::initialize() where we have the >>> other special treatment of UseMemSetInBOT for CMS and G1? Something >>> like: >>> >>> if (is_niagara()) { >>> ... >>> if (UseMemSetInBOT && (UseConcMarkSweepGC || UseG1GC)) { >>> if (FLAG_IS_DEFAULT(UseMemSetInBOT) { >>> FLAG_SET_DEFAULT(UseMemSetInBOT, false); >>> } else { >>> warning("Experimental flag -XX:+UseMemSetInBOT is known to >>> cause instability" >>> " on sun4v; please understand that you are using at >>> your own risk!"); >>> } >>> } >>> ... >>> } >>> >> >> This is a great idea. Done. >> >>> It would also have been nice to be able have a wrapper method for >>> the 4 places where we now have this type of code: >>> >>> if (UseMemSetInBOT) { >>> memset(&_offset_array[index_for(left)], offset, num_cards); >>> } else { >>> size_t i = index_for(left); >>> const size_t end = i + num_cards; >>> for (; i < end; i++) { >>> _offset_array[i] = offset; >>> } >>> } >>> >>> But I'm fine with leaving it as it is for now. >>> >>> >> I would prefer to leave it as it is for now. We do have a CR to >> reconcile G1's BOT with the that of the other collectors - I think >> that is when we can introduce the wrapper (and it would be back down >> to only two instances). I'll add a comment to that CR. >> >> JohnC >> > From staffan.larsen at oracle.com Sun Aug 19 19:52:52 2012 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Sun, 19 Aug 2012 21:52:52 +0200 Subject: JEP 158: Unified JVM logging In-Reply-To: <325CA5D7-13B4-4A72-ADA2-C7BFAA9F1353@kodewerk.com> References: <4E6D5DEC.8090109@oss.ntt.co.jp> <4E6DDB8F.4020301@oracle.com><3568943A-E6A4-4287-8319-8B0993F070F9@oracle.com><4E7A85AE.10507@oss.ntt.co.jp> <4E7B19EC.5070700@oracle.com> <4E7B2227.9070406@kippdata.de> <4E805629.7030703@oss.ntt.co.jp> <4E846158.3020606@oracle.com> <5017B761.5070203@lab.ntt.co.jp> <5F87EF2F-6D5D-41E3-A778-8692BA2B838A@oracle.com> <502A1853.70508@lab.ntt.co.jp> <502A25B0.20004@oracle.com> <502BB003.8020701@oracle.com> <2FF50E25-8A2C-4837-8C46-44E8C5CD6180@oracle.com> <325CA5D7-13B4-4A72-ADA2-C7BFAA9F1353@kodewerk.com> Message-ID: <026A78FD-9F3C-4332-8621-24BFF012B065@oracle.com> On 17 aug 2012, at 22:32, Kirk Pepperdine wrote: >> >> The suggestion in the JEP is that each category/component has several levels, so in that sense they are not independent. To enable logging you would specify a list of {component, level} tuples, like: >> >> gc:info, younggen:trace >> >> But to make it easier to type in the most common case the 'info' level is considered 'default' if nothing else is specified. So the above would be equivalent to saying just >> >> gc, younggen:trace > > Right, and this is at the heart of one of my objections is that the predefined levels used by everyone lack semantic meaning where as being able to define tags or groups of tags would allow one to define levels but not force one to use levels. There certainly is a case for levels and I would not want them precluded but I do not believe it is desirable for all components to hierarchically reduce log messages. > > If I had a better idea of the schedule for the JEP I'd have a better understanding of when I might be able to pitch in rather then just play seagull. The schedule is vague at the best. Originally we thought we could do this for jdk 8, but there aren't resources enough to do that. I'm still hoping for jdk 9, but the infrastructure needs to be done early in the jdk 9 schedule to give all groups time to adapt their logging. No resources (within Oracle) has yet been committed to the project. /Staffan From john.cuthbertson at oracle.com Tue Aug 21 16:50:13 2012 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Tue, 21 Aug 2012 09:50:13 -0700 Subject: RFR(S): 7192128: G1: Extend fix for 6948537 to G1's BOT In-Reply-To: <5031F367.3060608@oracle.com> References: <502D7DBF.50804@oracle.com> <502DE253.9030002@oracle.com> <502E955B.1050400@oracle.com> <502EB35A.2080106@oracle.com> <5031F367.3060608@oracle.com> Message-ID: <5033BC45.1090603@oracle.com> Hi Bengt, Thanks again for the review. I spoke to Vladimir about why we call VM_Version::initialize() twice on sparc. The main reason is because SPARC does not have an easy way to get the CPUID and so VM_Version::initialize is called before argument processing to initialize CPU flags like "is_niagara()" etc. JohnC On 08/20/12 01:20, Bengt Rutisson wrote: > > Hi John, > > "VM_Version_init gets called twice on sparc." > > I didn't dig any deeper into that, but it does sound like something > that should be cleaned up. But not as part of your changes. > > Going back to your original code is fine. I'm also fine with leaving > the other duplicated code until we remove the G1 BOT implementation. > > So, again: Ship it! :) > Bengt > > On 2012-08-17 23:10, John Cuthbertson wrote: >> Hi Bengt, >> >> I believe I'll have to leave the warnings being emitted from >> g1CollectedHeap.cpp and concurrentMarkSweepGeneration.cpp. Moving the >> warning into VM_Version::initialize() causes the warning to come out >> twice: >> >> dr-evil{jcuthber}:293> ./gamma -XX:+UnlockExperimentalVMOptions >> -XX:+UseG1GC -XX:+UseMemSetInBOT -version >> Using java runtime at: >> /net/jre.us.oracle.com/p/v42/jdk/7/fcs/b147/binaries/solaris-sparcv9/jre >> Java HotSpot(TM) 64-Bit Server VM warning: Experimental flag >> -XX:+UseMemSetInBOT is known to cause instability on sun4v; please >> understand that you are using at your own risk! >> Java HotSpot(TM) 64-Bit Server VM warning: Experimental flag >> -XX:+UseMemSetInBOT is known to cause instability on sun4v; please >> understand that you are using at your own risk! >> java version "1.7.0" >> Java(TM) SE Runtime Environment (build 1.7.0-b147) >> Java HotSpot(TM) 64-Bit Server VM (build 24.0-b20-internal-jvmg, >> mixed mode) >> >> It appears we are calling VM_Version::initialize() twice (for >> whatever reason). Once from Arguments::check_vm_args_consistency(): >> >> // Check the consistency of vm_init_args >> bool Arguments::check_vm_args_consistency() { >> // Method for adding checks for flag consistency. >> // The intent is to warn the user of all possible conflicts, >> // before returning an error. >> // Note: Needs platform-dependent factoring. >> bool status = true; >> >> #if ( (defined(COMPILER2) && defined(SPARC))) >> // NOTE: The call to VM_Version_init depends on the fact that >> VM_Version_init >> // on sparc doesn't require generation of a stub as is the case on, >> e.g., >> // x86. Normally, VM_Version_init must be called from init_globals in >> // init.cpp, which is called by the initial java thread *after* >> arguments >> // have been parsed. VM_Version_init gets called twice on sparc. >> extern void VM_Version_init(); >> VM_Version_init(); >> if (!VM_Version::has_v9()) { >> jio_fprintf(defaultStream::error_stream(), >> "V8 Machine detected, Server requires V9\n"); >> status = false; >> } >> #endif /* COMPILER2 && SPARC */ >> >> So I'll go back to the original code. >> >> JohnC >> >> On 08/17/12 12:02, John Cuthbertson wrote: >>> Hi Bengt, >>> >>> Thanks for reviewing the code so quickly. >>> >>> >>> On 08/16/12 23:18, Bengt Rutisson wrote: >>>> >>>> Hi John, >>>> >>>> Great that you found this issue and fixed it so quickly! >>>> >>>> I think the change looks good. >>>> >>>> An optional change that you can do if you would like to is: >>>> >>>> The G1CollectedHeap constructor now has this code: >>>> >>>> #ifdef SPARC >>>> // Issue a stern warning, but allow use for experimentation and >>>> debugging. >>>> if (VM_Version::is_sun4v() && UseMemSetInBOT) { >>>> assert(!FLAG_IS_DEFAULT(UseMemSetInBOT), "Error"); >>>> warning("Experimental flag -XX:+UseMemSetInBOT is known to >>>> cause instability" >>>> " on sun4v; please understand that you are using at >>>> your own risk!"); >>>> } >>>> #endif >>>> >>>> Which is duplicated code from the CMSCollector constructor. Could >>>> we move it to a common place? Maybe even include it in >>>> vm_version_sparc.cpp -> VM_Version::initialize() where we have the >>>> other special treatment of UseMemSetInBOT for CMS and G1? Something >>>> like: >>>> >>>> if (is_niagara()) { >>>> ... >>>> if (UseMemSetInBOT && (UseConcMarkSweepGC || UseG1GC)) { >>>> if (FLAG_IS_DEFAULT(UseMemSetInBOT) { >>>> FLAG_SET_DEFAULT(UseMemSetInBOT, false); >>>> } else { >>>> warning("Experimental flag -XX:+UseMemSetInBOT is known to >>>> cause instability" >>>> " on sun4v; please understand that you are using at >>>> your own risk!"); >>>> } >>>> } >>>> ... >>>> } >>>> >>> >>> This is a great idea. Done. >>> >>>> It would also have been nice to be able have a wrapper method for >>>> the 4 places where we now have this type of code: >>>> >>>> if (UseMemSetInBOT) { >>>> memset(&_offset_array[index_for(left)], offset, num_cards); >>>> } else { >>>> size_t i = index_for(left); >>>> const size_t end = i + num_cards; >>>> for (; i < end; i++) { >>>> _offset_array[i] = offset; >>>> } >>>> } >>>> >>>> But I'm fine with leaving it as it is for now. >>>> >>>> >>> I would prefer to leave it as it is for now. We do have a CR to >>> reconcile G1's BOT with the that of the other collectors - I think >>> that is when we can introduce the wrapper (and it would be back down >>> to only two instances). I'll add a comment to that CR. >>> >>> JohnC >>> >> > From john.cuthbertson at oracle.com Tue Aug 21 19:41:02 2012 From: john.cuthbertson at oracle.com (john.cuthbertson at oracle.com) Date: Tue, 21 Aug 2012 19:41:02 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 7192128: G1: Extend fix for 6948537 to G1's BOT Message-ID: <20120821194107.14F004766D@hg.openjdk.java.net> Changeset: f99a36499b8c Author: johnc Date: 2012-08-21 10:05 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/f99a36499b8c 7192128: G1: Extend fix for 6948537 to G1's BOT Summary: G1 does not appear to be immune to the issue described in CR 6948537 and increasing the size of old-generation PLABs appears to increase the liklihood of seeing the issue. Extend the fix for 6948537 to G1's BlockOffsetTable. Reviewed-by: brutisso, jmasa ! src/cpu/sparc/vm/vm_version_sparc.cpp ! src/share/vm/gc_implementation/g1/g1BlockOffsetTable.hpp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp From john.cuthbertson at oracle.com Tue Aug 21 22:42:04 2012 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Tue, 21 Aug 2012 15:42:04 -0700 Subject: RFR(S): 7041879: G1: introduce stress testing parameter to cause frequent evacuation failures In-Reply-To: <502CCFE9.2090101@oracle.com> References: <50085D41.90004@oracle.com> <502CCFE9.2090101@oracle.com> Message-ID: <50340EBC.4070006@oracle.com> Hi Bengt, Thanks for reviewing these changes. Detailed responses are inline.... On 08/16/12 03:48, Bengt Rutisson wrote: > > Hi John, > > Overall I think this looks fine. It's definitely a good stress option > to have. Thanks for fixing this! > > Functionally it looks good to me, but I have some style questions: > > * Why do we add the new methods as inline methods in the > g1CollectedHeap.inline.hpp file? These are only debug methods and they > are only used from g1CollectedHeap.cpp. I would prefer to place the > implementations in g1CollectedHeap.cpp and let the compiler decide > whether to inline or not. I followed the style of PromotionFailureALot. This is speculation... The code would be included in non-product builds (which includes jvmg, fastdebug, and optimized). If we are using fastdebug and/or optimized builds of hotspot to chase down a timing sensitive failure then adding overhead to the fast path of the copy code might affect our attempts to reproduce the failure. I don't think we can assume that all compilers will inline routines that are in the same cpp file (IIRC the Solaris Studio compiler only does it at certain optimization levels _and_ if it has already seen the definition of the callee routine). So by declaring the routines as "inline" and placing them in an include file, we don't need to rely upon the compiler (and the optimization level) to keep the overhead of the code as low as we can. As I said though, this is speculation. > > * In the constructor for G1CollectedHeap I think we could call > reset_evacuation_should_fail() instead of these lines: > > #ifndef PRODUCT > _evacuation_failure_alot_for_current_gc = false; > _evacuation_failure_alot_count = 0; > _evacuation_failure_alot_gc_number = 0; > #endif // !PRODUCT Good observation. Done. > > > * G1CollectedHeap::evacuation_should_fail() > Why do we need evacuation_should_fail(volatile size_t* count)? It is > always called with _evacuation_failure_alot_count as parameter. How > about using that directly instead of passing it as a parameter? Also, > the nesting level is getting three levels deep. We could flatten that > by bailing out early if we are not triggering evacuation failures. So, > I think I would prefer evacuation_should_fail() to look something like: > > bool G1CollectedHeap::evacuation_should_fail() { > if (!G1EvacuationFailureALot || > !_evacuation_failure_alot_for_current_gc) { > return false; > } > // Access to _evacuation_failure_alot_count is not atomic; the value > does not have to be exact. > if (++_evacuation_failure_alot_count >= G1EvacuationFailureALotCount) { > _evacuation_failure_alot_count = 0; > return true; > } > return false; > } > OK. How about also testing the negative of the second condition: inline bool G1CollectedHeap::evacuation_should_fail() { if (!G1EvacuationFailureALot || !_evacuation_failure_alot_for_current_gc) { return false; } // G1EvacuationFailureALot is in effect for current GC // Access to _evacuation_failure_alot_count is not atomic; // the value does not have to be exact. if (++_evacuation_failure_alot_count < G1EvacuationFailureALotCount) { return false; } _evacuation_failure_alot_count = 0; return true; } ? > > * G1CollectedHeap::evacuation_failure_alot_for_gc_type() and > G1CollectedHeap::set_evacuation_failure_alot_for_current_gc() > I am not sure that the method evacuation_failure_alot_for_gc_type() > actually makes the code easier to understand. I think I would prefer > to inline that logic in set_evacuation_failure_alot_for_current_gc(). > Something like: > > void G1CollectedHeap::set_evacuation_failure_alot_for_current_gc() { > if (G1EvacuationFailureALot) { > // Note we can't assert that _evacuation_failure_alot_for_current_gc > // is clear here. It may have been set during a previous GC but > that GC > // did not copy enough objects (i.e. G1EvacuationFailureALotCount) to > // trigger an evacuation failure and clear the flags and and counts. > _evacuation_failure_alot_for_current_gc = false; > > // Check if we have gone over the interval. > const size_t gc_num = total_collections(); > const size_t elapsed_gcs = gc_num - > _evacuation_failure_alot_gc_number; > > if (elapsed_gcs >= G1EvacuationFailureALotInterval) { > const bool gcs_are_young = g1_policy()->gcs_are_young(); > const bool during_im = g1_policy()->during_initial_mark_pause(); > const bool during_marking = mark_in_progress(); > > if (gcs_are_young) { > _evacuation_failure_alot_for_current_gc = > G1EvacuationFailureALotDuringYoungGC > || (during_marking && G1EvacuationFailureALotDuringConcMark) > || (during_im && G1EvacuationFailureALotDuringInitialMark); > } else { > _evacuation_failure_alot_for_current_gc = > G1EvacuationFailureALotDuringMixedGC; > } > } > } I disagree. The conditional statement above is more complicated IMO. In the current version you don't need to know that during_initial_mark will be true iff gcs_are_young is true and so you don't need to encode that in a nested if. It's sufficient say for an initial mark pause - check the value of G1EvacuationFailureALotDuringInitialMark. > > * If we keep any changes in g1CollectedHeap.inline.hpp it should get > its copyright year updated Oops. Thanks. Done. JohnC From john.cuthbertson at oracle.com Tue Aug 21 23:07:38 2012 From: john.cuthbertson at oracle.com (john.cuthbertson at oracle.com) Date: Tue, 21 Aug 2012 23:07:38 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 7185699: G1: Prediction model discrepancies Message-ID: <20120821230743.CE13D47671@hg.openjdk.java.net> Changeset: 7383557659bd Author: johnc Date: 2012-08-21 14:10 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/7383557659bd 7185699: G1: Prediction model discrepancies Summary: Correct the result value of G1CollectedHeap::pending_card_num(). Change the code that calculates the GC efficiency of a non-young heap region to use historical data from mixed GCs and the actual number of live bytes when predicting how long it would take to collect the region. Changes were also reviewed by Thomas Schatzl. Reviewed-by: azeemj, brutisso ! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp ! src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp ! src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp ! src/share/vm/gc_implementation/g1/g1ErgoVerbose.hpp ! src/share/vm/gc_implementation/g1/heapRegion.cpp From bengt.rutisson at oracle.com Wed Aug 22 06:48:23 2012 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 22 Aug 2012 08:48:23 +0200 Subject: RFR(S): 7041879: G1: introduce stress testing parameter to cause frequent evacuation failures In-Reply-To: <50340EBC.4070006@oracle.com> References: <50085D41.90004@oracle.com> <502CCFE9.2090101@oracle.com> <50340EBC.4070006@oracle.com> Message-ID: <503480B7.10008@oracle.com> Hi John, On 2012-08-22 00:42, John Cuthbertson wrote: > Hi Bengt, > > Thanks for reviewing these changes. Detailed responses are inline.... Thanks for your comments! With your proposed changes I think it looks good. Ship it! Bengt > > On 08/16/12 03:48, Bengt Rutisson wrote: >> >> Hi John, >> >> Overall I think this looks fine. It's definitely a good stress option >> to have. Thanks for fixing this! >> >> Functionally it looks good to me, but I have some style questions: >> >> * Why do we add the new methods as inline methods in the >> g1CollectedHeap.inline.hpp file? These are only debug methods and >> they are only used from g1CollectedHeap.cpp. I would prefer to place >> the implementations in g1CollectedHeap.cpp and let the compiler >> decide whether to inline or not. > > I followed the style of PromotionFailureALot. This is speculation... > > The code would be included in non-product builds (which includes jvmg, > fastdebug, and optimized). If we are using fastdebug and/or optimized > builds of hotspot to chase down a timing sensitive failure then adding > overhead to the fast path of the copy code might affect our attempts > to reproduce the failure. I don't think we can assume that all > compilers will inline routines that are in the same cpp file (IIRC the > Solaris Studio compiler only does it at certain optimization levels > _and_ if it has already seen the definition of the callee routine). So > by declaring the routines as "inline" and placing them in an include > file, we don't need to rely upon the compiler (and the optimization > level) to keep the overhead of the code as low as we can. As I said > though, this is speculation. >> >> * In the constructor for G1CollectedHeap I think we could call >> reset_evacuation_should_fail() instead of these lines: >> >> #ifndef PRODUCT >> _evacuation_failure_alot_for_current_gc = false; >> _evacuation_failure_alot_count = 0; >> _evacuation_failure_alot_gc_number = 0; >> #endif // !PRODUCT > > Good observation. Done. > >> >> >> * G1CollectedHeap::evacuation_should_fail() >> Why do we need evacuation_should_fail(volatile size_t* count)? It is >> always called with _evacuation_failure_alot_count as parameter. How >> about using that directly instead of passing it as a parameter? Also, >> the nesting level is getting three levels deep. We could flatten that >> by bailing out early if we are not triggering evacuation failures. >> So, I think I would prefer evacuation_should_fail() to look something >> like: >> >> bool G1CollectedHeap::evacuation_should_fail() { >> if (!G1EvacuationFailureALot || >> !_evacuation_failure_alot_for_current_gc) { >> return false; >> } >> // Access to _evacuation_failure_alot_count is not atomic; the >> value does not have to be exact. >> if (++_evacuation_failure_alot_count >= >> G1EvacuationFailureALotCount) { >> _evacuation_failure_alot_count = 0; >> return true; >> } >> return false; >> } >> > > OK. How about also testing the negative of the second condition: > > inline bool > G1CollectedHeap::evacuation_should_fail() { > if (!G1EvacuationFailureALot || > !_evacuation_failure_alot_for_current_gc) { > return false; > } > // G1EvacuationFailureALot is in effect for current GC > // Access to _evacuation_failure_alot_count is not atomic; > // the value does not have to be exact. > if (++_evacuation_failure_alot_count < G1EvacuationFailureALotCount) { > return false; > } > _evacuation_failure_alot_count = 0; > return true; > } > > ? > >> >> * G1CollectedHeap::evacuation_failure_alot_for_gc_type() and >> G1CollectedHeap::set_evacuation_failure_alot_for_current_gc() >> I am not sure that the method evacuation_failure_alot_for_gc_type() >> actually makes the code easier to understand. I think I would prefer >> to inline that logic in set_evacuation_failure_alot_for_current_gc(). >> Something like: >> >> void G1CollectedHeap::set_evacuation_failure_alot_for_current_gc() { >> if (G1EvacuationFailureALot) { >> // Note we can't assert that _evacuation_failure_alot_for_current_gc >> // is clear here. It may have been set during a previous GC but >> that GC >> // did not copy enough objects (i.e. >> G1EvacuationFailureALotCount) to >> // trigger an evacuation failure and clear the flags and and counts. >> _evacuation_failure_alot_for_current_gc = false; >> >> // Check if we have gone over the interval. >> const size_t gc_num = total_collections(); >> const size_t elapsed_gcs = gc_num - >> _evacuation_failure_alot_gc_number; >> >> if (elapsed_gcs >= G1EvacuationFailureALotInterval) { >> const bool gcs_are_young = g1_policy()->gcs_are_young(); >> const bool during_im = g1_policy()->during_initial_mark_pause(); >> const bool during_marking = mark_in_progress(); >> >> if (gcs_are_young) { >> _evacuation_failure_alot_for_current_gc = >> G1EvacuationFailureALotDuringYoungGC >> || (during_marking && G1EvacuationFailureALotDuringConcMark) >> || (during_im && G1EvacuationFailureALotDuringInitialMark); >> } else { >> _evacuation_failure_alot_for_current_gc = >> G1EvacuationFailureALotDuringMixedGC; >> } >> } >> } > > I disagree. The conditional statement above is more complicated IMO. > In the current version you don't need to know that during_initial_mark > will be true iff gcs_are_young is true and so you don't need to encode > that in a nested if. It's sufficient say for an initial mark pause - > check the value of G1EvacuationFailureALotDuringInitialMark. > >> >> * If we keep any changes in g1CollectedHeap.inline.hpp it should get >> its copyright year updated > > Oops. Thanks. Done. > > JohnC From bengt.rutisson at oracle.com Wed Aug 22 06:57:59 2012 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 22 Aug 2012 08:57:59 +0200 Subject: RFR(S): 7192128: G1: Extend fix for 6948537 to G1's BOT In-Reply-To: <5033BC45.1090603@oracle.com> References: <502D7DBF.50804@oracle.com> <502DE253.9030002@oracle.com> <502E955B.1050400@oracle.com> <502EB35A.2080106@oracle.com> <5031F367.3060608@oracle.com> <5033BC45.1090603@oracle.com> Message-ID: <503482F7.2060504@oracle.com> Hi John, On 2012-08-21 18:50, John Cuthbertson wrote: > Hi Bengt, > > Thanks again for the review. I spoke to Vladimir about why we call > VM_Version::initialize() twice on sparc. The main reason is because > SPARC does not have an easy way to get the CPUID and so > VM_Version::initialize is called before argument processing to > initialize CPU flags like "is_niagara()" etc. Thanks for investigating this further. I still think it would be better to split VM_Version::initialize() up into two methods on Sparc or at least have a guard if it has already been initialized. But that is unrelated to your change. Bengt > > JohnC > > On 08/20/12 01:20, Bengt Rutisson wrote: >> >> Hi John, >> >> "VM_Version_init gets called twice on sparc." >> >> I didn't dig any deeper into that, but it does sound like something >> that should be cleaned up. But not as part of your changes. >> >> Going back to your original code is fine. I'm also fine with leaving >> the other duplicated code until we remove the G1 BOT implementation. >> >> So, again: Ship it! :) >> Bengt >> >> On 2012-08-17 23:10, John Cuthbertson wrote: >>> Hi Bengt, >>> >>> I believe I'll have to leave the warnings being emitted from >>> g1CollectedHeap.cpp and concurrentMarkSweepGeneration.cpp. Moving >>> the warning into VM_Version::initialize() causes the warning to come >>> out twice: >>> >>> dr-evil{jcuthber}:293> ./gamma -XX:+UnlockExperimentalVMOptions >>> -XX:+UseG1GC -XX:+UseMemSetInBOT -version >>> Using java runtime at: >>> /net/jre.us.oracle.com/p/v42/jdk/7/fcs/b147/binaries/solaris-sparcv9/jre >>> >>> Java HotSpot(TM) 64-Bit Server VM warning: Experimental flag >>> -XX:+UseMemSetInBOT is known to cause instability on sun4v; please >>> understand that you are using at your own risk! >>> Java HotSpot(TM) 64-Bit Server VM warning: Experimental flag >>> -XX:+UseMemSetInBOT is known to cause instability on sun4v; please >>> understand that you are using at your own risk! >>> java version "1.7.0" >>> Java(TM) SE Runtime Environment (build 1.7.0-b147) >>> Java HotSpot(TM) 64-Bit Server VM (build 24.0-b20-internal-jvmg, >>> mixed mode) >>> >>> It appears we are calling VM_Version::initialize() twice (for >>> whatever reason). Once from Arguments::check_vm_args_consistency(): >>> >>> // Check the consistency of vm_init_args >>> bool Arguments::check_vm_args_consistency() { >>> // Method for adding checks for flag consistency. >>> // The intent is to warn the user of all possible conflicts, >>> // before returning an error. >>> // Note: Needs platform-dependent factoring. >>> bool status = true; >>> >>> #if ( (defined(COMPILER2) && defined(SPARC))) >>> // NOTE: The call to VM_Version_init depends on the fact that >>> VM_Version_init >>> // on sparc doesn't require generation of a stub as is the case on, >>> e.g., >>> // x86. Normally, VM_Version_init must be called from init_globals in >>> // init.cpp, which is called by the initial java thread *after* >>> arguments >>> // have been parsed. VM_Version_init gets called twice on sparc. >>> extern void VM_Version_init(); >>> VM_Version_init(); >>> if (!VM_Version::has_v9()) { >>> jio_fprintf(defaultStream::error_stream(), >>> "V8 Machine detected, Server requires V9\n"); >>> status = false; >>> } >>> #endif /* COMPILER2 && SPARC */ >>> >>> So I'll go back to the original code. >>> >>> JohnC >>> >>> On 08/17/12 12:02, John Cuthbertson wrote: >>>> Hi Bengt, >>>> >>>> Thanks for reviewing the code so quickly. >>>> >>>> >>>> On 08/16/12 23:18, Bengt Rutisson wrote: >>>>> >>>>> Hi John, >>>>> >>>>> Great that you found this issue and fixed it so quickly! >>>>> >>>>> I think the change looks good. >>>>> >>>>> An optional change that you can do if you would like to is: >>>>> >>>>> The G1CollectedHeap constructor now has this code: >>>>> >>>>> #ifdef SPARC >>>>> // Issue a stern warning, but allow use for experimentation and >>>>> debugging. >>>>> if (VM_Version::is_sun4v() && UseMemSetInBOT) { >>>>> assert(!FLAG_IS_DEFAULT(UseMemSetInBOT), "Error"); >>>>> warning("Experimental flag -XX:+UseMemSetInBOT is known to >>>>> cause instability" >>>>> " on sun4v; please understand that you are using at >>>>> your own risk!"); >>>>> } >>>>> #endif >>>>> >>>>> Which is duplicated code from the CMSCollector constructor. Could >>>>> we move it to a common place? Maybe even include it in >>>>> vm_version_sparc.cpp -> VM_Version::initialize() where we have the >>>>> other special treatment of UseMemSetInBOT for CMS and G1? >>>>> Something like: >>>>> >>>>> if (is_niagara()) { >>>>> ... >>>>> if (UseMemSetInBOT && (UseConcMarkSweepGC || UseG1GC)) { >>>>> if (FLAG_IS_DEFAULT(UseMemSetInBOT) { >>>>> FLAG_SET_DEFAULT(UseMemSetInBOT, false); >>>>> } else { >>>>> warning("Experimental flag -XX:+UseMemSetInBOT is known to >>>>> cause instability" >>>>> " on sun4v; please understand that you are using >>>>> at your own risk!"); >>>>> } >>>>> } >>>>> ... >>>>> } >>>>> >>>> >>>> This is a great idea. Done. >>>> >>>>> It would also have been nice to be able have a wrapper method for >>>>> the 4 places where we now have this type of code: >>>>> >>>>> if (UseMemSetInBOT) { >>>>> memset(&_offset_array[index_for(left)], offset, num_cards); >>>>> } else { >>>>> size_t i = index_for(left); >>>>> const size_t end = i + num_cards; >>>>> for (; i < end; i++) { >>>>> _offset_array[i] = offset; >>>>> } >>>>> } >>>>> >>>>> But I'm fine with leaving it as it is for now. >>>>> >>>>> >>>> I would prefer to leave it as it is for now. We do have a CR to >>>> reconcile G1's BOT with the that of the other collectors - I think >>>> that is when we can introduce the wrapper (and it would be back >>>> down to only two instances). I'll add a comment to that CR. >>>> >>>> JohnC >>>> >>> >> > From bengt.rutisson at oracle.com Wed Aug 22 07:46:19 2012 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 22 Aug 2012 09:46:19 +0200 Subject: Request for review (XXS): 7193157: G1: Make some develpflags available in product builds Message-ID: <50348E4B.50204@oracle.com> Hi all, Could I please have a couple of reviews for this really small fix? http://cr.openjdk.java.net/~brutisso/7193157/webrev.00/ We need to make some of the G1 develop flags available in product builds to allow the performance team to tune G1 properly. We decided to make the flags experimental to start with. After the performance team has evaluated the flags we might make some of them product flags. The current prime candidates for becoming product flags are: G1DefaultMinNewGenPercent G1DefaultMaxNewGenPercent G1OldCSetRegionLiveThresholdPercent Thanks, Bengt From ysr1729 at gmail.com Wed Aug 22 08:21:25 2012 From: ysr1729 at gmail.com (Srinivas Ramakrishna) Date: Wed, 22 Aug 2012 01:21:25 -0700 Subject: Request for review (XXS): 7193157: G1: Make some develpflags available in product builds In-Reply-To: <50348E4B.50204@oracle.com> References: <50348E4B.50204@oracle.com> Message-ID: Looks good to me. I am hoping the perf and gc teams will together also publish a G1 tuning white-paper soon! -- ramki On Wed, Aug 22, 2012 at 12:46 AM, Bengt Rutisson wrote: > > Hi all, > > Could I please have a couple of reviews for this really small fix? > http://cr.openjdk.java.net/~**brutisso/7193157/webrev.00/ > > We need to make some of the G1 develop flags available in product builds > to allow the performance team to tune G1 properly. We decided to make the > flags experimental to start with. After the performance team has evaluated > the flags we might make some of them product flags. The current prime > candidates for becoming product flags are: > > G1DefaultMinNewGenPercent > G1DefaultMaxNewGenPercent > G1OldCSetRegionLiveThresholdPe**rcent > > Thanks, > Bengt > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Wed Aug 22 08:24:11 2012 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 22 Aug 2012 10:24:11 +0200 Subject: Request for review (XXS): 7193157: G1: Make some develpflags available in product builds In-Reply-To: References: <50348E4B.50204@oracle.com> Message-ID: <5034972B.4080702@oracle.com> Thanks for the review, Ramki! You are right about the G1 tuning white-paper. We should definitely try to put something like that together. I think there will be a talk by the performance team about G1 tuning at JavaOne. Maybe that could be the basis for a white-paper? Bengt On 2012-08-22 10:21, Srinivas Ramakrishna wrote: > Looks good to me. I am hoping the perf and gc teams will together also > publish a G1 tuning white-paper soon! > > -- ramki > > On Wed, Aug 22, 2012 at 12:46 AM, Bengt Rutisson > > wrote: > > > Hi all, > > Could I please have a couple of reviews for this really small fix? > http://cr.openjdk.java.net/~brutisso/7193157/webrev.00/ > > > We need to make some of the G1 develop flags available in product > builds to allow the performance team to tune G1 properly. We > decided to make the flags experimental to start with. After the > performance team has evaluated the flags we might make some of > them product flags. The current prime candidates for becoming > product flags are: > > G1DefaultMinNewGenPercent > G1DefaultMaxNewGenPercent > G1OldCSetRegionLiveThresholdPercent > > Thanks, > Bengt > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Wed Aug 22 11:16:31 2012 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 22 Aug 2012 13:16:31 +0200 Subject: Request for review (M): 7178363 G1: Remove the serial code for PrintGCDetails and make it a special case of the parallel code In-Reply-To: <50094C6E.2030108@oracle.com> References: <5006C056.2080007@oracle.com> <5007CCE4.9070001@oracle.com> <50094C6E.2030108@oracle.com> Message-ID: <5034BF8F.7060400@oracle.com> Hi again, Here is an updated webrev based on comments from John Cuthbertson: http://cr.openjdk.java.net/~brutisso/7178363/webrev.02/ Here is a diff compared to the previous webrev: http://cr.openjdk.java.net/~brutisso/7178363/webrev.01-02-diff/ Thanks, Bengt On 2012-07-20 14:17, Bengt Rutisson wrote: > > Hi all, > > Here is an updated webrev for this change: > http://cr.openjdk.java.net/~brutisso/7178363/webrev.01/ > > It turns out the the earlier change for 7178361 had introduced two > more issues: the heap transition information for the PrintGC output > and the output for evacuation failures had both been moved in an > unintended way. The above webrev corrects both of these chagne too. > Thanks John Cuthbertson for pointing me to the evacuation failure output. > > Bengt > > > On 2012-07-19 11:01, Bengt Rutisson wrote: >> >> Hi again, >> >> Just in case anybody already started looking at this: I have updated >> the webrev since I had to make some changes to make it compile with >> the NMT fixes that have just made it into the hotspot-gc repository. >> Those updates were just to make it compile and not really related to >> my change, so I just overwrote the existing webrev. Just use the same >> link as before: >> >> http://cr.openjdk.java.net/~brutisso/7178363/webrev.00/ >> >> Also, if you want to see what the new output looks like I am >> attaching a file called new.txt with an example from running >> SpecJBB2005 with this command line: >> >> -XX:+UseG1GC -XX:ParallelGCThreads=4 -XX:+UnlockExperimentalVMOptions >> -XX:G1LogLevel=finest -XX:+TraceGen0Time -Xms256m -Xmx2G >> >> I am also attaching a file called old.txt with what the output, using >> the same command line, looked like before my change. As you can see >> the differences are what I listed in my earlier email. You will also >> notice that the "old" version has an entry for the SATB filtering, >> even though all the entries are 0 (we didn't do a concurrent cycle so >> there has been no SATB filtering). This was a bug I just introduced >> with my last change (for 7178361), so the new example is more correct. >> >> Thanks, >> Bengt >> >> On 2012-07-18 15:55, Bengt Rutisson wrote: >>> >>> Hi everyone, >>> >>> I would like some reviews of this change: >>> http://cr.openjdk.java.net/~brutisso/7178363/webrev.00/ >>> >>> This removes the special treatment for ParallelGCThreads=0 from the >>> G1 logging. I did keep the log output unchanged for that case. >>> Basically it just has one indentation level less and skips some >>> output. I am not sure this is really necessary since it is really a >>> special case. I'm open to change that special treatment too and just >>> have the same output as for ParallelGCThreads=1. >>> >>> The PrintGCDetails log output should be the same as before with >>> three minor adjustments: >>> >>> - The "Sum" is now not printed for the start and end values for GC >>> workers. This sum does not really make sense to me. >>> >>> - The "(ms)" unit was removed from output that aren't in >>> milliseconds (termination attempts for example). >>> >>> - The average value is now printed as a double for all types. >>> >>> I tried to clean up the code a bit and introduced a separate class, >>> Snippet WorkerDataArray, to keep track of the per thread logging. I >>> also introduced getters and setters to avoid having to make >>> G1CollectorPolicy and TraceGen0TimeData friend classes to >>> G1GCPhaseTimes. >>> >>> Thanks, >>> Bengt >> > From vitalyd at gmail.com Wed Aug 22 12:22:55 2012 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 22 Aug 2012 08:22:55 -0400 Subject: Request for review (M): 7178363 G1: Remove the serial code for PrintGCDetails and make it a special case of the parallel code In-Reply-To: <5034BF8F.7060400@oracle.com> References: <5006C056.2080007@oracle.com> <5007CCE4.9070001@oracle.com> <50094C6E.2030108@oracle.com> <5034BF8F.7060400@oracle.com> Message-ID: Hi Bengt, A few minor comments. In g1CollectedHeap.cpp: 1) probably doesn't matter much in its current form, but would it be better to call os::elapsedTime() right before prepare_verify() so that the timing info captures just the verification time and not printing to tty/gclog and constructing the HandleMark? This is in verify(). 2) it seems a bit weird to have the "guard" parameter in the verify() method. Why wouldn't caller just check the guard and then only call if it passes? Not a big deal though, just stylistic. 3) in verify_before/after(), would it make sense to only record the time if it's not 0.0? I'm thinking that the pointer chasing may possible get a cache miss only to record a 0. Again, pure speculation. 4) is os::elapsedTime() using a monotonic time source? If not, what would happen if you get a negative value in the timing? Looks good otherwise. Thanks Sent from my phone On Aug 22, 2012 7:21 AM, "Bengt Rutisson" wrote: > > Hi again, > > Here is an updated webrev based on comments from John Cuthbertson: > http://cr.openjdk.java.net/~**brutisso/7178363/webrev.02/ > > Here is a diff compared to the previous webrev: > http://cr.openjdk.java.net/~**brutisso/7178363/webrev.01-02-**diff/ > > Thanks, > Bengt > > On 2012-07-20 14:17, Bengt Rutisson wrote: > >> >> Hi all, >> >> Here is an updated webrev for this change: >> http://cr.openjdk.java.net/~**brutisso/7178363/webrev.01/ >> >> It turns out the the earlier change for 7178361 had introduced two more >> issues: the heap transition information for the PrintGC output and the >> output for evacuation failures had both been moved in an unintended way. >> The above webrev corrects both of these chagne too. Thanks John Cuthbertson >> for pointing me to the evacuation failure output. >> >> Bengt >> >> >> On 2012-07-19 11:01, Bengt Rutisson wrote: >> >>> >>> Hi again, >>> >>> Just in case anybody already started looking at this: I have updated the >>> webrev since I had to make some changes to make it compile with the NMT >>> fixes that have just made it into the hotspot-gc repository. Those updates >>> were just to make it compile and not really related to my change, so I just >>> overwrote the existing webrev. Just use the same link as before: >>> >>> http://cr.openjdk.java.net/~**brutisso/7178363/webrev.00/ >>> >>> Also, if you want to see what the new output looks like I am attaching a >>> file called new.txt with an example from running SpecJBB2005 with this >>> command line: >>> >>> -XX:+UseG1GC -XX:ParallelGCThreads=4 -XX:+**UnlockExperimentalVMOptions >>> -XX:G1LogLevel=finest -XX:+TraceGen0Time -Xms256m -Xmx2G >>> >>> I am also attaching a file called old.txt with what the output, using >>> the same command line, looked like before my change. As you can see the >>> differences are what I listed in my earlier email. You will also notice >>> that the "old" version has an entry for the SATB filtering, even though all >>> the entries are 0 (we didn't do a concurrent cycle so there has been no >>> SATB filtering). This was a bug I just introduced with my last change (for >>> 7178361), so the new example is more correct. >>> >>> Thanks, >>> Bengt >>> >>> On 2012-07-18 15:55, Bengt Rutisson wrote: >>> >>>> >>>> Hi everyone, >>>> >>>> I would like some reviews of this change: >>>> http://cr.openjdk.java.net/~**brutisso/7178363/webrev.00/ >>>> >>>> This removes the special treatment for ParallelGCThreads=0 from the G1 >>>> logging. I did keep the log output unchanged for that case. Basically it >>>> just has one indentation level less and skips some output. I am not sure >>>> this is really necessary since it is really a special case. I'm open to >>>> change that special treatment too and just have the same output as for >>>> ParallelGCThreads=1. >>>> >>>> The PrintGCDetails log output should be the same as before with three >>>> minor adjustments: >>>> >>>> - The "Sum" is now not printed for the start and end values for GC >>>> workers. This sum does not really make sense to me. >>>> >>>> - The "(ms)" unit was removed from output that aren't in milliseconds >>>> (termination attempts for example). >>>> >>>> - The average value is now printed as a double for all types. >>>> >>>> I tried to clean up the code a bit and introduced a separate class, >>>> Snippet WorkerDataArray, to keep track of the per thread logging. I also >>>> introduced getters and setters to avoid having to make G1CollectorPolicy >>>> and TraceGen0TimeData friend classes to G1GCPhaseTimes. >>>> >>>> Thanks, >>>> Bengt >>>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vitalyd at gmail.com Wed Aug 22 12:27:07 2012 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 22 Aug 2012 08:27:07 -0400 Subject: Request for review (XXS): 7193157: G1: Make some develpflags available in product builds In-Reply-To: <50348E4B.50204@oracle.com> References: <50348E4B.50204@oracle.com> Message-ID: Looks good. One small typo: experimental(uintx, G1OldCSetRegionLiveThresholdPercent, 90, "Threshold for regions to be added to the collection set. " "Regions with more live bytes that this will not be collected.") The "that this" should be "than this". Cheers Sent from my phone On Aug 22, 2012 3:50 AM, "Bengt Rutisson" wrote: > > Hi all, > > Could I please have a couple of reviews for this really small fix? > http://cr.openjdk.java.net/~**brutisso/7193157/webrev.00/ > > We need to make some of the G1 develop flags available in product builds > to allow the performance team to tune G1 properly. We decided to make the > flags experimental to start with. After the performance team has evaluated > the flags we might make some of them product flags. The current prime > candidates for becoming product flags are: > > G1DefaultMinNewGenPercent > G1DefaultMaxNewGenPercent > G1OldCSetRegionLiveThresholdPe**rcent > > Thanks, > Bengt > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vitalyd at gmail.com Wed Aug 22 13:17:05 2012 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 22 Aug 2012 09:17:05 -0400 Subject: Request for review (M): 7178363 G1: Remove the serial code for PrintGCDetails and make it a special case of the parallel code In-Reply-To: References: <5006C056.2080007@oracle.com> <5007CCE4.9070001@oracle.com> <50094C6E.2030108@oracle.com> <5034BF8F.7060400@oracle.com> Message-ID: I should also say that for #3 writing a 0.0 when it's already that will dirty the cache line. Again, ignore if this isn't an issue in the expected use cases. Sent from my phone On Aug 22, 2012 8:22 AM, "Vitaly Davidovich" wrote: > Hi Bengt, > > A few minor comments. > > In g1CollectedHeap.cpp: > > 1) probably doesn't matter much in its current form, but would it be > better to call os::elapsedTime() right before prepare_verify() so that the > timing info captures just the verification time and not printing to > tty/gclog and constructing the HandleMark? This is in verify(). > > 2) it seems a bit weird to have the "guard" parameter in the verify() > method. Why wouldn't caller just check the guard and then only call if it > passes? Not a big deal though, just stylistic. > > 3) in verify_before/after(), would it make sense to only record the time > if it's not 0.0? I'm thinking that the pointer chasing may possible get a > cache miss only to record a 0. Again, pure speculation. > > 4) is os::elapsedTime() using a monotonic time source? If not, what would > happen if you get a negative value in the timing? > > Looks good otherwise. > > Thanks > > Sent from my phone > On Aug 22, 2012 7:21 AM, "Bengt Rutisson" > wrote: > >> >> Hi again, >> >> Here is an updated webrev based on comments from John Cuthbertson: >> http://cr.openjdk.java.net/~**brutisso/7178363/webrev.02/ >> >> Here is a diff compared to the previous webrev: >> http://cr.openjdk.java.net/~**brutisso/7178363/webrev.01-02-**diff/ >> >> Thanks, >> Bengt >> >> On 2012-07-20 14:17, Bengt Rutisson wrote: >> >>> >>> Hi all, >>> >>> Here is an updated webrev for this change: >>> http://cr.openjdk.java.net/~**brutisso/7178363/webrev.01/ >>> >>> It turns out the the earlier change for 7178361 had introduced two more >>> issues: the heap transition information for the PrintGC output and the >>> output for evacuation failures had both been moved in an unintended way. >>> The above webrev corrects both of these chagne too. Thanks John Cuthbertson >>> for pointing me to the evacuation failure output. >>> >>> Bengt >>> >>> >>> On 2012-07-19 11:01, Bengt Rutisson wrote: >>> >>>> >>>> Hi again, >>>> >>>> Just in case anybody already started looking at this: I have updated >>>> the webrev since I had to make some changes to make it compile with the NMT >>>> fixes that have just made it into the hotspot-gc repository. Those updates >>>> were just to make it compile and not really related to my change, so I just >>>> overwrote the existing webrev. Just use the same link as before: >>>> >>>> http://cr.openjdk.java.net/~**brutisso/7178363/webrev.00/ >>>> >>>> Also, if you want to see what the new output looks like I am attaching >>>> a file called new.txt with an example from running SpecJBB2005 with this >>>> command line: >>>> >>>> -XX:+UseG1GC -XX:ParallelGCThreads=4 -XX:+**UnlockExperimentalVMOptions >>>> -XX:G1LogLevel=finest -XX:+TraceGen0Time -Xms256m -Xmx2G >>>> >>>> I am also attaching a file called old.txt with what the output, using >>>> the same command line, looked like before my change. As you can see the >>>> differences are what I listed in my earlier email. You will also notice >>>> that the "old" version has an entry for the SATB filtering, even though all >>>> the entries are 0 (we didn't do a concurrent cycle so there has been no >>>> SATB filtering). This was a bug I just introduced with my last change (for >>>> 7178361), so the new example is more correct. >>>> >>>> Thanks, >>>> Bengt >>>> >>>> On 2012-07-18 15:55, Bengt Rutisson wrote: >>>> >>>>> >>>>> Hi everyone, >>>>> >>>>> I would like some reviews of this change: >>>>> http://cr.openjdk.java.net/~**brutisso/7178363/webrev.00/ >>>>> >>>>> This removes the special treatment for ParallelGCThreads=0 from the G1 >>>>> logging. I did keep the log output unchanged for that case. Basically it >>>>> just has one indentation level less and skips some output. I am not sure >>>>> this is really necessary since it is really a special case. I'm open to >>>>> change that special treatment too and just have the same output as for >>>>> ParallelGCThreads=1. >>>>> >>>>> The PrintGCDetails log output should be the same as before with three >>>>> minor adjustments: >>>>> >>>>> - The "Sum" is now not printed for the start and end values for GC >>>>> workers. This sum does not really make sense to me. >>>>> >>>>> - The "(ms)" unit was removed from output that aren't in milliseconds >>>>> (termination attempts for example). >>>>> >>>>> - The average value is now printed as a double for all types. >>>>> >>>>> I tried to clean up the code a bit and introduced a separate class, >>>>> Snippet WorkerDataArray, to keep track of the per thread logging. I also >>>>> introduced getters and setters to avoid having to make G1CollectorPolicy >>>>> and TraceGen0TimeData friend classes to G1GCPhaseTimes. >>>>> >>>>> Thanks, >>>>> Bengt >>>>> >>>> >>>> >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Wed Aug 22 13:32:15 2012 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 22 Aug 2012 15:32:15 +0200 Subject: Request for review (XXS): 7193157: G1: Make some develpflags available in product builds In-Reply-To: References: <50348E4B.50204@oracle.com> Message-ID: <5034DF5F.5090902@oracle.com> Hi Vitaly, Thanks for looking at this. On 2012-08-22 14:27, Vitaly Davidovich wrote: > > Looks good. One small typo: > > experimental(uintx, G1OldCSetRegionLiveThresholdPercent, 90, > > "Threshold for regions to be added to the collection set. " > > "Regions with more live bytes that this will not be collected.") > > The "that this" should be "than this". > Absolutely. Fixed. Thanks, Bengt > Cheers > > Sent from my phone > > On Aug 22, 2012 3:50 AM, "Bengt Rutisson" > wrote: > > > Hi all, > > Could I please have a couple of reviews for this really small fix? > http://cr.openjdk.java.net/~brutisso/7193157/webrev.00/ > > > We need to make some of the G1 develop flags available in product > builds to allow the performance team to tune G1 properly. We > decided to make the flags experimental to start with. After the > performance team has evaluated the flags we might make some of > them product flags. The current prime candidates for becoming > product flags are: > > G1DefaultMinNewGenPercent > G1DefaultMaxNewGenPercent > G1OldCSetRegionLiveThresholdPercent > > Thanks, > Bengt > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Wed Aug 22 13:48:26 2012 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 22 Aug 2012 15:48:26 +0200 Subject: Request for review (M): 7178363 G1: Remove the serial code for PrintGCDetails and make it a special case of the parallel code In-Reply-To: References: <5006C056.2080007@oracle.com> <5007CCE4.9070001@oracle.com> <50094C6E.2030108@oracle.com> <5034BF8F.7060400@oracle.com> Message-ID: <5034E32A.9020206@oracle.com> Hi Vitaly, Thanks for looking at this change too! On 2012-08-22 14:22, Vitaly Davidovich wrote: > > Hi Bengt, > > A few minor comments. > > In g1CollectedHeap.cpp: > > 1) probably doesn't matter much in its current form, but would it be > better to call os::elapsedTime() right before prepare_verify() so that > the timing info captures just the verification time and not printing > to tty/gclog and constructing the HandleMark? This is in verify(). > Actually, I think it is good that the extra printing is included in the timing. We want to time the extra cost of turning on the verification, thus we should include the full cost of it. > 2) it seems a bit weird to have the "guard" parameter in the verify() > method. Why wouldn't caller just check the guard and then only call > if it passes? Not a big deal though, just stylistic. > I see your point. As you say, it is just a style question. If I remove the guard parameter to verify() I'd have to add the test to Snippet verify_before_gc() and verify_after_gc() and that kind of re-introduces some of the code duplication that I was aiming to remove. If you feel strongly about it I'll change. > 3) in verify_before/after(), would it make sense to only record the > time if it's not 0.0? I'm thinking that the pointer chasing may > possible get a cache miss only to record a 0. Again, pure speculation. > These values are set for each GC. If we don't set them to 0 somewhere we will get the values from the last GC. By always setting the value there is no risk that we get the wrong values. > 4) is os::elapsedTime() using a monotonic time source? If not, what > would happen if you get a negative value in the timing? > os::elapsedTime() is a monotonic time source. Thanks, Bengt > Looks good otherwise. > > Thanks > > Sent from my phone > > On Aug 22, 2012 7:21 AM, "Bengt Rutisson" > wrote: > > > Hi again, > > Here is an updated webrev based on comments from John Cuthbertson: > http://cr.openjdk.java.net/~brutisso/7178363/webrev.02/ > > > Here is a diff compared to the previous webrev: > http://cr.openjdk.java.net/~brutisso/7178363/webrev.01-02-diff/ > > > Thanks, > Bengt > > On 2012-07-20 14:17, Bengt Rutisson wrote: > > > Hi all, > > Here is an updated webrev for this change: > http://cr.openjdk.java.net/~brutisso/7178363/webrev.01/ > > > It turns out the the earlier change for 7178361 had introduced > two more issues: the heap transition information for the > PrintGC output and the output for evacuation failures had both > been moved in an unintended way. The above webrev corrects > both of these chagne too. Thanks John Cuthbertson for pointing > me to the evacuation failure output. > > Bengt > > > On 2012-07-19 11:01, Bengt Rutisson wrote: > > > Hi again, > > Just in case anybody already started looking at this: I > have updated the webrev since I had to make some changes > to make it compile with the NMT fixes that have just made > it into the hotspot-gc repository. Those updates were just > to make it compile and not really related to my change, so > I just overwrote the existing webrev. Just use the same > link as before: > > http://cr.openjdk.java.net/~brutisso/7178363/webrev.00/ > > > Also, if you want to see what the new output looks like I > am attaching a file called new.txt with an example from > running SpecJBB2005 with this command line: > > -XX:+UseG1GC -XX:ParallelGCThreads=4 > -XX:+UnlockExperimentalVMOptions -XX:G1LogLevel=finest > -XX:+TraceGen0Time -Xms256m -Xmx2G > > I am also attaching a file called old.txt with what the > output, using the same command line, looked like before my > change. As you can see the differences are what I listed > in my earlier email. You will also notice that the "old" > version has an entry for the SATB filtering, even though > all the entries are 0 (we didn't do a concurrent cycle so > there has been no SATB filtering). This was a bug I just > introduced with my last change (for 7178361), so the new > example is more correct. > > Thanks, > Bengt > > On 2012-07-18 15:55, Bengt Rutisson wrote: > > > Hi everyone, > > I would like some reviews of this change: > http://cr.openjdk.java.net/~brutisso/7178363/webrev.00/ > > This removes the special treatment for > ParallelGCThreads=0 from the G1 logging. I did keep > the log output unchanged for that case. Basically it > just has one indentation level less and skips some > output. I am not sure this is really necessary since > it is really a special case. I'm open to change that > special treatment too and just have the same output as > for ParallelGCThreads=1. > > The PrintGCDetails log output should be the same as > before with three minor adjustments: > > - The "Sum" is now not printed for the start and end > values for GC workers. This sum does not really make > sense to me. > > - The "(ms)" unit was removed from output that aren't > in milliseconds (termination attempts for example). > > - The average value is now printed as a double for all > types. > > I tried to clean up the code a bit and introduced a > separate class, Snippet WorkerDataArray, to keep track > of the per thread logging. I also introduced getters > and setters to avoid having to make G1CollectorPolicy > and TraceGen0TimeData friend classes to G1GCPhaseTimes. > > Thanks, > Bengt > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vitalyd at gmail.com Wed Aug 22 13:50:39 2012 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 22 Aug 2012 09:50:39 -0400 Subject: Request for review (M): 7178363 G1: Remove the serial code for PrintGCDetails and make it a special case of the parallel code In-Reply-To: <5034E32A.9020206@oracle.com> References: <5006C056.2080007@oracle.com> <5007CCE4.9070001@oracle.com> <50094C6E.2030108@oracle.com> <5034BF8F.7060400@oracle.com> <5034E32A.9020206@oracle.com> Message-ID: Thanks for the explanation Bengt - all good to me. Sent from my phone On Aug 22, 2012 9:39 AM, "Bengt Rutisson" wrote: > > Hi Vitaly, > > Thanks for looking at this change too! > > On 2012-08-22 14:22, Vitaly Davidovich wrote: > > Hi Bengt, > > A few minor comments. > > In g1CollectedHeap.cpp: > > 1) probably doesn't matter much in its current form, but would it be > better to call os::elapsedTime() right before prepare_verify() so that the > timing info captures just the verification time and not printing to > tty/gclog and constructing the HandleMark? This is in verify(). > > > Actually, I think it is good that the extra printing is included in the > timing. We want to time the extra cost of turning on the verification, thus > we should include the full cost of it. > > 2) it seems a bit weird to have the "guard" parameter in the verify() > method. Why wouldn't caller just check the guard and then only call if it > passes? Not a big deal though, just stylistic. > > > I see your point. As you say, it is just a style question. If I remove the > guard parameter to verify() I'd have to add the test to verify_before_gc() > and verify_after_gc() and that kind of re-introduces some of the code > duplication that I was aiming to remove. If you feel strongly about it I'll > change. > > 3) in verify_before/after(), would it make sense to only record the time > if it's not 0.0? I'm thinking that the pointer chasing may possible get a > cache miss only to record a 0. Again, pure speculation. > > > These values are set for each GC. If we don't set them to 0 somewhere we > will get the values from the last GC. By always setting the value there is > no risk that we get the wrong values. > > 4) is os::elapsedTime() using a monotonic time source? If not, what > would happen if you get a negative value in the timing? > > > os::elapsedTime() is a monotonic time source. > > Thanks, > Bengt > > Looks good otherwise. > > Thanks > > Sent from my phone > On Aug 22, 2012 7:21 AM, "Bengt Rutisson" > wrote: > >> >> Hi again, >> >> Here is an updated webrev based on comments from John Cuthbertson: >> http://cr.openjdk.java.net/~brutisso/7178363/webrev.02/ >> >> Here is a diff compared to the previous webrev: >> http://cr.openjdk.java.net/~brutisso/7178363/webrev.01-02-diff/ >> >> Thanks, >> Bengt >> >> On 2012-07-20 14:17, Bengt Rutisson wrote: >> >>> >>> Hi all, >>> >>> Here is an updated webrev for this change: >>> http://cr.openjdk.java.net/~brutisso/7178363/webrev.01/ >>> >>> It turns out the the earlier change for 7178361 had introduced two more >>> issues: the heap transition information for the PrintGC output and the >>> output for evacuation failures had both been moved in an unintended way. >>> The above webrev corrects both of these chagne too. Thanks John Cuthbertson >>> for pointing me to the evacuation failure output. >>> >>> Bengt >>> >>> >>> On 2012-07-19 11:01, Bengt Rutisson wrote: >>> >>>> >>>> Hi again, >>>> >>>> Just in case anybody already started looking at this: I have updated >>>> the webrev since I had to make some changes to make it compile with the NMT >>>> fixes that have just made it into the hotspot-gc repository. Those updates >>>> were just to make it compile and not really related to my change, so I just >>>> overwrote the existing webrev. Just use the same link as before: >>>> >>>> http://cr.openjdk.java.net/~brutisso/7178363/webrev.00/ >>>> >>>> Also, if you want to see what the new output looks like I am attaching >>>> a file called new.txt with an example from running SpecJBB2005 with this >>>> command line: >>>> >>>> -XX:+UseG1GC -XX:ParallelGCThreads=4 -XX:+UnlockExperimentalVMOptions >>>> -XX:G1LogLevel=finest -XX:+TraceGen0Time -Xms256m -Xmx2G >>>> >>>> I am also attaching a file called old.txt with what the output, using >>>> the same command line, looked like before my change. As you can see the >>>> differences are what I listed in my earlier email. You will also notice >>>> that the "old" version has an entry for the SATB filtering, even though all >>>> the entries are 0 (we didn't do a concurrent cycle so there has been no >>>> SATB filtering). This was a bug I just introduced with my last change (for >>>> 7178361), so the new example is more correct. >>>> >>>> Thanks, >>>> Bengt >>>> >>>> On 2012-07-18 15:55, Bengt Rutisson wrote: >>>> >>>>> >>>>> Hi everyone, >>>>> >>>>> I would like some reviews of this change: >>>>> http://cr.openjdk.java.net/~brutisso/7178363/webrev.00/ >>>>> >>>>> This removes the special treatment for ParallelGCThreads=0 from the G1 >>>>> logging. I did keep the log output unchanged for that case. Basically it >>>>> just has one indentation level less and skips some output. I am not sure >>>>> this is really necessary since it is really a special case. I'm open to >>>>> change that special treatment too and just have the same output as for >>>>> ParallelGCThreads=1. >>>>> >>>>> The PrintGCDetails log output should be the same as before with three >>>>> minor adjustments: >>>>> >>>>> - The "Sum" is now not printed for the start and end values for GC >>>>> workers. This sum does not really make sense to me. >>>>> >>>>> - The "(ms)" unit was removed from output that aren't in milliseconds >>>>> (termination attempts for example). >>>>> >>>>> - The average value is now printed as a double for all types. >>>>> >>>>> I tried to clean up the code a bit and introduced a separate class, >>>>> Snippet WorkerDataArray, to keep track of the per thread logging. I also >>>>> introduced getters and setters to avoid having to make G1CollectorPolicy >>>>> and TraceGen0TimeData friend classes to G1GCPhaseTimes. >>>>> >>>>> Thanks, >>>>> Bengt >>>>> >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vitalyd at gmail.com Wed Aug 22 14:12:46 2012 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 22 Aug 2012 10:12:46 -0400 Subject: Request for review (M): 7178363 G1: Remove the serial code for PrintGCDetails and make it a special case of the parallel code In-Reply-To: References: <5006C056.2080007@oracle.com> <5007CCE4.9070001@oracle.com> <50094C6E.2030108@oracle.com> <5034BF8F.7060400@oracle.com> <5034E32A.9020206@oracle.com> Message-ID: Bengt, Sorry, one follow-up after looking at the code again. In the guard case, if the VerifyXXX isn't set, then the value will always be 0.0, even in prior GCs. Are you saying these flags can be changed dynamically, perhaps by some management API? If not, then these methods will always produce and store 0, which seems a bit wasteful. Did I miss something? Sent from my phone On Aug 22, 2012 9:50 AM, "Vitaly Davidovich" wrote: > Thanks for the explanation Bengt - all good to me. > > Sent from my phone > On Aug 22, 2012 9:39 AM, "Bengt Rutisson" > wrote: > >> >> Hi Vitaly, >> >> Thanks for looking at this change too! >> >> On 2012-08-22 14:22, Vitaly Davidovich wrote: >> >> Hi Bengt, >> >> A few minor comments. >> >> In g1CollectedHeap.cpp: >> >> 1) probably doesn't matter much in its current form, but would it be >> better to call os::elapsedTime() right before prepare_verify() so that the >> timing info captures just the verification time and not printing to >> tty/gclog and constructing the HandleMark? This is in verify(). >> >> >> Actually, I think it is good that the extra printing is included in the >> timing. We want to time the extra cost of turning on the verification, thus >> we should include the full cost of it. >> >> 2) it seems a bit weird to have the "guard" parameter in the verify() >> method. Why wouldn't caller just check the guard and then only call if it >> passes? Not a big deal though, just stylistic. >> >> >> I see your point. As you say, it is just a style question. If I remove >> the guard parameter to verify() I'd have to add the test to >> verify_before_gc() and verify_after_gc() and that kind of re-introduces >> some of the code duplication that I was aiming to remove. If you feel >> strongly about it I'll change. >> >> 3) in verify_before/after(), would it make sense to only record the >> time if it's not 0.0? I'm thinking that the pointer chasing may possible >> get a cache miss only to record a 0. Again, pure speculation. >> >> >> These values are set for each GC. If we don't set them to 0 somewhere we >> will get the values from the last GC. By always setting the value there is >> no risk that we get the wrong values. >> >> 4) is os::elapsedTime() using a monotonic time source? If not, what >> would happen if you get a negative value in the timing? >> >> >> os::elapsedTime() is a monotonic time source. >> >> Thanks, >> Bengt >> >> Looks good otherwise. >> >> Thanks >> >> Sent from my phone >> On Aug 22, 2012 7:21 AM, "Bengt Rutisson" >> wrote: >> >>> >>> Hi again, >>> >>> Here is an updated webrev based on comments from John Cuthbertson: >>> http://cr.openjdk.java.net/~brutisso/7178363/webrev.02/ >>> >>> Here is a diff compared to the previous webrev: >>> http://cr.openjdk.java.net/~brutisso/7178363/webrev.01-02-diff/ >>> >>> Thanks, >>> Bengt >>> >>> On 2012-07-20 14:17, Bengt Rutisson wrote: >>> >>>> >>>> Hi all, >>>> >>>> Here is an updated webrev for this change: >>>> http://cr.openjdk.java.net/~brutisso/7178363/webrev.01/ >>>> >>>> It turns out the the earlier change for 7178361 had introduced two more >>>> issues: the heap transition information for the PrintGC output and the >>>> output for evacuation failures had both been moved in an unintended way. >>>> The above webrev corrects both of these chagne too. Thanks John Cuthbertson >>>> for pointing me to the evacuation failure output. >>>> >>>> Bengt >>>> >>>> >>>> On 2012-07-19 11:01, Bengt Rutisson wrote: >>>> >>>>> >>>>> Hi again, >>>>> >>>>> Just in case anybody already started looking at this: I have updated >>>>> the webrev since I had to make some changes to make it compile with the NMT >>>>> fixes that have just made it into the hotspot-gc repository. Those updates >>>>> were just to make it compile and not really related to my change, so I just >>>>> overwrote the existing webrev. Just use the same link as before: >>>>> >>>>> http://cr.openjdk.java.net/~brutisso/7178363/webrev.00/ >>>>> >>>>> Also, if you want to see what the new output looks like I am attaching >>>>> a file called new.txt with an example from running SpecJBB2005 with this >>>>> command line: >>>>> >>>>> -XX:+UseG1GC -XX:ParallelGCThreads=4 -XX:+UnlockExperimentalVMOptions >>>>> -XX:G1LogLevel=finest -XX:+TraceGen0Time -Xms256m -Xmx2G >>>>> >>>>> I am also attaching a file called old.txt with what the output, using >>>>> the same command line, looked like before my change. As you can see the >>>>> differences are what I listed in my earlier email. You will also notice >>>>> that the "old" version has an entry for the SATB filtering, even though all >>>>> the entries are 0 (we didn't do a concurrent cycle so there has been no >>>>> SATB filtering). This was a bug I just introduced with my last change (for >>>>> 7178361), so the new example is more correct. >>>>> >>>>> Thanks, >>>>> Bengt >>>>> >>>>> On 2012-07-18 15:55, Bengt Rutisson wrote: >>>>> >>>>>> >>>>>> Hi everyone, >>>>>> >>>>>> I would like some reviews of this change: >>>>>> http://cr.openjdk.java.net/~brutisso/7178363/webrev.00/ >>>>>> >>>>>> This removes the special treatment for ParallelGCThreads=0 from the >>>>>> G1 logging. I did keep the log output unchanged for that case. Basically it >>>>>> just has one indentation level less and skips some output. I am not sure >>>>>> this is really necessary since it is really a special case. I'm open to >>>>>> change that special treatment too and just have the same output as for >>>>>> ParallelGCThreads=1. >>>>>> >>>>>> The PrintGCDetails log output should be the same as before with three >>>>>> minor adjustments: >>>>>> >>>>>> - The "Sum" is now not printed for the start and end values for GC >>>>>> workers. This sum does not really make sense to me. >>>>>> >>>>>> - The "(ms)" unit was removed from output that aren't in milliseconds >>>>>> (termination attempts for example). >>>>>> >>>>>> - The average value is now printed as a double for all types. >>>>>> >>>>>> I tried to clean up the code a bit and introduced a separate class, >>>>>> Snippet WorkerDataArray, to keep track of the per thread logging. I also >>>>>> introduced getters and setters to avoid having to make G1CollectorPolicy >>>>>> and TraceGen0TimeData friend classes to G1GCPhaseTimes. >>>>>> >>>>>> Thanks, >>>>>> Bengt >>>>>> >>>>> >>>>> >>>> >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.cuthbertson at oracle.com Wed Aug 22 16:49:43 2012 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Wed, 22 Aug 2012 09:49:43 -0700 Subject: Request for review (XXS): 7193157: G1: Make some develpflags available in product builds In-Reply-To: <50348E4B.50204@oracle.com> References: <50348E4B.50204@oracle.com> Message-ID: <50350DA7.4010501@oracle.com> Hi Bengt, Looks good. Ship it. Thanks, JohnC On 08/22/12 00:46, Bengt Rutisson wrote: > > Hi all, > > Could I please have a couple of reviews for this really small fix? > http://cr.openjdk.java.net/~brutisso/7193157/webrev.00/ > > We need to make some of the G1 develop flags available in product > builds to allow the performance team to tune G1 properly. We decided > to make the flags experimental to start with. After the performance > team has evaluated the flags we might make some of them product flags. > The current prime candidates for becoming product flags are: > > G1DefaultMinNewGenPercent > G1DefaultMaxNewGenPercent > G1OldCSetRegionLiveThresholdPercent > > Thanks, > Bengt From john.cuthbertson at oracle.com Wed Aug 22 17:23:50 2012 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Wed, 22 Aug 2012 10:23:50 -0700 Subject: Request for review (XXS): 7193157: G1: Make some develpflags available in product builds In-Reply-To: References: <50348E4B.50204@oracle.com> Message-ID: <503515A6.2070404@oracle.com> Hi Ramki, Once we're happy with the policies, and default values and G1 is mostly meeting peoples' expectations, we definitely should work a tuning guide. Any outside help and suggestions (hint, hint) will be more than welcome. The perf team has been working some what-if scenarios and that could be used as a basis. There has also been mention of contributing to a future edition of Charlie Hunt's excellent book (if there is one). IMO we need to improve the mixed GC times. During a typical mixed GC sequence the pause times are typically higher than the typical young GC and, in the latter stages of mixed GC sequence, the pause times go up significantly as the cost of collecting the regions goes up. I think we can reduce this through changing default flag settings (for example a 90% occupancy might result in too many "expensive" candidate regions). For increased RSet updating times - which is typically the case with a mixed GC because of the increased number of filled updated buffers by the GC itself - it might be worth investigating starting the concurrent refine threads more aggressively to process the higher number of deferred updates. I also think there may be an issue in the prediction logic for the RSet scanning time based upon RSet size - I don't think it handles coarse RSet entries very well. Also there were a few times where we weren't meeting the pause time goal because we couldn't size the young generation small enough - we were running into the 20% lower limit of the heap. As a result of Bengt's change we have provided a few more knobs for people to play with. Thanks, JohnC On 08/22/12 01:21, Srinivas Ramakrishna wrote: > Looks good to me. I am hoping the perf and gc teams will together also > publish a G1 tuning white-paper soon! > > -- ramki > > On Wed, Aug 22, 2012 at 12:46 AM, Bengt Rutisson > > wrote: > > > Hi all, > > Could I please have a couple of reviews for this really small fix? > http://cr.openjdk.java.net/~brutisso/7193157/webrev.00/ > > > We need to make some of the G1 develop flags available in product > builds to allow the performance team to tune G1 properly. We > decided to make the flags experimental to start with. After the > performance team has evaluated the flags we might make some of > them product flags. The current prime candidates for becoming > product flags are: > > G1DefaultMinNewGenPercent > G1DefaultMaxNewGenPercent > G1OldCSetRegionLiveThresholdPercent > > Thanks, > Bengt > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon.masamitsu at oracle.com Wed Aug 22 17:34:46 2012 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 22 Aug 2012 10:34:46 -0700 Subject: Request for review (XXS): 7193157: G1: Make some develpflags available in product builds In-Reply-To: <50348E4B.50204@oracle.com> References: <50348E4B.50204@oracle.com> Message-ID: <50351836.7030707@oracle.com> Looks good. On 8/22/2012 12:46 AM, Bengt Rutisson wrote: > > Hi all, > > Could I please have a couple of reviews for this really small fix? > http://cr.openjdk.java.net/~brutisso/7193157/webrev.00/ > > We need to make some of the G1 develop flags available in product > builds to allow the performance team to tune G1 properly. We decided > to make the flags experimental to start with. After the performance > team has evaluated the flags we might make some of them product flags. > The current prime candidates for becoming product flags are: > > G1DefaultMinNewGenPercent > G1DefaultMaxNewGenPercent > G1OldCSetRegionLiveThresholdPercent > > Thanks, > Bengt From bengt.rutisson at oracle.com Thu Aug 23 03:43:46 2012 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 23 Aug 2012 05:43:46 +0200 Subject: Request for review (XXS): 7193157: G1: Make some develpflags available in product builds In-Reply-To: <50351836.7030707@oracle.com> References: <50348E4B.50204@oracle.com> <50351836.7030707@oracle.com> Message-ID: <5035A6F2.9030600@oracle.com> Thanks Ramki, Vitaly, John and Jon for the reviews! All set to push this now. Bengt On 2012-08-22 19:34, Jon Masamitsu wrote: > Looks good. > > On 8/22/2012 12:46 AM, Bengt Rutisson wrote: >> >> Hi all, >> >> Could I please have a couple of reviews for this really small fix? >> http://cr.openjdk.java.net/~brutisso/7193157/webrev.00/ >> >> We need to make some of the G1 develop flags available in product >> builds to allow the performance team to tune G1 properly. We decided >> to make the flags experimental to start with. After the performance >> team has evaluated the flags we might make some of them product >> flags. The current prime candidates for becoming product flags are: >> >> G1DefaultMinNewGenPercent >> G1DefaultMaxNewGenPercent >> G1OldCSetRegionLiveThresholdPercent >> >> Thanks, >> Bengt From john.cuthbertson at oracle.com Thu Aug 23 05:18:20 2012 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Wed, 22 Aug 2012 22:18:20 -0700 Subject: Request for review (M): 7178363 G1: Remove the serial code for PrintGCDetails and make it a special case of the parallel code In-Reply-To: <5034BF8F.7060400@oracle.com> References: <5006C056.2080007@oracle.com> <5007CCE4.9070001@oracle.com> <50094C6E.2030108@oracle.com> <5034BF8F.7060400@oracle.com> Message-ID: <5035BD1C.2060906@oracle.com> Hi Bengt, Looks good to me. Some really good refactoring in there. Some minor nits: g1GCPhaseTimes.cpp, line 205: On my screen it looks like the indentation is incorrect. You might want to check it. Also would it read better to use a local to hold the difference between the end and start times? Your choice. Also your patch comment has an extra capital letter: WorkerDataARray. Ship it! JohnC On 8/22/2012 4:16 AM, Bengt Rutisson wrote: > > Hi again, > > Here is an updated webrev based on comments from John Cuthbertson: > http://cr.openjdk.java.net/~brutisso/7178363/webrev.02/ > > Here is a diff compared to the previous webrev: > http://cr.openjdk.java.net/~brutisso/7178363/webrev.01-02-diff/ > > Thanks, > Bengt > > On 2012-07-20 14:17, Bengt Rutisson wrote: >> >> Hi all, >> >> Here is an updated webrev for this change: >> http://cr.openjdk.java.net/~brutisso/7178363/webrev.01/ >> >> It turns out the the earlier change for 7178361 had introduced two >> more issues: the heap transition information for the PrintGC output >> and the output for evacuation failures had both been moved in an >> unintended way. The above webrev corrects both of these chagne too. >> Thanks John Cuthbertson for pointing me to the evacuation failure >> output. >> >> Bengt >> >> >> On 2012-07-19 11:01, Bengt Rutisson wrote: >>> >>> Hi again, >>> >>> Just in case anybody already started looking at this: I have updated >>> the webrev since I had to make some changes to make it compile with >>> the NMT fixes that have just made it into the hotspot-gc repository. >>> Those updates were just to make it compile and not really related to >>> my change, so I just overwrote the existing webrev. Just use the >>> same link as before: >>> >>> http://cr.openjdk.java.net/~brutisso/7178363/webrev.00/ >>> >>> Also, if you want to see what the new output looks like I am >>> attaching a file called new.txt with an example from running >>> SpecJBB2005 with this command line: >>> >>> -XX:+UseG1GC -XX:ParallelGCThreads=4 >>> -XX:+UnlockExperimentalVMOptions -XX:G1LogLevel=finest >>> -XX:+TraceGen0Time -Xms256m -Xmx2G >>> >>> I am also attaching a file called old.txt with what the output, >>> using the same command line, looked like before my change. As you >>> can see the differences are what I listed in my earlier email. You >>> will also notice that the "old" version has an entry for the SATB >>> filtering, even though all the entries are 0 (we didn't do a >>> concurrent cycle so there has been no SATB filtering). This was a >>> bug I just introduced with my last change (for 7178361), so the new >>> example is more correct. >>> >>> Thanks, >>> Bengt >>> >>> On 2012-07-18 15:55, Bengt Rutisson wrote: >>>> >>>> Hi everyone, >>>> >>>> I would like some reviews of this change: >>>> http://cr.openjdk.java.net/~brutisso/7178363/webrev.00/ >>>> >>>> This removes the special treatment for ParallelGCThreads=0 from the >>>> G1 logging. I did keep the log output unchanged for that case. >>>> Basically it just has one indentation level less and skips some >>>> output. I am not sure this is really necessary since it is really a >>>> special case. I'm open to change that special treatment too and >>>> just have the same output as for ParallelGCThreads=1. >>>> >>>> The PrintGCDetails log output should be the same as before with >>>> three minor adjustments: >>>> >>>> - The "Sum" is now not printed for the start and end values for GC >>>> workers. This sum does not really make sense to me. >>>> >>>> - The "(ms)" unit was removed from output that aren't in >>>> milliseconds (termination attempts for example). >>>> >>>> - The average value is now printed as a double for all types. >>>> >>>> I tried to clean up the code a bit and introduced a separate class, >>>> Snippet WorkerDataArray, to keep track of the per thread logging. I >>>> also introduced getters and setters to avoid having to make >>>> G1CollectorPolicy and TraceGen0TimeData friend classes to >>>> G1GCPhaseTimes. >>>> >>>> Thanks, >>>> Bengt >>> >> > From bengt.rutisson at oracle.com Thu Aug 23 05:39:39 2012 From: bengt.rutisson at oracle.com (bengt.rutisson at oracle.com) Date: Thu, 23 Aug 2012 05:39:39 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 7193157: G1: Make some develpflags available in product builds Message-ID: <20120823053943.3E4F0476A5@hg.openjdk.java.net> Changeset: 3650da95d2ee Author: brutisso Date: 2012-08-23 05:25 +0200 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/3650da95d2ee 7193157: G1: Make some develpflags available in product builds Summary: Also reviewed by: vitalyd at gmail.com. Make G1DefaultMinNewGenPercent, G1DefaultMaxNewGenPercent, G1OldCSetRegionLiveThresholdPercent and G1OldCSetRegionThresholdPercent experimental flags Reviewed-by: ysr, johnc, jmasa ! src/share/vm/gc_implementation/g1/g1_globals.hpp From ysr1729 at gmail.com Thu Aug 23 05:47:59 2012 From: ysr1729 at gmail.com (Srinivas Ramakrishna) Date: Wed, 22 Aug 2012 22:47:59 -0700 Subject: CMSWaitDuration unstable behavior In-Reply-To: <1DDCA93502632C4DA22E9EE199CE907C4FD16487@SE002593.cs.commerzbank.com> References: <1DDCA93502632C4DA22E9EE199CE907C4FD1647A@SE002593.cs.commerzbank.com> <5022818B.6070502@oracle.com> <1DDCA93502632C4DA22E9EE199CE907C4FD16487@SE002593.cs.commerzbank.com> Message-ID: Hi Michal -- thanks for drawing my attention to your response, which I had somehow missed in my overflowing mailbox... On Thu, Aug 9, 2012 at 4:32 AM, Frajt, Michal < Michal.Frajt at partner.commerzbank.com> wrote: > Hi Ramki,**** > > ** ** > > The current CMSWaitDuration implementation (in the normal mode only) is > not always waiting the maximum specified time for a scavenge to occur. > There is just a simple CGC_lock mutex wait call with the CMSWaitDuration > parameter but nobody checks how much time passed when the method returns. > The code expects that the mutex is notified after the scavenge and when the > CMS collector thread is to terminate. Unfortunately there are, to me > unknown, other mutex notifications coming from the desynchronize method > invocation. The CMSWaitDuration is currently only a maximum time for which > the CMS collector can wait before it checks if a new CMS cycle should > start. It is not directly related to the scavenge occurrence - even > occasionally it would hit it correctly. > Thanks for this observation. I think you are right. AFAIK, the "descynchronize" occurs at each global safepoint. And you are right that safepoints can occur for other reasons than GC (for example they used to occur for some kinds of deoptimization, although i am not sure if that's the case today, and they do occur for bulk(?) biased lock revocation). So you are completely correct that the notifications on that wait can occur for other reasons than just scavenge. Given that this is the case, and given that you have fixed the issue in your own JVM based on HotSpot, it would be great if you are able to share your fix to this problem as a patch on this list so it could be reviewed for inclusion into HotSpot, since teh community of CMS users would also gain from this fix. Thanks again for finding the problem and fixing it!! -- ramki The abortable preclean phase is calling the wait_on_cms_lock with the > CMSAbortablePrecleanWaitMillis timeout just to react on CMS thread > termination while taking a short break between preclean iterations. The > CMSWaitDuration time is exclusively specific to the new CMS cycle start > (hence initiating a new initial-mark pause).**** > > ** ** > > The idea of the CMSScavengeBeforeInitialMark might be easier to implement > but we would strongly prefer not to invoke yet another scavenge explicitly > as it is unbalancing young objects aging and leads to unwanted promotions. > Even the CMSWaitDuration was never meant to control the scheduling of the > initial mark pause in relation to the scavenge, it would be still better to > behave the way it was never meant to, than to have unplanned scavenge > invocations. Someone could even think about a combined solution when it > first waits for the specified duration and if there is no scavenge > occurring it is explicitly invoking a scavenge before the inital-mark phase > (or better pause) starts. Both configurable by the existing CMSWaitDuration > and the new CMSScavengeBeforeInitialMark parameters. **** > > ** ** > > We understand that the most experienced CMS engineer left Oracle. We had > the chance to speak with him, review our CMS observations, listen to a > wonderful presentation of the G1 collector. Since we try every year to > change to the G1 collector but without much success. Either it crashes or > runs far slower than the current CMS collector with the parallel new > generation. Solving or helping both the STW phases of the CMS collector > might be still beneficial for many clients who are not able to move to the > G1 collector yet.**** > > ** ** > > Regards,**** > > Michal**** > > ** ** > > ** ** > > *From:* Srinivas Ramakrishna [mailto:ysr1729 at gmail.com] > *Sent:* Mittwoch, 8. August 2012 20:57 > *To:* Jon Masamitsu > *Cc:* hotspot-gc-dev at openjdk.java.net; Frajt, Michal > *Subject:* Re: CMSWaitDuration unstable behavior**** > > ** ** > > Hi Michal -- > > There's an RFE (lost in the mists of time) to piggyback initial marking > work on a ParNew collection (and hence do it multi-threaded > rather than single-threaded as is the case currently). But it never got > implemented, unfortunately. > > That said, I understand your motivation is to reduce the duration of the > initial mark pause in the face of using a large Eden space > which is currently marked single-threaded. > > Unfortunately, CMSWaitDuration was never meant to control the scheduling > of the initial mark pause in relation to the scavenge. > Rather it was meant to be a maximum wait time for which the CMS collector > would wait for a scavenge to occur -- if a scavenge > did not occur within that time, CMS might decide to unequivocally take the > action it might otherwise have taken immediately after > the scavenge -- such as polling the old generation occupancy to decide if > a new CMS cycle should start (hence initiating a new > initial-mark pause), or if the "abortable preclean phase" should be exited > in the absence of a scavenge occurring suitably soon. > Thus, trying to retrofit CMSWaitDuration to meet your purpose of > co-scheduling initial mark after scavenge is probably not the right thing > to do. > > I think the easiest thing to do, as Jon suggested, is to have an explicit > flag such as CMSScavengeBeforeInitialMark which would be > analogous to the current role of CMSScavengeBeforeRemark. Here, ICMS wakes > up at the normal time and takes control, but > instead of doing an initial mark straightaay, it first initiates a > parallel scavenge and follows that up with a single-threaded initial mark. > Granted this will not cause an initial mark step to occur immediately > after a "normal" scavenge as we really want, but rather cause > an additional scavenge to happen just before an initial mark pause is > scheduled in ICMS (exactly as is the case with the current > CMSScavengeBeforeRemark where an extra scavenge occurs which is not very > pleasant), but it would be far easier to implement > without making any other changes in the system. > > The best solution of course is to implement the RFE to do the initial mark > in parallel piggybacked on the scavenge and all > your problems go away (ICMS may need a very minor adjustment for that). > Anyone want to take a stab at parallelizing > and piggybacking initial mark on scavenge? It would be a matter of > extending the scavenge object and root scanning closures > to new closures so as to not skip the references that point outside of > young gen as is done for the normal parnew scanning closures, > but to mark the appropriate bits in the CMS marking bit map. That's really > theoretically all it will take. > > PS: Jon, if Michal takes the approach of CMSScavengeBeforeInitialMark, I'd > say it would be useful to the broader community (not > just ICMS users) if that were integrated into the main-line code, as it > would be a via-media for CMS scaling in the absence of the > piggybacking RFE which is really the best solution here. > > thanks! > -- ramki**** > > On Wed, Aug 8, 2012 at 8:11 AM, Jon Masamitsu > wrote:**** > > Michal, > > The engineer with the most experience on CMS left Oracle > and I suspect this is not going to get fixed in the way you want. > > I've create CR 7189971 to capture your comments and it will be > reviewed along with other RFE's for CMS but I would not be > optimistic. > > Since you are customizing your own VM, did you consider > explicitly invoking a young collection before the initial mark > the way that it is done for the remark phase with the flag > > CMSScavengeBeforeRemark > > Jon**** > > > > On 8/7/2012 6:16 AM, Frajt, Michal wrote:**** > > Hi all, > > We are using the incremental CMS collector for many years. We have a > distributed application framework based on the subscribe-unsubscribe model > where the data unsubscriptions are handled by the application layer just > forgetting the strong reference to the distributed data. The underlying > application framework layer is using weak references to trace the data > requirement from the application layer. We keep the old generation > processed permanently (incrementally) to get the week references released > and reported within a short period of time (minutes). > > Unfortunately the incremental mode is missing the support for the > CMSWaitDuration to place the initial mark phase right after the young space > collection. With some new gen sizing optimization we went to a situation > when the new gen is more or less big enough to keep the most of live > objects with only a few promotions to the old gen. The incremental CMS is > then started every minute in a random moment with pretty garbaged new gen. > The initial mark takes 20-50 times more than a single new gen processing > (40ms new gen, initial mark 1100ms). > > We decided to customize the OpenJDK 6 by adding the incremental mode > CMSWaitDuration support. We took the same approach as the wait_on_cms_lock > method does with the CGC_lock object. Unfortunately we realized that the > CGC_lock mutex is additionally notified in some other situation than the > young space collection finishing. The young space collection unrelated > notifications are coming from the desynchronize method invocations. These > unrelated notifications are causing the wait_on_cms_lock to return earlier > than required. The initial mark phase is started before the young space > collection even there is enough wait duration time specified to wait. We > have fixed it by waiting again if the > GenCollectedHeap::heap()->total_collections() counter is not changed after > the CGC_long->wait method returns but not longer than the CMSWaitDuration > in total. The initial mark is then always placed (if CMSWaitDuration is > long enough) after the young space collection. Every initial mark phase > takes no longer than 17ms (previously 1100ms). > > We tested the CMSWaitDuration behavior in the normal CMS mode. We > specified the -XX:+UseCMSInitiatingOccupancyOnly and > -XX:CMSInitiatingOccupancyFraction=10 to force the CMS running permanently > (shouldConcurrentCollect should be returning true). The CMS initial-mark is > many times started without waiting for the young space collection which > makes the initial marking running 20-50 longer. We find this as unstable > behavior of the CMSWaitDuration implementation related to the problem of > the wait-notify signaling on the CGC_lock object. We disabled the explicit > GC invocation (-XX:+DisableExplicitGC) to be sure there is no other reason > to start the CMS initial mark phase before the young space collection. > > Is there any plan to get the CMSWaitDuration supported in the incremental > mode and/or get it fixed in the normal mode? > > Thanks, > Michal Frajt > > **** > > ** ** > > ** ** > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Thu Aug 23 06:42:27 2012 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 23 Aug 2012 08:42:27 +0200 Subject: Request for review (M): 7178363 G1: Remove the serial code for PrintGCDetails and make it a special case of the parallel code In-Reply-To: <5035BD1C.2060906@oracle.com> References: <5006C056.2080007@oracle.com> <5007CCE4.9070001@oracle.com> <50094C6E.2030108@oracle.com> <5034BF8F.7060400@oracle.com> <5035BD1C.2060906@oracle.com> Message-ID: <5035D0D3.8020200@oracle.com> Hi John, Thanks for looking at this so quickly! On 2012-08-23 07:18, John Cuthbertson wrote: > Hi Bengt, > > Looks good to me. Some really good refactoring in there. Some minor nits: > > g1GCPhaseTimes.cpp, line 205: On my screen it looks like the > indentation is incorrect. You might want to check it. Also would it > read better to use a local to hold the difference between the end and > start times? Your choice. Good catch. I fixed the indentation and I added locals to hold both the worker time and the other time. Looks better now I think. Here's what the for loop looks like now: for (uint i = 0; i < _active_gc_threads; i++) { double worker_time = _last_gc_worker_end_times_ms.get(i) - _last_gc_worker_start_times_ms.get(i); _last_gc_worker_times_ms.set(i, worker_time); double worker_known_time = _last_ext_root_scan_times_ms.get(i) + _last_satb_filtering_times_ms.get(i) + _last_update_rs_times_ms.get(i) + _last_scan_rs_times_ms.get(i) + _last_obj_copy_times_ms.get(i) + _last_termination_times_ms.get(i); double worker_other_time = worker_time - worker_known_time; _last_gc_worker_other_times_ms.set(i, worker_other_time); } > > Also your patch comment has an extra capital letter: WorkerDataARray. Thanks for catching this. I'll fix it before I push. > Ship it! Thanks again for looking at this! Bengt > > JohnC > > On 8/22/2012 4:16 AM, Bengt Rutisson wrote: >> >> Hi again, >> >> Here is an updated webrev based on comments from John Cuthbertson: >> http://cr.openjdk.java.net/~brutisso/7178363/webrev.02/ >> >> Here is a diff compared to the previous webrev: >> http://cr.openjdk.java.net/~brutisso/7178363/webrev.01-02-diff/ >> >> Thanks, >> Bengt >> >> On 2012-07-20 14:17, Bengt Rutisson wrote: >>> >>> Hi all, >>> >>> Here is an updated webrev for this change: >>> http://cr.openjdk.java.net/~brutisso/7178363/webrev.01/ >>> >>> It turns out the the earlier change for 7178361 had introduced two >>> more issues: the heap transition information for the PrintGC output >>> and the output for evacuation failures had both been moved in an >>> unintended way. The above webrev corrects both of these chagne too. >>> Thanks John Cuthbertson for pointing me to the evacuation failure >>> output. >>> >>> Bengt >>> >>> >>> On 2012-07-19 11:01, Bengt Rutisson wrote: >>>> >>>> Hi again, >>>> >>>> Just in case anybody already started looking at this: I have >>>> updated the webrev since I had to make some changes to make it >>>> compile with the NMT fixes that have just made it into the >>>> hotspot-gc repository. Those updates were just to make it compile >>>> and not really related to my change, so I just overwrote the >>>> existing webrev. Just use the same link as before: >>>> >>>> http://cr.openjdk.java.net/~brutisso/7178363/webrev.00/ >>>> >>>> Also, if you want to see what the new output looks like I am >>>> attaching a file called new.txt with an example from running >>>> SpecJBB2005 with this command line: >>>> >>>> -XX:+UseG1GC -XX:ParallelGCThreads=4 >>>> -XX:+UnlockExperimentalVMOptions -XX:G1LogLevel=finest >>>> -XX:+TraceGen0Time -Xms256m -Xmx2G >>>> >>>> I am also attaching a file called old.txt with what the output, >>>> using the same command line, looked like before my change. As you >>>> can see the differences are what I listed in my earlier email. You >>>> will also notice that the "old" version has an entry for the SATB >>>> filtering, even though all the entries are 0 (we didn't do a >>>> concurrent cycle so there has been no SATB filtering). This was a >>>> bug I just introduced with my last change (for 7178361), so the new >>>> example is more correct. >>>> >>>> Thanks, >>>> Bengt >>>> >>>> On 2012-07-18 15:55, Bengt Rutisson wrote: >>>>> >>>>> Hi everyone, >>>>> >>>>> I would like some reviews of this change: >>>>> http://cr.openjdk.java.net/~brutisso/7178363/webrev.00/ >>>>> >>>>> This removes the special treatment for ParallelGCThreads=0 from >>>>> the G1 logging. I did keep the log output unchanged for that case. >>>>> Basically it just has one indentation level less and skips some >>>>> output. I am not sure this is really necessary since it is really >>>>> a special case. I'm open to change that special treatment too and >>>>> just have the same output as for ParallelGCThreads=1. >>>>> >>>>> The PrintGCDetails log output should be the same as before with >>>>> three minor adjustments: >>>>> >>>>> - The "Sum" is now not printed for the start and end values for GC >>>>> workers. This sum does not really make sense to me. >>>>> >>>>> - The "(ms)" unit was removed from output that aren't in >>>>> milliseconds (termination attempts for example). >>>>> >>>>> - The average value is now printed as a double for all types. >>>>> >>>>> I tried to clean up the code a bit and introduced a separate >>>>> class, Snippet WorkerDataArray, to keep track of the per thread >>>>> logging. I also introduced getters and setters to avoid having to >>>>> make G1CollectorPolicy and TraceGen0TimeData friend classes to >>>>> G1GCPhaseTimes. >>>>> >>>>> Thanks, >>>>> Bengt >>>> >>> >> From bengt.rutisson at oracle.com Thu Aug 23 08:42:35 2012 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 23 Aug 2012 10:42:35 +0200 Subject: Request for review (M): 7178363 G1: Remove the serial code for PrintGCDetails and make it a special case of the parallel code In-Reply-To: References: <5006C056.2080007@oracle.com> <5007CCE4.9070001@oracle.com> <50094C6E.2030108@oracle.com> <5034BF8F.7060400@oracle.com> <5034E32A.9020206@oracle.com> Message-ID: <5035ECFB.9080809@oracle.com> Hi Vitaly, On 2012-08-22 16:12, Vitaly Davidovich wrote: > > Bengt, > > Sorry, one follow-up after looking at the code again. In the guard > case, if the VerifyXXX isn't set, then the value will always be 0.0, > even in prior GCs. Are you saying these flags can be changed > dynamically, perhaps by some management API? If not, then these > methods will always produce and store 0, which seems a bit wasteful. > Did I miss something? > Currently the Verify* flags can not be turned on or off at runtime. There is however the VerifyGCStartAt to consider, so even when the Verify* flags are on it is not sure that we do verification every GC. It seems safer and simpler to me to always set the values. We only do this once per GC so I think the potential performance effects are very limited. Always setting the value will also be robust if we decide to make the Verify* flags manageable in the future. Cheers, Bengt > Sent from my phone > > On Aug 22, 2012 9:50 AM, "Vitaly Davidovich" > wrote: > > Thanks for the explanation Bengt - all good to me. > > Sent from my phone > > On Aug 22, 2012 9:39 AM, "Bengt Rutisson" > > wrote: > > > Hi Vitaly, > > Thanks for looking at this change too! > > On 2012-08-22 14:22, Vitaly Davidovich wrote: >> >> Hi Bengt, >> >> A few minor comments. >> >> In g1CollectedHeap.cpp: >> >> 1) probably doesn't matter much in its current form, but >> would it be better to call os::elapsedTime() right before >> prepare_verify() so that the timing info captures just the >> verification time and not printing to tty/gclog and >> constructing the HandleMark? This is in verify(). >> > > Actually, I think it is good that the extra printing is > included in the timing. We want to time the extra cost of > turning on the verification, thus we should include the full > cost of it. > >> 2) it seems a bit weird to have the "guard" parameter in the >> verify() method. Why wouldn't caller just check the guard >> and then only call if it passes? Not a big deal though, just >> stylistic. >> > > I see your point. As you say, it is just a style question. If > I remove the guard parameter to verify() I'd have to add the > test to verify_before_gc() and verify_after_gc() and that kind > of re-introduces some of the code duplication that I was > aiming to remove. If you feel strongly about it I'll change. > >> 3) in verify_before/after(), would it make sense to only >> record the time if it's not 0.0? I'm thinking that the >> pointer chasing may possible get a cache miss only to record >> a 0. Again, pure speculation. >> > > These values are set for each GC. If we don't set them to 0 > somewhere we will get the values from the last GC. By always > setting the value there is no risk that we get the wrong values. > >> 4) is os::elapsedTime() using a monotonic time source? If >> not, what would happen if you get a negative value in the timing? >> > > os::elapsedTime() is a monotonic time source. > > Thanks, > Bengt > >> Looks good otherwise. >> >> Thanks >> >> Sent from my phone >> >> On Aug 22, 2012 7:21 AM, "Bengt Rutisson" >> > > wrote: >> >> >> Hi again, >> >> Here is an updated webrev based on comments from John >> Cuthbertson: >> http://cr.openjdk.java.net/~brutisso/7178363/webrev.02/ >> >> >> Here is a diff compared to the previous webrev: >> http://cr.openjdk.java.net/~brutisso/7178363/webrev.01-02-diff/ >> >> >> Thanks, >> Bengt >> >> On 2012-07-20 14:17, Bengt Rutisson wrote: >> >> >> Hi all, >> >> Here is an updated webrev for this change: >> http://cr.openjdk.java.net/~brutisso/7178363/webrev.01/ >> >> >> It turns out the the earlier change for 7178361 had >> introduced two more issues: the heap transition >> information for the PrintGC output and the output for >> evacuation failures had both been moved in an >> unintended way. The above webrev corrects both of >> these chagne too. Thanks John Cuthbertson for >> pointing me to the evacuation failure output. >> >> Bengt >> >> >> On 2012-07-19 11:01, Bengt Rutisson wrote: >> >> >> Hi again, >> >> Just in case anybody already started looking at >> this: I have updated the webrev since I had to >> make some changes to make it compile with the NMT >> fixes that have just made it into the hotspot-gc >> repository. Those updates were just to make it >> compile and not really related to my change, so I >> just overwrote the existing webrev. Just use the >> same link as before: >> >> http://cr.openjdk.java.net/~brutisso/7178363/webrev.00/ >> >> >> Also, if you want to see what the new output >> looks like I am attaching a file called new.txt >> with an example from running SpecJBB2005 with >> this command line: >> >> -XX:+UseG1GC -XX:ParallelGCThreads=4 >> -XX:+UnlockExperimentalVMOptions >> -XX:G1LogLevel=finest -XX:+TraceGen0Time -Xms256m >> -Xmx2G >> >> I am also attaching a file called old.txt with >> what the output, using the same command line, >> looked like before my change. As you can see the >> differences are what I listed in my earlier >> email. You will also notice that the "old" >> version has an entry for the SATB filtering, even >> though all the entries are 0 (we didn't do a >> concurrent cycle so there has been no SATB >> filtering). This was a bug I just introduced with >> my last change (for 7178361), so the new example >> is more correct. >> >> Thanks, >> Bengt >> >> On 2012-07-18 15:55, Bengt Rutisson wrote: >> >> >> Hi everyone, >> >> I would like some reviews of this change: >> http://cr.openjdk.java.net/~brutisso/7178363/webrev.00/ >> >> >> This removes the special treatment for >> ParallelGCThreads=0 from the G1 logging. I >> did keep the log output unchanged for that >> case. Basically it just has one indentation >> level less and skips some output. I am not >> sure this is really necessary since it is >> really a special case. I'm open to change >> that special treatment too and just have the >> same output as for ParallelGCThreads=1. >> >> The PrintGCDetails log output should be the >> same as before with three minor adjustments: >> >> - The "Sum" is now not printed for the start >> and end values for GC workers. This sum does >> not really make sense to me. >> >> - The "(ms)" unit was removed from output >> that aren't in milliseconds (termination >> attempts for example). >> >> - The average value is now printed as a >> double for all types. >> >> I tried to clean up the code a bit and >> introduced a separate class, Snippet >> WorkerDataArray, to keep track of the per >> thread logging. I also introduced getters and >> setters to avoid having to make >> G1CollectorPolicy and TraceGen0TimeData >> friend classes to G1GCPhaseTimes. >> >> Thanks, >> Bengt >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Thu Aug 23 08:44:08 2012 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 23 Aug 2012 10:44:08 +0200 Subject: Request for review (M): 7178363 G1: Remove the serial code for PrintGCDetails and make it a special case of the parallel code In-Reply-To: <5035D0D3.8020200@oracle.com> References: <5006C056.2080007@oracle.com> <5007CCE4.9070001@oracle.com> <50094C6E.2030108@oracle.com> <5034BF8F.7060400@oracle.com> <5035BD1C.2060906@oracle.com> <5035D0D3.8020200@oracle.com> Message-ID: <5035ED58.8090009@oracle.com> Thanks John, Mikael Gerdin and Vitaly for reviewing this! All set to push now. Bengt On 2012-08-23 08:42, Bengt Rutisson wrote: > > Hi John, > > Thanks for looking at this so quickly! > > On 2012-08-23 07:18, John Cuthbertson wrote: >> Hi Bengt, >> >> Looks good to me. Some really good refactoring in there. Some minor >> nits: >> >> g1GCPhaseTimes.cpp, line 205: On my screen it looks like the >> indentation is incorrect. You might want to check it. Also would it >> read better to use a local to hold the difference between the end and >> start times? Your choice. > > Good catch. I fixed the indentation and I added locals to hold both > the worker time and the other time. Looks better now I think. Here's > what the for loop looks like now: > > for (uint i = 0; i < _active_gc_threads; i++) { > double worker_time = _last_gc_worker_end_times_ms.get(i) - > _last_gc_worker_start_times_ms.get(i); > _last_gc_worker_times_ms.set(i, worker_time); > > double worker_known_time = _last_ext_root_scan_times_ms.get(i) + > _last_satb_filtering_times_ms.get(i) + > _last_update_rs_times_ms.get(i) + > _last_scan_rs_times_ms.get(i) + > _last_obj_copy_times_ms.get(i) + > _last_termination_times_ms.get(i); > > double worker_other_time = worker_time - worker_known_time; > _last_gc_worker_other_times_ms.set(i, worker_other_time); > } > >> >> Also your patch comment has an extra capital letter: WorkerDataARray. > > Thanks for catching this. I'll fix it before I push. > >> Ship it! > > Thanks again for looking at this! > Bengt > >> >> JohnC >> >> On 8/22/2012 4:16 AM, Bengt Rutisson wrote: >>> >>> Hi again, >>> >>> Here is an updated webrev based on comments from John Cuthbertson: >>> http://cr.openjdk.java.net/~brutisso/7178363/webrev.02/ >>> >>> Here is a diff compared to the previous webrev: >>> http://cr.openjdk.java.net/~brutisso/7178363/webrev.01-02-diff/ >>> >>> Thanks, >>> Bengt >>> >>> On 2012-07-20 14:17, Bengt Rutisson wrote: >>>> >>>> Hi all, >>>> >>>> Here is an updated webrev for this change: >>>> http://cr.openjdk.java.net/~brutisso/7178363/webrev.01/ >>>> >>>> It turns out the the earlier change for 7178361 had introduced two >>>> more issues: the heap transition information for the PrintGC output >>>> and the output for evacuation failures had both been moved in an >>>> unintended way. The above webrev corrects both of these chagne too. >>>> Thanks John Cuthbertson for pointing me to the evacuation failure >>>> output. >>>> >>>> Bengt >>>> >>>> >>>> On 2012-07-19 11:01, Bengt Rutisson wrote: >>>>> >>>>> Hi again, >>>>> >>>>> Just in case anybody already started looking at this: I have >>>>> updated the webrev since I had to make some changes to make it >>>>> compile with the NMT fixes that have just made it into the >>>>> hotspot-gc repository. Those updates were just to make it compile >>>>> and not really related to my change, so I just overwrote the >>>>> existing webrev. Just use the same link as before: >>>>> >>>>> http://cr.openjdk.java.net/~brutisso/7178363/webrev.00/ >>>>> >>>>> Also, if you want to see what the new output looks like I am >>>>> attaching a file called new.txt with an example from running >>>>> SpecJBB2005 with this command line: >>>>> >>>>> -XX:+UseG1GC -XX:ParallelGCThreads=4 >>>>> -XX:+UnlockExperimentalVMOptions -XX:G1LogLevel=finest >>>>> -XX:+TraceGen0Time -Xms256m -Xmx2G >>>>> >>>>> I am also attaching a file called old.txt with what the output, >>>>> using the same command line, looked like before my change. As you >>>>> can see the differences are what I listed in my earlier email. You >>>>> will also notice that the "old" version has an entry for the SATB >>>>> filtering, even though all the entries are 0 (we didn't do a >>>>> concurrent cycle so there has been no SATB filtering). This was a >>>>> bug I just introduced with my last change (for 7178361), so the >>>>> new example is more correct. >>>>> >>>>> Thanks, >>>>> Bengt >>>>> >>>>> On 2012-07-18 15:55, Bengt Rutisson wrote: >>>>>> >>>>>> Hi everyone, >>>>>> >>>>>> I would like some reviews of this change: >>>>>> http://cr.openjdk.java.net/~brutisso/7178363/webrev.00/ >>>>>> >>>>>> This removes the special treatment for ParallelGCThreads=0 from >>>>>> the G1 logging. I did keep the log output unchanged for that >>>>>> case. Basically it just has one indentation level less and skips >>>>>> some output. I am not sure this is really necessary since it is >>>>>> really a special case. I'm open to change that special treatment >>>>>> too and just have the same output as for ParallelGCThreads=1. >>>>>> >>>>>> The PrintGCDetails log output should be the same as before with >>>>>> three minor adjustments: >>>>>> >>>>>> - The "Sum" is now not printed for the start and end values for >>>>>> GC workers. This sum does not really make sense to me. >>>>>> >>>>>> - The "(ms)" unit was removed from output that aren't in >>>>>> milliseconds (termination attempts for example). >>>>>> >>>>>> - The average value is now printed as a double for all types. >>>>>> >>>>>> I tried to clean up the code a bit and introduced a separate >>>>>> class, Snippet WorkerDataArray, to keep track of the per thread >>>>>> logging. I also introduced getters and setters to avoid having to >>>>>> make G1CollectorPolicy and TraceGen0TimeData friend classes to >>>>>> G1GCPhaseTimes. >>>>>> >>>>>> Thanks, >>>>>> Bengt >>>>> >>>> >>> > From john.coomes at oracle.com Fri Aug 24 03:46:47 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 24 Aug 2012 03:46:47 +0000 Subject: hg: hsx/hotspot-gc: Added tag jdk8-b53 for changeset febd7ff52800 Message-ID: <20120824034647.C9864476CC@hg.openjdk.java.net> Changeset: c1a277c6022a Author: katleman Date: 2012-08-23 12:27 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/rev/c1a277c6022a Added tag jdk8-b53 for changeset febd7ff52800 ! .hgtags From john.coomes at oracle.com Fri Aug 24 03:46:52 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 24 Aug 2012 03:46:52 +0000 Subject: hg: hsx/hotspot-gc/corba: Added tag jdk8-b53 for changeset 63aeb7a2472f Message-ID: <20120824034655.4A561476CD@hg.openjdk.java.net> Changeset: 16c82fc74695 Author: katleman Date: 2012-08-23 12:27 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/corba/rev/16c82fc74695 Added tag jdk8-b53 for changeset 63aeb7a2472f ! .hgtags From john.coomes at oracle.com Fri Aug 24 03:47:00 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 24 Aug 2012 03:47:00 +0000 Subject: hg: hsx/hotspot-gc/jaxp: Added tag jdk8-b53 for changeset 2c566f25c39f Message-ID: <20120824034710.D4650476CE@hg.openjdk.java.net> Changeset: 7dd81ccb7c11 Author: katleman Date: 2012-08-23 12:27 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jaxp/rev/7dd81ccb7c11 Added tag jdk8-b53 for changeset 2c566f25c39f ! .hgtags From john.coomes at oracle.com Fri Aug 24 03:47:17 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 24 Aug 2012 03:47:17 +0000 Subject: hg: hsx/hotspot-gc/jaxws: Added tag jdk8-b53 for changeset 8a35fd644d3c Message-ID: <20120824034724.A75E5476CF@hg.openjdk.java.net> Changeset: 91970935926a Author: katleman Date: 2012-08-23 12:27 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jaxws/rev/91970935926a Added tag jdk8-b53 for changeset 8a35fd644d3c ! .hgtags From john.coomes at oracle.com Fri Aug 24 03:47:35 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 24 Aug 2012 03:47:35 +0000 Subject: hg: hsx/hotspot-gc/jdk: Added tag jdk8-b53 for changeset 2c6933c5106b Message-ID: <20120824034833.5B5EA476D0@hg.openjdk.java.net> Changeset: 156ab3c38556 Author: katleman Date: 2012-08-23 12:27 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jdk/rev/156ab3c38556 Added tag jdk8-b53 for changeset 2c6933c5106b ! .hgtags From john.coomes at oracle.com Fri Aug 24 03:49:47 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 24 Aug 2012 03:49:47 +0000 Subject: hg: hsx/hotspot-gc/langtools: Added tag jdk8-b53 for changeset d3d0b9cd76e0 Message-ID: <20120824034954.DED9F476D1@hg.openjdk.java.net> Changeset: 9cf72631baf5 Author: katleman Date: 2012-08-23 12:27 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/langtools/rev/9cf72631baf5 Added tag jdk8-b53 for changeset d3d0b9cd76e0 ! .hgtags From alejandro.murillo at oracle.com Sat Aug 25 06:01:32 2012 From: alejandro.murillo at oracle.com (alejandro.murillo at oracle.com) Date: Sat, 25 Aug 2012 06:01:32 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 26 new changesets Message-ID: <20120825060225.C51A147703@hg.openjdk.java.net> Changeset: 54240c1b8e87 Author: katleman Date: 2012-08-16 11:43 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/54240c1b8e87 Added tag jdk8-b52 for changeset 6d0436885201 ! .hgtags Changeset: de2aa86e037d Author: katleman Date: 2012-08-23 12:27 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/de2aa86e037d Added tag jdk8-b53 for changeset 54240c1b8e87 ! .hgtags Changeset: 6acee021f5ac Author: coleenp Date: 2012-08-01 16:52 -0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/6acee021f5ac 7129723: MAC: Some regression tests need to recognize Mac OS X platform Summary: Add Darwin like Linux to shell scripts Reviewed-by: kvn, kamg, dholmes ! test/compiler/6894807/Test6894807.sh ! test/gc/6941923/test6941923.sh ! test/runtime/6626217/Test6626217.sh ! test/runtime/6878713/Test6878713.sh ! test/runtime/6929067/Test6929067.sh ! test/runtime/7051189/Xchecksig.sh ! test/runtime/7110720/Test7110720.sh ! test/runtime/7158800/Test7158800.sh ! test/runtime/7158988/TestFieldMonitor.sh Changeset: 4acebbe310e1 Author: zgu Date: 2012-08-01 17:19 -0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/4acebbe310e1 7185614: NMT ON: "check by caller" assertion failed on nsk ThreadMXBean test 7187429: NMT ON: Merge failure should cause NMT to shutdown Summary: Fixed NMT assertion failures Reviewed-by: acorn, kvn ! src/share/vm/services/memSnapshot.cpp ! src/share/vm/services/memSnapshot.hpp ! src/share/vm/services/memTrackWorker.cpp ! src/share/vm/services/memTracker.hpp Changeset: b27675afea11 Author: zgu Date: 2012-08-01 15:00 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/b27675afea11 Merge Changeset: 8e69438de9c6 Author: zgu Date: 2012-08-01 16:16 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/8e69438de9c6 Merge Changeset: 282abd0fd878 Author: dcubed Date: 2012-08-02 14:54 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/282abd0fd878 7188168: 7071904 broke the DEBUG_BINARIES option on Linux Summary: Change DEBUG_BINARIES option logic to be more clear. Reviewed-by: fparain, andrew ! make/linux/makefiles/adlc.make ! make/linux/makefiles/gcc.make Changeset: 0d8e265ba727 Author: dcubed Date: 2012-08-03 18:34 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/0d8e265ba727 7181175: Enable builds on Windows with MinGW/MSYS Summary: This fix is the minimum number of Makefile changes to enable building HotSpot with MinGW/MSYS Reviewed-by: jcoomes, dcubed, tbell, ohair Contributed-by: volker.simonis at gmail.com ! make/windows/makefiles/defs.make ! make/windows/makefiles/rules.make ! make/windows/makefiles/sa.make ! make/windows/makefiles/shared.make Changeset: c3c2141203e7 Author: dcubed Date: 2012-08-06 09:34 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/c3c2141203e7 Merge Changeset: 4ee06e614636 Author: kamg Date: 2012-08-06 15:54 -0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/4ee06e614636 7116786: RFE: Detailed information on VerifyErrors Summary: Provide additional detail in VerifyError messages Reviewed-by: sspitsyn, acorn ! src/os/solaris/vm/dtraceJSDT_solaris.cpp ! src/share/vm/classfile/stackMapFrame.cpp ! src/share/vm/classfile/stackMapFrame.hpp ! src/share/vm/classfile/stackMapTable.cpp ! src/share/vm/classfile/stackMapTable.hpp ! src/share/vm/classfile/stackMapTableFormat.hpp ! src/share/vm/classfile/verificationType.cpp ! src/share/vm/classfile/verificationType.hpp ! src/share/vm/classfile/verifier.cpp ! src/share/vm/classfile/verifier.hpp ! src/share/vm/runtime/globals.hpp ! src/share/vm/runtime/relocator.cpp ! src/share/vm/utilities/ostream.cpp ! src/share/vm/utilities/ostream.hpp + test/runtime/7116786/Test7116786.java + test/runtime/7116786/testcases.jar Changeset: 98625323d3a3 Author: tbell Date: 2012-08-10 23:16 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/98625323d3a3 7190512: Fix for 7181175 broke hotspot/make/windows/create.bat builds Summary: Add some quotes around the classpath in the project file rule. Reviewed-by: dcubed ! make/windows/projectfiles/common/Makefile Changeset: e5bf1c79ed5b Author: zgu Date: 2012-08-14 13:56 -0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/e5bf1c79ed5b 7191124: Optimized build is broken due to inconsistent use of DEBUG_ONLY and NOT_PRODUCT macros in NMT Summary: Updated all related variables and methods to use NOT_PRODUCT macros Reviewed-by: coleenp, acorn, kvn ! src/share/vm/services/memPtr.cpp ! src/share/vm/services/memPtr.hpp ! src/share/vm/services/memPtrArray.hpp ! src/share/vm/services/memRecorder.hpp ! src/share/vm/services/memSnapshot.cpp ! src/share/vm/services/memTracker.cpp Changeset: fce6d7280776 Author: dcubed Date: 2012-08-17 11:57 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/fce6d7280776 Merge ! src/share/vm/classfile/verifier.cpp ! src/share/vm/runtime/globals.hpp Changeset: b63c0564035a Author: dcubed Date: 2012-08-21 19:25 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/b63c0564035a Merge Changeset: ce0254b13eb8 Author: brutisso Date: 2012-08-24 09:45 +0200 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/ce0254b13eb8 Merge Changeset: 006050192a5a Author: kvn Date: 2012-08-20 09:07 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/006050192a5a 6340864: Implement vectorization optimizations in hotspot-server Summary: Added asm encoding and mach nodes for vector arithmetic instructions on x86. Reviewed-by: roland ! src/cpu/x86/vm/assembler_x86.cpp ! src/cpu/x86/vm/assembler_x86.hpp ! src/cpu/x86/vm/x86.ad ! src/cpu/x86/vm/x86_32.ad ! src/cpu/x86/vm/x86_64.ad ! src/share/vm/opto/classes.hpp ! src/share/vm/opto/loopnode.cpp ! src/share/vm/opto/superword.cpp ! src/share/vm/opto/vectornode.cpp ! src/share/vm/opto/vectornode.hpp + test/compiler/6340864/TestByteVect.java + test/compiler/6340864/TestDoubleVect.java + test/compiler/6340864/TestFloatVect.java + test/compiler/6340864/TestIntVect.java + test/compiler/6340864/TestLongVect.java + test/compiler/6340864/TestShortVect.java Changeset: 09aad8452938 Author: kvn Date: 2012-08-20 09:58 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/09aad8452938 7190310: Inlining WeakReference.get(), and hoisting $referent may lead to non-terminating loops Summary: In C2 add software membar after load from Reference.referent field to prevent commoning of loads across safepoint since GC can change its value. In C1 always generate Reference.get() intrinsic. Reviewed-by: roland, twisti, dholmes, johnc ! src/cpu/sparc/vm/c1_CodeStubs_sparc.cpp ! src/cpu/x86/vm/c1_CodeStubs_x86.cpp ! src/share/vm/c1/c1_CodeStubs.hpp ! src/share/vm/c1/c1_GraphBuilder.cpp ! src/share/vm/c1/c1_LIRGenerator.cpp ! src/share/vm/opto/idealKit.cpp ! src/share/vm/opto/library_call.cpp + test/compiler/7190310/Test7190310.java + test/compiler/7190310/Test7190310_unsafe.java Changeset: 7a302948f5a4 Author: twisti Date: 2012-08-21 10:48 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/7a302948f5a4 7192167: JSR 292: C1 has old broken code which needs to be removed Reviewed-by: kvn, roland, jrose ! src/share/vm/c1/c1_GraphBuilder.cpp ! src/share/vm/c1/c1_Instruction.cpp ! src/share/vm/c1/c1_LIRAssembler.cpp ! src/share/vm/c1/c1_LIRGenerator.cpp ! src/share/vm/opto/callGenerator.cpp Changeset: 4b0d6fd74911 Author: kvn Date: 2012-08-21 14:50 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/4b0d6fd74911 7192964: assert(false) failed: bad AD file Summary: Shifts with loop variant counts "a[i]=1<= 1) failed: illegal call to offset() Summary: C1 value numbering hits unloaded klass. Reviewed-by: kvn, twisti ! src/share/vm/c1/c1_ValueMap.cpp ! src/share/vm/c1/c1_ValueMap.hpp Changeset: 5af51c882207 Author: kvn Date: 2012-08-22 11:55 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/5af51c882207 7192963: assert(_in[req-1] == this) failed: Must pass arg count to 'new' Summary: Fixed Pack node generation. Not vectorize shift instructions if count is not the same for all shifts and if count is vector. Reviewed-by: twisti ! src/share/vm/opto/compile.cpp ! src/share/vm/opto/superword.cpp ! src/share/vm/opto/vectornode.cpp ! src/share/vm/opto/vectornode.hpp + test/compiler/7192963/TestByteVect.java + test/compiler/7192963/TestDoubleVect.java + test/compiler/7192963/TestFloatVect.java + test/compiler/7192963/TestIntVect.java + test/compiler/7192963/TestLongVect.java + test/compiler/7192963/TestShortVect.java Changeset: f7cd53cedd78 Author: kvn Date: 2012-08-23 09:13 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/f7cd53cedd78 7192965: assert(is_aligned_sets(size)) failed: mask is not aligned, adjacent sets Summary: Change pair check to vector check in RA bias coloring code. Reviewed-by: jrose, twisti ! src/share/vm/opto/chaitin.cpp ! src/share/vm/opto/output.cpp Changeset: c32dee9b8023 Author: twisti Date: 2012-08-24 11:48 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/c32dee9b8023 Merge Changeset: 9e3ae661284d Author: amurillo Date: 2012-08-24 15:51 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/9e3ae661284d Merge - src/share/vm/gc_implementation/parNew/parGCAllocBuffer.cpp - src/share/vm/gc_implementation/parNew/parGCAllocBuffer.hpp Changeset: e8fb566b9466 Author: amurillo Date: 2012-08-24 15:51 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/e8fb566b9466 Added tag hs24-b21 for changeset 9e3ae661284d ! .hgtags Changeset: 153776c4cb6f Author: amurillo Date: 2012-08-24 16:23 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/153776c4cb6f 7194004: new hotspot build - hs24-b22 Reviewed-by: jcoomes ! make/hotspot_version From bengt.rutisson at oracle.com Sun Aug 26 21:40:35 2012 From: bengt.rutisson at oracle.com (bengt.rutisson at oracle.com) Date: Sun, 26 Aug 2012 21:40:35 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 7178363: G1: Remove the serial code for PrintGCDetails and make it a special case of the parallel code Message-ID: <20120826214040.B24C447728@hg.openjdk.java.net> Changeset: bb3f6194fedb Author: brutisso Date: 2012-08-23 10:21 +0200 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/bb3f6194fedb 7178363: G1: Remove the serial code for PrintGCDetails and make it a special case of the parallel code Summary: Also reviewed by vitalyd at gmail.com. Introduced the WorkerDataArray class. Fixed some minor logging bugs. Reviewed-by: johnc, mgerdin ! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp ! src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp ! src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp ! src/share/vm/gc_implementation/g1/g1GCPhaseTimes.cpp ! src/share/vm/gc_implementation/g1/g1GCPhaseTimes.hpp ! src/share/vm/gc_implementation/g1/g1RemSet.cpp ! src/share/vm/runtime/arguments.cpp From michal at frajt.eu Mon Aug 27 17:17:40 2012 From: michal at frajt.eu (Michal Frajt) Date: Mon, 27 Aug 2012 19:17:40 +0200 Subject: [PATCH] 7189971: Implement CMSWaitDuration for non-incremental mode of CMS Message-ID: Hi Ramki / Jon, Please find the patch for the CMSWaitDuration unstable behavior issue. The patch keeps the method wait_on_cms_lock untouched for the calls from the abortable_preclean phase (not very correct behaviour but still acceptable for the abortable preclean 'short break' calls between the preclean work iterations). The new method wait_on_cms_lock_for_scavenge has been added. The method monitors the CGC_lock for notifications, handles the full wait time interval, checks the scavenge occurrence by the total_collections counter changing its value. When reviewing please mind that the allowed locking order in the CMS thread should be FreelistHolder -> Heap_lock -> CGC_lock (based on a source code comment but the collect_in_background method is using reverted order between the Freelist and the Heap_lock ??). The sleepBeforeNextCycle method is now using the new wait_on_cms_lock_for_scavenge method for both the normal and the incremental CMS mode. The patch has been prepared for the openjdk6 and openjdk7u. The openjdk6 got compiled and tested on solaris-amd64 platform. The openjdk7u got compiled without much testing (we have jdk6 application environment only). You additionally suggested to have an explicit flag such as CMSScavengeBeforeInitialMark. I already replied to it but it did not get into the posting list (sent from another email address). The idea of the CMSScavengeBeforeInitialMark could be easier to implement but we strongly prefer not to invoke yet another scavenge explicitly as it is unbalancing young objects aging and leads to unwanted promotions. I could think about a combined solution when it first waits for the CMSWaitDuration and, if there is no scavenge occurring, it is explicitly invoking a scavenge before the inital-mark phase (or better pause) starts. Regards, Michal -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: openjdk6-hostspot-7189971.patch Type: application/octet-stream Size: 4421 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: openjdk7u-hostspot-7189971.patch Type: application/octet-stream Size: 4421 bytes Desc: not available URL: From john.cuthbertson at oracle.com Tue Aug 28 00:36:26 2012 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Mon, 27 Aug 2012 17:36:26 -0700 Subject: RFR(S): 7190666: G1: assert(_unused == 0) failed: Inconsistency in PLAB stats Message-ID: <503C128A.3060909@oracle.com> Hi Everyone, Can I have a couple of volunteers review the changes for this CR? The webrev can be found at: http://cr.openjdk.java.net/~johnc/7190666/webrev.0/ Summary: The value of PLABStats::_allocated was overflowing and the failed assertion detected when it overflowed to 0. When we retired an individual allocation buffer, we were flushing some accumulated values to the associated PLABStats instance. This was artificially inflating the values in the PLABStats instance since we were not reseting the accumulated values in the ParGCAllocBuffer after we flushed. Ordinarily this would not cause an issue (other than the values being too large) but with this particular test case we obtained an evacuation failure. As a result we were executing the GC allocation slow-path, and flushing the accumulated values, for every failed attempted object allocation (even though we were unable to allocate a new buffer), and we overflowed. Reseting the sensor values in the ParGCAllocBuffer instance after flushing prevents the artificial inflation and overflow. Additionally we should not be flushing the values to the PLABStats instance on every buffer retirement (though it is not stated in the code). Flushing the stats values on every retirement is unnecessary and, in the case of an evacuation, adds a fair amount of additional work for each failed object copy. Instead we should only be flushing the accumulated sensor values when we retire the final buffers prior to disposing the G1ParScanThreadState object. Testing: The failing test case; the GC test suite with +PrintPLAB, and jprt. Note while testing this I ran into some assertion and guarantee failures from G1's block offset table. I've only seen and been able (so far) to reproduce these failures on a single machine in the jprt pool. I will be submitting a new CR for these failures. I do not believe that the failures are related to this fix (or the change that enabled resize-able PLABS) as I've been able to reproduce the failures with disabling ResizePLAB and setting OldPLABSize=8k, 16k, and 32k. Thanks, JohnC From jesper.wilhelmsson at oracle.com Tue Aug 28 07:44:54 2012 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Tue, 28 Aug 2012 09:44:54 +0200 Subject: RFR(S): 7190666: G1: assert(_unused == 0) failed: Inconsistency in PLAB stats In-Reply-To: <503C128A.3060909@oracle.com> References: <503C128A.3060909@oracle.com> Message-ID: <503C76F6.3070408@oracle.com> Looks good! I especially like that you added more information to the assert message. That will be helpful the next time the assert triggers. /Jesper On 2012-08-28 02:36, John Cuthbertson wrote: > Hi Everyone, > > Can I have a couple of volunteers review the changes for this CR? The webrev > can be found at: http://cr.openjdk.java.net/~johnc/7190666/webrev.0/ > > Summary: > The value of PLABStats::_allocated was overflowing and the failed assertion > detected when it overflowed to 0. When we retired an individual allocation > buffer, we were flushing some accumulated values to the associated PLABStats > instance. This was artificially inflating the values in the PLABStats > instance since we were not reseting the accumulated values in the > ParGCAllocBuffer after we flushed. Ordinarily this would not cause an issue > (other than the values being too large) but with this particular test case we > obtained an evacuation failure. As a result we were executing the GC > allocation slow-path, and flushing the accumulated values, for every failed > attempted object allocation (even though we were unable to allocate a new > buffer), and we overflowed. Reseting the sensor values in the > ParGCAllocBuffer instance after flushing prevents the artificial inflation > and overflow. > > Additionally we should not be flushing the values to the PLABStats instance > on every buffer retirement (though it is not stated in the code). Flushing > the stats values on every retirement is unnecessary and, in the case of an > evacuation, adds a fair amount of additional work for each failed object > copy. Instead we should only be flushing the accumulated sensor values when > we retire the final buffers prior to disposing the G1ParScanThreadState object. > > Testing: > The failing test case; the GC test suite with +PrintPLAB, and jprt. > > Note while testing this I ran into some assertion and guarantee failures from > G1's block offset table. I've only seen and been able (so far) to reproduce > these failures on a single machine in the jprt pool. I will be submitting a > new CR for these failures. I do not believe that the failures are related to > this fix (or the change that enabled resize-able PLABS) as I've been able to > reproduce the failures with disabling ResizePLAB and setting OldPLABSize=8k, > 16k, and 32k. > > Thanks, > > JohnC -------------- next part -------------- A non-text attachment was scrubbed... Name: jesper_wilhelmsson.vcf Type: text/x-vcard Size: 251 bytes Desc: not available URL: From thomas.schatzl at jku.at Tue Aug 28 08:37:53 2012 From: thomas.schatzl at jku.at (Thomas Schatzl) Date: Tue, 28 Aug 2012 10:37:53 +0200 Subject: RFR(S): 7190666: G1: assert(_unused == 0) failed: Inconsistency in PLAB stats In-Reply-To: <503C128A.3060909@oracle.com> References: <503C128A.3060909@oracle.com> Message-ID: <1346143073.27667.131.camel@gremio> Hi, On Mon, 2012-08-27 at 17:36 -0700, John Cuthbertson wrote: > Hi Everyone, > > Can I have a couple of volunteers review the changes for this CR? The > webrev can be found at: http://cr.openjdk.java.net/~johnc/7190666/webrev.0/ > > > Note while testing this I ran into some assertion and guarantee failures > from G1's block offset table. I've only seen and been able (so far) to > reproduce these failures on a single machine in the jprt pool. I will be > submitting a new CR for these failures. I do not believe that the > failures are related to this fix (or the change that enabled resize-able > PLABS) as I've been able to reproduce the failures with disabling > ResizePLAB and setting OldPLABSize=8k, 16k, and 32k. Could you describe the failure/assertion and possibly any circumstances it occurs? I've run into some card table issues some time ago too. (or the CR to see the description) Maybe, coincidently, they're the same. Thomas From jon.masamitsu at oracle.com Tue Aug 28 17:00:21 2012 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Tue, 28 Aug 2012 10:00:21 -0700 Subject: RFR(S): 7190666: G1: assert(_unused == 0) failed: Inconsistency in PLAB stats In-Reply-To: <503C128A.3060909@oracle.com> References: <503C128A.3060909@oracle.com> Message-ID: <503CF925.1070501@oracle.com> Looks good. On 8/27/2012 5:36 PM, John Cuthbertson wrote: > Hi Everyone, > > Can I have a couple of volunteers review the changes for this CR? The > webrev can be found at: > http://cr.openjdk.java.net/~johnc/7190666/webrev.0/ > > Summary: > The value of PLABStats::_allocated was overflowing and the failed > assertion detected when it overflowed to 0. When we retired an > individual allocation buffer, we were flushing some accumulated values > to the associated PLABStats instance. This was artificially inflating > the values in the PLABStats instance since we were not reseting the > accumulated values in the ParGCAllocBuffer after we flushed. > Ordinarily this would not cause an issue (other than the values being > too large) but with this particular test case we obtained an > evacuation failure. As a result we were executing the GC allocation > slow-path, and flushing the accumulated values, for every failed > attempted object allocation (even though we were unable to allocate a > new buffer), and we overflowed. Reseting the sensor values in the > ParGCAllocBuffer instance after flushing prevents the artificial > inflation and overflow. > > Additionally we should not be flushing the values to the PLABStats > instance on every buffer retirement (though it is not stated in the > code). Flushing the stats values on every retirement is unnecessary > and, in the case of an evacuation, adds a fair amount of additional > work for each failed object copy. Instead we should only be flushing > the accumulated sensor values when we retire the final buffers prior > to disposing the G1ParScanThreadState object. > > Testing: > The failing test case; the GC test suite with +PrintPLAB, and jprt. > > Note while testing this I ran into some assertion and guarantee > failures from G1's block offset table. I've only seen and been able > (so far) to reproduce these failures on a single machine in the jprt > pool. I will be submitting a new CR for these failures. I do not > believe that the failures are related to this fix (or the change that > enabled resize-able PLABS) as I've been able to reproduce the failures > with disabling ResizePLAB and setting OldPLABSize=8k, 16k, and 32k. > > Thanks, > > JohnC From john.cuthbertson at oracle.com Tue Aug 28 17:58:26 2012 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Tue, 28 Aug 2012 10:58:26 -0700 Subject: RFR(XXXS): 7194409: os::javaTimeNanos() shows hot on CPU_CLK_UNHALTED profiles Message-ID: <503D06C2.6080602@oracle.com> Hi Everyone, Can I have another volunteer to review the changes for this CR (which were contributed by Brandon Mitchell at Twitter)? The webrev can be found at: http://cr.openjdk.java.net/~johnc/7194409/webrev.0/ Here's the description, according to Brandon: > os::javaTimeNanos() shows hot on CPU_CLK_UNHALTED profiles due to > unnecessary function setup/teardown code around > Linux::supports_monotonic_clock(). I've added an inline annotation to > simplify the funcall to a NULL check, and verified the change using > both gdb disas and additional profiling. I observed a 2-3% CPU time > delta in my profile data for os::javaTimeNanos(). > Thanks, JohnC From mikael.vidstedt at oracle.com Tue Aug 28 18:36:12 2012 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Tue, 28 Aug 2012 11:36:12 -0700 Subject: RFR(XXXS): 7194409: os::javaTimeNanos() shows hot on CPU_CLK_UNHALTED profiles In-Reply-To: <503D06C2.6080602@oracle.com> References: <503D06C2.6080602@oracle.com> Message-ID: <503D0F9C.2030209@oracle.com> Looks good. If this function is indeed this hot it sure would be interesting to see what the effects would be of inlining os::javaTimeNanos too. Cheers, Mikael On 8/28/2012 10:58 AM, John Cuthbertson wrote: > Hi Everyone, > > Can I have another volunteer to review the changes for this CR (which > were contributed by Brandon Mitchell at Twitter)? The webrev can be > found at: http://cr.openjdk.java.net/~johnc/7194409/webrev.0/ > > Here's the description, according to Brandon: > >> os::javaTimeNanos() shows hot on CPU_CLK_UNHALTED profiles due to >> unnecessary function setup/teardown code around >> Linux::supports_monotonic_clock(). I've added an inline annotation to >> simplify the funcall to a NULL check, and verified the change using >> both gdb disas and additional profiling. I observed a 2-3% CPU time >> delta in my profile data for os::javaTimeNanos(). > Thanks, > > JohnC From azeem.jiva at oracle.com Tue Aug 28 18:45:56 2012 From: azeem.jiva at oracle.com (Azeem Jiva) Date: Tue, 28 Aug 2012 13:45:56 -0500 Subject: RFR(XXXS): 7194409: os::javaTimeNanos() shows hot on CPU_CLK_UNHALTED profiles In-Reply-To: <503D06C2.6080602@oracle.com> References: <503D06C2.6080602@oracle.com> Message-ID: <503D11E4.4000706@oracle.com> Looks good to me Azeem Jiva @javawithjiva On 08/28/2012 12:58 PM, John Cuthbertson wrote: > Hi Everyone, > > Can I have another volunteer to review the changes for this CR (which > were contributed by Brandon Mitchell at Twitter)? The webrev can be > found at: http://cr.openjdk.java.net/~johnc/7194409/webrev.0/ > > Here's the description, according to Brandon: > >> os::javaTimeNanos() shows hot on CPU_CLK_UNHALTED profiles due to >> unnecessary function setup/teardown code around >> Linux::supports_monotonic_clock(). I've added an inline annotation to >> simplify the funcall to a NULL check, and verified the change using >> both gdb disas and additional profiling. I observed a 2-3% CPU time >> delta in my profile data for os::javaTimeNanos(). > Thanks, > > JohnC -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.cuthbertson at oracle.com Tue Aug 28 20:10:35 2012 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Tue, 28 Aug 2012 13:10:35 -0700 Subject: RFR(S): 7190666: G1: assert(_unused == 0) failed: Inconsistency in PLAB stats In-Reply-To: <1346143073.27667.131.camel@gremio> References: <503C128A.3060909@oracle.com> <1346143073.27667.131.camel@gremio> Message-ID: <503D25BB.2050909@oracle.com> Hi Thomas. The CR is 7194633. It should be available externally by the time you read this. Regards, JohnC On 08/28/12 01:37, Thomas Schatzl wrote: > Hi, > > On Mon, 2012-08-27 at 17:36 -0700, John Cuthbertson wrote: > >> Hi Everyone, >> >> Can I have a couple of volunteers review the changes for this CR? The >> webrev can be found at: http://cr.openjdk.java.net/~johnc/7190666/webrev.0/ >> >> >> Note while testing this I ran into some assertion and guarantee failures >> from G1's block offset table. I've only seen and been able (so far) to >> reproduce these failures on a single machine in the jprt pool. I will be >> submitting a new CR for these failures. I do not believe that the >> failures are related to this fix (or the change that enabled resize-able >> PLABS) as I've been able to reproduce the failures with disabling >> ResizePLAB and setting OldPLABSize=8k, 16k, and 32k. >> > > Could you describe the failure/assertion and possibly any circumstances > it occurs? I've run into some card table issues some time ago too. > (or the CR to see the description) > > Maybe, coincidently, they're the same. > > Thomas > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.cuthbertson at oracle.com Tue Aug 28 20:19:14 2012 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Tue, 28 Aug 2012 13:19:14 -0700 Subject: RFR(S): 7190666: G1: assert(_unused == 0) failed: Inconsistency in PLAB stats In-Reply-To: <503C76F6.3070408@oracle.com> References: <503C128A.3060909@oracle.com> <503C76F6.3070408@oracle.com> Message-ID: <503D27C2.4030508@oracle.com> Hi Jesper, Thanks for the review. The additional info was very helpful and, since _allocated, _wasted, and _unused are updated together, led me to the overflow conclusion. We have a CR to extend various asserts and guarantees in G1 with more descriptive error messages (6949301). One of the engineers at Twitter has volunteered to lead the effort. Thanks, JohnC On 08/28/12 00:44, Jesper Wilhelmsson wrote: > Looks good! > > I especially like that you added more information to the assert > message. That will be helpful the next time the assert triggers. > /Jesper > > On 2012-08-28 02:36, John Cuthbertson wrote: >> Hi Everyone, >> >> Can I have a couple of volunteers review the changes for this CR? The >> webrev can be found at: >> http://cr.openjdk.java.net/~johnc/7190666/webrev.0/ >> >> Summary: >> The value of PLABStats::_allocated was overflowing and the failed >> assertion detected when it overflowed to 0. When we retired an >> individual allocation buffer, we were flushing some accumulated >> values to the associated PLABStats instance. This was artificially >> inflating the values in the PLABStats instance since we were not >> reseting the accumulated values in the ParGCAllocBuffer after we >> flushed. Ordinarily this would not cause an issue (other than the >> values being too large) but with this particular test case we >> obtained an evacuation failure. As a result we were executing the GC >> allocation slow-path, and flushing the accumulated values, for every >> failed attempted object allocation (even though we were unable to >> allocate a new buffer), and we overflowed. Reseting the sensor values >> in the ParGCAllocBuffer instance after flushing prevents the >> artificial inflation and overflow. >> >> Additionally we should not be flushing the values to the PLABStats >> instance on every buffer retirement (though it is not stated in the >> code). Flushing the stats values on every retirement is unnecessary >> and, in the case of an evacuation, adds a fair amount of additional >> work for each failed object copy. Instead we should only be flushing >> the accumulated sensor values when we retire the final buffers prior >> to disposing the G1ParScanThreadState object. >> >> Testing: >> The failing test case; the GC test suite with +PrintPLAB, and jprt. >> >> Note while testing this I ran into some assertion and guarantee >> failures from G1's block offset table. I've only seen and been able >> (so far) to reproduce these failures on a single machine in the jprt >> pool. I will be submitting a new CR for these failures. I do not >> believe that the failures are related to this fix (or the change that >> enabled resize-able PLABS) as I've been able to reproduce the >> failures with disabling ResizePLAB and setting OldPLABSize=8k, 16k, >> and 32k. >> >> Thanks, >> >> JohnC > From john.cuthbertson at oracle.com Tue Aug 28 20:20:09 2012 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Tue, 28 Aug 2012 13:20:09 -0700 Subject: RFR(S): 7190666: G1: assert(_unused == 0) failed: Inconsistency in PLAB stats In-Reply-To: <503CF925.1070501@oracle.com> References: <503C128A.3060909@oracle.com> <503CF925.1070501@oracle.com> Message-ID: <503D27F9.9000202@oracle.com> Hi Jon, Thanks for the review. JohnC On 08/28/12 10:00, Jon Masamitsu wrote: > Looks good. > > On 8/27/2012 5:36 PM, John Cuthbertson wrote: >> Hi Everyone, >> >> Can I have a couple of volunteers review the changes for this CR? The >> webrev can be found at: >> http://cr.openjdk.java.net/~johnc/7190666/webrev.0/ >> >> Summary: >> The value of PLABStats::_allocated was overflowing and the failed >> assertion detected when it overflowed to 0. When we retired an >> individual allocation buffer, we were flushing some accumulated >> values to the associated PLABStats instance. This was artificially >> inflating the values in the PLABStats instance since we were not >> reseting the accumulated values in the ParGCAllocBuffer after we >> flushed. Ordinarily this would not cause an issue (other than the >> values being too large) but with this particular test case we >> obtained an evacuation failure. As a result we were executing the GC >> allocation slow-path, and flushing the accumulated values, for every >> failed attempted object allocation (even though we were unable to >> allocate a new buffer), and we overflowed. Reseting the sensor values >> in the ParGCAllocBuffer instance after flushing prevents the >> artificial inflation and overflow. >> >> Additionally we should not be flushing the values to the PLABStats >> instance on every buffer retirement (though it is not stated in the >> code). Flushing the stats values on every retirement is unnecessary >> and, in the case of an evacuation, adds a fair amount of additional >> work for each failed object copy. Instead we should only be flushing >> the accumulated sensor values when we retire the final buffers prior >> to disposing the G1ParScanThreadState object. >> >> Testing: >> The failing test case; the GC test suite with +PrintPLAB, and jprt. >> >> Note while testing this I ran into some assertion and guarantee >> failures from G1's block offset table. I've only seen and been able >> (so far) to reproduce these failures on a single machine in the jprt >> pool. I will be submitting a new CR for these failures. I do not >> believe that the failures are related to this fix (or the change that >> enabled resize-able PLABS) as I've been able to reproduce the >> failures with disabling ResizePLAB and setting OldPLABSize=8k, 16k, >> and 32k. >> >> Thanks, >> >> JohnC From vitalyd at gmail.com Tue Aug 28 22:50:09 2012 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 28 Aug 2012 18:50:09 -0400 Subject: RFR(XXXS): 7194409: os::javaTimeNanos() shows hot on CPU_CLK_UNHALTED profiles In-Reply-To: <503D06C2.6080602@oracle.com> References: <503D06C2.6080602@oracle.com> Message-ID: Looks good. Not sure if it's hot but should supports_fast_thread_cpu_time() get the same treatment as it's very small? I'm curious if anyone knows why the compiler didn't inline it on its own (I'm assuming this is some not-too-ancient gcc for the Linux build)? Sent from my phone On Aug 28, 2012 1:59 PM, "John Cuthbertson" wrote: > Hi Everyone, > > Can I have another volunteer to review the changes for this CR (which were > contributed by Brandon Mitchell at Twitter)? The webrev can be found at: > http://cr.openjdk.java.net/~**johnc/7194409/webrev.0/ > > Here's the description, according to Brandon: > > os::javaTimeNanos() shows hot on CPU_CLK_UNHALTED profiles due to >> unnecessary function setup/teardown code around >> Linux::supports_monotonic_**clock(). I've added an inline annotation to >> simplify the funcall to a NULL check, and verified the change using >> both gdb disas and additional profiling. I observed a 2-3% CPU time >> delta in my profile data for os::javaTimeNanos(). >> >> > Thanks, > > JohnC > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.cuthbertson at oracle.com Wed Aug 29 02:36:17 2012 From: john.cuthbertson at oracle.com (john.cuthbertson at oracle.com) Date: Wed, 29 Aug 2012 02:36:17 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 7041879: G1: introduce stress testing parameter to cause frequent evacuation failures Message-ID: <20120829023621.9DAC747792@hg.openjdk.java.net> Changeset: c9814fadeb38 Author: johnc Date: 2012-08-28 15:20 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/c9814fadeb38 7041879: G1: introduce stress testing parameter to cause frequent evacuation failures Summary: Add the flags G1EvacuationFailureALot flag (and supporting flags) to force trigger evacuation failures. The support flags control how often to trigger an evacuation failure and during which types of evacuation pause. This functionality is analogous to that of PromotionFailureALot for the other collectors. Reviewed-by: brutisso ! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.inline.hpp ! src/share/vm/gc_implementation/g1/g1_globals.hpp From ysr1729 at gmail.com Wed Aug 29 08:43:33 2012 From: ysr1729 at gmail.com (Srinivas Ramakrishna) Date: Wed, 29 Aug 2012 01:43:33 -0700 Subject: RFR(S): 7190666: G1: assert(_unused == 0) failed: Inconsistency in PLAB stats In-Reply-To: <503D27F9.9000202@oracle.com> References: <503C128A.3060909@oracle.com> <503CF925.1070501@oracle.com> <503D27F9.9000202@oracle.com> Message-ID: Hi John -- Good catch... I suppose we have been getting too large PLABs for so many years and perhaps wasting space to fragmentation as a result of this so far (hmm.. although perhaps the waste calculation in the next cycle allows us to adjust down, but i guess we are prone to oscillate because of the sensor value corruption...) I can imagine that in the case of ParNew+CMS this has been wasting space (because of said oscillations) in the survivor spaces. Thanks for the fix, and it looks good to me too. Any chance that the PLAB stats portion of the fix at least might also be backported to JDK 7, so the performance benefit of a more correct calculation accrues there as well? May be under an MR? (PS: I am curious to know if this showed any change in (gc) performance on any of the usual benchmarks...) thanks! -- ramki On Tue, Aug 28, 2012 at 1:20 PM, John Cuthbertson < john.cuthbertson at oracle.com> wrote: > Hi Jon, > > Thanks for the review. > > JohnC > > > On 08/28/12 10:00, Jon Masamitsu wrote: > >> Looks good. >> >> On 8/27/2012 5:36 PM, John Cuthbertson wrote: >> >>> Hi Everyone, >>> >>> Can I have a couple of volunteers review the changes for this CR? The >>> webrev can be found at: http://cr.openjdk.java.net/~** >>> johnc/7190666/webrev.0/ >>> >>> Summary: >>> The value of PLABStats::_allocated was overflowing and the failed >>> assertion detected when it overflowed to 0. When we retired an individual >>> allocation buffer, we were flushing some accumulated values to the >>> associated PLABStats instance. This was artificially inflating the values >>> in the PLABStats instance since we were not reseting the accumulated values >>> in the ParGCAllocBuffer after we flushed. Ordinarily this would not cause >>> an issue (other than the values being too large) but with this particular >>> test case we obtained an evacuation failure. As a result we were executing >>> the GC allocation slow-path, and flushing the accumulated values, for every >>> failed attempted object allocation (even though we were unable to allocate >>> a new buffer), and we overflowed. Reseting the sensor values in the >>> ParGCAllocBuffer instance after flushing prevents the artificial inflation >>> and overflow. >>> >>> Additionally we should not be flushing the values to the PLABStats >>> instance on every buffer retirement (though it is not stated in the code). >>> Flushing the stats values on every retirement is unnecessary and, in the >>> case of an evacuation, adds a fair amount of additional work for each >>> failed object copy. Instead we should only be flushing the accumulated >>> sensor values when we retire the final buffers prior to disposing the >>> G1ParScanThreadState object. >>> >>> Testing: >>> The failing test case; the GC test suite with +PrintPLAB, and jprt. >>> >>> Note while testing this I ran into some assertion and guarantee failures >>> from G1's block offset table. I've only seen and been able (so far) to >>> reproduce these failures on a single machine in the jprt pool. I will be >>> submitting a new CR for these failures. I do not believe that the failures >>> are related to this fix (or the change that enabled resize-able PLABS) as >>> I've been able to reproduce the failures with disabling ResizePLAB and >>> setting OldPLABSize=8k, 16k, and 32k. >>> >>> Thanks, >>> >>> JohnC >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lberk at redhat.com Wed Aug 29 13:16:41 2012 From: lberk at redhat.com (Lukas Berk) Date: Wed, 29 Aug 2012 09:16:41 -0400 Subject: [RFC] Adding dtrace probe points for more detailed GC information In-Reply-To: <20120719230324.GE26228@redhat.com> References: <20120709135433.GA7093@redhat.com> <20120719230324.GE26228@redhat.com> Message-ID: <20120829131640.GA27472@redhat.com> Hey List, I have a slightly updated patch, splitting collections into begin/end probe points where possible. I've also attached the systemtap tapset I've been using for reference (please note that this patch should work without systemtap dependencies, and as-is with dtrace). This patch features probe points for: - Mark Sweep Collections and compactions - Parallel Collections - G1 collections - Object moves - Tenured Collections - Parallel Scavenges Any comments regarding the patch would be appreciated. Is there anything I can do try and get these changes commited to openjdk? Cheers, Lukas * Lukas Berk [2012-07-19 19:04]: > Thanks for the feedback I've gotten, I've updated my patch, adding in > scavenges, as well as looked into > collect_in_foregroud/collect_in_background which I've added in invoke(). > Where applicable I've also added the gc_cause for collections. > > I'll still add probe points for collecting permanent generations if > needed, however due to the plan to remove them I've held off on adding > them for now. Any comments on the patch would be appreciated. > > Thanks, > > Lukas -------------- next part -------------- +--- openjdk.orig/hotspot/src/share/vm/compiler/oopMap.cpp 2012-06-26 09:24:22.390325184 -0400 ++++ openjdk/hotspot/src/share/vm/compiler/oopMap.cpp 2012-07-06 10:12:44.981413003 -0400 +@@ -33,9 +33,13 @@ + #include "memory/resourceArea.hpp" + #include "runtime/frame.inline.hpp" + #include "runtime/signature.hpp" ++#include "utilities/dtrace.hpp" + #ifdef COMPILER1 + #include "c1/c1_Defs.hpp" + #endif ++#ifndef USDT2 ++ HS_DTRACE_PROBE_DECL1(provider, gc__collection__delete, *uintptr_t); ++#endif /* !USDT2 */ + + // OopMapStream + +@@ -677,6 +681,9 @@ + " - Derived: " INTPTR_FORMAT " Base: " INTPTR_FORMAT " (Offset: %d)", + derived_loc, (address)*derived_loc, (address)base, offset); + } ++#ifndef USDT2 ++ HS_DTRACE_PROBE1(hotspot, gc__collection__delete, entry); ++#endif /* !USDT2 */ + + // Delete entry + delete entry; +--- openjdk.orig/hotspot/src/share/vm/gc_implementation/parallelScavenge/psParallelCompact.cpp 2012-07-12 09:48:40.349999515 -0400 ++++ openjdk/hotspot/src/share/vm/gc_implementation/parallelScavenge/psParallelCompact.cpp 2012-07-19 18:38:07.560757426 -0400 +@@ -53,11 +53,18 @@ + #include "runtime/vmThread.hpp" + #include "services/management.hpp" + #include "services/memoryService.hpp" ++#include "utilities/dtrace.hpp" + #include "utilities/events.hpp" + #include "utilities/stack.inline.hpp" + + #include + ++#ifndef USDT2 ++ HS_DTRACE_PROBE_DECL2(provider, gc__collection__ParallelCompact__clear, *uintptr_t, *uintptr_t); ++ HS_DTRACE_PROBE_DECL2(provider, gc__collection__parallel__collect, *uintptr_t, *uintptr_t); ++ HS_DTRACE_PROBE_DECL4(provider, gc__collection__move, *uintptr_t, *uintptr_t, *uintptr_t, *uintptr_t); ++#endif /* !USDT2 */ ++ + // All sizes are in HeapWords. + const size_t ParallelCompactData::Log2RegionSize = 9; // 512 words + const size_t ParallelCompactData::RegionSize = (size_t)1 << Log2RegionSize; +@@ -433,6 +439,9 @@ + + void ParallelCompactData::clear() + { ++#ifndef USDT2 ++ HS_DTRACE_PROBE2(hotspot, gc__collection__ParallelCompact__clear, &_region_data, _region_data->data_location()); ++#endif /* !USDT2 */ + memset(_region_data, 0, _region_vspace->committed_size()); + } + +@@ -1970,6 +1979,9 @@ + "should be in vm thread"); + + ParallelScavengeHeap* heap = gc_heap(); ++#ifndef USDT2 ++ HS_DTRACE_PROBE2(hotspot, gc__collection__parallel__collect, heap, heap->gc_cause()); ++#endif /* !USDT2 */ + GCCause::Cause gc_cause = heap->gc_cause(); + assert(!heap->is_gc_active(), "not reentrant"); + +@@ -3376,6 +3388,9 @@ + // past the end of the partial object entering the region (if any). + HeapWord* const dest_addr = sd.partial_obj_end(dp_region); + HeapWord* const new_top = _space_info[space_id].new_top(); ++#ifndef USDT2 ++ HS_DTRACE_PROBE4(hotspot, gc__collection__move, &beg_addr, &end_addr, &dest_addr, &new_top); ++#endif /* !USDT2 */ + assert(new_top >= dest_addr, "bad new_top value"); + const size_t words = pointer_delta(new_top, dest_addr); + +--- openjdk.orig/hotspot/src/share/vm/gc_implementation/g1/g1MarkSweep.cpp 2012-08-15 12:04:43.837439833 -0400 ++++ openjdk/hotspot/src/share/vm/gc_implementation/g1/g1MarkSweep.cpp 2012-08-15 12:01:47.897745719 -0400 +@@ -45,8 +45,13 @@ + #include "runtime/thread.hpp" + #include "runtime/vmThread.hpp" + #include "utilities/copy.hpp" ++#include "utilities/dtrace.hpp" + #include "utilities/events.hpp" + ++#ifndef USDT2 ++ HS_DTRACE_PROBE_DECL2(provider, gc__collection__G1__begin, *uintptr_t, *uintptr_t); ++ HS_DTRACE_PROBE_DECL2(provider, gc__collection__G1__end, *uintptr_t, *uintptr_t); ++ #endif /* !USDT2 */ + class HeapRegion; + + void G1MarkSweep::invoke_at_safepoint(ReferenceProcessor* rp, +@@ -84,6 +89,9 @@ + // The marking doesn't preserve the marks of biased objects. + BiasedLocking::preserve_marks(); + ++#ifndef USDT2 ++ HS_DTRACE_PROBE2(hotspot, gc__collection__G1__begin, &sh, sh->gc_cause()); ++#endif /* !USDT2 */ + mark_sweep_phase1(marked_for_unloading, clear_all_softrefs); + + mark_sweep_phase2(); +@@ -103,6 +111,9 @@ + GenRemSet* rs = sh->rem_set(); + rs->invalidate(sh->perm_gen()->used_region(), true /*whole_heap*/); + ++#ifndef USDT2 ++ HS_DTRACE_PROBE2(hotspot, gc__collection__G1__end, &sh, sh->gc_cause()); ++#endif /* !USDT2 */ + // "free at last gc" is calculated from these. + // CHF: cheating for now!!! + // Universe::set_heap_capacity_at_last_gc(Universe::heap()->capacity()); +--- openjdk.orig/hotspot/src/share/vm/memory/tenuredGeneration.cpp 2012-08-15 12:03:43.009543167 -0400 ++++ openjdk/hotspot/src/share/vm/memory/tenuredGeneration.cpp 2012-08-15 12:14:25.414381449 -0400 +@@ -33,6 +33,12 @@ + #include "memory/tenuredGeneration.hpp" + #include "oops/oop.inline.hpp" + #include "runtime/java.hpp" ++#include "utilities/dtrace.hpp" ++ ++#ifndef USDT2 ++ HS_DTRACE_PROBE_DECL4(provider, gc__collection__tenured__begin, bool, bool, size_t, bool); ++ HS_DTRACE_PROBE_DECL4(provider, gc__collection__tenured__end, bool, bool, size_t, bool); ++#endif /* !USDT2 */ + + TenuredGeneration::TenuredGeneration(ReservedSpace rs, + size_t initial_byte_size, int level, +@@ -307,8 +313,14 @@ + size_t size, + bool is_tlab) { + retire_alloc_buffers_before_full_gc(); ++#ifndef USDT2 ++ HS_DTRACE_PROBE4(hotspot, gc__collection__tenured__begin, full, clear_all_soft_refs, size, is_tlab); ++#endif /* !USDT2 */ + OneContigSpaceCardGeneration::collect(full, clear_all_soft_refs, + size, is_tlab); ++#ifndef USDT2 ++ HS_DTRACE_PROBE4(hotspot, gc__collection__tenured__end, full, clear_all_soft_refs, size, is_tlab); ++#endif /* !USDT2 */ + } + + void TenuredGeneration::update_gc_stats(int current_level, +--- openjdk.orig/hotspot/src/share/vm/gc_implementation/parNew/parNewGeneration.cpp 2012-08-15 12:03:43.039543116 -0400 ++++ openjdk/hotspot/src/share/vm/gc_implementation/parNew/parNewGeneration.cpp 2012-08-15 12:18:57.181932342 -0400 +@@ -49,6 +49,12 @@ + #include "utilities/copy.hpp" + #include "utilities/globalDefinitions.hpp" + #include "utilities/workgroup.hpp" ++#include "utilities/dtrace.hpp" ++ ++#ifndef USDT2 ++ HS_DTRACE_PROBE_DECL4(provider, gc__collection__parnew__begin, bool, bool, size_t, bool); ++ HS_DTRACE_PROBE_DECL4(provider, gc__collection__parnew__end, bool, bool, size_t, bool); ++#endif /* !USDT2 */ + + #ifdef _MSC_VER + #pragma warning( push ) +@@ -878,6 +884,9 @@ + bool clear_all_soft_refs, + size_t size, + bool is_tlab) { ++#ifndef USDT2 ++ HS_DTRACE_PROBE4(hotspot, gc__collection__parnew__begin, full, clear_all_soft_refs, size, is_tlab); ++#endif /* !USDT2 */ + assert(full || size > 0, "otherwise we don't want to collect"); + GenCollectedHeap* gch = GenCollectedHeap::heap(); + assert(gch->kind() == CollectedHeap::GenCollectedHeap, +@@ -1032,6 +1041,10 @@ + gch->print_heap_change(gch_prev_used); + } + ++#ifndef USDT2 ++ HS_DTRACE_PROBE4(hotspot, gc__collection__parnew__end, full, clear_all_soft_refs, size, is_tlab); ++#endif /* !USDT2 */ ++ + if (PrintGCDetails && ParallelGCVerbose) { + TASKQUEUE_STATS_ONLY(thread_state_set.print_termination_stats()); + TASKQUEUE_STATS_ONLY(thread_state_set.print_taskqueue_stats()); +--- openjdk.orig/hotspot/src/share/vm/memory/defNewGeneration.cpp 2012-08-15 12:03:43.010543164 -0400 ++++ openjdk/hotspot/src/share/vm/memory/defNewGeneration.cpp 2012-08-15 12:21:41.076673646 -0400 +@@ -38,6 +38,7 @@ + #include "oops/oop.inline.hpp" + #include "runtime/java.hpp" + #include "utilities/copy.hpp" ++#include "utilities/dtrace.hpp" + #include "utilities/stack.inline.hpp" + #ifdef TARGET_OS_FAMILY_linux + # include "thread_linux.inline.hpp" +@@ -51,7 +52,10 @@ + #ifdef TARGET_OS_FAMILY_bsd + # include "thread_bsd.inline.hpp" + #endif +- ++#ifndef USDT2 ++ HS_DTRACE_PROBE_DECL4(provider, gc__collection__defnew__begin, bool, bool, size_t, bool); ++ HS_DTRACE_PROBE_DECL4(provider, gc__collection__defnew__end, bool, bool, size_t, bool); ++#endif /* !USDT2 */ + // + // DefNewGeneration functions. + +@@ -528,6 +532,9 @@ + bool clear_all_soft_refs, + size_t size, + bool is_tlab) { ++#ifndef USDT2 ++ HS_DTRACE_PROBE4(hotspot, gc__collection__defnew__begin, full, clear_all_soft_refs, size, is_tlab); ++#endif /* !USDT2 */ + assert(full || size > 0, "otherwise we don't want to collect"); + GenCollectedHeap* gch = GenCollectedHeap::heap(); + _next_gen = gch->next_gen(this); +@@ -661,6 +668,10 @@ + // does not guarantee monotonicity. + jlong now = os::javaTimeNanos() / NANOSECS_PER_MILLISEC; + update_time_of_last_gc(now); ++#ifndef USDT2 ++ HS_DTRACE_PROBE4(hotspot, gc__collection__defnew__end, full, clear_all_soft_refs, size, is_tlab); ++#endif /* !USDT2 */ ++ + } + + class RemoveForwardPointerClosure: public ObjectClosure { +--- openjdk.orig/hotspot/src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp 2012-08-15 12:03:43.044543106 -0400 ++++ openjdk/hotspot/src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp 2012-08-15 12:25:26.632316692 -0400 +@@ -55,6 +55,12 @@ + #include "runtime/vmThread.hpp" + #include "services/memoryService.hpp" + #include "services/runtimeService.hpp" ++#include "utilities/dtrace.hpp" ++ ++#ifndef USDT2 ++ HS_DTRACE_PROBE_DECL4(provider, gc__collection__contig__begin, bool, bool, size_t, bool); ++ HS_DTRACE_PROBE_DECL4(provider, gc__collection__contig__end, bool, bool, size_t, bool); ++#endif /* !USDT2 */ + + // statics + CMSCollector* ConcurrentMarkSweepGeneration::_collector = NULL; +@@ -1647,7 +1653,13 @@ + size_t size, + bool tlab) + { ++#ifndef USDT2 ++ HS_DTRACE_PROBE4(hotspot, gc__collection__contig__begin, full, clear_all_soft_refs, size, tlab); ++#endif /* !USDT2 */ + collector()->collect(full, clear_all_soft_refs, size, tlab); ++#ifndef USDT2 ++ HS_DTRACE_PROBE4(hotspot, gc__collection__contig__end, full, clear_all_soft_refs, size, tlab); ++#endif /* !USDT2 */ + } + + void CMSCollector::collect(bool full, +--- openjdk.orig/hotspot/src/share/vm/memory/generation.cpp 2012-08-15 12:03:43.009543167 -0400 ++++ openjdk/hotspot/src/share/vm/memory/generation.cpp 2012-08-15 12:27:46.378095083 -0400 +@@ -39,8 +39,14 @@ + #include "oops/oop.inline.hpp" + #include "runtime/java.hpp" + #include "utilities/copy.hpp" ++#include "utilities/dtrace.hpp" + #include "utilities/events.hpp" + ++#ifndef USDT2 ++ HS_DTRACE_PROBE_DECL4(provider, gc__collection__contig__begin, bool, bool, size_t, bool); ++ HS_DTRACE_PROBE_DECL4(provider, gc__collection__contig__end, bool, bool, size_t, bool); ++#endif /* !USDT2 */ ++ + Generation::Generation(ReservedSpace rs, size_t initial_size, int level) : + _level(level), + _ref_processor(NULL) { +@@ -470,7 +476,13 @@ + // refs discovery is over the entire heap, not just this generation + ReferenceProcessorSpanMutator + x(ref_processor(), GenCollectedHeap::heap()->reserved_region()); ++#ifndef USDT2 ++ HS_DTRACE_PROBE4(hotspot, gc__collection__contig__begin, full, clear_all_soft_refs, size, is_tlab); ++#endif /* !USDT2 */ + GenMarkSweep::invoke_at_safepoint(_level, ref_processor(), clear_all_soft_refs); ++#ifndef USDT2 ++ HS_DTRACE_PROBE4(hotspot, gc__collection__contig__end, full, clear_all_soft_refs, size, is_tlab); ++#endif /* !USDT2 */ + SpecializationStats::print(); + } + +--- openjdk.orig/hotspot/src/share/vm/gc_implementation/parallelScavenge/parallelScavengeHeap.cpp 2012-08-15 12:03:43.046543102 -0400 ++++ openjdk/hotspot/src/share/vm/gc_implementation/parallelScavenge/parallelScavengeHeap.cpp 2012-08-15 12:53:47.762647059 -0400 +@@ -40,8 +40,13 @@ + #include "runtime/handles.inline.hpp" + #include "runtime/java.hpp" + #include "runtime/vmThread.hpp" ++#include "utilities/dtrace.hpp" + #include "utilities/vmError.hpp" + ++#ifndef USDT2 ++ HS_DTRACE_PROBE_DECL2(provider, gc__collection__parscavenge, *uintptr_t, *uintptr_t); ++#endif /* !USDT2 */ ++ + PSYoungGen* ParallelScavengeHeap::_young_gen = NULL; + PSOldGen* ParallelScavengeHeap::_old_gen = NULL; + PSPermGen* ParallelScavengeHeap::_perm_gen = NULL; +@@ -806,6 +811,9 @@ + } + + VM_ParallelGCSystemGC op(gc_count, full_gc_count, cause); ++#ifndef USDT2 ++ HS_DTRACE_PROBE2(hotspot, gc__collection__parscavenge, &op, cause); ++#endif /* !USDT2 */ + VMThread::execute(&op); + } + +--- openjdk.orig/hotspot/src/share/vm/gc_implementation/parallelScavenge/psScavenge.cpp 2012-07-25 13:24:07.000000000 -0400 ++++ openjdk/hotspot/src/share/vm/gc_implementation/parallelScavenge/psScavenge.cpp 2012-08-17 10:26:44.181117802 -0400 +@@ -51,8 +51,17 @@ + #include "runtime/vmThread.hpp" + #include "runtime/vm_operations.hpp" + #include "services/memoryService.hpp" ++#include "utilities/dtrace.hpp" + #include "utilities/stack.inline.hpp" + ++#ifndef USDT2 ++ HS_DTRACE_PROBE_DECL2(provider, gc__collection__PSScavenge__begin, *uintptr_t, *uintptr_t); ++ HS_DTRACE_PROBE_DECL2(provider, gc__collection__PSScavenge__end, *uintptr_t, *uintptr_t); ++ HS_DTRACE_PROBE_DECL2(provider, gc__collection__PSParallelCompact__begin, *uintptr_t, *uintptr_t); ++ HS_DTRACE_PROBE_DECL2(provider, gc__collection__PSParallelCompact__end, *uintptr_t, *uintptr_t); ++ HS_DTRACE_PROBE_DECL2(provider, gc__collection__PSMarkSweep__begin, *uintptr_t, *uintptr_t); ++ HS_DTRACE_PROBE_DECL2(provider, gc__collection__PSMarkSweep__end, *uintptr_t, *uintptr_t); ++#endif /* !USDT2 */ + + HeapWord* PSScavenge::_to_space_top_before_gc = NULL; + int PSScavenge::_consecutive_skipped_scavenges = 0; +@@ -226,7 +235,13 @@ + PSAdaptiveSizePolicy* policy = heap->size_policy(); + IsGCActiveMark mark; + ++#ifndef USDT2 ++ HS_DTRACE_PROBE2(hotspot, gc__collection__PSScavenge__begin, *heap, heap->gc_cause()); ++#endif /* !USDT2 */ + const bool scavenge_done = PSScavenge::invoke_no_policy(); ++#ifndef USDT2 ++ HS_DTRACE_PROBE2(hotspot, gc__collection__PSScavenge__end, *heap, heap->gc_cause()); ++#endif /* !USDT2 */ + const bool need_full_gc = !scavenge_done || + policy->should_full_GC(heap->old_gen()->free_in_bytes()); + bool full_gc_done = false; +@@ -243,9 +258,21 @@ + const bool clear_all_softrefs = cp->should_clear_all_soft_refs(); + + if (UseParallelOldGC) { ++#ifndef USDT2 ++ HS_DTRACE_PROBE2(hotspot, gc__collection__PSParallelCompact__begin, *heap, heap->gc_cause()); ++#endif /* !USDT2 */ + full_gc_done = PSParallelCompact::invoke_no_policy(clear_all_softrefs); ++#ifndef USDT2 ++ HS_DTRACE_PROBE2(hotspot, gc__collection__PSParallelCompact__end, *heap, heap->gc_cause()); ++#endif /* !USDT2 */ + } else { ++#ifndef USDT2 ++ HS_DTRACE_PROBE2(hotspot, gc__collection__PSMarkSweep__begin, *heap, heap->gc_cause()); ++#endif /* !USDT2 */ + full_gc_done = PSMarkSweep::invoke_no_policy(clear_all_softrefs); ++#ifndef USDT2 ++ HS_DTRACE_PROBE2(hotspot, gc__collection__PSMarkSweep__end, *heap, heap->gc_cause()); ++#endif /* !USDT2 */ + } + } + -------------- next part -------------- /* * probe - gc_collect_contig_begin * * @name: gc_collect_contig_begin * @is_full: If TRUE, attempt a full collection of the generation. * Else; perform a scavenge * @size: Word size of the object to be collected. * @is_tlab: Is this a Thread Local Allocation Buffer? * * Description: This marks the start of a contiguous space generation collection. * */ probe hotspot.gc_collect_contig_begin = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__contig__begin") { name = "gc_collect_contig_begin"; is_full = $arg2; size = $arg3; is_tlab = $arg4; probestr = sprintf("%s(is_full='%d', size='%d', is_tlab='%d')", name, is_full, size, is_tlab); } /* * probe - gc_collect_contig_end * * @name: gc_collect_contig_end_ * @is_full: If TRUE, attempt a full collection of the generation. * Else; perform a scavenge. * @size: Word size of the object to be collected. * @is_tlab: Is this a Thread Local Allocation Buffer? * * Description: This marks the end of a contiguous space generation collection. * */ probe hotspot.gc_collect_contig_end = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__contig__end") { name = "gc_collect_contig_end"; is_full = $arg2; size = $arg3; is_tlab = $arg4; probestr = sprintf("%s(is_full='%d', size='%d', is_tlab='%d')", name, is_full, size, is_tlab); } /* * probe - gc_collect_parnew_begin * * @name: gc_collect_parnew_begin * @is_full: If TRUE, attempt a full collection of the generation. * Else; perform a scavenge * @size: Word size of the object to be collected. * @is_tlab: Is this a Thread Local Allocation Buffer? * * Description: This marks the beginning of a parallel collection of a new * generation. * */ probe hotspot.gc_collect_parnew = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__parnew__begin") { name = "gc_collect_parnew_begin"; is_full = $arg2; size = $arg3; is_tlab = $arg4; probestr = sprintf("%s(is_full='%d', size='%d', is_tlab='%d')", name, is_full, size, is_tlab); } /* * probe - gc_collect_parnew_end * * @name: gc_collect_parnew_end * @is_full: If TRUE, attempt a full collection of the generation. * Else; perform a scavenge * @size: Word size of the object to be collected. * @is_tlab: Is this a Thread Local Allocation Buffer? * * Description: This marks the end of a parallel collection of a new * generation. * */ probe hotspot.gc_collect_parnew_end = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__parnew__end") { name = "gc_collect_parnew_end"; is_full = $arg2; size = $arg3; is_tlab = $arg4; probestr = sprintf("%s(is_full='%d', size='%d', is_tlab='%d')", name, is_full, size, is_tlab); } /* * probe - gc_collect_defnew_begin * * @name: gc_collect_defnew_begin * @is_full: If TRUE, attempt a full collection of the generation. * Else; perform a scavenge * @size: Word size of the object to be collected. * @is_tlab: Is this a Thread Local Allocation Buffer? * * Description: This marks the start of a newly defined generation * collection * */ probe hotspot.gc_collect_defnew_begin = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__defnew__begin") { name = "gc_collect_defnew_begin"; is_full = $arg2; size = $arg3; is_tlab = $arg4; probestr = sprintf("%s(is_full='%d', size='%d', is_tlab='%d')", name, is_full, size, is_tlab); } /* * probe - gc_collect_defnew_end * * @name: gc_collect_defnew_end * @is_full: If TRUE, attempt a full collection of the generation. * Else; perform a scavenge * @size: Word size of the object to be collected. * @is_tlab: Is this a Thread Local Allocation Buffer? * * Description: This marks the end of a newly defined generation * collection * */ probe hotspot.gc_collect_defnew_end = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__defnew__end") { name = "gc_collect_defnew_end"; is_full = $arg2; size = $arg3; is_tlab = $arg4; probestr = sprintf("%s(is_full='%d', size='%d', is_tlab='%d')", name, is_full, size, is_tlab); } /* * probe - gc_collect_tenured_begin * * @name: gc_collect_tenured_begin * @is_full: If TRUE, attempt a full collection of the generation. * Else; perform a scavenge * @size: Word size of the object to be collected. * @is_tlab: Is this a Thread Local Allocation Buffer? * * Description: This is the start of a collection of a tenured generation * (a generation that has survived multiple garbage collections and is * now in a 'tenured' object space. * */ probe hotspot.gc_collect_tenured_begin = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__tenured__begin") { name = "gc_collect_tenured_begin"; is_full = $arg2; size = $arg3; is_tlab = $arg4; probestr = sprintf("%s(is_full='%d', size='%d', is_tlab='%d')", name, is_full, size, is_tlab); } /* * probe - gc_collect_tenured_end * * @name: gc_collect_tenured_end * @is_full: If TRUE, attempt a full collection of the generation. * Else; perform a scavenge * @size: Word size of the object to be collected. * @is_tlab: Is this a Thread Local Allocation Buffer? * * Description: This is the end of a collection of a tenured generation * (a generation that has survived multiple garbage collections and is * now in a 'tenured' object space. * */ probe hotspot.gc_collect_tenured_end = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__tenured__end") { name = "gc_collect_tenured_end"; is_full = $arg2; size = $arg3; is_tlab = $arg4; probestr = sprintf("%s(is_full='%d', size='%d', is_tlab='%d')", name, is_full, size, is_tlab); } /* * probe - gc_collect_parallel_scavenge * * @name: gc_collect_parallel_scavenge * @address: Address of object being collected. * @cause: Cause of the collection. * * Description: This is a parallel scavenge, where the jvm process doesn't * have to halt while the gc is being completed. */ probe hotspot.gc_collect_parallel_scavenge = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__parscavenge") { name = "gc_collect_parallel_scavenge"; address = sprintf("0x%x", $arg1); cause = $arg2; probestr = sprintf("%s(address='%s', cause='%d')", name, address, cause); } /* * probe - gc_collect_parallel_collect * * @name: gc_collect_parallel_collect * @address: Address of object being collected. * @cause: Cause of the collection. * * Description: This marks a parallel collection. * */ probe hotspot.gc_collect_parallel_collect = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__parallel__collect") { name = "gc_collect_parallel_collect"; address = sprintf("0x%x", $arg1); cause = $arg2; probestr = sprintf("%s(address='%s', cause='%d')", name, address, cause); } /* * probe - gc_collect_g1_begin * * @name: gc_collect_g1_begin * @address: Address of object being collected. * @cause: Cause of the collection. * * Description: This marks the start of a G1 style garbage collection * (Garbage-First Garbage Collector). * */ probe hotspot.gc_collect_g1_begin = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__G1__begin") { name = "gc_collect_g1_begin"; address = sprintf("0x%x", $arg1); cause = $arg2; probestr = sprintf("%s(address='%s', cause='%d')", name, address, cause); } /* * probe - gc_collect_g1_end * * @name: gc_collect_g1_end * @address: Address of object being collected. * @cause: Cause of the collection. * * Description: This marks then end of a G1 style garbage collection * (Garbage-First Garbage Collector). * */ probe hotspot.gc_collect_g1_end = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__G1__end") { name = "gc_collect_g1_end"; address = sprintf("0x%x", $arg1); cause = $arg2; probestr = sprintf("%s(address='%s', cause='%d')", name, address, cause); } /* * probe - gc_collect_delete * * @name: gc_collect_delete * @address: Address of object being collected. * @cause: Cause of the collection. * * Description: A delete statement of an object. * */ probe hotspot.gc_collect_delete = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__delete") { name = "gc_collect_delete"; address = sprintf("0x%x", $arg1); probestr = sprintf("%s(address='%s')", name, address); } /* * probe - gc_collect_PSScavenge_begin * * @name: gc_collect_PSScavenge_begin * @address: Address of scavenge * @cause: Cause of the collection. * * Description: A parallel scavenge begins. A scavenge is a partial garbage * collection which should be much more common than a full garbage collection * throughout the course of the java program. * */ probe hotspot.gc_collect_PSScavenge_begin = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__PSScavenge__begin") { name = "gc_collect_PSScavenge_begin"; address = sprintf("0x%x", $arg1); cause = $arg2; probestr = sprintf("%s(address='%s', cause='%d')", name, address, cause); } /* * probe - gc_collect_PSScavenge_end * * @name: gc_collect_PSScavenge_end * @address: Address of scavenge. * @cause: Cause of the collection. * * Description: The end of the parallel scavenge. The beginning and end of * the scavenge is noted due to the possbility of multiple scavenges occuring * at the same time. * */ probe hotspot.gc_collect_PSScavenge_end = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__PSScavenge__end") { name = "gc_collect_PSScavenge_end"; address = sprintf("0x%x", $arg1); cause = $arg2; probestr = sprintf("%s(address='%s', cause='%d')", name, address, cause); } /* * probe - gc_collect_PSParallelCompact_begin * * @name: gc_collect_PSParallelCompact_begin * @address: Address of compaction. * @cause: Cause of the collection. * * Description: This marks the start of a parallel compaction. * */ probe hotspot.gc_collect_PSParallelCompact_begin = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__PSParallelCompact__begin") { name = "gc_collect_PSParallelCompact_begin"; address = sprintf("0x%x", $arg1); cause = $arg2; probestr = sprintf("%s(address='%s', cause='%d')", name, address, cause); } /* * probe - gc_collect_PSParallelCompact_end * * @name: gc_collect_PSParallelCompact_end * @address: Address of compaction. * @cause: Cause of the collection. * * Description: This marks the end of a parallel compaction. * */ probe hotspot.gc_collect_PSParallelCompact_end = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__PSParallelCompact__end") { name = "gc_collect_PSParallelCompact_end"; address = sprintf("0x%x", $arg1); cause = $arg2; probestr = sprintf("%s(address='%s', cause='%d')", name, address, cause); } /* * probe - gc_collect_PSMarkSweep_begin * * @name: gc_collect_PSMarkSweep_begin * @address: Address of parallel mark sweep process. * @cause: Cause of the collection. * * Description: This marks the start of a parallel mark sweep for * objects that require collection. * */ probe hotspot.gc_collect_PSMarkSweep_begin = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__PSMarkSweep__begin") { name = "gc_collect_PSMarkSweep_begin"; address = sprintf("0x%x", $arg1); cause = $arg2; probestr = sprintf("%s(address='%s', cause='%d')", name, address, cause); } /* * probe - gc_collect_PSMarkSweep_end * * @name: gc_collect_PSMarkSweep_end * @address: Address of parallel mark sweep process. * @cause: Cause of the collection. * * Description: This marks the start of a parallel mark sweep for * objects that require collection. * */ probe hotspot.gc_collect_PSMarkSweep_end = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__PSMarkSweep__end") { name = "gc_collect_PSMarkSweep_end"; address = sprintf("0x%x", $arg1); cause = $arg2; probestr = sprintf("%s(address='%s', cause='%d')", name, address, cause); } /* * probe - gc_collect_move * * @name: gc_collect_move * @from_bottom_address: The bottom address of the object being moved. * @from_top_address: The top address of the object being moved. * @to_bottom_address: The bottom address of where the object is being moved to. * @to_top_address: The top address of where the object is being moved to. * @cause: Cause of the collection. * * Description: During garbage collections there are times where objects or * blocks of memory need to be moved. This probe will detail from where * the memory is moving and where to. * */ probe hotspot.gc_collect_move = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__move") { name = "gc_collect_move"; from_bottom_address = sprintf("0x%x", $arg1); from_top_address = sprintf("0x%x", $arg2); to_bottom_address = sprintf("0x%x", $arg3); to_top_address = sprintf("0x%x", $arg4); probestr = sprintf("%s(from_bottom_address='%s', from_top_address='%s', to_bottom_address='%s', to_top_address='%s')", name, from_bottom_address, from_top_address, to_bottom_address, to_top_address); } /* * probe - gc_collect_clear * * @name: gc_collect_clear * @address: Address of object being collected. * @cause: Cause of the collection. * * Description: This probe dictates the region of data that needs to be * cleared in a compaction action. * */ probe hotspot.gc_collect_clear = process("/home/lberk/code/icedtea7/build/openjdk.build-debug/j2sdk-image/jre/lib/amd64/server/libjvm.so").mark("gc__collection__ParallelCompact__clear") { name = "gc_collect_clear"; region_data = sprintf("0x%x", $arg1); data_location = sprintf("0x%x", $arg2); probestr = sprintf("%s(region_data='%s', data_location='%s')", name, region_data, data_location); } -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From john.cuthbertson at oracle.com Wed Aug 29 17:23:10 2012 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Wed, 29 Aug 2012 10:23:10 -0700 Subject: RFR(S): 7190666: G1: assert(_unused == 0) failed: Inconsistency in PLAB stats In-Reply-To: References: <503C128A.3060909@oracle.com> <503CF925.1070501@oracle.com> <503D27F9.9000202@oracle.com> Message-ID: <503E4FFE.8060805@oracle.com> Hi Ramki, Thanks for the review. I don't think we need to back port the change in PLABStats. CMS/ParNew was only flushing the accumulated values once rather than at every allocation buffer retirement and wasn't affected by the "double" add that inflated the value. The real fix for G1 was to do the same. The only reason for the change in PLABStats was to make it a bit more robust so that we could flush the stats every n-th buffer retirement if we wanted. I haven't run performance numbers yet - I've been distracted with the assertions and guarantee failures from G1's BOT that I've seen with a jprt-built binary. It is a good idea to run refworkload. My expectation is that ParNew/CMS will be a wash; with G1 I would expect the deviation to narrow. Thanks, JohnC On 8/29/2012 1:43 AM, Srinivas Ramakrishna wrote: > Hi John -- > > Good catch... I suppose we have been getting too large PLABs for so > many years and perhaps wasting space to fragmentation as a result of > this so far (hmm.. although perhaps the waste calculation in the next > cycle allows us to adjust down, but i guess we are prone to oscillate > because of the sensor value corruption...) I can imagine that in the > case of ParNew+CMS this has been wasting space (because of said > oscillations) in the survivor spaces. > > Thanks for the fix, and it looks good to me too. Any chance that the > PLAB stats portion of the fix at least might also be backported to JDK > 7, so the performance benefit of a more correct calculation accrues > there as well? May be under an MR? (PS: I am curious to know if this > showed any change in (gc) performance on any of the usual benchmarks...) > > thanks! > -- ramki > > On Tue, Aug 28, 2012 at 1:20 PM, John Cuthbertson > > wrote: > > Hi Jon, > > Thanks for the review. > > JohnC > > > On 08/28/12 10:00, Jon Masamitsu wrote: > > Looks good. > > On 8/27/2012 5:36 PM, John Cuthbertson wrote: > > Hi Everyone, > > Can I have a couple of volunteers review the changes for > this CR? The webrev can be found at: > http://cr.openjdk.java.net/~johnc/7190666/webrev.0/ > > > Summary: > The value of PLABStats::_allocated was overflowing and the > failed assertion detected when it overflowed to 0. When we > retired an individual allocation buffer, we were flushing > some accumulated values to the associated PLABStats > instance. This was artificially inflating the values in > the PLABStats instance since we were not reseting the > accumulated values in the ParGCAllocBuffer after we > flushed. Ordinarily this would not cause an issue (other > than the values being too large) but with this particular > test case we obtained an evacuation failure. As a result > we were executing the GC allocation slow-path, and > flushing the accumulated values, for every failed > attempted object allocation (even though we were unable to > allocate a new buffer), and we overflowed. Reseting the > sensor values in the ParGCAllocBuffer instance after > flushing prevents the artificial inflation and overflow. > > Additionally we should not be flushing the values to the > PLABStats instance on every buffer retirement (though it > is not stated in the code). Flushing the stats values on > every retirement is unnecessary and, in the case of an > evacuation, adds a fair amount of additional work for each > failed object copy. Instead we should only be flushing the > accumulated sensor values when we retire the final buffers > prior to disposing the G1ParScanThreadState object. > > Testing: > The failing test case; the GC test suite with +PrintPLAB, > and jprt. > > Note while testing this I ran into some assertion and > guarantee failures from G1's block offset table. I've only > seen and been able (so far) to reproduce these failures on > a single machine in the jprt pool. I will be submitting a > new CR for these failures. I do not believe that the > failures are related to this fix (or the change that > enabled resize-able PLABS) as I've been able to reproduce > the failures with disabling ResizePLAB and setting > OldPLABSize=8k, 16k, and 32k. > > Thanks, > > JohnC > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.cuthbertson at oracle.com Thu Aug 30 00:27:14 2012 From: john.cuthbertson at oracle.com (john.cuthbertson at oracle.com) Date: Thu, 30 Aug 2012 00:27:14 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 7194409: os::javaTimeNanos() shows hot on CPU_CLK_UNHALTED profiles Message-ID: <20120830002719.827D0477DE@hg.openjdk.java.net> Changeset: fa9253dcd4df Author: johnc Date: 2012-08-29 13:02 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/fa9253dcd4df 7194409: os::javaTimeNanos() shows hot on CPU_CLK_UNHALTED profiles Summary: Add inline directives to os::Linux::supports_monotonic_clock() and os::Bsd::supports_monotonic_clock(). Reviewed-by: johnc, azeemj, mikael Contributed-by: Brandon Mitchell ! src/os/bsd/vm/os_bsd.hpp ! src/os/linux/vm/os_linux.hpp From ysr1729 at gmail.com Thu Aug 30 07:51:16 2012 From: ysr1729 at gmail.com (Srinivas Ramakrishna) Date: Thu, 30 Aug 2012 00:51:16 -0700 Subject: RFR(S): 7190666: G1: assert(_unused == 0) failed: Inconsistency in PLAB stats In-Reply-To: <503E4FFE.8060805@oracle.com> References: <503C128A.3060909@oracle.com> <503CF925.1070501@oracle.com> <503D27F9.9000202@oracle.com> <503E4FFE.8060805@oracle.com> Message-ID: On Wed, Aug 29, 2012 at 10:23 AM, John Cuthbertson < john.cuthbertson at oracle.com> wrote: > ** > Hi Ramki, > > Thanks for the review. > > I don't think we need to back port the change in PLABStats. CMS/ParNew was > only flushing the accumulated values once rather than at every allocation > buffer retirement and wasn't affected by the "double" add that inflated the > value. > ah, you are right ... > The real fix for G1 was to do the same. The only reason for the change in > PLABStats was to make it a bit more robust so that we could flush the stats > every n-th buffer retirement if we wanted. > Makes sense, and sorry for my confusion. > > I haven't run performance numbers yet - I've been distracted with the > assertions and guarantee failures from G1's BOT that I've seen with a > jprt-built binary. It is a good idea to run refworkload. My expectation is > that ParNew/CMS will be a wash; with G1 I would expect the deviation to > narrow. > I'd agree wrt parnew/cms, now that i understand the code better in light of your correction of my confusion. thanks! -- ramki > > Thanks, > > JohnC > > > On 8/29/2012 1:43 AM, Srinivas Ramakrishna wrote: > > Hi John -- > > Good catch... I suppose we have been getting too large PLABs for so many > years and perhaps wasting space to fragmentation as a result of this so far > (hmm.. although perhaps the waste calculation in the next cycle allows us > to adjust down, but i guess we are prone to oscillate because of the sensor > value corruption...) I can imagine that in the case of ParNew+CMS this has > been wasting space (because of said oscillations) in the survivor spaces. > > Thanks for the fix, and it looks good to me too. Any chance that the PLAB > stats portion of the fix at least might also be backported to JDK 7, so the > performance benefit of a more correct calculation accrues there as well? > May be under an MR? (PS: I am curious to know if this showed any change in > (gc) performance on any of the usual benchmarks...) > > thanks! > -- ramki > > On Tue, Aug 28, 2012 at 1:20 PM, John Cuthbertson < > john.cuthbertson at oracle.com> wrote: > >> Hi Jon, >> >> Thanks for the review. >> >> JohnC >> >> >> On 08/28/12 10:00, Jon Masamitsu wrote: >> >>> Looks good. >>> >>> On 8/27/2012 5:36 PM, John Cuthbertson wrote: >>> >>>> Hi Everyone, >>>> >>>> Can I have a couple of volunteers review the changes for this CR? The >>>> webrev can be found at: >>>> http://cr.openjdk.java.net/~johnc/7190666/webrev.0/ >>>> >>>> Summary: >>>> The value of PLABStats::_allocated was overflowing and the failed >>>> assertion detected when it overflowed to 0. When we retired an individual >>>> allocation buffer, we were flushing some accumulated values to the >>>> associated PLABStats instance. This was artificially inflating the values >>>> in the PLABStats instance since we were not reseting the accumulated values >>>> in the ParGCAllocBuffer after we flushed. Ordinarily this would not cause >>>> an issue (other than the values being too large) but with this particular >>>> test case we obtained an evacuation failure. As a result we were executing >>>> the GC allocation slow-path, and flushing the accumulated values, for every >>>> failed attempted object allocation (even though we were unable to allocate >>>> a new buffer), and we overflowed. Reseting the sensor values in the >>>> ParGCAllocBuffer instance after flushing prevents the artificial inflation >>>> and overflow. >>>> >>>> Additionally we should not be flushing the values to the PLABStats >>>> instance on every buffer retirement (though it is not stated in the code). >>>> Flushing the stats values on every retirement is unnecessary and, in the >>>> case of an evacuation, adds a fair amount of additional work for each >>>> failed object copy. Instead we should only be flushing the accumulated >>>> sensor values when we retire the final buffers prior to disposing the >>>> G1ParScanThreadState object. >>>> >>>> Testing: >>>> The failing test case; the GC test suite with +PrintPLAB, and jprt. >>>> >>>> Note while testing this I ran into some assertion and guarantee >>>> failures from G1's block offset table. I've only seen and been able (so >>>> far) to reproduce these failures on a single machine in the jprt pool. I >>>> will be submitting a new CR for these failures. I do not believe that the >>>> failures are related to this fix (or the change that enabled resize-able >>>> PLABS) as I've been able to reproduce the failures with disabling >>>> ResizePLAB and setting OldPLABSize=8k, 16k, and 32k. >>>> >>>> Thanks, >>>> >>>> JohnC >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ysr1729 at gmail.com Thu Aug 30 08:05:52 2012 From: ysr1729 at gmail.com (Srinivas Ramakrishna) Date: Thu, 30 Aug 2012 01:05:52 -0700 Subject: [PATCH] 7189971: Implement CMSWaitDuration for non-incremental mode of CMS In-Reply-To: References: Message-ID: Hi Michal -- Thanks so much for the patch... (hopefully you have or will complete the contributor agreement that will allow the patch to be used). I will definitely try and review the patch over the next day or two as soon as i find a few spare cycles. thanks! -- ramki On Mon, Aug 27, 2012 at 10:17 AM, Michal Frajt wrote: > Hi Ramki / Jon, > > Please find the patch for the CMSWaitDuration unstable behavior issue. The > patch keeps the method wait_on_cms_lock untouched for the calls from the > abortable_preclean phase (not very correct behaviour but still acceptable > for the abortable preclean 'short break' calls between the preclean work > iterations). The new method wait_on_cms_lock_for_scavenge has been added. > The method monitors the CGC_lock for notifications, handles the full wait > time interval, checks the scavenge occurrence by the total_collections > counter changing its value. When reviewing please mind that the allowed > locking order in the CMS thread should be FreelistHolder -> Heap_lock -> > CGC_lock (based on a source code comment but the collect_in_background > method is using reverted order between the Freelist and the Heap_lock ??). > The sleepBeforeNextCycle method is now using the new > wait_on_cms_lock_for_scavenge method for both the normal and the > incremental CMS mode. > > The patch has been prepared for the openjdk6 and openjdk7u. The openjdk6 > got compiled and tested on solaris-amd64 platform. The openjdk7u got > compiled without much testing (we have jdk6 application environment only). > > You additionally suggested to have an explicit flag such as > CMSScavengeBeforeInitialMark. I already replied to it but it did not get > into the posting list (sent from another email address). The idea of the > CMSScavengeBeforeInitialMark could be easier to implement but we strongly > prefer not to invoke yet another scavenge explicitly as it is unbalancing > young objects aging and leads to unwanted promotions. I could think about a > combined solution when it first waits for the CMSWaitDuration and, if there > is no scavenge occurring, it is explicitly invoking a scavenge before the > inital-mark phase (or better pause) starts. > > Regards, > Michal > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.coomes at oracle.com Fri Aug 31 08:36:43 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 31 Aug 2012 08:36:43 +0000 Subject: hg: hsx/hotspot-gc: Added tag jdk8-b54 for changeset c1a277c6022a Message-ID: <20120831083643.55EBB47828@hg.openjdk.java.net> Changeset: d5e73011bde2 Author: katleman Date: 2012-08-30 10:26 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/rev/d5e73011bde2 Added tag jdk8-b54 for changeset c1a277c6022a ! .hgtags From john.coomes at oracle.com Fri Aug 31 08:36:47 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 31 Aug 2012 08:36:47 +0000 Subject: hg: hsx/hotspot-gc/corba: Added tag jdk8-b54 for changeset 16c82fc74695 Message-ID: <20120831083650.80AF047829@hg.openjdk.java.net> Changeset: 6b2a363213f4 Author: katleman Date: 2012-08-30 10:27 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/corba/rev/6b2a363213f4 Added tag jdk8-b54 for changeset 16c82fc74695 ! .hgtags From john.coomes at oracle.com Fri Aug 31 08:36:55 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 31 Aug 2012 08:36:55 +0000 Subject: hg: hsx/hotspot-gc/jaxp: Added tag jdk8-b54 for changeset 7dd81ccb7c11 Message-ID: <20120831083703.72AE74782A@hg.openjdk.java.net> Changeset: 0cb5f2471628 Author: katleman Date: 2012-08-30 10:27 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jaxp/rev/0cb5f2471628 Added tag jdk8-b54 for changeset 7dd81ccb7c11 ! .hgtags From john.coomes at oracle.com Fri Aug 31 08:37:08 2012 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 31 Aug 2012 08:37:08 +0000 Subject: hg: hsx/hotspot-gc/jaxws: Added tag jdk8-b54 for changeset 91970935926a Message-ID: <20120831083713.B16E24782B@hg.openjdk.java.net> Changeset: 109c9e1f2d85 Author: katleman Date: 2012-08-30 10:27 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/jaxws/rev/109c9e1f2d85 Added tag jdk8-b54 for changeset 91970935926a ! .hgtags From kirk at kodewerk.com Fri Aug 31 08:49:39 2012 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Fri, 31 Aug 2012 09:49:39 +0100 Subject: NewRatio behaviour and some interesting PSYoungGen Message-ID: Hi all, I was spelunking about in PSYoungGen.cpp trying to sort out how the survivor space max size was being calculated with regards to how tenuring thresholds are being calculated. Point being that in this case I was trying to sort out why my app was rolling back the tenuring threshold about 90% of the time when there should have been plenty of been plenty of memory to accommodate incoming bytes. Some details, -Xmx36g -XX:NewRatio=4. Typical log record looks like this. 47877.205: [GC Desired survivor size 80216064 bytes, new threshold 1 (max 15) [PSYoungGen: 401014K->3647K(477568K)] 3983447K->3588145K(4129920K), 0.0097340 secs] [Times: user=0.13 sys=0.01, real=0.01 secs] At one point in time when there was a higher demand for memory, survivor spaces where big enough to accommodate incoming and surviving bytes. But as demand drew down adaptive sizing drew down on heap size and that in turn drew down on survivor space sizing. The problem here is full gc pause times. I'd prefer not to fix the size of young and consequently survivor but I fear that unless I can stop the VM from undersizing survivor I will need to. The reason the heap is (too) big is that we are battling another issue to do with serialization (inadvertent construction of 1 gig byte arrays where only ~35K is ever used) and until that problem is solved the workaround is an excessively large heap with parallel collectors. In theory this combination should allow us to over size the heap and not take the penalty when things are working normally. In practice the premature promotion problem is causing too frequent full gcs with pause times that are longer than they should be. Scavenge pauses are acceptably short so no problems there. Question is; is there is a way to give adaptive sizing a hint that survivor spaces need to larger while still allowing it to draw back on eden's capacity? One other thing that I found interesting is that when using ParNew with CMS in combination with NewRatio. What I've seen is that when the VM is loaded in low ram Heap will resize young gen. However, for some unknown reason, the resize never happens when the VM is loaded higher up in memory. I first noticed this while running a benchmark at a conference. The demo worked as expected prior to my talk, failed during the talk, and then worked after the talk. The difference in the runs was that during the talk I had keynote and Chrome and a PDF reader and a few other things running. They were idle but they were (of course) consuming quite a bit of memory. After a little bit of experimentation I noted that the adaptivity of the JVM is some affected by where it lies in memory. In my case it wasn't swapping but maybe being higher ram causes enough extra work with the pointers that you end up with this side effect????? Pure speculation but would be interested in any thoughts. One last thing, while spelunking through the code I ran into this. // This method currently does not expect to expand into eden (i.e., // the virtual space boundaries is expected to be consistent // with the eden boundaries.. void PSYoungGen::post_resize() { assert_locked_or_safepoint(Heap_lock); assert((eden_space()->bottom() < to_space()->bottom()) && (eden_space()->bottom() < from_space()->bottom()), "Eden is assumed to be below the survivor spaces"); MemRegion cmr((HeapWord*)virtual_space()->low(), (HeapWord*)virtual_space()->high()); Universe::heap()->barrier_set()->resize_covered_region(cmr); space_invariants(); } The assertion, as I understand it, is asking if the bottom of to_space and from_space are both higher up in memory than the bottom of eden. Wouldn't you want to make sure that to or from is higher in memory than the top of eden? Regards, Kirk -------------- next part -------------- An HTML attachment was scrubbed... URL: