From erik.osterlund at oracle.com Thu Mar 1 15:46:58 2018 From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=) Date: Thu, 1 Mar 2018 16:46:58 +0100 Subject: RFR (XS): 8198896: Remove dead code: cardTableModRefBSForCTRS.hpp Message-ID: <06EB2BA8-321E-4647-81A8-D943D95D5C70@oracle.com> Hi, The cardTableModRefBSForCTRS.hpp file is no longer referenced from anywhere and may be removed. Webrev: http://cr.openjdk.java.net/~eosterlund/8198896/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8198896 Thanks, /Erik -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Thu Mar 1 21:39:23 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 1 Mar 2018 16:39:23 -0500 Subject: RFR (XS): 8198896: Remove dead code: cardTableModRefBSForCTRS.hpp In-Reply-To: <06EB2BA8-321E-4647-81A8-D943D95D5C70@oracle.com> References: <06EB2BA8-321E-4647-81A8-D943D95D5C70@oracle.com> Message-ID: <9E1FDCF9-20C9-4331-AF3B-B8EDB9C7F6C3@oracle.com> > On Mar 1, 2018, at 10:46 AM, Erik ?sterlund wrote: > > Hi, > > The cardTableModRefBSForCTRS.hpp file is no longer referenced from anywhere and may be removed. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8198896/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8198896 > > Thanks, > /Erik Looks good. From per.liden at oracle.com Fri Mar 2 06:45:44 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 2 Mar 2018 07:45:44 +0100 Subject: RFR (XS): 8198896: Remove dead code: cardTableModRefBSForCTRS.hpp In-Reply-To: <06EB2BA8-321E-4647-81A8-D943D95D5C70@oracle.com> References: <06EB2BA8-321E-4647-81A8-D943D95D5C70@oracle.com> Message-ID: <88df8a71-1b9f-0ff9-1df0-85c776799b17@oracle.com> Looks good! /Per On 03/01/2018 04:46 PM, Erik ?sterlund wrote: > Hi, > > The cardTableModRefBSForCTRS.hpp file is no longer referenced from > anywhere and may be removed. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8198896/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8198896 > > Thanks, > /Erik From erik.osterlund at oracle.com Fri Mar 2 07:11:40 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Fri, 2 Mar 2018 08:11:40 +0100 Subject: RFR (XS): 8198896: Remove dead code: cardTableModRefBSForCTRS.hpp In-Reply-To: <9E1FDCF9-20C9-4331-AF3B-B8EDB9C7F6C3@oracle.com> References: <06EB2BA8-321E-4647-81A8-D943D95D5C70@oracle.com> <9E1FDCF9-20C9-4331-AF3B-B8EDB9C7F6C3@oracle.com> Message-ID: Hi Kim, Thanks for the review. /Erik On 1 Mar 2018, at 22:39, Kim Barrett wrote: >> On Mar 1, 2018, at 10:46 AM, Erik ?sterlund wrote: >> >> Hi, >> >> The cardTableModRefBSForCTRS.hpp file is no longer referenced from anywhere and may be removed. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8198896/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8198896 >> >> Thanks, >> /Erik > > Looks good. > From erik.osterlund at oracle.com Fri Mar 2 07:12:20 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Fri, 2 Mar 2018 08:12:20 +0100 Subject: RFR (XS): 8198896: Remove dead code: cardTableModRefBSForCTRS.hpp In-Reply-To: <88df8a71-1b9f-0ff9-1df0-85c776799b17@oracle.com> References: <06EB2BA8-321E-4647-81A8-D943D95D5C70@oracle.com> <88df8a71-1b9f-0ff9-1df0-85c776799b17@oracle.com> Message-ID: <6F663A75-7BD6-4FE4-A5C5-F86B256F94EF@oracle.com> Hi Per, Thanks for the review. /Erik > On 2 Mar 2018, at 07:45, Per Liden wrote: > > Looks good! > > /Per > >> On 03/01/2018 04:46 PM, Erik ?sterlund wrote: >> Hi, >> The cardTableModRefBSForCTRS.hpp file is no longer referenced from anywhere and may be removed. >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8198896/webrev.00/ >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8198896 >> Thanks, >> /Erik From erik.helin at oracle.com Fri Mar 2 10:39:55 2018 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 2 Mar 2018 11:39:55 +0100 Subject: RFR: 8198948: Remove unused parameter evacuation_info from G1CollectedHeap::evacuate_collection_set Message-ID: Hi all, while cruising through the code on this fine Friday morning I noticed that the parameter evacuation_info for G1CollectedHeap::evacuate_collection_set is not used, so I simply removed it :) Issue: https://bugs.openjdk.java.net/browse/JDK-8198948 Webrev: http://cr.openjdk.java.net/~ehelin/8198948/00/ Testing: - make run-test-tier1 on Linux x86-64 Thanks, Erik From stefan.johansson at oracle.com Fri Mar 2 10:45:53 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Fri, 2 Mar 2018 11:45:53 +0100 Subject: RFR: 8198948: Remove unused parameter evacuation_info from G1CollectedHeap::evacuate_collection_set In-Reply-To: References: Message-ID: On 2018-03-02 11:39, Erik Helin wrote: > Hi all, > > while cruising through the code on this fine Friday morning I noticed > that the parameter evacuation_info for > G1CollectedHeap::evacuate_collection_set is not used, so I simply > removed it :) > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8198948 > > Webrev: > http://cr.openjdk.java.net/~ehelin/8198948/00/ > Looks good, thanks for cleaning this up. Cheers, Stefan > Testing: > - make run-test-tier1 on Linux x86-64 > > Thanks, > Erik From erik.helin at oracle.com Fri Mar 2 11:12:58 2018 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 2 Mar 2018 12:12:58 +0100 Subject: RFR: 8197843: Remove unused method G1EvacuationRootClosures::create_root_closures_ext Message-ID: Hi all, this patch removes the unused method G1EvacuationRootClosures::create_root_closures_ext. Since this method is the sole content of the file g1RootClosures_ext.cpp, that file is also removed. Issue: https://bugs.openjdk.java.net/browse/JDK-8197843 Patch: http://cr.openjdk.java.net/~ehelin/8197843/00/ Testing: - make run-test-tier1 on Linux x86-64 Thanks, Erik From thomas.schatzl at oracle.com Fri Mar 2 11:42:15 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 02 Mar 2018 12:42:15 +0100 Subject: RFR: 8198948: Remove unused parameter evacuation_info from G1CollectedHeap::evacuate_collection_set In-Reply-To: References: Message-ID: <1519990935.5140.1.camel@oracle.com> Hi, On Fri, 2018-03-02 at 11:39 +0100, Erik Helin wrote: > Hi all, > > while cruising through the code on this fine Friday morning I > noticed > that the parameter evacuation_info for > G1CollectedHeap::evacuate_collection_set is not used, so I simply > removed it :) > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8198948 > > Webrev: > http://cr.openjdk.java.net/~ehelin/8198948/00/ > > Testing: > - make run-test-tier1 on Linux x86-64 looks good. Thomas From stefan.johansson at oracle.com Fri Mar 2 12:26:11 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Fri, 2 Mar 2018 13:26:11 +0100 Subject: RFR: 8197843: Remove unused method G1EvacuationRootClosures::create_root_closures_ext In-Reply-To: References: Message-ID: On 2018-03-02 12:12, Erik Helin wrote: > Hi all, > > this patch removes the unused method > G1EvacuationRootClosures::create_root_closures_ext. Since this method > is the sole content of the file g1RootClosures_ext.cpp, that file is > also removed. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8197843 > > Patch: > http://cr.openjdk.java.net/~ehelin/8197843/00/ Looks good, Stefan > > Testing: > - make run-test-tier1 on Linux x86-64 > > Thanks, > Erik From erik.osterlund at oracle.com Fri Mar 2 12:32:17 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 2 Mar 2018 13:32:17 +0100 Subject: RFR: 8197843: Remove unused method G1EvacuationRootClosures::create_root_closures_ext In-Reply-To: References: Message-ID: <5A994451.2000106@oracle.com> Hi Erik, Looks good. Thanks, /Erik On 2018-03-02 12:12, Erik Helin wrote: > Hi all, > > this patch removes the unused method > G1EvacuationRootClosures::create_root_closures_ext. Since this method > is the sole content of the file g1RootClosures_ext.cpp, that file is > also removed. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8197843 > > Patch: > http://cr.openjdk.java.net/~ehelin/8197843/00/ > > Testing: > - make run-test-tier1 on Linux x86-64 > > Thanks, > Erik From erik.helin at oracle.com Fri Mar 2 13:30:50 2018 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 2 Mar 2018 14:30:50 +0100 Subject: RFR: 8197847: Remove unused file g1ParScanThreadState_ext.cpp Message-ID: <7bd6d3d1-c166-ef35-25d3-e6858e88a5f2@oracle.com> Hi all, this patch removes the unused file g1ParScanThreadState_ext.cpp and also all the mechanics involved with using these extension points. Issue: https://bugs.openjdk.java.net/browse/JDK-8197847 Patch: http://cr.openjdk.java.net/~ehelin/8197847/00/ Testing: - make run-test-tier1 on Linux x86-64 Thanks, Erik From stefan.johansson at oracle.com Fri Mar 2 14:39:00 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Fri, 2 Mar 2018 15:39:00 +0100 Subject: RFR: 8197847: Remove unused file g1ParScanThreadState_ext.cpp In-Reply-To: <7bd6d3d1-c166-ef35-25d3-e6858e88a5f2@oracle.com> References: <7bd6d3d1-c166-ef35-25d3-e6858e88a5f2@oracle.com> Message-ID: On 2018-03-02 14:30, Erik Helin wrote: > Hi all, > > this patch removes the unused file g1ParScanThreadState_ext.cpp and > also all the mechanics involved with using these extension points. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8197847 > > Patch: > http://cr.openjdk.java.net/~ehelin/8197847/00/ > Looks good, Stefan > Testing: > - make run-test-tier1 on Linux x86-64 > > Thanks, > Erik From erik.osterlund at oracle.com Fri Mar 2 15:12:43 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 2 Mar 2018 16:12:43 +0100 Subject: RFR: 8197847: Remove unused file g1ParScanThreadState_ext.cpp In-Reply-To: <7bd6d3d1-c166-ef35-25d3-e6858e88a5f2@oracle.com> References: <7bd6d3d1-c166-ef35-25d3-e6858e88a5f2@oracle.com> Message-ID: <5A9969EB.50104@oracle.com> Hi Erik, Looks good. Thanks, /Erik On 2018-03-02 14:30, Erik Helin wrote: > Hi all, > > this patch removes the unused file g1ParScanThreadState_ext.cpp and > also all the mechanics involved with using these extension points. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8197847 > > Patch: > http://cr.openjdk.java.net/~ehelin/8197847/00/ > > Testing: > - make run-test-tier1 on Linux x86-64 > > Thanks, > Erik From erik.helin at oracle.com Fri Mar 2 15:53:42 2018 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 2 Mar 2018 16:53:42 +0100 Subject: RFR: 8198957: Remove unused function ArgumentsExt::set_gc_specific_flags Message-ID: <366e468e-0283-b338-77d8-97df0c1fc905@oracle.com> Hi all, this small patch removes the unused function ArgumentsExt::set_gc_specific_flags. Issue: https://bugs.openjdk.java.net/browse/JDK-8198957 Patch: http://cr.openjdk.java.net/~ehelin/8198957/00/ Testing: - make run-test-tier1 on Linux x86-64 Thanks, Erik From stefan.johansson at oracle.com Fri Mar 2 16:04:12 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Fri, 2 Mar 2018 17:04:12 +0100 Subject: RFR: 8198957: Remove unused function ArgumentsExt::set_gc_specific_flags In-Reply-To: <366e468e-0283-b338-77d8-97df0c1fc905@oracle.com> References: <366e468e-0283-b338-77d8-97df0c1fc905@oracle.com> Message-ID: <93606fe4-f2c7-6543-a8ef-9ca351c388f3@oracle.com> Looks awesome! Stefan On 2018-03-02 16:53, Erik Helin wrote: > Hi all, > > this small patch removes the unused function > ArgumentsExt::set_gc_specific_flags. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8198957 > > Patch: > http://cr.openjdk.java.net/~ehelin/8198957/00/ > > Testing: > - make run-test-tier1 on Linux x86-64 > > Thanks, > Erik From erik.helin at oracle.com Fri Mar 2 16:04:11 2018 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 2 Mar 2018 17:04:11 +0100 Subject: RFR: 8197845: Remove unnecessary method G1CollectedHeap::create_g1_policy Message-ID: <028ebe1d-c200-a990-d844-5917a039f1fd@oracle.com> Hi all, this patch removes the no longer needed method G1CollectedHeap::create_g1_policy. Since that method is the sole content of the file g1CollectedHeap_ext.cpp, that file is also removed as part of this patch. Issue: https://bugs.openjdk.java.net/browse/JDK-8197845 Patch: http://cr.openjdk.java.net/~ehelin/8197845/00/ Testing: - make run-test-tier1 on Linux x86-64 Thanks, Erik From thomas.schatzl at oracle.com Mon Mar 5 07:53:25 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 05 Mar 2018 08:53:25 +0100 Subject: RFR: 8198957: Remove unused function ArgumentsExt::set_gc_specific_flags In-Reply-To: <366e468e-0283-b338-77d8-97df0c1fc905@oracle.com> References: <366e468e-0283-b338-77d8-97df0c1fc905@oracle.com> Message-ID: <1520236405.2532.0.camel@oracle.com> Hi, On Fri, 2018-03-02 at 16:53 +0100, Erik Helin wrote: > Hi all, > > this small patch removes the unused function > ArgumentsExt::set_gc_specific_flags. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8198957 > > Patch: > http://cr.openjdk.java.net/~ehelin/8198957/00/ > > Testing: > - make run-test-tier1 on Linux x86-64 looks good. Thomas From stefan.johansson at oracle.com Mon Mar 5 08:18:13 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 5 Mar 2018 09:18:13 +0100 Subject: RFR: 8197845: Remove unnecessary method G1CollectedHeap::create_g1_policy In-Reply-To: <028ebe1d-c200-a990-d844-5917a039f1fd@oracle.com> References: <028ebe1d-c200-a990-d844-5917a039f1fd@oracle.com> Message-ID: <2f24383e-b180-91f2-d851-40882af43cd3@oracle.com> Looks good, Stefan On 2018-03-02 17:04, Erik Helin wrote: > Hi all, > > this patch removes the no longer needed method > G1CollectedHeap::create_g1_policy. Since that method is the sole > content of the file g1CollectedHeap_ext.cpp, that file is also removed > as part of this patch. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8197845 > > Patch: > http://cr.openjdk.java.net/~ehelin/8197845/00/ > > Testing: > - make run-test-tier1 on Linux x86-64 > > Thanks, > Erik From thomas.schatzl at oracle.com Mon Mar 5 08:31:50 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 05 Mar 2018 09:31:50 +0100 Subject: RFR: 8197845: Remove unnecessary method G1CollectedHeap::create_g1_policy In-Reply-To: <028ebe1d-c200-a990-d844-5917a039f1fd@oracle.com> References: <028ebe1d-c200-a990-d844-5917a039f1fd@oracle.com> Message-ID: <1520238710.2532.3.camel@oracle.com> Hi, On Fri, 2018-03-02 at 17:04 +0100, Erik Helin wrote: > Hi all, > > this patch removes the no longer needed method > G1CollectedHeap::create_g1_policy. Since that method is the sole > content of the file g1CollectedHeap_ext.cpp, that file is also > removed as part of this patch. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8197845 > > Patch: > http://cr.openjdk.java.net/~ehelin/8197845/00/ > Looks good. Thanks, Thomas From thomas.schatzl at oracle.com Mon Mar 5 14:09:45 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 05 Mar 2018 15:09:45 +0100 Subject: RFR [0/7]: Rebuild remembered sets concurrently Message-ID: <1520258985.2532.40.camel@oracle.com> Hi all, I would like to ask for reviews for a significant change in how G1 manages its remembered sets by rebuilding its remembered sets concurrently. Currently G1 maintains the remembered sets for all regions all the time. This has several drawbacks: - memory consumption for remembered sets is very high all the time, not only due to actually storing these remembered sets, but also due to additional memory wastage due to fragmentation. - the mechanism to combat this memory fragmentation (somewhat) has been to periodically "scrub" remembered sets, at high costs during the "Cleanup" pause. - maintenance of ultimately unused remembered sets decreases overall throughput. With these changes, G1 will only maintain remembered sets that will ever be considered for evacuation ("candidate collection set regions") only during marking and the following mixed gc phase. This not only to a very significant degree decreases average memory consumption (e.g. during the young-only phase mainly remembered sets for young-only regions are maintained), but also maximum memory consumption, as there is little fragmentation while remembered sets are maintained. There is no need for scrubbing the remembered sets any more either, making the Cleanup pause very fast. This allows us to actually remove some previously (very small) concurrent phase. As a beneficial side-effect we noticed that particularly mixed gc performance may increase significantly (up to like 40%-50% faster) due to improved remembered set memory locality. Particularly "big data" workloads where the remembered sets take a lot of memory and maintenance cpu resources benefit most, but we have also noticed significant throughput improvements when running with G1 in other cases. The only drawback we are aware of is that the delay from start of marking to the first mixed gc is longer. G1's mechanism to adaptively start marking seems to be able to manage this change nicely, but some people that already set a very high specific IHOP manually may experience full gcs. Due to the large benefits we do not intend to provide the original behavior, although it could be implemented as the decisions which regions to maintain the remembered sets are encapsulated into a single class. For more information look at my somewhat recent FOSDEM talk explaining this change in more details [1]. For further review purposes this change will be split into seven distinct changesets that will be organized into a thread below this post, each tagged with a running number. Obviously they are dependent on each other. These are in order: 8197569: Refactor eager reclaim for concurrent remembered set rebuilding 8197570: Make rules for choosing collection set candidates more explicit 8197850: Calculate liveness in regions during marking 8180415: Rebuild remembered sets during concurrent mark (main change) 8197573: Remove concurrent cleanup and secondary free list handling 8197928: Only enqueue deferred cards with references into regions that have a tracked remembered set during GC 8197932: Better split work in rebuild remembered sets phase Each of those changesets will have its own webrev, and there will be notes about each of the changes. However, for reference a webrev containing all the changes can be found here: http://cr.openjdk.java.net/~tschatzl/rebuild-all/webrev/ Testing: hs-tier1-5, internal perf tests Enjoy! :) Thomas [1] https://fosdem.org/2018/schedule/event/g1/ From thomas.schatzl at oracle.com Mon Mar 5 14:18:19 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 05 Mar 2018 15:18:19 +0100 Subject: RFR [1/7]: 8197569: Refactor eager reclaim for concurrent remembered set rebuilding In-Reply-To: <1520258985.2532.40.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> Message-ID: <1520259499.2532.44.camel@oracle.com> Hi all, can I have some reviews for the following change that refactors the eager reclaim condition so that it can be later used for the policy that determines for which regions G1 should rebuild the remembered set? (And it's a reasonable refactoring on its own btw). CR: https://bugs.openjdk.java.net/browse/JDK-8197569 Webrev: http://cr.openjdk.java.net/~tschatzl/8197569/webrev/ Testing: hs-tier 1-5, mostly as part of other related changes Thanks, Thomas From thomas.schatzl at oracle.com Mon Mar 5 14:22:15 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 05 Mar 2018 15:22:15 +0100 Subject: RFR [S][2/7] 8197570: Make rules for choosing collection set candidates more explicit In-Reply-To: <1520258985.2532.40.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> Message-ID: <1520259735.2532.48.camel@oracle.com> Hi all, can I have reviews for this change that makes the rule (occupancy) for choosing collection set candidates more explicit, i.e. factor it into a single method. This is needed for selecting the regions we want to rebuild the remembered sets concurrently (JDK-8180415) more straightforward. It also adds a method to allow iteration over the regions in the collection set - I did not see that worth an extra webrev. CR: https://bugs.openjdk.java.net/browse/JDK-8197570 Webrev: http://cr.openjdk.java.net/~tschatzl/8197570/webrev/ Testing: hs-tier 1-5, mostly in conjunction with related changes Thanks, Thomas From thomas.schatzl at oracle.com Mon Mar 5 14:41:42 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 05 Mar 2018 15:41:42 +0100 Subject: RFR [L][3/7]: 8197850: Calculate liveness in regions during marking In-Reply-To: <1520258985.2532.40.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> Message-ID: <1520260902.2532.66.camel@oracle.com> Hi all, can I have reviews for this change that calculates the liveness in regions (below nTAMS) during marking? This is required to be more clever about selecting the regions G1 is going to rebuild the remembered sets for (i.e. the well-known G1MixedGCLiveThresholdPercent threshold). This slows down the marking part of the concurrent phase a bit, but all alternatives were worse: - initially rebuild everything and then drop remembered sets for regions for too highly occupied regions at cleanup: wastes lots of space and memory maintaining these superfluous remembered sets. - choose regions using previous marking cycle data: this is generally not a good idea if your marking cycles are infrequent (which should be the normal case), as it tends to make too few regions available for evacuation (e.g. eager reclaim may already reclaim a lot). I.e. you need extra headroom, or be more aggressive with rebuilding. The main change, next to the calls that add the liveness to some internal data structure every time an object is marked, is some per- thread cache that (G1RegionMarkStatsCache). It helps decreasing the synchronization overhead the (atomic) increments in the global result cause. It is a very simple fixed size hashmap where increments to the same entries are collected, and on collision evicts the previous entry, doing the atomic operation on the global result. There is also some unfortunate moving of code that is required to flush these caches correctly during mark overflow handling. CR: https://bugs.openjdk.java.net/browse/JDK-8197850 Webrev: http://cr.openjdk.java.net/~tschatzl/8197850/webrev/index.html Testing: hs-tier 1-5; the next change will add an assert that verifies that this concurrently calculated value is correct at the end of rebuild, i.e. the results will always verified. Thanks, Thomas From erik.helin at oracle.com Mon Mar 5 15:02:54 2018 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 5 Mar 2018 16:02:54 +0100 Subject: RFR: 8197852: Move G1DefaultPolicy into G1Policy Message-ID: Hi all, this fairly small patch re-unites G1Policy and G1DefaultPolicy, there is no need for having both classes. With this patch we only have G1Policy (and of course G1CollectorPolicy as the little CollectorPolicy shim). Issue: https://bugs.openjdk.java.net/browse/JDK-8197852 Patch: http://cr.openjdk.java.net/~ehelin/8197852/00/ Testing: - make run-test-tier1 on Linux, Windows, Mac, Solaris (SPARC and x86-64) Thanks, Erik From thomas.schatzl at oracle.com Mon Mar 5 15:07:58 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 05 Mar 2018 16:07:58 +0100 Subject: RFR [L][4/7]: 8180415: Rebuild remembered sets during the concurrent cycle In-Reply-To: <1520258985.2532.40.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> Message-ID: <1520262478.2532.91.camel@oracle.com> Hi all, can I have reviews for this change that implements the bulk of the "rebuild remembered sets concurrently" mechanism? This change modifies the management of remembered sets so that those are only maintained when they are useful. This roughly means: - always maintain remembered sets for young gen regions - (almost) always maintain remembered sets for humongous regions (for eager reclaim) as long as they meet the eager reclaim criteria. - never create remembered sets for archive regions - do not create remembered sets for any regions during full gc - do not create remembered sets for regions with evacuation failure (The latter two decisions may not be optimal, but the idea was to strictly only maintain remembered sets for regions we are going to try to evacuate soon) - create remembered sets for old gen regions after finishing marking (starting at the Remark pause), iff - the liveness of these regions indicate so (liveness <= G1MixedGCLiveThresholdPercent) - any (non-objArray) humongous region that does not have a remembered set yet (eg. because of full gc) (Please see the new G1RemSetTrackingPolicy implementation for exact details) During the following "Rebuild Remembered Set" phase, that replaces the "Create Live Data" phase, all non-young regions that may contain outgoing pointers (i.e. excluding closed archive regions), G1 scans the live objects in the regions between bottom and TARS ("top_at_rebuild_start" - yes, that's new :) for inter-region references and add them to the appropriate remembered set. The TARS, as the name indicates, is the top-value at the start of remark, and the regions contain live data between this value and bottom that needs to be scanned during this rebuild phase. The reason for a new indicator is that nTAMS is *not* sufficient: during marking there might have been more allocations in the old gen that also need to be scanned for new references. All references above TARS (and below the region's top) will be handled by the existing barrier mechanism. After rebuild, during the Cleanup pause, we create the new (candidate) collection set, eventually filtering out more regions, like humongous ones that exceed the eager reclaim criteria. The current state of the remembered set for a given region is tracked within the remembered set for that region (HeapRegionRememberedSet class): a remembered set can be either Untracked, Updating or Complete. Untracked means that we do not manage a remembered set for this region, Complete means just that, and Updating means that we are currently in the process of updating the remembered set. This distinction is relevant for evacuation, e.g. we obviously must not evacuate not- Complete remembered sets. Note that this change contains one single temporary ugly hack that allows G1 to work until the next change :) - so in this change G1 determines whether there will be a mixed gc phase during the Cleanup pause. It needs to determine this information at that time, instead of previously during concurrent cleanup, to, if there is no mixed gc coming, drop all remembered sets to not maintain them further. For this reason, and to avoid duplicate recalculation that may yield different results after dropping remembered sets, there is a new flag mixed_gc_pending in CollectorState, making it even more complicated. That one is going away in the next change, promised. :) Some imho useful messages about remembered set tracking are printed using "gc, remset, tracking" log tags, both in debug and trace level. CR: https://bugs.openjdk.java.net/browse/JDK-8180415 Webrev: http://cr.openjdk.java.net/~tschatzl/8180415/webrev/index.html Testing: hs-tier 1-5, etc. etc. Thanks, Thomas From stefan.johansson at oracle.com Mon Mar 5 15:11:53 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 5 Mar 2018 16:11:53 +0100 Subject: RFR [1/7]: 8197569: Refactor eager reclaim for concurrent remembered set rebuilding In-Reply-To: <1520259499.2532.44.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520259499.2532.44.camel@oracle.com> Message-ID: <8d58ae9c-1c18-c317-a40c-66241faa9a81@oracle.com> Thanks for splitting this change up Thomas, On 2018-03-05 15:18, Thomas Schatzl wrote: > Hi all, > > can I have some reviews for the following change that refactors the > eager reclaim condition so that it can be later used for the policy > that determines for which regions G1 should rebuild the remembered set? > (And it's a reasonable refactoring on its own btw). > > CR: > https://bugs.openjdk.java.net/browse/JDK-8197569 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8197569/webrev/ Look good, just one thing: src/hotspot/share/gc/g1/g1CollectedHeap.cpp 2542?? bool humongous_region_is_candidate(G1CollectedHeap* heap, HeapRegion* region) const { ... 2594???? return obj->is_typeArray() && 2595 G1CollectedHeap::heap()->is_potential_eager_reclaim_candidate(region); You could use the passed in heap parameter instead of using G1CollectedHeap::heap(). Another, maybe even nicer, solution would be to move is_potential_eager_reclaim_candidate() to HeapRegion, and just call region->is_potential_eager_reclaim_candidate(). Thanks, Stefan > Testing: > hs-tier 1-5, mostly as part of other related changes > > Thanks, > Thomas From thomas.schatzl at oracle.com Mon Mar 5 15:16:35 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 05 Mar 2018 16:16:35 +0100 Subject: RFR [S][5/7]: 8197573: Remove concurrent cleanup and secondary free list handling In-Reply-To: <1520258985.2532.40.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> Message-ID: <1520262995.2532.99.camel@oracle.com> Hi all, can I have reviews for this change that removes the concurrent cleanup phase and the secondary free list handling entirely. It moves leftover work into the Cleanup pause. In the previous change the remembered set scrubbing, which took in cases where it mattered like 95%+ of time, has been removed because it is not required any more. The reasons for removal are: - due to JDK-8180415 *freeing* remembered sets (as opposed to calculating occupancy) is really fast, so this phase has never been very long for a long time. - it adds a lock in the LAB allocation path It also allows us to remove the "mixed_gc_pending" flag in the CollectorState, as there is no further time to wait until G1 can schedule the "last young gc". CR: https://bugs.openjdk.java.net/browse/JDK-8197573 Webrev: http://cr.openjdk.java.net/~tschatzl/8197573/webrev/index.html Testing: hs-tier 1-3, .... (I consider this an "S" changeset because it is mostly removal of code). Thanks, Thomas From stefan.johansson at oracle.com Mon Mar 5 15:19:31 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 5 Mar 2018 16:19:31 +0100 Subject: RFR [S][2/7] 8197570: Make rules for choosing collection set candidates more explicit In-Reply-To: <1520259735.2532.48.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520259735.2532.48.camel@oracle.com> Message-ID: <172e3f7e-e326-274b-f20f-90167b6aaeb9@oracle.com> Hi Thomas, On 2018-03-05 15:22, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that makes the rule (occupancy) > for choosing collection set candidates more explicit, i.e. factor it > into a single method. This is needed for selecting the regions we want > to rebuild the remembered sets concurrently (JDK-8180415) more > straightforward. > > It also adds a method to allow iteration over the regions in the > collection set - I did not see that worth an extra webrev. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8197570 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8197570/webrev/ Looks good, Stefan > Testing: > hs-tier 1-5, mostly in conjunction with related changes > > Thanks, > Thomas > From thomas.schatzl at oracle.com Mon Mar 5 15:25:15 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 05 Mar 2018 16:25:15 +0100 Subject: RFR [XS][6/7]: 8197928: Only enqueue deferred cards with references into regions that have a tracked remembered set during GC In-Reply-To: <1520258985.2532.40.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> Message-ID: <1520263515.2532.106.camel@oracle.com> Hi all, can I have reviews for this tiny change that optimizes evacuation a little: in particular, when updating the remembered sets (enqueuing cards), we do not need to do this for regions where we do not track the remembered sets. The suggested implementation is rather primitive, I have not noticed pause time impact on doing so (of course it can lessen the concurrent work later). An alternative implementation I considered has been to create a per- region map with a single flag that contains the "needs-update/does-not- need-update" information, like the _fast_in_cset_state table. However creating such a table requires G1 to iterate over all regions before evacuation, and that would be something that needs to be added as G1 does not do that yet. Another would be to move remembered set state/remembered sets outside of the region, so it can be accessed quickly during GC. In any case I thought this implementation is better than not doing it, hoping for some input here. :) CR: https://bugs.openjdk.java.net/browse/JDK-8197928 Webrev: http://cr.openjdk.java.net/~tschatzl/8197928/webrev/ Testing: hs-tier1-5, ... Thanks, Thomas From thomas.schatzl at oracle.com Mon Mar 5 15:32:43 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 05 Mar 2018 16:32:43 +0100 Subject: RFR [M][7/7]: Rebuild remembered sets concurrently In-Reply-To: <1520258985.2532.40.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> Message-ID: <1520263963.2532.113.camel@oracle.com> Hi all, (last one for now :)) please have a look at this change that improves TTSP during the rebuild remembered set phase. So in the original change G1 yields only after processing a particular region - in case of humongous objArrays at e.g. 32M region size with lots of references this can take a bit. This change modifies the rebuild remembered set closure to yield more finely grained. Still a single thread processes a single region, there is a different CR out there for improving that a bit. The default "chunk size" of 256K of references has been measured to take <1ms in the large majority of cases on my setup on some of these densely occupied humongous objArrays, there are log messages for that, and is configurable. CR: https://bugs.openjdk.java.net/browse/JDK-8197932 Webrev: http://cr.openjdk.java.net/~tschatzl/8197932/webrev/ Testing: see main thread Thanks, Thomas From thomas.schatzl at oracle.com Mon Mar 5 15:45:28 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 05 Mar 2018 16:45:28 +0100 Subject: RFR [L][4/7]: 8180415: Rebuild remembered sets during the concurrent cycle In-Reply-To: <1520262478.2532.91.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520262478.2532.91.camel@oracle.com> Message-ID: <1520264728.2532.124.camel@oracle.com> Hi all, just a note: alternatives to this 2-pass scheme were considered and implemented, however overall performed worse. Also alternatives to "only" fixing the long Cleanup/Scrubbing times were considered but rejected due to similar reasons (e.g. JDK-7187490, which contains a lot of comments about this). One issue is that the remembered set add operation itself just does not scale too well (which needs significant love anyway for other reasons too, but that is a completely different issue), and in conjunction with the already ongoing (already bandwidth intensive) marking the overall result is a longer total cycle time. There is already the already mentioned issue that during marking you do not have too good information about the heap too, so you may end up kind of blindly rebuilding remembered sets for the wrong regions. A separate pass may in the future also allow gathering more data to improve the selection of the rebuild region set (see e.g. JDK-8198419). Of course, alternative suggestions and ideas welcome :) Thanks, Thomas On Mon, 2018-03-05 at 16:07 +0100, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that implements the bulk of the > "rebuild remembered sets concurrently" mechanism? > > This change modifies the management of remembered sets so that those > are only maintained when they are useful. This roughly means: > - always maintain remembered sets for young gen regions > - (almost) always maintain remembered sets for humongous regions (for > eager reclaim) as long as they meet the eager reclaim criteria. > - never create remembered sets for archive regions > - do not create remembered sets for any regions during full gc > - do not create remembered sets for regions with evacuation failure > > (The latter two decisions may not be optimal, but the idea was to > strictly only maintain remembered sets for regions we are going to > try > to evacuate soon) > > - create remembered sets for old gen regions after finishing marking > (starting at the Remark pause), iff > > - the liveness of these regions indicate so (liveness <= > G1MixedGCLiveThresholdPercent) > - any (non-objArray) humongous region that does not have a > remembered > set yet (eg. because of full gc) > > (Please see the new G1RemSetTrackingPolicy implementation for exact > details) > > During the following "Rebuild Remembered Set" phase, that replaces > the > "Create Live Data" phase, all non-young regions that may contain > outgoing pointers (i.e. excluding closed archive regions), G1 scans > the > live objects in the regions between bottom and TARS > ("top_at_rebuild_start" - yes, that's new :) for inter-region > references and add them to the appropriate remembered set. > > The TARS, as the name indicates, is the top-value at the start of > remark, and the regions contain live data between this value and > bottom > that needs to be scanned during this rebuild phase. > > The reason for a new indicator is that nTAMS is *not* sufficient: > during marking there might have been more allocations in the old gen > that also need to be scanned for new references. > > All references above TARS (and below the region's top) will be > handled > by the existing barrier mechanism. > > After rebuild, during the Cleanup pause, we create the new > (candidate) > collection set, eventually filtering out more regions, like humongous > ones that exceed the eager reclaim criteria. > > The current state of the remembered set for a given region is tracked > within the remembered set for that region (HeapRegionRememberedSet > class): a remembered set can be either Untracked, Updating or > Complete. > Untracked means that we do not manage a remembered set for this > region, > Complete means just that, and Updating means that we are currently in > the process of updating the remembered set. This distinction is > relevant for evacuation, e.g. we obviously must not evacuate not- > Complete remembered sets. > > Note that this change contains one single temporary ugly hack that > allows G1 to work until the next change :) - so in this change G1 > determines whether there will be a mixed gc phase during the Cleanup > pause. It needs to determine this information at that time, instead > of > previously during concurrent cleanup, to, if there is no mixed gc > coming, drop all remembered sets to not maintain them further. > > For this reason, and to avoid duplicate recalculation that may yield > different results after dropping remembered sets, there is a new flag > mixed_gc_pending in CollectorState, making it even more complicated. > That one is going away in the next change, promised. :) > > Some imho useful messages about remembered set tracking are printed > using "gc, remset, tracking" log tags, both in debug and trace level. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8180415 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8180415/webrev/index.html > Testing: > hs-tier 1-5, etc. etc. > > Thanks, > Thomas From thomas.schatzl at oracle.com Mon Mar 5 15:51:55 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 05 Mar 2018 16:51:55 +0100 Subject: RFR [M][7/7]: 8197932: Better split work in rebuild remembered sets phase In-Reply-To: <1520258985.2532.40.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> Message-ID: <1520265115.2532.126.camel@oracle.com> Hi all, (last one for now :)) (resend because I forgot to change the subject) please have a look at this change that improves TTSP during the rebuild remembered set phase. So in the original change G1 yields only after processing a particular region - in case of humongous objArrays at e.g. 32M region size with lots of references this can take a bit. This change modifies the rebuild remembered set closure to yield more finely grained. Still a single thread processes a single region, there is a different CR out there for improving that a bit. The default "chunk size" of 256K of references has been measured to take <1ms in the large majority of cases on my setup on some of these densely occupied humongous objArrays, there are log messages for that, and is configurable. CR: https://bugs.openjdk.java.net/browse/JDK-8197932 Webrev: http://cr.openjdk.java.net/~tschatzl/8197932/webrev/ Testing: see main thread Thanks, Thomas From erik.helin at oracle.com Mon Mar 5 16:44:07 2018 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 5 Mar 2018 17:44:07 +0100 Subject: RFR: 8199024: Remove unneccessary protected and virtual modifiers from G1CollectedHeap Message-ID: Hi all, this tiny patch cleans up the visibility for members in G1CollectedHeap a bit. There is no need for protected members, I also removed two unnecessary virtual modifiers. Issue: https://bugs.openjdk.java.net/browse/JDK-8199024 Patch: http://cr.openjdk.java.net/~ehelin/8199024/00/ Testing: - make run-test-tier1 on Linux, macOS, Windows (x86-64) and Solaris (SPARC and x86-64) Thanks, Erik From thomas.stuefe at gmail.com Mon Mar 5 18:47:39 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 05 Mar 2018 18:47:39 +0000 Subject: RFR: 8199024: Remove unneccessary protected and virtual modifiers from G1CollectedHeap In-Reply-To: References: Message-ID: Hi Erik, Looks good :) ..Thomas On Mon 5. Mar 2018 at 17:44, Erik Helin wrote: > Hi all, > > this tiny patch cleans up the visibility for members in G1CollectedHeap > a bit. There is no need for protected members, I also removed two > unnecessary virtual modifiers. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8199024 > > Patch: > http://cr.openjdk.java.net/~ehelin/8199024/00/ > > Testing: > - make run-test-tier1 on Linux, macOS, Windows (x86-64) and > Solaris (SPARC and x86-64) > > Thanks, > Erik > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Mon Mar 5 23:27:23 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 5 Mar 2018 18:27:23 -0500 Subject: RFR: 8199024: Remove unneccessary protected and virtual modifiers from G1CollectedHeap In-Reply-To: References: Message-ID: <56AB7ED0-2BDA-4814-9486-8386A65BD7A6@oracle.com> > On Mar 5, 2018, at 11:44 AM, Erik Helin wrote: > > Hi all, > > this tiny patch cleans up the visibility for members in G1CollectedHeap a bit. There is no need for protected members, I also removed two unnecessary virtual modifiers. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8199024 > > Patch: > http://cr.openjdk.java.net/~ehelin/8199024/00/ > > Testing: > - make run-test-tier1 on Linux, macOS, Windows (x86-64) and > Solaris (SPARC and x86-64) > > Thanks, > Erik Looks good. The (pre-existing) placement of the declarations for pending_card_num() and _max_heap_capacity at the end of the class seems odd. I don?t need a new webrev if you want to move them to some more logical place in the class. From thomas.schatzl at oracle.com Tue Mar 6 11:32:38 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 06 Mar 2018 12:32:38 +0100 Subject: RFR (XXS): 8198546: [Redo] JDK-8196883 G1RemSet::refine_card_concurrently doesn't need to check for cards in collection set Message-ID: <1520335958.2360.14.camel@oracle.com> Hi all, can I have reviews for this redo of an earlier change that has been accidentally pushed to the jdk10 repo, backed out there, and everything only recently been merged with jdk 11? The change itself is unmodified. CR: https://bugs.openjdk.java.net/browse/JDK-8198546 Webrev: http://cr.openjdk.java.net/~tschatzl/8198546/webrev/ Testing: local compilation Thanks, Thomas From rkennke at redhat.com Tue Mar 6 11:56:50 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 6 Mar 2018 12:56:50 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy Message-ID: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> Currently, the Access API is only used for oop-arraycopy, but not for primitive arraycopy. GCs might want to intercept this too, e.g. resolve src and dst arrays. There *is* an implementation of primitive arraycopy in the Access API, but it doesn't even compile, because Raw::arraycopy() does not take src and dst oop operands, but it's called like that. The only reason why this does not blow up (I think) is that because nobody calls it, the compiler doesn't even get there. This change fixes the Access API/impl and adds the relevant calls into it (in C1 and runtime land). C2 uses arraycopy stubs (which cannot be handled here) or calls out to the ArrayKlass::copy_array(), which should be covered with this change. It should be possible to use the same Access API for Java-array <-> native-array bulk transfers, which currently use the rather ugly typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address that in a separate change though. http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ Tests: tier1 ok Please review! Thanks, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From erik.osterlund at oracle.com Tue Mar 6 12:10:05 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 6 Mar 2018 13:10:05 +0100 Subject: RFR (XXS): 8198546: [Redo] JDK-8196883 G1RemSet::refine_card_concurrently doesn't need to check for cards in collection set In-Reply-To: <1520335958.2360.14.camel@oracle.com> References: <1520335958.2360.14.camel@oracle.com> Message-ID: <5A9E851D.1010108@oracle.com> Hi Thomas, Looks good. Thanks, /Erik On 2018-03-06 12:32, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this redo of an earlier change that has been > accidentally pushed to the jdk10 repo, backed out there, and everything > only recently been merged with jdk 11? > > The change itself is unmodified. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8198546 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8198546/webrev/ > Testing: > local compilation > > Thanks, > Thomas From thomas.schatzl at oracle.com Tue Mar 6 12:25:50 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 06 Mar 2018 13:25:50 +0100 Subject: RFR (XXS): 8198546: [Redo] JDK-8196883 G1RemSet::refine_card_concurrently doesn't need to check for cards in collection set In-Reply-To: <5A9E851D.1010108@oracle.com> References: <1520335958.2360.14.camel@oracle.com> <5A9E851D.1010108@oracle.com> Message-ID: <1520339150.2378.1.camel@oracle.com> Hi, On Tue, 2018-03-06 at 13:10 +0100, Erik ?sterlund wrote: > Hi Thomas, > > Looks good. thanks. I will push this right away due to triviality, nothing changed since last time, and already having been tested (accidentally) quite thoroughly over time. I will add the original reviewers too. Thanks a lot, Thomas > > Thanks, > /Erik > > On 2018-03-06 12:32, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this redo of an earlier change that has > > been > > accidentally pushed to the jdk10 repo, backed out there, and > > everything > > only recently been merged with jdk 11? > > > > The change itself is unmodified. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8198546 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8198546/webrev/ > > Testing: > > local compilation > > > > Thanks, > > Thomas > > From stefan.johansson at oracle.com Tue Mar 6 12:31:31 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 6 Mar 2018 13:31:31 +0100 Subject: RFR [L][3/7]: 8197850: Calculate liveness in regions during marking In-Reply-To: <1520260902.2532.66.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520260902.2532.66.camel@oracle.com> Message-ID: Hi Thomas, On 2018-03-05 15:41, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that calculates the liveness in > regions (below nTAMS) during marking? > > This is required to be more clever about selecting the regions G1 is > going to rebuild the remembered sets for (i.e. the well-known > G1MixedGCLiveThresholdPercent threshold). > > This slows down the marking part of the concurrent phase a bit, but all > alternatives were worse: > > - initially rebuild everything and then drop remembered sets for > regions for too highly occupied regions at cleanup: wastes lots of > space and memory maintaining these superfluous remembered sets. > > - choose regions using previous marking cycle data: this is generally > not a good idea if your marking cycles are infrequent (which should be > the normal case), as it tends to make too few regions available for > evacuation (e.g. eager reclaim may already reclaim a lot). I.e. you > need extra headroom, or be more aggressive with rebuilding. > > The main change, next to the calls that add the liveness to some > internal data structure every time an object is marked, is some per- > thread cache that (G1RegionMarkStatsCache). It helps decreasing the > synchronization overhead the (atomic) increments in the global result > cause. > > It is a very simple fixed size hashmap where increments to the same > entries are collected, and on collision evicts the previous entry, > doing the atomic operation on the global result. > > There is also some unfortunate moving of code that is required to flush > these caches correctly during mark overflow handling. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8197850 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8197850/webrev/index.html Look good, just some minor things. src/hotspot/share/gc/g1/g1ConcurrentMark.cpp 394?? _region_mark_stats(NEW_C_HEAP_ARRAY(G1RegionMarkStats, max_regions, mtGC)) max_regions is passed into the constructor but everywhere else in G1ConcurrentMark we use g1h->max_regions(). I would consider either calling g1h->max_regions() in the constructor too, or save the value in a member and use it everywhere. --- src/hotspot/share/gc/g1/g1_globals.hpp ?262?? experimental(size_t, G1RegionMarkStatsCacheSize, 1024, ?... ?265?????????? range(128, (max_juint / 2) + 1) Do we need this flag or should we scale it against the number of regions? If we want to keep it I think we should have a somewhat more restrictive range. --- src/hotspot/share/gc/g1/g1RegionMarkStatsCache.hpp In G1RegionMarkStatsCacheEntry _region_idx is set to 0 when clearing. I don't see a problem with it right now, but it feels a bit wrong since index 0 is a valid region index. Maybe we could use another cleared marker. --- Thanks, Stefan > Testing: > hs-tier 1-5; the next change will add an assert that verifies that this > concurrently calculated value is correct at the end of rebuild, i.e. > the results will always verified. > > Thanks, > Thomas > From per.liden at oracle.com Tue Mar 6 12:47:16 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 6 Mar 2018 13:47:16 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> Message-ID: <41715172-d010-c5ef-c4b0-66f393601051@oracle.com> Hi Roman, On 03/06/2018 12:56 PM, Roman Kennke wrote: > Currently, the Access API is only used for oop-arraycopy, but not for > primitive arraycopy. GCs might want to intercept this too, e.g. resolve > src and dst arrays. > > There *is* an implementation of primitive arraycopy in the Access API, > but it doesn't even compile, because Raw::arraycopy() does not take src > and dst oop operands, but it's called like that. The only reason why > this does not blow up (I think) is that because nobody calls it, the > compiler doesn't even get there. > > This change fixes the Access API/impl and adds the relevant calls into > it (in C1 and runtime land). C2 uses arraycopy stubs (which cannot be > handled here) or calls out to the ArrayKlass::copy_array(), which should > be covered with this change. > > It should be possible to use the same Access API for Java-array <-> > native-array bulk transfers, which currently use the rather ugly > typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address that in a > separate change though. > > http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ src/hotspot/share/c1/c1_Runtime1.cpp ------------------------------------ - memmove(dst_addr, src_addr, length << l2es); - return ac_ok; + return HeapAccess<>::arraycopy(arrayOop(src), arrayOop(dst), src_addr, dst_addr, length << l2es) ? ac_ok : ac_failed; I think we should always return ac_ok here, and change Heap<>::arraycopy() to return void, since it will always return true for primitive types. src/hotspot/share/gc/shared/barrierSet.hpp ------------------------------------------ - return Raw::arraycopy(src_obj, dst_obj, src, dst, length); + return Raw::arraycopy(src, dst, length); I'd suggest that we adjust the Raw::arraycopy() signature instead to accept these arguments (even if they are later unused), to keep this symmetric with oop_arraycopy() and avoid doing these types of "decisions" in this layer. src/hotspot/share/oops/accessBackend.cpp ---------------------------------------- + void arraycopy_conjoint(char* src, char* dst, size_t length) { + Copy::conjoint_jbytes(src, dst, length); + } This looks like an unnecessary addition. I suggest you adjust the code in src/hotspot/share/c1/c1_Runtime1.cpp to use jbyte* instead char* in the address calculations. I.e. these lines: char* src_addr = (char*) ((oopDesc**)src + ihs) + (src_pos << l2es); char* dst_addr = (char*) ((oopDesc**)dst + ihs) + (dst_pos << l2es); cheers, Per > > Tests: tier1 ok > > Please review! > Thanks, Roman > From david.holmes at oracle.com Tue Mar 6 12:48:31 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 6 Mar 2018 22:48:31 +1000 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> Message-ID: Hi Roman, Not a review as I'm not familiar enough with the Access API, but in src/hotspot/share/c1/c1_Runtime1.cpp the comments above the changed code need updating - probably deleting. I assume the Access API arraycopy is atomic? Thanks, David On 6/03/2018 9:56 PM, Roman Kennke wrote: > Currently, the Access API is only used for oop-arraycopy, but not for > primitive arraycopy. GCs might want to intercept this too, e.g. resolve > src and dst arrays. > > There *is* an implementation of primitive arraycopy in the Access API, > but it doesn't even compile, because Raw::arraycopy() does not take src > and dst oop operands, but it's called like that. The only reason why > this does not blow up (I think) is that because nobody calls it, the > compiler doesn't even get there. > > This change fixes the Access API/impl and adds the relevant calls into > it (in C1 and runtime land). C2 uses arraycopy stubs (which cannot be > handled here) or calls out to the ArrayKlass::copy_array(), which should > be covered with this change. > > It should be possible to use the same Access API for Java-array <-> > native-array bulk transfers, which currently use the rather ugly > typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address that in a > separate change though. > > http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ > > Tests: tier1 ok > > Please review! > Thanks, Roman > From per.liden at oracle.com Tue Mar 6 12:51:24 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 6 Mar 2018 13:51:24 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> Message-ID: <7213ece8-4604-5410-aaab-1fcca23abd33@oracle.com> On 03/06/2018 01:48 PM, David Holmes wrote: > Hi Roman, > > Not a review as I'm not familiar enough with the Access API, but in > src/hotspot/share/c1/c1_Runtime1.cpp the comments above the changed code > need updating - probably deleting. I assume the Access API arraycopy is > atomic? Agree. We should understand of that comment is correct, and if so use the ARRAYCOPY_ATOMIC decorator, i.e. HeapAccess::arraycopy(). /Per > > Thanks, > David > > On 6/03/2018 9:56 PM, Roman Kennke wrote: >> Currently, the Access API is only used for oop-arraycopy, but not for >> primitive arraycopy. GCs might want to intercept this too, e.g. resolve >> src and dst arrays. >> >> There *is* an implementation of primitive arraycopy in the Access API, >> but it doesn't even compile, because Raw::arraycopy() does not take src >> and dst oop operands, but it's called like that. The only reason why >> this does not blow up (I think) is that because nobody calls it, the >> compiler doesn't even get there. >> >> This change fixes the Access API/impl and adds the relevant calls into >> it (in C1 and runtime land). C2 uses arraycopy stubs (which cannot be >> handled here) or calls out to the ArrayKlass::copy_array(), which should >> be covered with this change. >> >> It should be possible to use the same Access API for Java-array <-> >> native-array bulk transfers, which currently use the rather ugly >> typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address that in a >> separate change though. >> >> http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ >> >> Tests: tier1 ok >> >> Please review! >> Thanks, Roman >> From david.holmes at oracle.com Tue Mar 6 12:56:22 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 6 Mar 2018 22:56:22 +1000 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: <5A9E8F76.1050809@oracle.com> References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> <5A9E8F76.1050809@oracle.com> Message-ID: <83de65bd-5d3f-78df-02d2-eb4c53aa0c62@oracle.com> On 6/03/2018 10:54 PM, Erik ?sterlund wrote: > Hi David, > > It is atomic with the ARRAYCOPY_ATOMIC decorator. If that comment is > correct (I do not know if it is), then the ARRAYCOPY_ATOMIC decorator > should probably be used here. If that code implements a Java array copy then yes it is required to be 32-bit atomic. Do you need the decorator to get 32-bit atomicity? David > Thanks, > /Erik > > On 2018-03-06 13:48, David Holmes wrote: >> Hi Roman, >> >> Not a review as I'm not familiar enough with the Access API, but in >> src/hotspot/share/c1/c1_Runtime1.cpp the comments above the changed >> code need updating - probably deleting. I assume the Access API >> arraycopy is atomic? >> >> Thanks, >> David >> >> On 6/03/2018 9:56 PM, Roman Kennke wrote: >>> Currently, the Access API is only used for oop-arraycopy, but not for >>> primitive arraycopy. GCs might want to intercept this too, e.g. resolve >>> src and dst arrays. >>> >>> There *is* an implementation of primitive arraycopy in the Access API, >>> but it doesn't even compile, because Raw::arraycopy() does not take src >>> and dst oop operands, but it's called like that. The only reason why >>> this does not blow up (I think) is that because nobody calls it, the >>> compiler doesn't even get there. >>> >>> This change fixes the Access API/impl and adds the relevant calls into >>> it (in C1 and runtime land). C2 uses arraycopy stubs (which cannot be >>> handled here) or calls out to the ArrayKlass::copy_array(), which should >>> be covered with this change. >>> >>> It should be possible to use the same Access API for Java-array <-> >>> native-array bulk transfers, which currently use the rather ugly >>> typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address that in a >>> separate change though. >>> >>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ >>> >>> Tests: tier1 ok >>> >>> Please review! >>> Thanks, Roman >>> > From erik.osterlund at oracle.com Tue Mar 6 12:54:14 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 6 Mar 2018 13:54:14 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> Message-ID: <5A9E8F76.1050809@oracle.com> Hi David, It is atomic with the ARRAYCOPY_ATOMIC decorator. If that comment is correct (I do not know if it is), then the ARRAYCOPY_ATOMIC decorator should probably be used here. Thanks, /Erik On 2018-03-06 13:48, David Holmes wrote: > Hi Roman, > > Not a review as I'm not familiar enough with the Access API, but in > src/hotspot/share/c1/c1_Runtime1.cpp the comments above the changed > code need updating - probably deleting. I assume the Access API > arraycopy is atomic? > > Thanks, > David > > On 6/03/2018 9:56 PM, Roman Kennke wrote: >> Currently, the Access API is only used for oop-arraycopy, but not for >> primitive arraycopy. GCs might want to intercept this too, e.g. resolve >> src and dst arrays. >> >> There *is* an implementation of primitive arraycopy in the Access API, >> but it doesn't even compile, because Raw::arraycopy() does not take src >> and dst oop operands, but it's called like that. The only reason why >> this does not blow up (I think) is that because nobody calls it, the >> compiler doesn't even get there. >> >> This change fixes the Access API/impl and adds the relevant calls into >> it (in C1 and runtime land). C2 uses arraycopy stubs (which cannot be >> handled here) or calls out to the ArrayKlass::copy_array(), which should >> be covered with this change. >> >> It should be possible to use the same Access API for Java-array <-> >> native-array bulk transfers, which currently use the rather ugly >> typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address that in a >> separate change though. >> >> http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ >> >> Tests: tier1 ok >> >> Please review! >> Thanks, Roman >> From thomas.schatzl at oracle.com Tue Mar 6 13:17:53 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 06 Mar 2018 14:17:53 +0100 Subject: RFR [1/7]: 8197569: Refactor eager reclaim for concurrent remembered set rebuilding In-Reply-To: <8d58ae9c-1c18-c317-a40c-66241faa9a81@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520259499.2532.44.camel@oracle.com> <8d58ae9c-1c18-c317-a40c-66241faa9a81@oracle.com> Message-ID: <1520342273.2378.15.camel@oracle.com> Hi Stefan, thanks for looking into this. On Mon, 2018-03-05 at 16:11 +0100, Stefan Johansson wrote: > Thanks for splitting this change up Thomas, > > On 2018-03-05 15:18, Thomas Schatzl wrote: > > Hi all, > > > > can I have some reviews for the following change that refactors > > the > > eager reclaim condition so that it can be later used for the policy > > that determines for which regions G1 should rebuild the remembered > > set? > > (And it's a reasonable refactoring on its own btw). > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8197569 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8197569/webrev/ > > Look good, just one thing: > src/hotspot/share/gc/g1/g1CollectedHeap.cpp > 2542 bool humongous_region_is_candidate(G1CollectedHeap* heap, > HeapRegion* region) const { > ... > 2594 return obj->is_typeArray() && > 2595 G1CollectedHeap::heap()- > >is_potential_eager_reclaim_candidate(region); > > You could use the passed in heap parameter instead of using > G1CollectedHeap::heap(). Done. Also renamed the parameter to "g1h" as used everywhere else. > Another, maybe even nicer, solution would be to > move is_potential_eager_reclaim_candidate() to HeapRegion, and just > call region->is_potential_eager_reclaim_candidate(). I did not do this for the following reasons: - is_potential_eager_reclaim_candidate may depend on global state in the future. E.g. when trying to eagerly reclaim objArrays or similar the result depends on whether we are currently marking or not. I do not like making HeapRegion dependent on e.g. G1CollectorState :) (any more than it already may be). - (almost) all eager reclaim related code has until now been in G1CollectedHeap. Such a change would go against this, and run kind of contrary to my intent to move away from using HeapRegion as dumping ground for anything loosely related to it out of convenience, but only as a representation for that memory. Either way is kind of fine for me right now, so let's see what others say? New webrevs: http://cr.openjdk.java.net/~tschatzl/8197569/webrev.0_to_1/ (diff) http://cr.openjdk.java.net/~tschatzl/8197569/webrev.1/ (full) Thanks, Thomas From per.liden at oracle.com Tue Mar 6 13:18:21 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 6 Mar 2018 14:18:21 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: <41715172-d010-c5ef-c4b0-66f393601051@oracle.com> References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> <41715172-d010-c5ef-c4b0-66f393601051@oracle.com> Message-ID: <1721af77-7378-0e24-fab0-bdaa7614e925@oracle.com> On 03/06/2018 01:47 PM, Per Liden wrote: > Hi Roman, > > On 03/06/2018 12:56 PM, Roman Kennke wrote: >> Currently, the Access API is only used for oop-arraycopy, but not for >> primitive arraycopy. GCs might want to intercept this too, e.g. resolve >> src and dst arrays. >> >> There *is* an implementation of primitive arraycopy in the Access API, >> but it doesn't even compile, because Raw::arraycopy() does not take src >> and dst oop operands, but it's called like that. The only reason why >> this does not blow up (I think) is that because nobody calls it, the >> compiler doesn't even get there. >> >> This change fixes the Access API/impl and adds the relevant calls into >> it (in C1 and runtime land). C2 uses arraycopy stubs (which cannot be >> handled here) or calls out to the ArrayKlass::copy_array(), which should >> be covered with this change. >> >> It should be possible to use the same Access API for Java-array <-> >> native-array bulk transfers, which currently use the rather ugly >> typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address that in a >> separate change though. >> >> http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ > > src/hotspot/share/c1/c1_Runtime1.cpp > ------------------------------------ > > -??? memmove(dst_addr, src_addr, length << l2es); > -??? return ac_ok; > +??? return HeapAccess<>::arraycopy(arrayOop(src), arrayOop(dst), > src_addr, dst_addr, length << l2es) ? ac_ok : ac_failed; > > I think we should always return ac_ok here, and change > Heap<>::arraycopy() to return void, since it will always return true for > primitive types. > > > src/hotspot/share/gc/shared/barrierSet.hpp > ------------------------------------------ > -????? return Raw::arraycopy(src_obj, dst_obj, src, dst, length); > +????? return Raw::arraycopy(src, dst, length); > > I'd suggest that we adjust the Raw::arraycopy() signature instead to > accept these arguments (even if they are later unused), to keep this > symmetric with oop_arraycopy() and avoid doing these types of > "decisions" in this layer. > > > src/hotspot/share/oops/accessBackend.cpp > ---------------------------------------- > > +? void arraycopy_conjoint(char* src, char* dst, size_t length) { > +??? Copy::conjoint_jbytes(src, dst, length); > +? } > > This looks like an unnecessary addition. I suggest you adjust the code > in src/hotspot/share/c1/c1_Runtime1.cpp to use jbyte* instead char* in > the address calculations. I.e. these lines: > > ???? char* src_addr = (char*) ((oopDesc**)src + ihs) + (src_pos << l2es); > ???? char* dst_addr = (char*) ((oopDesc**)dst + ihs) + (dst_pos << l2es); > I missed this one... src/hotspot/share/oops/typeArrayKlass.cpp ----------------------------------------- - Copy::conjoint_memory_atomic(src, dst, (size_t)length << l2es); + HeapAccess<>::arraycopy(s, d, src, dst, (size_t)length << l2es); You're loosing the atomic part here, which should translate to an ARRAYCOPY_ATOMIC decorator. Erik will send out a proposal for how to deal with the atomic stuff when we've erased the type information, like in C1's stub. /Per > cheers, > Per > >> >> Tests: tier1 ok >> >> Please review! >> Thanks, Roman >> From thomas.schatzl at oracle.com Tue Mar 6 13:18:37 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 06 Mar 2018 14:18:37 +0100 Subject: RFR [S][2/7] 8197570: Make rules for choosing collection set candidates more explicit In-Reply-To: <172e3f7e-e326-274b-f20f-90167b6aaeb9@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520259735.2532.48.camel@oracle.com> <172e3f7e-e326-274b-f20f-90167b6aaeb9@oracle.com> Message-ID: <1520342317.2378.16.camel@oracle.com> Hi Stefan, On Mon, 2018-03-05 at 16:19 +0100, Stefan Johansson wrote: > Hi Thomas, > > On 2018-03-05 15:22, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this change that makes the rule > > (occupancy) for choosing collection set candidates more explicit, > > i.e. factor it into a single method. This is needed for selecting > > the regions we want to rebuild the remembered sets concurrently > > (JDK-8180415) more straightforward. > > > > It also adds a method to allow iteration over the regions in the > > collection set - I did not see that worth an extra webrev. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8197570 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8197570/webrev/ > > Looks good, thanks for your review. Thomas From erik.osterlund at oracle.com Tue Mar 6 13:21:28 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 6 Mar 2018 14:21:28 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: <83de65bd-5d3f-78df-02d2-eb4c53aa0c62@oracle.com> References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> <5A9E8F76.1050809@oracle.com> <83de65bd-5d3f-78df-02d2-eb4c53aa0c62@oracle.com> Message-ID: <5A9E95D8.6080602@oracle.com> Hi David, The ARRAYCOPY_ATOMIC decorator makes the arraycopy atomic over the size of the passed in elements. In this case, it looks like the address has been type erased to void*, and hence lost what the element size was. There is currently no overload accepted for type erased element - only accurate elements {jbyte, jshort, jint, jlong}. So it looks like an overload must be added to accept type erased void* elements and make that call conjoint_memory_atomic when the ARRAYCOPY_ATOMIC decorator is passed in. Thanks, /Erik On 2018-03-06 13:56, David Holmes wrote: > On 6/03/2018 10:54 PM, Erik ?sterlund wrote: >> Hi David, >> >> It is atomic with the ARRAYCOPY_ATOMIC decorator. If that comment is >> correct (I do not know if it is), then the ARRAYCOPY_ATOMIC decorator >> should probably be used here. > > If that code implements a Java array copy then yes it is required to > be 32-bit atomic. Do you need the decorator to get 32-bit atomicity? > > David > >> Thanks, >> /Erik >> >> On 2018-03-06 13:48, David Holmes wrote: >>> Hi Roman, >>> >>> Not a review as I'm not familiar enough with the Access API, but in >>> src/hotspot/share/c1/c1_Runtime1.cpp the comments above the changed >>> code need updating - probably deleting. I assume the Access API >>> arraycopy is atomic? >>> >>> Thanks, >>> David >>> >>> On 6/03/2018 9:56 PM, Roman Kennke wrote: >>>> Currently, the Access API is only used for oop-arraycopy, but not for >>>> primitive arraycopy. GCs might want to intercept this too, e.g. >>>> resolve >>>> src and dst arrays. >>>> >>>> There *is* an implementation of primitive arraycopy in the Access API, >>>> but it doesn't even compile, because Raw::arraycopy() does not take >>>> src >>>> and dst oop operands, but it's called like that. The only reason why >>>> this does not blow up (I think) is that because nobody calls it, the >>>> compiler doesn't even get there. >>>> >>>> This change fixes the Access API/impl and adds the relevant calls into >>>> it (in C1 and runtime land). C2 uses arraycopy stubs (which cannot be >>>> handled here) or calls out to the ArrayKlass::copy_array(), which >>>> should >>>> be covered with this change. >>>> >>>> It should be possible to use the same Access API for Java-array <-> >>>> native-array bulk transfers, which currently use the rather ugly >>>> typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address that in a >>>> separate change though. >>>> >>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ >>>> >>>> Tests: tier1 ok >>>> >>>> Please review! >>>> Thanks, Roman >>>> >> From rkennke at redhat.com Tue Mar 6 13:35:02 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 6 Mar 2018 14:35:02 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: <5A9E95D8.6080602@oracle.com> References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> <5A9E8F76.1050809@oracle.com> <83de65bd-5d3f-78df-02d2-eb4c53aa0c62@oracle.com> <5A9E95D8.6080602@oracle.com> Message-ID: <2cc9a407-aba8-209a-7ef7-4ab26b88fef6@redhat.com> Makes me wonder: why attempt to be smart in c1_Runtime1.cpp, when we can just as well call typeArrayOop::copy_array() and have it do the right thing? Or go even further and also do it for oop-arraycopy? Roman > > The ARRAYCOPY_ATOMIC decorator makes the arraycopy atomic over the size > of the passed in elements. > In this case, it looks like the address has been type erased to void*, > and hence lost what the element size was. There is currently no overload > accepted for type erased element - only accurate elements {jbyte, > jshort, jint, jlong}. > > So it looks like an overload must be added to accept type erased void* > elements and make that call conjoint_memory_atomic when the > ARRAYCOPY_ATOMIC decorator is passed in. > > Thanks, > /Erik > > On 2018-03-06 13:56, David Holmes wrote: >> On 6/03/2018 10:54 PM, Erik ?sterlund wrote: >>> Hi David, >>> >>> It is atomic with the ARRAYCOPY_ATOMIC decorator. If that comment is >>> correct (I do not know if it is), then the ARRAYCOPY_ATOMIC decorator >>> should probably be used here. >> >> If that code implements a Java array copy then yes it is required to >> be 32-bit atomic. Do you need the decorator to get 32-bit atomicity? >> >> David >> >>> Thanks, >>> /Erik >>> >>> On 2018-03-06 13:48, David Holmes wrote: >>>> Hi Roman, >>>> >>>> Not a review as I'm not familiar enough with the Access API, but in >>>> src/hotspot/share/c1/c1_Runtime1.cpp the comments above the changed >>>> code need updating - probably deleting. I assume the Access API >>>> arraycopy is atomic? >>>> >>>> Thanks, >>>> David >>>> >>>> On 6/03/2018 9:56 PM, Roman Kennke wrote: >>>>> Currently, the Access API is only used for oop-arraycopy, but not for >>>>> primitive arraycopy. GCs might want to intercept this too, e.g. >>>>> resolve >>>>> src and dst arrays. >>>>> >>>>> There *is* an implementation of primitive arraycopy in the Access API, >>>>> but it doesn't even compile, because Raw::arraycopy() does not take >>>>> src >>>>> and dst oop operands, but it's called like that. The only reason why >>>>> this does not blow up (I think) is that because nobody calls it, the >>>>> compiler doesn't even get there. >>>>> >>>>> This change fixes the Access API/impl and adds the relevant calls into >>>>> it (in C1 and runtime land). C2 uses arraycopy stubs (which cannot be >>>>> handled here) or calls out to the ArrayKlass::copy_array(), which >>>>> should >>>>> be covered with this change. >>>>> >>>>> It should be possible to use the same Access API for Java-array <-> >>>>> native-array bulk transfers, which currently use the rather ugly >>>>> typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address that in a >>>>> separate change though. >>>>> >>>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ >>>>> >>>>> Tests: tier1 ok >>>>> >>>>> Please review! >>>>> Thanks, Roman >>>>> >>> > From thomas.schatzl at oracle.com Tue Mar 6 13:33:54 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 06 Mar 2018 14:33:54 +0100 Subject: RFR [L][3/7]: 8197850: Calculate liveness in regions during marking In-Reply-To: References: <1520258985.2532.40.camel@oracle.com> <1520260902.2532.66.camel@oracle.com> Message-ID: <1520343234.2378.29.camel@oracle.com> On Tue, 2018-03-06 at 13:31 +0100, Stefan Johansson wrote: > Hi Thomas, > > On 2018-03-05 15:41, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this change that calculates the liveness > > in regions (below nTAMS) during marking? > > > > This is required to be more clever about selecting the regions G1 > > is going to rebuild the remembered sets for (i.e. the well-known > > G1MixedGCLiveThresholdPercent threshold). > > > > This slows down the marking part of the concurrent phase a bit, but > > all alternatives were worse: > > > > - initially rebuild everything and then drop remembered sets for > > regions for too highly occupied regions at cleanup: wastes lots of > > space and memory maintaining these superfluous remembered sets. > > > > - choose regions using previous marking cycle data: this is > > generally not a good idea if your marking cycles are infrequent > > (which should be the normal case), as it tends to make too few > > regions available for evacuation (e.g. eager reclaim may already > > reclaim a lot). I.e. you need extra headroom, or be more aggressive > > with rebuilding. > > > > The main change, next to the calls that add the liveness to some > > internal data structure every time an object is marked, is some > > per- thread cache that (G1RegionMarkStatsCache). It helps > > decreasing the synchronization overhead the (atomic) increments in > > the global result cause. > > > > It is a very simple fixed size hashmap where increments to the same > > entries are collected, and on collision evicts the previous entry, > > doing the atomic operation on the global result. > > > > There is also some unfortunate moving of code that is required to > > flush these caches correctly during mark overflow handling. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8197850 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8197850/webrev/index.html > > Look good, just some minor things. > > src/hotspot/share/gc/g1/g1ConcurrentMark.cpp > 394 _region_mark_stats(NEW_C_HEAP_ARRAY(G1RegionMarkStats, > max_regions, mtGC)) > > max_regions is passed into the constructor but everywhere else in > G1ConcurrentMark we use g1h->max_regions(). I would consider either > calling g1h->max_regions() in the constructor too, or save the value > in a member and use it everywhere. Done using option 1), use g1h->max_regions() everywhere. > --- >src/hotspot/share/gc/g1/g1_globals.hpp > 262 experimental(size_t, G1RegionMarkStatsCacheSize, 1024, > ... > 265 range(128, (max_juint / 2) + 1) > > Do we need this flag or should we scale it against the number of > regions? If we want to keep it I think we should have a somewhat > more restrictive range. I would not want this to scale according to the number of heap regions, because that is going to have a significant impact on pause times when all of the thread's caches are evicted (The O(#regions) part). The flag has only been added (rather late, admittedly) to fix potential issues with marking time with really large heaps with tens of thousands of regions. Let me do some more measurements, I will get back to you with them before giving you a new webrev. > --- > src/hotspot/share/gc/g1/g1RegionMarkStatsCache.hpp > > In G1RegionMarkStatsCacheEntry _region_idx is set to 0 when clearing. > I don't see a problem with it right now, but it feels a bit wrong > since index 0 is a valid region index. Maybe we could use another > cleared marker. Unfortunately there is none (one could use something like -1, but that one is valid too), and it seems a waste of memory to reserve more space for it. I see your point that this is a bit ugly, but there should never be any issue because of the initialization of the value part of an entry with zero. It simply has no impact on totals if you add zero. The code could initialize the cache with region indices from 0 to cache size - 1, which are valid values given the hash function, what do you think? However that would slow down clearing a little (the compiler will just compile it to a memset() in the loop right now). Thanks, Thomas From stefan.johansson at oracle.com Tue Mar 6 13:49:20 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 6 Mar 2018 14:49:20 +0100 Subject: RFR [1/7]: 8197569: Refactor eager reclaim for concurrent remembered set rebuilding In-Reply-To: <1520342273.2378.15.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520259499.2532.44.camel@oracle.com> <8d58ae9c-1c18-c317-a40c-66241faa9a81@oracle.com> <1520342273.2378.15.camel@oracle.com> Message-ID: On 2018-03-06 14:17, Thomas Schatzl wrote: > Hi Stefan, > > thanks for looking into this. > > On Mon, 2018-03-05 at 16:11 +0100, Stefan Johansson wrote: >> Thanks for splitting this change up Thomas, >> >> On 2018-03-05 15:18, Thomas Schatzl wrote: >>> Hi all, >>> >>> can I have some reviews for the following change that refactors >>> the >>> eager reclaim condition so that it can be later used for the policy >>> that determines for which regions G1 should rebuild the remembered >>> set? >>> (And it's a reasonable refactoring on its own btw). >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8197569 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8197569/webrev/ >> Look good, just one thing: >> src/hotspot/share/gc/g1/g1CollectedHeap.cpp >> 2542 bool humongous_region_is_candidate(G1CollectedHeap* heap, >> HeapRegion* region) const { >> ... >> 2594 return obj->is_typeArray() && >> 2595 G1CollectedHeap::heap()- >>> is_potential_eager_reclaim_candidate(region); >> You could use the passed in heap parameter instead of using >> G1CollectedHeap::heap(). > Done. Also renamed the parameter to "g1h" as used everywhere else. > >> Another, maybe even nicer, solution would be to >> move is_potential_eager_reclaim_candidate() to HeapRegion, and just >> call region->is_potential_eager_reclaim_candidate(). > I did not do this for the following reasons: > > - is_potential_eager_reclaim_candidate may depend on global state in > the future. E.g. when trying to eagerly reclaim objArrays or similar the result depends on whether we are currently marking or not. > > I do not like making HeapRegion dependent on e.g. G1CollectorState :) > (any more than it already may be). > > - (almost) all eager reclaim related code has until now been in > G1CollectedHeap. Such a change would go against this, and run kind of > contrary to my intent to move away from using HeapRegion as dumping > ground for anything loosely related to it out of convenience, but only > as a representation for that memory. > > Either way is kind of fine for me right now, so let's see what others > say? I'm fine with this as well. Consider it reviewed :) Stefan > New webrevs: > http://cr.openjdk.java.net/~tschatzl/8197569/webrev.0_to_1/ (diff) > http://cr.openjdk.java.net/~tschatzl/8197569/webrev.1/ (full) > > Thanks, > Thomas > From stefan.johansson at oracle.com Tue Mar 6 14:03:11 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 6 Mar 2018 15:03:11 +0100 Subject: RFR [L][3/7]: 8197850: Calculate liveness in regions during marking In-Reply-To: <1520343234.2378.29.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520260902.2532.66.camel@oracle.com> <1520343234.2378.29.camel@oracle.com> Message-ID: <8f7f25ef-f0c6-20f6-1584-b615dba234e1@oracle.com> On 2018-03-06 14:33, Thomas Schatzl wrote: > On Tue, 2018-03-06 at 13:31 +0100, Stefan Johansson wrote: >> Hi Thomas, >> >> On 2018-03-05 15:41, Thomas Schatzl wrote: >>> Hi all, >>> >>> ... >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8197850 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8197850/webrev/index.html >> Look good, just some minor things. >> >> src/hotspot/share/gc/g1/g1ConcurrentMark.cpp >> 394 _region_mark_stats(NEW_C_HEAP_ARRAY(G1RegionMarkStats, >> max_regions, mtGC)) >> >> max_regions is passed into the constructor but everywhere else in >> G1ConcurrentMark we use g1h->max_regions(). I would consider either >> calling g1h->max_regions() in the constructor too, or save the value >> in a member and use it everywhere. > Done using option 1), use g1h->max_regions() everywhere. Thanks. >> --- >> src/hotspot/share/gc/g1/g1_globals.hpp >> 262 experimental(size_t, G1RegionMarkStatsCacheSize, 1024, >> ... >> 265 range(128, (max_juint / 2) + 1) >> >> Do we need this flag or should we scale it against the number of >> regions? If we want to keep it I think we should have a somewhat >> more restrictive range. > I would not want this to scale according to the number of heap regions, > because that is going to have a significant impact on pause times when > all of the thread's caches are evicted (The O(#regions) part). True, just a thought since 1024 is roughly half the number regions we aim for maybe this could be used together with a reasonable upper bound. > The flag has only been added (rather late, admittedly) to fix potential > issues with marking time with really large heaps with tens of thousands > of regions. > Let me do some more measurements, I will get back to you with them > before giving you a new webrev. Sound good. >> --- >> src/hotspot/share/gc/g1/g1RegionMarkStatsCache.hpp >> >> In G1RegionMarkStatsCacheEntry _region_idx is set to 0 when clearing. >> I don't see a problem with it right now, but it feels a bit wrong >> since index 0 is a valid region index. Maybe we could use another >> cleared marker. > Unfortunately there is none (one could use something like -1, but that > one is valid too), and it seems a waste of memory to reserve more space > for it. > > I see your point that this is a bit ugly, but there should never be any > issue because of the initialization of the value part of an entry with > zero. It simply has no impact on totals if you add zero. > > The code could initialize the cache with region indices from 0 to cache > size - 1, which are valid values given the hash function, what do you > think? > > However that would slow down clearing a little (the compiler will just > compile it to a memset() in the loop right now). You are correct that -1 is a valid index, but we seldom see heaps with so many regions. I'm fine with leaving it as is, if you dislike using -1 and if it will slow down clearing. Thanks, Stefan > > Thanks, > Thomas > From martin.doerr at sap.com Tue Mar 6 14:37:07 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 6 Mar 2018 14:37:07 +0000 Subject: RFR(S): 8198510: Enable UseDynamicNumberOfGCThreads by default Message-ID: <96da428b596e4a729612b32415d2ed11@sap.com> Hi, I'd like to solve the problem that an unnecessarily high number of GC threads get started in some situations. This wastes resources. The proposal is to enable the existing flag UseDynamicNumberOfGCThreads by default. Bug: https://bugs.openjdk.java.net/browse/JDK-8198510 CSR: https://bugs.openjdk.java.net/browse/JDK-8198547 Webrev: http://cr.openjdk.java.net/~mdoerr/8198510_gc_threads/webrev.01/ Please review. Thanks and best regards, Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Tue Mar 6 15:01:04 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 06 Mar 2018 16:01:04 +0100 Subject: RFR(S): 8198510: Enable UseDynamicNumberOfGCThreads by default In-Reply-To: <96da428b596e4a729612b32415d2ed11@sap.com> References: <96da428b596e4a729612b32415d2ed11@sap.com> Message-ID: <1520348464.2378.39.camel@oracle.com> Hi, On Tue, 2018-03-06 at 14:37 +0000, Doerr, Martin wrote: > Hi, > > I?d like to solve the problem that an unnecessarily high number of GC > threads get started in some situations. This wastes resources. > The proposal is to enable the existing flag > UseDynamicNumberOfGCThreads by default. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8198510 > > CSR: > https://bugs.openjdk.java.net/browse/JDK-8198547 > > Webrev: > http://cr.openjdk.java.net/~mdoerr/8198510_gc_threads/webrev.01/ > > Please review. > looks good to me :) Thomas From rkennke at redhat.com Tue Mar 6 16:03:28 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 6 Mar 2018 17:03:28 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: <2cc9a407-aba8-209a-7ef7-4ab26b88fef6@redhat.com> References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> <5A9E8F76.1050809@oracle.com> <83de65bd-5d3f-78df-02d2-eb4c53aa0c62@oracle.com> <5A9E95D8.6080602@oracle.com> <2cc9a407-aba8-209a-7ef7-4ab26b88fef6@redhat.com> Message-ID: <6d8435af-5bec-da70-fa8a-60d708937f44@redhat.com> Am 06.03.2018 um 14:35 schrieb Roman Kennke: > Makes me wonder: why attempt to be smart in c1_Runtime1.cpp, when we can > just as well call typeArrayOop::copy_array() and have it do the right > thing? Or go even further and also do it for oop-arraycopy? Something like: http://cr.openjdk.java.net/~rkennke/8198445/8198445-1.patch This wouldn't compile because of bunch of missing arraycopy_conjoint_atomic defintions for extra types like jfloat, jdouble, jboolean, etc, which in turn would be missing the same Copy::conjoint_jFluffs_atomic() which drags in a bunch of platform specific stuff... and my question before I go there is: do we want all that? Or can you think of a better way to solve it? Roman > > Roman > >> >> The ARRAYCOPY_ATOMIC decorator makes the arraycopy atomic over the size >> of the passed in elements. >> In this case, it looks like the address has been type erased to void*, >> and hence lost what the element size was. There is currently no overload >> accepted for type erased element - only accurate elements {jbyte, >> jshort, jint, jlong}. >> >> So it looks like an overload must be added to accept type erased void* >> elements and make that call conjoint_memory_atomic when the >> ARRAYCOPY_ATOMIC decorator is passed in. >> >> Thanks, >> /Erik >> >> On 2018-03-06 13:56, David Holmes wrote: >>> On 6/03/2018 10:54 PM, Erik ?sterlund wrote: >>>> Hi David, >>>> >>>> It is atomic with the ARRAYCOPY_ATOMIC decorator. If that comment is >>>> correct (I do not know if it is), then the ARRAYCOPY_ATOMIC decorator >>>> should probably be used here. >>> >>> If that code implements a Java array copy then yes it is required to >>> be 32-bit atomic. Do you need the decorator to get 32-bit atomicity? >>> >>> David >>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2018-03-06 13:48, David Holmes wrote: >>>>> Hi Roman, >>>>> >>>>> Not a review as I'm not familiar enough with the Access API, but in >>>>> src/hotspot/share/c1/c1_Runtime1.cpp the comments above the changed >>>>> code need updating - probably deleting. I assume the Access API >>>>> arraycopy is atomic? >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> On 6/03/2018 9:56 PM, Roman Kennke wrote: >>>>>> Currently, the Access API is only used for oop-arraycopy, but not for >>>>>> primitive arraycopy. GCs might want to intercept this too, e.g. >>>>>> resolve >>>>>> src and dst arrays. >>>>>> >>>>>> There *is* an implementation of primitive arraycopy in the Access API, >>>>>> but it doesn't even compile, because Raw::arraycopy() does not take >>>>>> src >>>>>> and dst oop operands, but it's called like that. The only reason why >>>>>> this does not blow up (I think) is that because nobody calls it, the >>>>>> compiler doesn't even get there. >>>>>> >>>>>> This change fixes the Access API/impl and adds the relevant calls into >>>>>> it (in C1 and runtime land). C2 uses arraycopy stubs (which cannot be >>>>>> handled here) or calls out to the ArrayKlass::copy_array(), which >>>>>> should >>>>>> be covered with this change. >>>>>> >>>>>> It should be possible to use the same Access API for Java-array <-> >>>>>> native-array bulk transfers, which currently use the rather ugly >>>>>> typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address that in a >>>>>> separate change though. >>>>>> >>>>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ >>>>>> >>>>>> Tests: tier1 ok >>>>>> >>>>>> Please review! >>>>>> Thanks, Roman >>>>>> >>>> >> > From erik.osterlund at oracle.com Tue Mar 6 16:10:22 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 6 Mar 2018 17:10:22 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: <6d8435af-5bec-da70-fa8a-60d708937f44@redhat.com> References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> <5A9E8F76.1050809@oracle.com> <83de65bd-5d3f-78df-02d2-eb4c53aa0c62@oracle.com> <5A9E95D8.6080602@oracle.com> <2cc9a407-aba8-209a-7ef7-4ab26b88fef6@redhat.com> <6d8435af-5bec-da70-fa8a-60d708937f44@redhat.com> Message-ID: <5A9EBD6E.3070005@oracle.com> Hi Roman, My spontaneous idea was to have an overload for type erased void* that when called with ARRAYCOPY_ATOMIC maps to Copy::conjoint_memory_atomic which AFAIK is conservatively atomic regardless of what the precise element type is. But if you have better ideas, I am open for suggestions. Thanks, /Erik On 2018-03-06 17:03, Roman Kennke wrote: > Am 06.03.2018 um 14:35 schrieb Roman Kennke: >> Makes me wonder: why attempt to be smart in c1_Runtime1.cpp, when we can >> just as well call typeArrayOop::copy_array() and have it do the right >> thing? Or go even further and also do it for oop-arraycopy? > Something like: > > http://cr.openjdk.java.net/~rkennke/8198445/8198445-1.patch > > This wouldn't compile because of bunch of missing > arraycopy_conjoint_atomic defintions for extra types like jfloat, > jdouble, jboolean, etc, which in turn would be missing the same > Copy::conjoint_jFluffs_atomic() which drags in a bunch of platform > specific stuff... and my question before I go there is: do we want all > that? Or can you think of a better way to solve it? > > Roman > >> Roman >> >>> The ARRAYCOPY_ATOMIC decorator makes the arraycopy atomic over the size >>> of the passed in elements. >>> In this case, it looks like the address has been type erased to void*, >>> and hence lost what the element size was. There is currently no overload >>> accepted for type erased element - only accurate elements {jbyte, >>> jshort, jint, jlong}. >>> >>> So it looks like an overload must be added to accept type erased void* >>> elements and make that call conjoint_memory_atomic when the >>> ARRAYCOPY_ATOMIC decorator is passed in. >>> >>> Thanks, >>> /Erik >>> >>> On 2018-03-06 13:56, David Holmes wrote: >>>> On 6/03/2018 10:54 PM, Erik ?sterlund wrote: >>>>> Hi David, >>>>> >>>>> It is atomic with the ARRAYCOPY_ATOMIC decorator. If that comment is >>>>> correct (I do not know if it is), then the ARRAYCOPY_ATOMIC decorator >>>>> should probably be used here. >>>> If that code implements a Java array copy then yes it is required to >>>> be 32-bit atomic. Do you need the decorator to get 32-bit atomicity? >>>> >>>> David >>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>> On 2018-03-06 13:48, David Holmes wrote: >>>>>> Hi Roman, >>>>>> >>>>>> Not a review as I'm not familiar enough with the Access API, but in >>>>>> src/hotspot/share/c1/c1_Runtime1.cpp the comments above the changed >>>>>> code need updating - probably deleting. I assume the Access API >>>>>> arraycopy is atomic? >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>> On 6/03/2018 9:56 PM, Roman Kennke wrote: >>>>>>> Currently, the Access API is only used for oop-arraycopy, but not for >>>>>>> primitive arraycopy. GCs might want to intercept this too, e.g. >>>>>>> resolve >>>>>>> src and dst arrays. >>>>>>> >>>>>>> There *is* an implementation of primitive arraycopy in the Access API, >>>>>>> but it doesn't even compile, because Raw::arraycopy() does not take >>>>>>> src >>>>>>> and dst oop operands, but it's called like that. The only reason why >>>>>>> this does not blow up (I think) is that because nobody calls it, the >>>>>>> compiler doesn't even get there. >>>>>>> >>>>>>> This change fixes the Access API/impl and adds the relevant calls into >>>>>>> it (in C1 and runtime land). C2 uses arraycopy stubs (which cannot be >>>>>>> handled here) or calls out to the ArrayKlass::copy_array(), which >>>>>>> should >>>>>>> be covered with this change. >>>>>>> >>>>>>> It should be possible to use the same Access API for Java-array <-> >>>>>>> native-array bulk transfers, which currently use the rather ugly >>>>>>> typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address that in a >>>>>>> separate change though. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ >>>>>>> >>>>>>> Tests: tier1 ok >>>>>>> >>>>>>> Please review! >>>>>>> Thanks, Roman >>>>>>> From sangheon.kim at oracle.com Tue Mar 6 23:28:46 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Tue, 6 Mar 2018 15:28:46 -0800 Subject: RFR [1/7]: 8197569: Refactor eager reclaim for concurrent remembered set rebuilding In-Reply-To: <1520342273.2378.15.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520259499.2532.44.camel@oracle.com> <8d58ae9c-1c18-c317-a40c-66241faa9a81@oracle.com> <1520342273.2378.15.camel@oracle.com> Message-ID: <35988ffd-383e-307e-f297-8abaad1dd43b@oracle.com> Hi Thomas, Thank you for these bunch of patches for rebuilding RemSet concurrently. This is really nice. On 03/06/2018 05:17 AM, Thomas Schatzl wrote: > Hi Stefan, > > thanks for looking into this. > > On Mon, 2018-03-05 at 16:11 +0100, Stefan Johansson wrote: >> Thanks for splitting this change up Thomas, >> >> On 2018-03-05 15:18, Thomas Schatzl wrote: >>> Hi all, >>> >>> can I have some reviews for the following change that refactors >>> the >>> eager reclaim condition so that it can be later used for the policy >>> that determines for which regions G1 should rebuild the remembered >>> set? >>> (And it's a reasonable refactoring on its own btw). >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8197569 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8197569/webrev/ >> Look good, just one thing: >> src/hotspot/share/gc/g1/g1CollectedHeap.cpp >> 2542 bool humongous_region_is_candidate(G1CollectedHeap* heap, >> HeapRegion* region) const { >> ... >> 2594 return obj->is_typeArray() && >> 2595 G1CollectedHeap::heap()- >>> is_potential_eager_reclaim_candidate(region); >> You could use the passed in heap parameter instead of using >> G1CollectedHeap::heap(). > Done. Also renamed the parameter to "g1h" as used everywhere else. Good but still there are some places using 'g1'. It would be good idea to fix, of course in other CR. > >> Another, maybe even nicer, solution would be to >> move is_potential_eager_reclaim_candidate() to HeapRegion, and just >> call region->is_potential_eager_reclaim_candidate(). > I did not do this for the following reasons: > > - is_potential_eager_reclaim_candidate may depend on global state in > the future. E.g. when trying to eagerly reclaim objArrays or similar the result depends on whether we are currently marking or not. > > I do not like making HeapRegion dependent on e.g. G1CollectorState :) > (any more than it already may be). > > - (almost) all eager reclaim related code has until now been in > G1CollectedHeap. Such a change would go against this, and run kind of > contrary to my intent to move away from using HeapRegion as dumping > ground for anything loosely related to it out of convenience, but only > as a representation for that memory. > > Either way is kind of fine for me right now, so let's see what others > say? Sounds reasonable to me. > > New webrevs: > http://cr.openjdk.java.net/~tschatzl/8197569/webrev.0_to_1/ (diff) > http://cr.openjdk.java.net/~tschatzl/8197569/webrev.1/ (full) webrev.1 seems good to me. Thanks, Sangheon > > Thanks, > Thomas > From sangheon.kim at oracle.com Wed Mar 7 00:02:05 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Tue, 6 Mar 2018 16:02:05 -0800 Subject: RFR [S][2/7] 8197570: Make rules for choosing collection set candidates more explicit In-Reply-To: <1520259735.2532.48.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520259735.2532.48.camel@oracle.com> Message-ID: Hi Thomas, On 03/05/2018 06:22 AM, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that makes the rule (occupancy) > for choosing collection set candidates more explicit, i.e. factor it > into a single method. This is needed for selecting the regions we want > to rebuild the remembered sets concurrently (JDK-8180415) more > straightforward. > > It also adds a method to allow iteration over the regions in the > collection set - I did not see that worth an extra webrev. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8197570 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8197570/webrev/ Looks good. One minor nit: You can add 'const' qualifier to the newly added method, CollectionSetChooser::should_add(HeapRegion* hr). I don't need a new webrev for this. Thanks, Sangheon > Testing: > hs-tier 1-5, mostly in conjunction with related changes > > Thanks, > Thomas > From stefan.johansson at oracle.com Wed Mar 7 08:25:35 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 7 Mar 2018 09:25:35 +0100 Subject: RFR: 8197852: Move G1DefaultPolicy into G1Policy In-Reply-To: References: Message-ID: <045edcd2-884a-a0a6-a3b6-49d2a7810de7@oracle.com> Looks good, StefanJ On 2018-03-05 16:02, Erik Helin wrote: > Hi all, > > this fairly small patch re-unites G1Policy and G1DefaultPolicy, there > is no need for having both classes. With this patch we only have > G1Policy (and of course G1CollectorPolicy as the little > CollectorPolicy shim). > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8197852 > > Patch: > http://cr.openjdk.java.net/~ehelin/8197852/00/ > > Testing: > - make run-test-tier1 on Linux, Windows, Mac, Solaris (SPARC and x86-64) > > Thanks, > Erik From thomas.schatzl at oracle.com Wed Mar 7 08:48:41 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 07 Mar 2018 09:48:41 +0100 Subject: RFR [L][3/7]: 8197850: Calculate liveness in regions during marking In-Reply-To: <8f7f25ef-f0c6-20f6-1584-b615dba234e1@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520260902.2532.66.camel@oracle.com> <1520343234.2378.29.camel@oracle.com> <8f7f25ef-f0c6-20f6-1584-b615dba234e1@oracle.com> Message-ID: <1520412521.2346.4.camel@oracle.com> Hi, On Tue, 2018-03-06 at 15:03 +0100, Stefan Johansson wrote: > > On 2018-03-06 14:33, Thomas Schatzl wrote: > > On Tue, 2018-03-06 at 13:31 +0100, Stefan Johansson wrote: > > > Hi Thomas, > > > > > > On 2018-03-05 15:41, Thomas Schatzl wrote: > > > > Hi all, > > > > > > > > ... > > > > > > > > CR: > > > > https://bugs.openjdk.java.net/browse/JDK-8197850 > > > > Webrev: > > > > http://cr.openjdk.java.net/~tschatzl/8197850/webrev/index.html > > > > > > Look good, just some minor things. > > > > > > src/hotspot/share/gc/g1/g1ConcurrentMark.cpp > > > 394 _region_mark_stats(NEW_C_HEAP_ARRAY(G1RegionMarkStats, > > > max_regions, mtGC)) > > > > > > max_regions is passed into the constructor but everywhere else in > > > G1ConcurrentMark we use g1h->max_regions(). I would consider > > > either > > > calling g1h->max_regions() in the constructor too, or save the > > > value > > > in a member and use it everywhere. > > > > Done using option 1), use g1h->max_regions() everywhere. > > Thanks. > > > --- > > > src/hotspot/share/gc/g1/g1_globals.hpp > > > 262 experimental(size_t, G1RegionMarkStatsCacheSize, 1024, > > > ... > > > 265 range(128, (max_juint / 2) + 1) > > > > > > Do we need this flag or should we scale it against the number of > > > regions? If we want to keep it I think we should have a somewhat > > > more restrictive range. > > > > I would not want this to scale according to the number of heap > > regions, > > because that is going to have a significant impact on pause times > > when > > all of the thread's caches are evicted (The O(#regions) part). > > True, just a thought since 1024 is roughly half the number regions > we aim for maybe this could be used together with a reasonable upper > bound. > > > The flag has only been added (rather late, admittedly) to fix > > potential issues with marking time with really large heaps with > > tens of thousands of regions. > > Let me do some more measurements, I will get back to you with them > > before giving you a new webrev. > > Sound good. I did some measurements with 100k regions, and I think we can just remove the flag. There is always logging available (gc+mark+stats) that shows cache hit ratio to diagnose issues. > > > --- > > > src/hotspot/share/gc/g1/g1RegionMarkStatsCache.hpp > > > > > > In G1RegionMarkStatsCacheEntry _region_idx is set to 0 when > > > clearing. I don't see a problem with it right now, but it feels a > > > bit wrong since index 0 is a valid region index. Maybe we could > > > use another cleared marker. > > > > Unfortunately there is none (one could use something like -1, but > > that one is valid too), and it seems a waste of memory to reserve > > more space for it. > > > > I see your point that this is a bit ugly, but there should never be > > any issue because of the initialization of the value part of an > > entry with zero. It simply has no impact on totals if you add zero. > > > > The code could initialize the cache with region indices from 0 to > > cache size - 1, which are valid values given the hash function, > > what do you think? > > > > However that would slow down clearing a little (the compiler will > > just compile it to a memset() in the loop right now). > > You are correct that -1 is a valid index, but we seldom see heaps > with so many regions. I'm fine with leaving it as is, if you dislike > using -1 and if it will slow down clearing. Just a bit. It seems as bad to use -1 or another random value, so I kept it. Webrevs: http://cr.openjdk.java.net/~tschatzl/8197850/webrev.0_to_1/index.html (diff) http://cr.openjdk.java.net/~tschatzl/8197850/webrev.1/index.html (full) Testing: This change ran, with some changes to 8180415 due to internal comments (I will post in a few minutes) through hs-tier 1-5 over night. Thanks, Thomas From thomas.schatzl at oracle.com Wed Mar 7 08:59:36 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 07 Mar 2018 09:59:36 +0100 Subject: RFR [L][4/7]: 8180415: Rebuild remembered sets during the concurrent cycle In-Reply-To: <1520262478.2532.91.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520262478.2532.91.camel@oracle.com> Message-ID: <1520413176.2346.9.camel@oracle.com> Hi all, Stefan had some comments about the HeapRegionType parameter in the G1RemSetTrackingPolicy::update_at_allocate() method - that one is not really required when placing the call to this method correctly and just using the HeapRegion's type directly. Changing this removes another 40 LOC of changes. There has been another bug introduced by me during cleaning up for final review: G1RemSetTrackingPolicy::update_at_free() is empty in this change, which is wrong for this change - in this change we still do not free the remembered sets during the cleanup pause, only in the Concurrent Cleanup phase. When doing this concurrently, an assert triggers when setting the remembered set state to empty outside a safepoint. The fix is to make the remembered set untracked during the Cleanup phase still. New webrevs: http://cr.openjdk.java.net/~tschatzl/8180415/webrev.0_to_1 (diff) http://cr.openjdk.java.net/~tschatzl/8180415/webrev.1 (full) Thanks, Thomas On Mon, 2018-03-05 at 16:07 +0100, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that implements the bulk of the > "rebuild remembered sets concurrently" mechanism? > > This change modifies the management of remembered sets so that those > are only maintained when they are useful. This roughly means: > - always maintain remembered sets for young gen regions > - (almost) always maintain remembered sets for humongous regions (for > eager reclaim) as long as they meet the eager reclaim criteria. > - never create remembered sets for archive regions > - do not create remembered sets for any regions during full gc > - do not create remembered sets for regions with evacuation failure > > (The latter two decisions may not be optimal, but the idea was to > strictly only maintain remembered sets for regions we are going to > try > to evacuate soon) > > - create remembered sets for old gen regions after finishing marking > (starting at the Remark pause), iff > > - the liveness of these regions indicate so (liveness <= > G1MixedGCLiveThresholdPercent) > - any (non-objArray) humongous region that does not have a > remembered > set yet (eg. because of full gc) > > (Please see the new G1RemSetTrackingPolicy implementation for exact > details) > > During the following "Rebuild Remembered Set" phase, that replaces > the > "Create Live Data" phase, all non-young regions that may contain > outgoing pointers (i.e. excluding closed archive regions), G1 scans > the > live objects in the regions between bottom and TARS > ("top_at_rebuild_start" - yes, that's new :) for inter-region > references and add them to the appropriate remembered set. > > The TARS, as the name indicates, is the top-value at the start of > remark, and the regions contain live data between this value and > bottom > that needs to be scanned during this rebuild phase. > > The reason for a new indicator is that nTAMS is *not* sufficient: > during marking there might have been more allocations in the old gen > that also need to be scanned for new references. > > All references above TARS (and below the region's top) will be > handled > by the existing barrier mechanism. > > After rebuild, during the Cleanup pause, we create the new > (candidate) > collection set, eventually filtering out more regions, like humongous > ones that exceed the eager reclaim criteria. > > The current state of the remembered set for a given region is tracked > within the remembered set for that region (HeapRegionRememberedSet > class): a remembered set can be either Untracked, Updating or > Complete. > Untracked means that we do not manage a remembered set for this > region, > Complete means just that, and Updating means that we are currently in > the process of updating the remembered set. This distinction is > relevant for evacuation, e.g. we obviously must not evacuate not- > Complete remembered sets. > > Note that this change contains one single temporary ugly hack that > allows G1 to work until the next change :) - so in this change G1 > determines whether there will be a mixed gc phase during the Cleanup > pause. It needs to determine this information at that time, instead > of > previously during concurrent cleanup, to, if there is no mixed gc > coming, drop all remembered sets to not maintain them further. > > For this reason, and to avoid duplicate recalculation that may yield > different results after dropping remembered sets, there is a new flag > mixed_gc_pending in CollectorState, making it even more complicated. > That one is going away in the next change, promised. :) > > Some imho useful messages about remembered set tracking are printed > using "gc, remset, tracking" log tags, both in debug and trace level. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8180415 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8180415/webrev/index.html > Testing: > hs-tier 1-5, etc. etc. > > Thanks, > Thomas From stefan.johansson at oracle.com Wed Mar 7 09:29:07 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 7 Mar 2018 10:29:07 +0100 Subject: RFR(S): 8198510: Enable UseDynamicNumberOfGCThreads by default In-Reply-To: <96da428b596e4a729612b32415d2ed11@sap.com> References: <96da428b596e4a729612b32415d2ed11@sap.com> Message-ID: <23a06a29-d8c1-3f85-06be-93a8ca5380a7@oracle.com> Hi Martin, On 2018-03-06 15:37, Doerr, Martin wrote: > > Hi, > > I?d like to solve the problem that an unnecessarily high number of GC > threads get started in some situations. This wastes resources. > > The proposal is to enable the existing flag > UseDynamicNumberOfGCThreads by default. > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8198510 > > CSR: > > https://bugs.openjdk.java.net/browse/JDK-8198547 > > Webrev: > > http://cr.openjdk.java.net/~mdoerr/8198510_gc_threads/webrev.01/ > > Change looks good and I also reviewed the CSR. So feel free to move it to proposed (or finalized if you want to fast-track it). Thanks, Stefan > Please review. > > Thanks and best regards, > > Martin > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Wed Mar 7 10:00:09 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 07 Mar 2018 11:00:09 +0100 Subject: RFR [S][2/7] 8197570: Make rules for choosing collection set candidates more explicit In-Reply-To: References: <1520258985.2532.40.camel@oracle.com> <1520259735.2532.48.camel@oracle.com> Message-ID: <1520416809.2346.16.camel@oracle.com> Hi, thanks for your review. I created webrevs anyway: http://cr.openjdk.java.net/~tschatzl/8197570/webrev.0_to_1/ (diff) http://cr.openjdk.java.net/~tschatzl/8197570/webrev.1/ (full) Thanks, Thomas On Tue, 2018-03-06 at 16:02 -0800, sangheon.kim wrote: > Hi Thomas, > > On 03/05/2018 06:22 AM, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this change that makes the rule > > (occupancy) > > for choosing collection set candidates more explicit, i.e. factor > > it > > into a single method. This is needed for selecting the regions we > > want > > to rebuild the remembered sets concurrently (JDK-8180415) more > > straightforward. > > > > It also adds a method to allow iteration over the regions in the > > collection set - I did not see that worth an extra webrev. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8197570 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8197570/webrev/ > > Looks good. > > One minor nit: > You can add 'const' qualifier to the newly added method, > CollectionSetChooser::should_add(HeapRegion* hr). > > I don't need a new webrev for this. > > Thanks, > Sangheon > > > > Testing: > > hs-tier 1-5, mostly in conjunction with related changes > > > > Thanks, > > Thomas > > > > From thomas.schatzl at oracle.com Wed Mar 7 10:04:14 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 07 Mar 2018 11:04:14 +0100 Subject: RFR: 8197852: Move G1DefaultPolicy into G1Policy In-Reply-To: References: Message-ID: <1520417054.2346.19.camel@oracle.com> Hi, On Mon, 2018-03-05 at 16:02 +0100, Erik Helin wrote: > Hi all, > > this fairly small patch re-unites G1Policy and G1DefaultPolicy, there > is no need for having both classes. With this patch we only have > G1Policy (and of course G1CollectorPolicy as the little > CollectorPolicy shim). > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8197852 > > Patch: > http://cr.openjdk.java.net/~ehelin/8197852/00/ > > Testing: > - make run-test-tier1 on Linux, Windows, Mac, Solaris (SPARC and x86- > 64) in g1Policy.hpp, if you update the copyright notice, please use the correct year (2018) :) Is there a way to move g1DefaultPolicy.hpp into g1Policy.hpp while retaining history? I.e. delete the latter first, and then rename? I do not need to re-review this change. Thanks, Thomas From stefan.johansson at oracle.com Wed Mar 7 10:12:10 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 7 Mar 2018 11:12:10 +0100 Subject: RFR [L][3/7]: 8197850: Calculate liveness in regions during marking In-Reply-To: <1520412521.2346.4.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520260902.2532.66.camel@oracle.com> <1520343234.2378.29.camel@oracle.com> <8f7f25ef-f0c6-20f6-1584-b615dba234e1@oracle.com> <1520412521.2346.4.camel@oracle.com> Message-ID: <0cd6cca6-cace-4e4f-7f9c-152e20757dfe@oracle.com> Thanks Thomas, This looks good to me, reviewed. Stefan On 2018-03-07 09:48, Thomas Schatzl wrote: > Hi, > > On Tue, 2018-03-06 at 15:03 +0100, Stefan Johansson wrote: >> On 2018-03-06 14:33, Thomas Schatzl wrote: >>> On Tue, 2018-03-06 at 13:31 +0100, Stefan Johansson wrote: >>>> Hi Thomas, >>>> >>>> On 2018-03-05 15:41, Thomas Schatzl wrote: >>>>> Hi all, >>>>> >>>>> ... >>>>> >>>>> CR: >>>>> https://bugs.openjdk.java.net/browse/JDK-8197850 >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~tschatzl/8197850/webrev/index.html >>>> Look good, just some minor things. >>>> >>>> src/hotspot/share/gc/g1/g1ConcurrentMark.cpp >>>> 394 _region_mark_stats(NEW_C_HEAP_ARRAY(G1RegionMarkStats, >>>> max_regions, mtGC)) >>>> >>>> max_regions is passed into the constructor but everywhere else in >>>> G1ConcurrentMark we use g1h->max_regions(). I would consider >>>> either >>>> calling g1h->max_regions() in the constructor too, or save the >>>> value >>>> in a member and use it everywhere. >>> Done using option 1), use g1h->max_regions() everywhere. >> Thanks. >>>> --- >>>> src/hotspot/share/gc/g1/g1_globals.hpp >>>> 262 experimental(size_t, G1RegionMarkStatsCacheSize, 1024, >>>> ... >>>> 265 range(128, (max_juint / 2) + 1) >>>> >>>> Do we need this flag or should we scale it against the number of >>>> regions? If we want to keep it I think we should have a somewhat >>>> more restrictive range. >>> I would not want this to scale according to the number of heap >>> regions, >>> because that is going to have a significant impact on pause times >>> when >>> all of the thread's caches are evicted (The O(#regions) part). >> True, just a thought since 1024 is roughly half the number regions >> we aim for maybe this could be used together with a reasonable upper >> bound. >> >>> The flag has only been added (rather late, admittedly) to fix >>> potential issues with marking time with really large heaps with >>> tens of thousands of regions. >>> Let me do some more measurements, I will get back to you with them >>> before giving you a new webrev. >> Sound good. > I did some measurements with 100k regions, and I think we can just > remove the flag. There is always logging available (gc+mark+stats) that > shows cache hit ratio to diagnose issues. > >>>> --- >>>> src/hotspot/share/gc/g1/g1RegionMarkStatsCache.hpp >>>> >>>> In G1RegionMarkStatsCacheEntry _region_idx is set to 0 when >>>> clearing. I don't see a problem with it right now, but it feels a >>>> bit wrong since index 0 is a valid region index. Maybe we could >>>> use another cleared marker. >>> Unfortunately there is none (one could use something like -1, but >>> that one is valid too), and it seems a waste of memory to reserve >>> more space for it. >>> >>> I see your point that this is a bit ugly, but there should never be >>> any issue because of the initialization of the value part of an >>> entry with zero. It simply has no impact on totals if you add zero. >>> >>> The code could initialize the cache with region indices from 0 to >>> cache size - 1, which are valid values given the hash function, >>> what do you think? >>> >>> However that would slow down clearing a little (the compiler will >>> just compile it to a memset() in the loop right now). >> You are correct that -1 is a valid index, but we seldom see heaps >> with so many regions. I'm fine with leaving it as is, if you dislike >> using -1 and if it will slow down clearing. > Just a bit. It seems as bad to use -1 or another random value, so I > kept it. > > Webrevs: > http://cr.openjdk.java.net/~tschatzl/8197850/webrev.0_to_1/index.html > (diff) > http://cr.openjdk.java.net/~tschatzl/8197850/webrev.1/index.html (full) > > Testing: > This change ran, with some changes to 8180415 due to internal comments > (I will post in a few minutes) through hs-tier 1-5 over night. > > Thanks, > Thomas From erik.helin at oracle.com Wed Mar 7 10:39:44 2018 From: erik.helin at oracle.com (Erik Helin) Date: Wed, 7 Mar 2018 11:39:44 +0100 Subject: RFR: 8197852: Move G1DefaultPolicy into G1Policy In-Reply-To: <1520417054.2346.19.camel@oracle.com> References: <1520417054.2346.19.camel@oracle.com> Message-ID: <5587e8c6-2a1c-4c23-dd2b-896cb2876e25@oracle.com> On 03/07/2018 11:04 AM, Thomas Schatzl wrote: > Hi, > > On Mon, 2018-03-05 at 16:02 +0100, Erik Helin wrote: >> Hi all, >> >> this fairly small patch re-unites G1Policy and G1DefaultPolicy, there >> is no need for having both classes. With this patch we only have >> G1Policy (and of course G1CollectorPolicy as the little >> CollectorPolicy shim). >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8197852 >> >> Patch: >> http://cr.openjdk.java.net/~ehelin/8197852/00/ >> >> Testing: >> - make run-test-tier1 on Linux, Windows, Mac, Solaris (SPARC and x86- >> 64) > > in g1Policy.hpp, if you update the copyright notice, please use the > correct year (2018) :) Oops, my bad, I will fix before I push :) > Is there a way to move g1DefaultPolicy.hpp into g1Policy.hpp while > retaining history? I.e. delete the latter first, and then rename? Yep, the commit does that, it is just not reflected in the webrev. The "trick" is to `hg mv g1DefaultPolicy.hpp g1Policy.hpp` and then make the changes, that keeps the history. Thanks, Erik > I do not need to re-review this change. > > Thanks, > Thomas > From thomas.schatzl at oracle.com Wed Mar 7 11:47:34 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 07 Mar 2018 12:47:34 +0100 Subject: RFR: 8197852: Move G1DefaultPolicy into G1Policy In-Reply-To: <5587e8c6-2a1c-4c23-dd2b-896cb2876e25@oracle.com> References: <1520417054.2346.19.camel@oracle.com> <5587e8c6-2a1c-4c23-dd2b-896cb2876e25@oracle.com> Message-ID: <1520423254.2346.34.camel@oracle.com> Hi, On Wed, 2018-03-07 at 11:39 +0100, Erik Helin wrote: > On 03/07/2018 11:04 AM, Thomas Schatzl wrote: > > Hi, > > > > On Mon, 2018-03-05 at 16:02 +0100, Erik Helin wrote: > > > Hi all, > > > > > > this fairly small patch re-unites G1Policy and G1DefaultPolicy, > > > there > > > is no need for having both classes. With this patch we only have > > > G1Policy (and of course G1CollectorPolicy as the little > > > CollectorPolicy shim). > > > > > > Issue: > > > https://bugs.openjdk.java.net/browse/JDK-8197852 > > > > > > Patch: > > > http://cr.openjdk.java.net/~ehelin/8197852/00/ > > > > > > Testing: > > > - make run-test-tier1 on Linux, Windows, Mac, Solaris (SPARC and > > > x86- > > > 64) > > > > in g1Policy.hpp, if you update the copyright notice, please use > > the correct year (2018) :) > > Oops, my bad, I will fix before I push :) > > > Is there a way to move g1DefaultPolicy.hpp into g1Policy.hpp while > > retaining history? I.e. delete the latter first, and then rename? > > Yep, the commit does that, it is just not reflected in the webrev. > The "trick" is to `hg mv g1DefaultPolicy.hpp g1Policy.hpp` and then > make the changes, that keeps the history. okay, thanks for the notice! Thomas From yasuenag at gmail.com Wed Mar 7 12:18:41 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Wed, 7 Mar 2018 21:18:41 +0900 Subject: PING: RFR: JDK-8153333: [REDO] STW phases at Concurrent GC should count in PerfCounter In-Reply-To: <9183dfd3-522f-fd73-a681-5ea957f1d717@gmail.com> References: <2f4e0901-1602-6276-e7fd-84e168e7b317@gmail.com> <3210723e-c5f9-4277-a97a-be61e10e6b3d@oracle.com> <89a1a161-874d-4743-e52b-7ab202fd8976@gmail.com> <9799a2ad-6727-cae9-312b-fd9de13c8203@oracle.com> <9183dfd3-522f-fd73-a681-5ea957f1d717@gmail.com> Message-ID: PING: Could you review it? http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.08/ JBS: https://bugs.openjdk.java.net/browse/JDK-8153333 CSR: https://bugs.openjdk.java.net/browse/JDK-8196862 This change has passed Mach5 on submit repo. Also it has passed hotspot/jtreg/:hotspot_serviceability and jdk/:jdk_tools jtreg tests. We need one more reviewer. Thanks, Yasumasa On 2018/02/21 21:14, Yasumasa Suenaga wrote: > PING: Could you review it? > >> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.07/ > > JBS: https://bugs.openjdk.java.net/browse/JDK-8153333 > CSR: https://bugs.openjdk.java.net/browse/JDK-8196862 > > > Yasumasa > > > On 2018/02/15 10:23, Yasumasa Suenaga wrote: >> Hi all, >> >> CSR for this issue [1] has been approved. >> This webrev has been reviewed by Stefan, but we need one more >> reviewer. Could you review it? >> >> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.07/ >> >> >> Thanks, >> >> Yasumasa >> >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8196862 >> >> >> >> 2018-02-06 22:33 GMT+09:00 Yasumasa Suenaga : >>> Hi Stefan, >>> >>>> This looks good to me, will do some more testing while waiting for a >>>> second reviewer and I can sponsor the change once it's ready to go. >>> >>> >>> Thanks! I'm waiting for second reviewer. >>> >>>>> What should I do to get CSR approve? >>>> >>>> In the bug system under "More" you can choose "Create CSR" which is the >>>> first step. More information can be found on the wiki: >>>> https://wiki.openjdk.java.net/display/csr/CSR+FAQs >>> >>> >>> I filed new CSR: >>> ?? https://bugs.openjdk.java.net/browse/JDK-8196862 >>> >>> >>> Yasumasa >>> >>> >>> >>> On 2018/02/06 21:55, Stefan Johansson wrote: >>>> >>>> >>>> >>>> On 2018-02-06 06:10, Yasumasa Suenaga wrote: >>>>> >>>>> Hi Stefan, >>>>> >>>>>> I agree, for G1 this should not be controlled. Maybe I was a bit >>>>>> unclear, I >>>>>> was wondering why we want to control it for CMS. >>>>> >>>>> I said to remove -XX:EnableConcGCPerfCounter in two years ago. I've >>>>> missed it :-) >>>>> >>>>> >>>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/017125.html >>>>> >>>>> So I uploaded new webrev. This change includes copyright year updates. >>>>> >>>>> ??? http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.06/ >>>> >>>> Thanks Yasumasa, >>>> >>>> This looks good to me, will do some more testing while waiting for a >>>> second reviewer and I can sponsor the change once it's ready to go. >>>>> >>>>> >>>>> This change passes all tests on submit repo, and >>>>> :hotspot_serviceability :jdk_tools tests on my laptop. >>>>> >>>>> >>>>> http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180206-0222-10428 >>>>> >>>>> >>>>>> If we do the change for CMS, we should >>>>>> probably also do a CSR, but that should be fairly straight forward. >>>>> >>>>> What should I do to get CSR approve? >>>> >>>> In the bug system under "More" you can choose "Create CSR" which is the >>>> first step. More information can be found on the wiki: >>>> https://wiki.openjdk.java.net/display/csr/CSR+FAQs >>>> >>>> Cheers, >>>> Stefan >>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> 2018-02-06 0:33 GMT+09:00 Stefan Johansson : >>>>>> >>>>>> >>>>>> On 2018-02-03 06:40, Yasumasa Suenaga wrote: >>>>>>> >>>>>>> On 2018/02/02 23:38, Stefan Johansson wrote: >>>>>>>> >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> The changes doesn't apply clean on the latest jdk/hs, can you provide >>>>>>>> an >>>>>>>> updated webrev? >>>>>>> >>>>>>> >>>>>>> I uploaded webrev for jdk-hs: >>>>>>> ??? cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.05/ >>>>>>> >>>>>> Thanks, I've kicked off a testing job now to verify nothing unexpected >>>>>> fails. >>>>>>> >>>>>>> >>>>>>>> The testing done by the submit repo doesn't cover the tests you have >>>>>>>> update so I plan to take the change for a spin and make sure the >>>>>>>> correct >>>>>>>> tests are run and verified in Mach 5. >>>>>>> >>>>>>> >>>>>>> I've also tested hotspot/jtreg/:hotspot_serviceability and >>>>>>> jdk/:jdk_tools >>>>>>> on my laptop. >>>>>>> I did not see any errors / failures which are related to this change. >>>>>> >>>>>> I also ran some local tests on this and it looks good. >>>>>>> >>>>>>> >>>>>>> >>>>>>>> Also a question about the change. Why do we need a special flag for >>>>>>>> CMS? >>>>>>>> I see that the original bug report refers to the flag as being a way >>>>>>>> to turn >>>>>>>> on and off the feature but the current implementation only consider >>>>>>>> the flag >>>>>>>> for CMS. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/016774.html >>>>>>> >>>>>>> Originally, STW phases (Remark and Cleanup) at G1 are not counted in >>>>>>> jstat >>>>>>> FGC column. >>>>>>> So I think we need not to control the behavior of PerfCounter for G1. >>>>>>> >>>>>> I agree, for G1 this should not be controlled. Maybe I was a bit >>>>>> unclear, I >>>>>> was wondering why we want to control it for CMS. I think either we >>>>>> should >>>>>> change the behavior without guarding it by a flag or just skip updating >>>>>> CMS >>>>>> (and leave the pauses in FGC). If we do the change for CMS, we should >>>>>> probably also do a CSR, but that should be fairly straight forward. >>>>>> >>>>>> I also found the old review thread where Jon M had the same comment >>>>>> (removing the flag) and it looks like all agreed on that: >>>>>> >>>>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/017118.html >>>>>> >>>>>> Thanks, >>>>>> Stefan >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>>> Thanks, >>>>>>>> Stefan >>>>>>>> >>>>>>>> On 2018-02-01 14:58, Yasumasa Suenaga wrote: >>>>>>>>> >>>>>>>>> PING: Could you review and sponsor it? >>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>>>>> >>>>>>>>> >>>>>>>>> This change has been passed Mach 5 via submit repo: >>>>>>>>> >>>>>>>>> >>>>>>>>> http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180201-0805-10101 >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2017/11/01 22:02, Yasumasa Suenaga wrote: >>>>>>>>>> >>>>>>>>>> PING: Could you review and sponsor it? >>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Also I need JPRT results of this change. >>>>>>>>>> Could you cooperate? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2017/09/27 0:08, Yasumasa Suenaga wrote: >>>>>>>>>>> >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> I uploaded new webrev to be adapted to jdk10/hs: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>>>>>>> >>>>>>>>>>> I want to check this patch via JPRT, but I cannot access it. >>>>>>>>>>> Could you cooperate? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2017/09/21 7:46, Yasumasa Suenaga wrote: >>>>>>>>>>>> >>>>>>>>>>>> PING: >>>>>>>>>>>> >>>>>>>>>>>> Have you checked this issue? >>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2017/07/01 23:44, Yasumasa Suenaga wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> PING: >>>>>>>>>>>>> >>>>>>>>>>>>> Have you checked this issue? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2017/06/14 13:22, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I changed PerfCounter to show CGC STW phase in jstat in >>>>>>>>>>>>>> JDK-8151674. >>>>>>>>>>>>>> However, it occurred several jtreg test failure, so it was >>>>>>>>>>>>>> back-outed. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I want to resume to work for this issue. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> These changes are work fine on jtreg test as below: >>>>>>>>>>>>>> >>>>>>>>>>>>>> ???? hotspot/test/serviceability/tmtools/jstat >>>>>>>>>>>>>> ???? jdk/test/sun/tools >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Since JDK 9, default GC algorithm is set to G1. >>>>>>>>>>>>>> So I think this change is useful to watch GC behavior through >>>>>>>>>>>>>> jstat. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I cannot access JPRT. Could you help? >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>> >>> From thomas.schatzl at oracle.com Thu Mar 8 08:59:16 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 08 Mar 2018 09:59:16 +0100 Subject: RFR (S): 8196485: FromCardCache default card index can cause crashes Message-ID: <1520499556.3121.8.camel@oracle.com> Hi all, can I have reviews for the following change that fixes a potential crash due to missing remembered set entries on huge (2TB+) heaps, or heaps that cross the 2TB address boundary (set via HeapBaseAddress)? The reason is that in the FromCardCache that is used to filter out repeated additions of the same card G1 uses 32 bits to represent that card, with a default value of 2^32-1. So if the first card we add to the remembered set has the index -1, G1 assumes that we already added that value and wrongly bails out. 2TB+ because 2^32 + 2^9 (card size) = 2^41 which is 2TB. Note that some assertions in that code already catch this issue. The fix is to extend the size of a FromCardCacheEntry to 64 bits, which allows 2^73 byte sized heaps - which hopefully is sufficient for some time :) On my machine the passing test takes 8s (in debug mode), so I added it to the regular tests run everytime. CR: https://bugs.openjdk.java.net/browse/JDK-8196485 Webrev: http://cr.openjdk.java.net/~tschatzl/8196485/webrev/ Testing: hs-tier1+2, new jtreg test asserting without the patch, running through with the patch. Thanks, Thomas From stefan.johansson at oracle.com Thu Mar 8 09:15:46 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 8 Mar 2018 10:15:46 +0100 Subject: RFR [L][4/7]: 8180415: Rebuild remembered sets during the concurrent cycle In-Reply-To: <1520413176.2346.9.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520262478.2532.91.camel@oracle.com> <1520413176.2346.9.camel@oracle.com> Message-ID: <1f345c49-7793-7925-2c15-281f24d28e74@oracle.com> Thanks for addressing my initial comments Thomas, On 2018-03-07 09:59, Thomas Schatzl wrote: > Hi all, > > Stefan had some comments about the HeapRegionType parameter in the > G1RemSetTrackingPolicy::update_at_allocate() method - that one is not > really required when placing the call to this method correctly and just > using the HeapRegion's type directly. > > Changing this removes another 40 LOC of changes. > > There has been another bug introduced by me during cleaning up for > final review: G1RemSetTrackingPolicy::update_at_free() is empty in this > change, which is wrong for this change - in this change we still do not > free the remembered sets during the cleanup pause, only in the > Concurrent Cleanup phase. When doing this concurrently, an assert > triggers when setting the remembered set state to empty outside a > safepoint. > The fix is to make the remembered set untracked during the Cleanup > phase still. > > New webrevs: > http://cr.openjdk.java.net/~tschatzl/8180415/webrev.0_to_1 (diff) > http://cr.openjdk.java.net/~tschatzl/8180415/webrev.1 (full) The change looks good. Just some additional comments: src/hotspot/share/gc/g1/g1ConcurrentMark.cpp 1255???? GCTraceTime(Debug, gc)("Complete Remembered Set Tracking"); ... 1291???? GCTraceTime(Debug, gc)("Finalize Concurrent Mark Cleanup"); Please add the "phases" tag to the above logging to be more consistent with the remark and other pauses. --- src/hotspot/share/gc/g1/g1FullGCOopClosures.inline.hpp 52 template inline void G1AdjustClosure::adjust_pointer(T* p) In this method the comments should be updated since we no longer return anything. --- src/hotspot/share/gc/g1/g1RemSet.cpp 757???????? HeapRegion* start_region = hr->humongous_start_region(); start_region is only used to get bottom and crate an oop, so maybe instead create the oop directly, like: oop humongous = oop(hr->humongous_start_region()->bottom()); ?761???????? // A humongous object is live (with respect to the scanning) either ?762???????? // a) it is marked on the bitmap as such ?763???????? // b) its TARS is larger than nTAMS, i.e. has been allocated during marking. ?764???????? if (_cm->next_mark_bitmap()->is_marked(start_region->bottom()) || ?765?????????? (top_at_rebuild_start > hr->next_top_at_mark_start())) { Break this check out to get something like: is_live(humongous, tars, ntams). There is also the marked_bytes counting in this method that makes the code a bit harder to follow. I've discussed this with Thomas offline and he has an idea of how to avoid this. --- Also in G1FullCollector::phase3_adjust_pointers() there is a GCTraceTime that needs updating, since we no longer rebuild remember sets. Thanks, Stefan > Thanks, > Thomas > > On Mon, 2018-03-05 at 16:07 +0100, Thomas Schatzl wrote: >> Hi all, >> >> can I have reviews for this change that implements the bulk of the >> "rebuild remembered sets concurrently" mechanism? >> >> This change modifies the management of remembered sets so that those >> are only maintained when they are useful. This roughly means: >> - always maintain remembered sets for young gen regions >> - (almost) always maintain remembered sets for humongous regions (for >> eager reclaim) as long as they meet the eager reclaim criteria. >> - never create remembered sets for archive regions >> - do not create remembered sets for any regions during full gc >> - do not create remembered sets for regions with evacuation failure >> >> (The latter two decisions may not be optimal, but the idea was to >> strictly only maintain remembered sets for regions we are going to >> try >> to evacuate soon) >> >> - create remembered sets for old gen regions after finishing marking >> (starting at the Remark pause), iff >> >> - the liveness of these regions indicate so (liveness <= >> G1MixedGCLiveThresholdPercent) >> - any (non-objArray) humongous region that does not have a >> remembered >> set yet (eg. because of full gc) >> >> (Please see the new G1RemSetTrackingPolicy implementation for exact >> details) >> >> During the following "Rebuild Remembered Set" phase, that replaces >> the >> "Create Live Data" phase, all non-young regions that may contain >> outgoing pointers (i.e. excluding closed archive regions), G1 scans >> the >> live objects in the regions between bottom and TARS >> ("top_at_rebuild_start" - yes, that's new :) for inter-region >> references and add them to the appropriate remembered set. >> >> The TARS, as the name indicates, is the top-value at the start of >> remark, and the regions contain live data between this value and >> bottom >> that needs to be scanned during this rebuild phase. >> >> The reason for a new indicator is that nTAMS is *not* sufficient: >> during marking there might have been more allocations in the old gen >> that also need to be scanned for new references. >> >> All references above TARS (and below the region's top) will be >> handled >> by the existing barrier mechanism. >> >> After rebuild, during the Cleanup pause, we create the new >> (candidate) >> collection set, eventually filtering out more regions, like humongous >> ones that exceed the eager reclaim criteria. >> >> The current state of the remembered set for a given region is tracked >> within the remembered set for that region (HeapRegionRememberedSet >> class): a remembered set can be either Untracked, Updating or >> Complete. >> Untracked means that we do not manage a remembered set for this >> region, >> Complete means just that, and Updating means that we are currently in >> the process of updating the remembered set. This distinction is >> relevant for evacuation, e.g. we obviously must not evacuate not- >> Complete remembered sets. >> >> Note that this change contains one single temporary ugly hack that >> allows G1 to work until the next change :) - so in this change G1 >> determines whether there will be a mixed gc phase during the Cleanup >> pause. It needs to determine this information at that time, instead >> of >> previously during concurrent cleanup, to, if there is no mixed gc >> coming, drop all remembered sets to not maintain them further. >> >> For this reason, and to avoid duplicate recalculation that may yield >> different results after dropping remembered sets, there is a new flag >> mixed_gc_pending in CollectorState, making it even more complicated. >> That one is going away in the next change, promised. :) >> >> Some imho useful messages about remembered set tracking are printed >> using "gc, remset, tracking" log tags, both in debug and trace level. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8180415 >> Webrev: >> http://cr.openjdk.java.net/~tschatzl/8180415/webrev/index.html >> Testing: >> hs-tier 1-5, etc. etc. >> >> Thanks, >> Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From rkennke at redhat.com Thu Mar 8 10:25:42 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 8 Mar 2018 11:25:42 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: <5A9EBD6E.3070005@oracle.com> References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> <5A9E8F76.1050809@oracle.com> <83de65bd-5d3f-78df-02d2-eb4c53aa0c62@oracle.com> <5A9E95D8.6080602@oracle.com> <2cc9a407-aba8-209a-7ef7-4ab26b88fef6@redhat.com> <6d8435af-5bec-da70-fa8a-60d708937f44@redhat.com> <5A9EBD6E.3070005@oracle.com> Message-ID: <2612309c-868b-6094-5944-e9de54daf8d2@redhat.com> Wouldn't that lead to mis-aligned accesses when you try to copy an array of, e.g., bytes, but shoehorn it to pointer-size? Or am I misunderstanding? An idea would be to copy equal-sized elements using the corresponding Copy::conjoint_XXX_atomic() calls, e.g. char and short would use the short version, boolean and byte would use the byte version (or is boolean sometimes int?!), float and int would use the int version and long and double the long version. WDYT? Roman > My spontaneous idea was to have an overload for type erased void* that > when called with ARRAYCOPY_ATOMIC maps to Copy::conjoint_memory_atomic > which AFAIK is conservatively atomic regardless of what the precise > element type is. > > But if you have better ideas, I am open for suggestions. > > Thanks, > /Erik > > On 2018-03-06 17:03, Roman Kennke wrote: >> Am 06.03.2018 um 14:35 schrieb Roman Kennke: >>> Makes me wonder: why attempt to be smart in c1_Runtime1.cpp, when we can >>> just as well call typeArrayOop::copy_array() and have it do the right >>> thing? Or go even further and also do it for oop-arraycopy? >> Something like: >> >> http://cr.openjdk.java.net/~rkennke/8198445/8198445-1.patch >> >> This wouldn't compile because of bunch of missing >> arraycopy_conjoint_atomic defintions for extra types like jfloat, >> jdouble, jboolean, etc, which in turn would be missing the same >> Copy::conjoint_jFluffs_atomic() which drags in a bunch of platform >> specific stuff... and my question before I go there is: do we want all >> that? Or can you think of a better way to solve it? >> >> Roman >> >>> Roman >>> >>>> The ARRAYCOPY_ATOMIC decorator makes the arraycopy atomic over the size >>>> of the passed in elements. >>>> In this case, it looks like the address has been type erased to void*, >>>> and hence lost what the element size was. There is currently no >>>> overload >>>> accepted for type erased element - only accurate elements {jbyte, >>>> jshort, jint, jlong}. >>>> >>>> So it looks like an overload must be added to accept type erased void* >>>> elements and make that call conjoint_memory_atomic when the >>>> ARRAYCOPY_ATOMIC decorator is passed in. >>>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2018-03-06 13:56, David Holmes wrote: >>>>> On 6/03/2018 10:54 PM, Erik ?sterlund wrote: >>>>>> Hi David, >>>>>> >>>>>> It is atomic with the ARRAYCOPY_ATOMIC decorator. If that comment is >>>>>> correct (I do not know if it is), then the ARRAYCOPY_ATOMIC decorator >>>>>> should probably be used here. >>>>> If that code implements a Java array copy then yes it is required to >>>>> be 32-bit atomic. Do you need the decorator to get 32-bit atomicity? >>>>> >>>>> David >>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>>>> On 2018-03-06 13:48, David Holmes wrote: >>>>>>> Hi Roman, >>>>>>> >>>>>>> Not a review as I'm not familiar enough with the Access API, but in >>>>>>> src/hotspot/share/c1/c1_Runtime1.cpp the comments above the changed >>>>>>> code need updating - probably deleting. I assume the Access API >>>>>>> arraycopy is atomic? >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>> On 6/03/2018 9:56 PM, Roman Kennke wrote: >>>>>>>> Currently, the Access API is only used for oop-arraycopy, but >>>>>>>> not for >>>>>>>> primitive arraycopy. GCs might want to intercept this too, e.g. >>>>>>>> resolve >>>>>>>> src and dst arrays. >>>>>>>> >>>>>>>> There *is* an implementation of primitive arraycopy in the >>>>>>>> Access API, >>>>>>>> but it doesn't even compile, because Raw::arraycopy() does not take >>>>>>>> src >>>>>>>> and dst oop operands, but it's called like that. The only reason >>>>>>>> why >>>>>>>> this does not blow up (I think) is that because nobody calls it, >>>>>>>> the >>>>>>>> compiler doesn't even get there. >>>>>>>> >>>>>>>> This change fixes the Access API/impl and adds the relevant >>>>>>>> calls into >>>>>>>> it (in C1 and runtime land). C2 uses arraycopy stubs (which >>>>>>>> cannot be >>>>>>>> handled here) or calls out to the ArrayKlass::copy_array(), which >>>>>>>> should >>>>>>>> be covered with this change. >>>>>>>> >>>>>>>> It should be possible to use the same Access API for Java-array <-> >>>>>>>> native-array bulk transfers, which currently use the rather ugly >>>>>>>> typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address that in a >>>>>>>> separate change though. >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ >>>>>>>> >>>>>>>> Tests: tier1 ok >>>>>>>> >>>>>>>> Please review! >>>>>>>> Thanks, Roman >>>>>>>> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Thu Mar 8 11:49:20 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 08 Mar 2018 12:49:20 +0100 Subject: RFR [L][4/7]: 8180415: Rebuild remembered sets during the concurrent cycle In-Reply-To: <1f345c49-7793-7925-2c15-281f24d28e74@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520262478.2532.91.camel@oracle.com> <1520413176.2346.9.camel@oracle.com> <1f345c49-7793-7925-2c15-281f24d28e74@oracle.com> Message-ID: <1520509760.3121.15.camel@oracle.com> Hi Stefan, On Thu, 2018-03-08 at 10:15 +0100, Stefan Johansson wrote: > Thanks for addressing my initial comments Thomas, > > On 2018-03-07 09:59, Thomas Schatzl wrote: > > Hi all, > > > > Stefan had some comments about the HeapRegionType parameter in > > the G1RemSetTrackingPolicy::update_at_allocate() method - that one > > is not really required when placing the call to this method > > correctly and just using the HeapRegion's type directly. > > > > Changing this removes another 40 LOC of changes. > > > > There has been another bug introduced by me during cleaning up for > > final review: G1RemSetTrackingPolicy::update_at_free() is empty in > > this change, which is wrong for this change - in this change we > > still do not free the remembered sets during the cleanup pause, > > only in the Concurrent Cleanup phase. When doing this concurrently, > > an assert triggers when setting the remembered set state to empty > > outside a safepoint. > > The fix is to make the remembered set untracked during the Cleanup > > phase still. > > > > New webrevs: > > http://cr.openjdk.java.net/~tschatzl/8180415/webrev.0_to_1 (diff) > > http://cr.openjdk.java.net/~tschatzl/8180415/webrev.1 (full) > The change looks good. Just some additional comments: > src/hotspot/share/gc/g1/g1ConcurrentMark.cpp > 1255 GCTraceTime(Debug, gc)("Complete Remembered Set Tracking"); > ... > 1291 GCTraceTime(Debug, gc)("Finalize Concurrent Mark Cleanup"); > > Please add the "phases" tag to the above logging to be more > consistent with the remark and other pauses. > --- > src/hotspot/share/gc/g1/g1FullGCOopClosures.inline.hpp > 52 template inline void G1AdjustClosure::adjust_pointer(T* > p) > > In this method the comments should be updated since we no longer > return anything. > --- > src/hotspot/share/gc/g1/g1RemSet.cpp > 757 HeapRegion* start_region = hr->humongous_start_region(); > > start_region is only used to get bottom and crate an oop, so maybe > instead create the oop directly, like: > oop humongous = oop(hr->humongous_start_region()->bottom()); > > 761 // A humongous object is live (with respect to the > scanning) either > 762 // a) it is marked on the bitmap as such > 763 // b) its TARS is larger than nTAMS, i.e. has been > allocated during marking. > 764 if (_cm->next_mark_bitmap()->is_marked(start_region- > >bottom()) || > 765 (top_at_rebuild_start > hr->next_top_at_mark_start())) > { > > Break this check out to get something like: is_live(humongous, tars, > ntams). Fixed. > There is also the marked_bytes counting in this method that makes the > code a bit harder to follow. I've discussed this with Thomas offline > and he has an idea of how to avoid this. As discussed I would like to postpone this for now as it is much harder than it looks and warrants its own CR (actually two or three). > Also in G1FullCollector::phase3_adjust_pointers() there is a > GCTraceTime that needs updating, since we no longer rebuild remember > sets. All done. New webrevs: http://cr.openjdk.java.net/~tschatzl/8180415/webrev.1_to_2/ (diff) http://cr.openjdk.java.net/~tschatzl/8180415/webrev.2/ (full) Thomas From thomas.schatzl at oracle.com Thu Mar 8 11:59:44 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 08 Mar 2018 12:59:44 +0100 Subject: RFR [S][5/7]: 8197573: Remove concurrent cleanup and secondary free list handling In-Reply-To: <1520262995.2532.99.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520262995.2532.99.camel@oracle.com> Message-ID: <1520510384.3121.17.camel@oracle.com> Hi, On Mon, 2018-03-05 at 16:16 +0100, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that removes the concurrent > cleanup phase and the secondary free list handling entirely. It moves > leftover work into the Cleanup pause. > > In the previous change the remembered set scrubbing, which took in > cases where it mattered like 95%+ of time, has been removed because > it > is not required any more. > > The reasons for removal are: > - due to JDK-8180415 *freeing* remembered sets (as opposed to > calculating occupancy) is really fast, so this phase has never been > very long for a long time. > - it adds a lock in the LAB allocation path > > It also allows us to remove the "mixed_gc_pending" flag in the > CollectorState, as there is no further time to wait until G1 can > schedule the "last young gc". > > CR: > https://bugs.openjdk.java.net/browse/JDK-8197573 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8197573/webrev/index.html > Testing: > hs-tier 1-3, .... > > (I consider this an "S" changeset because it is mostly removal of > code). > Some ripple-through after latest changes to JDK-8180415 requires an update of the webrevs: http://cr.openjdk.java.net/~tschatzl/8197573/webrev.0_to_1 (diff) http://cr.openjdk.java.net/~tschatzl/8197573/webrev.1 (full) I also found some more opportunity to do minor cleanup after Stefan's last remark about log messages... Thanks, Thomas From rkennke at redhat.com Thu Mar 8 12:27:31 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 8 Mar 2018 13:27:31 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: <5A9EBD6E.3070005@oracle.com> References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> <5A9E8F76.1050809@oracle.com> <83de65bd-5d3f-78df-02d2-eb4c53aa0c62@oracle.com> <5A9E95D8.6080602@oracle.com> <2cc9a407-aba8-209a-7ef7-4ab26b88fef6@redhat.com> <6d8435af-5bec-da70-fa8a-60d708937f44@redhat.com> <5A9EBD6E.3070005@oracle.com> Message-ID: <1c75416f-d27e-c7a2-537d-743a77407812@redhat.com> The following changeset addresses all the comments that have been brought up so far in review. It also makes the arraycopy word-atomic. It casts pointers to elements of same size to their respective counterparts in Copy::conjoint_XXX_atomic(), i.e. jboolean* -> jbyte*, jchar* -> jshort*, jfloat* -> jint* and jdouble* -> jlong*. Differential: http://cr.openjdk.java.net/~rkennke/8198445/webrev.01.diff/ Full: http://cr.openjdk.java.net/~rkennke/8198445/webrev.01/ Still passes tier1 tests. OK to go? Roman > Hi Roman, > > My spontaneous idea was to have an overload for type erased void* that > when called with ARRAYCOPY_ATOMIC maps to Copy::conjoint_memory_atomic > which AFAIK is conservatively atomic regardless of what the precise > element type is. > > But if you have better ideas, I am open for suggestions. > > Thanks, > /Erik > > On 2018-03-06 17:03, Roman Kennke wrote: >> Am 06.03.2018 um 14:35 schrieb Roman Kennke: >>> Makes me wonder: why attempt to be smart in c1_Runtime1.cpp, when we can >>> just as well call typeArrayOop::copy_array() and have it do the right >>> thing? Or go even further and also do it for oop-arraycopy? >> Something like: >> >> http://cr.openjdk.java.net/~rkennke/8198445/8198445-1.patch >> >> This wouldn't compile because of bunch of missing >> arraycopy_conjoint_atomic defintions for extra types like jfloat, >> jdouble, jboolean, etc, which in turn would be missing the same >> Copy::conjoint_jFluffs_atomic() which drags in a bunch of platform >> specific stuff... and my question before I go there is: do we want all >> that? Or can you think of a better way to solve it? >> >> Roman >> >>> Roman >>> >>>> The ARRAYCOPY_ATOMIC decorator makes the arraycopy atomic over the size >>>> of the passed in elements. >>>> In this case, it looks like the address has been type erased to void*, >>>> and hence lost what the element size was. There is currently no >>>> overload >>>> accepted for type erased element - only accurate elements {jbyte, >>>> jshort, jint, jlong}. >>>> >>>> So it looks like an overload must be added to accept type erased void* >>>> elements and make that call conjoint_memory_atomic when the >>>> ARRAYCOPY_ATOMIC decorator is passed in. >>>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2018-03-06 13:56, David Holmes wrote: >>>>> On 6/03/2018 10:54 PM, Erik ?sterlund wrote: >>>>>> Hi David, >>>>>> >>>>>> It is atomic with the ARRAYCOPY_ATOMIC decorator. If that comment is >>>>>> correct (I do not know if it is), then the ARRAYCOPY_ATOMIC decorator >>>>>> should probably be used here. >>>>> If that code implements a Java array copy then yes it is required to >>>>> be 32-bit atomic. Do you need the decorator to get 32-bit atomicity? >>>>> >>>>> David >>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>>>> On 2018-03-06 13:48, David Holmes wrote: >>>>>>> Hi Roman, >>>>>>> >>>>>>> Not a review as I'm not familiar enough with the Access API, but in >>>>>>> src/hotspot/share/c1/c1_Runtime1.cpp the comments above the changed >>>>>>> code need updating - probably deleting. I assume the Access API >>>>>>> arraycopy is atomic? >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>> On 6/03/2018 9:56 PM, Roman Kennke wrote: >>>>>>>> Currently, the Access API is only used for oop-arraycopy, but >>>>>>>> not for >>>>>>>> primitive arraycopy. GCs might want to intercept this too, e.g. >>>>>>>> resolve >>>>>>>> src and dst arrays. >>>>>>>> >>>>>>>> There *is* an implementation of primitive arraycopy in the >>>>>>>> Access API, >>>>>>>> but it doesn't even compile, because Raw::arraycopy() does not take >>>>>>>> src >>>>>>>> and dst oop operands, but it's called like that. The only reason >>>>>>>> why >>>>>>>> this does not blow up (I think) is that because nobody calls it, >>>>>>>> the >>>>>>>> compiler doesn't even get there. >>>>>>>> >>>>>>>> This change fixes the Access API/impl and adds the relevant >>>>>>>> calls into >>>>>>>> it (in C1 and runtime land). C2 uses arraycopy stubs (which >>>>>>>> cannot be >>>>>>>> handled here) or calls out to the ArrayKlass::copy_array(), which >>>>>>>> should >>>>>>>> be covered with this change. >>>>>>>> >>>>>>>> It should be possible to use the same Access API for Java-array <-> >>>>>>>> native-array bulk transfers, which currently use the rather ugly >>>>>>>> typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address that in a >>>>>>>> separate change though. >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ >>>>>>>> >>>>>>>> Tests: tier1 ok >>>>>>>> >>>>>>>> Please review! >>>>>>>> Thanks, Roman >>>>>>>> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From stefan.johansson at oracle.com Thu Mar 8 12:28:56 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 8 Mar 2018 13:28:56 +0100 Subject: RFR [S][5/7]: 8197573: Remove concurrent cleanup and secondary free list handling In-Reply-To: <1520510384.3121.17.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520262995.2532.99.camel@oracle.com> <1520510384.3121.17.camel@oracle.com> Message-ID: On 2018-03-08 12:59, Thomas Schatzl wrote: > Hi, > > On Mon, 2018-03-05 at 16:16 +0100, Thomas Schatzl wrote: >> Hi all, >> >> can I have reviews for this change that removes the concurrent >> cleanup phase and the secondary free list handling entirely. It moves >> leftover work into the Cleanup pause. >> >> In the previous change the remembered set scrubbing, which took in >> cases where it mattered like 95%+ of time, has been removed because >> it >> is not required any more. >> >> The reasons for removal are: >> - due to JDK-8180415 *freeing* remembered sets (as opposed to >> calculating occupancy) is really fast, so this phase has never been >> very long for a long time. >> - it adds a lock in the LAB allocation path >> >> It also allows us to remove the "mixed_gc_pending" flag in the >> CollectorState, as there is no further time to wait until G1 can >> schedule the "last young gc". >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8197573 >> Webrev: >> http://cr.openjdk.java.net/~tschatzl/8197573/webrev/index.html >> Testing: >> hs-tier 1-3, .... >> >> (I consider this an "S" changeset because it is mostly removal of >> code). >> > Some ripple-through after latest changes to JDK-8180415 requires an > update of the webrevs: > http://cr.openjdk.java.net/~tschatzl/8197573/webrev.0_to_1 (diff) > http://cr.openjdk.java.net/~tschatzl/8197573/webrev.1 (full) Looks lovely, just some minor things: src/hotspot/share/gc/g1/concurrentMarkThread.cpp 436?????????? log_info(gc, marking)("Concurrent Mark Abort"); This line was removed, and it looks like will never log a concurrent mark abort now. Looks like this could be added to G1ConcurrentMark::concurrent_cycle_end() where we trace a potential cm failure. --- src/hotspot/share/gc/g1/g1FullGCPrepareTask.cpp 106?? _g1h->free_humongous_region(hr, &dummy_free_list); Should be safe to call free_humongous_region with locked = true here, since this will be the only thread working on that region. 115?? _g1h->card_table()->clear(MemRegion(hr->bottom(), hr->end())); Pre-existing, but since you have added the clear_cardtable() function we could use it here as well, seems to be the only place where the old "scheme" is used. --- > I also found some more opportunity to do minor cleanup after Stefan's > last remark about log messages... Nice, and you also restructured the code exactly like I planned to comment :) Thanks, Stefan > Thanks, > Thomas > From stefan.johansson at oracle.com Thu Mar 8 12:52:56 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 8 Mar 2018 13:52:56 +0100 Subject: RFR [XS][6/7]: 8197928: Only enqueue deferred cards with references into regions that have a tracked remembered set during GC In-Reply-To: <1520263515.2532.106.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520263515.2532.106.camel@oracle.com> Message-ID: Hi Thomas, On 2018-03-05 16:25, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this tiny change that optimizes evacuation a > little: in particular, when updating the remembered sets (enqueuing > cards), we do not need to do this for regions where we do not track the > remembered sets. > > The suggested implementation is rather primitive, I have not noticed > pause time impact on doing so (of course it can lessen the concurrent > work later). > > An alternative implementation I considered has been to create a per- > region map with a single flag that contains the "needs-update/does-not- > need-update" information, like the _fast_in_cset_state table. > > However creating such a table requires G1 to iterate over all regions > before evacuation, and that would be something that needs to be added > as G1 does not do that yet. > > Another would be to move remembered set state/remembered sets outside > of the region, so it can be accessed quickly during GC. > > In any case I thought this implementation is better than not doing it, > hoping for some input here. :) I agree, let's start out with this approach and we can re-visit this later if needed. Reviewed, Stefan > CR: > https://bugs.openjdk.java.net/browse/JDK-8197928 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8197928/webrev/ > Testing: > hs-tier1-5, ... > > Thanks, > Thomas > From erik.osterlund at oracle.com Thu Mar 8 13:13:13 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 8 Mar 2018 14:13:13 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: <2612309c-868b-6094-5944-e9de54daf8d2@redhat.com> References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> <5A9E8F76.1050809@oracle.com> <83de65bd-5d3f-78df-02d2-eb4c53aa0c62@oracle.com> <5A9E95D8.6080602@oracle.com> <2cc9a407-aba8-209a-7ef7-4ab26b88fef6@redhat.com> <6d8435af-5bec-da70-fa8a-60d708937f44@redhat.com> <5A9EBD6E.3070005@oracle.com> <2612309c-868b-6094-5944-e9de54daf8d2@redhat.com> Message-ID: <5AA136E9.9080100@oracle.com> Hi Roman, On 2018-03-08 11:25, Roman Kennke wrote: > Wouldn't that lead to mis-aligned accesses when you try to copy an array > of, e.g., bytes, but shoehorn it to pointer-size? Or am I misunderstanding? No it would not. The element type is void, not void*. Because the element type is type erased. And that would map to Copy::conjoint_memory_atomic, which is precisely what typeArrayKlass.cpp maps such an arraycopy to today. Assuming elements are aligned to their natural size (which they are), Copy::conjoint_memory_atomic is guaranteed to be at least as atomic as you need your elements to be, up to an element size of 8. > An idea would be to copy equal-sized elements using the corresponding > Copy::conjoint_XXX_atomic() calls, e.g. char and short would use the > short version, boolean and byte would use the byte version (or is > boolean sometimes int?!), float and int would use the int version and > long and double the long version. WDYT? I'm not sure I see what the advantage of that would be. Thanks, /Erik > Roman > > >> My spontaneous idea was to have an overload for type erased void* that >> when called with ARRAYCOPY_ATOMIC maps to Copy::conjoint_memory_atomic >> which AFAIK is conservatively atomic regardless of what the precise >> element type is. >> >> But if you have better ideas, I am open for suggestions. >> >> Thanks, >> /Erik >> >> On 2018-03-06 17:03, Roman Kennke wrote: >>> Am 06.03.2018 um 14:35 schrieb Roman Kennke: >>>> Makes me wonder: why attempt to be smart in c1_Runtime1.cpp, when we can >>>> just as well call typeArrayOop::copy_array() and have it do the right >>>> thing? Or go even further and also do it for oop-arraycopy? >>> Something like: >>> >>> http://cr.openjdk.java.net/~rkennke/8198445/8198445-1.patch >>> >>> This wouldn't compile because of bunch of missing >>> arraycopy_conjoint_atomic defintions for extra types like jfloat, >>> jdouble, jboolean, etc, which in turn would be missing the same >>> Copy::conjoint_jFluffs_atomic() which drags in a bunch of platform >>> specific stuff... and my question before I go there is: do we want all >>> that? Or can you think of a better way to solve it? >>> >>> Roman >>> >>>> Roman >>>> >>>>> The ARRAYCOPY_ATOMIC decorator makes the arraycopy atomic over the size >>>>> of the passed in elements. >>>>> In this case, it looks like the address has been type erased to void*, >>>>> and hence lost what the element size was. There is currently no >>>>> overload >>>>> accepted for type erased element - only accurate elements {jbyte, >>>>> jshort, jint, jlong}. >>>>> >>>>> So it looks like an overload must be added to accept type erased void* >>>>> elements and make that call conjoint_memory_atomic when the >>>>> ARRAYCOPY_ATOMIC decorator is passed in. >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>> On 2018-03-06 13:56, David Holmes wrote: >>>>>> On 6/03/2018 10:54 PM, Erik ?sterlund wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> It is atomic with the ARRAYCOPY_ATOMIC decorator. If that comment is >>>>>>> correct (I do not know if it is), then the ARRAYCOPY_ATOMIC decorator >>>>>>> should probably be used here. >>>>>> If that code implements a Java array copy then yes it is required to >>>>>> be 32-bit atomic. Do you need the decorator to get 32-bit atomicity? >>>>>> >>>>>> David >>>>>> >>>>>>> Thanks, >>>>>>> /Erik >>>>>>> >>>>>>> On 2018-03-06 13:48, David Holmes wrote: >>>>>>>> Hi Roman, >>>>>>>> >>>>>>>> Not a review as I'm not familiar enough with the Access API, but in >>>>>>>> src/hotspot/share/c1/c1_Runtime1.cpp the comments above the changed >>>>>>>> code need updating - probably deleting. I assume the Access API >>>>>>>> arraycopy is atomic? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>> On 6/03/2018 9:56 PM, Roman Kennke wrote: >>>>>>>>> Currently, the Access API is only used for oop-arraycopy, but >>>>>>>>> not for >>>>>>>>> primitive arraycopy. GCs might want to intercept this too, e.g. >>>>>>>>> resolve >>>>>>>>> src and dst arrays. >>>>>>>>> >>>>>>>>> There *is* an implementation of primitive arraycopy in the >>>>>>>>> Access API, >>>>>>>>> but it doesn't even compile, because Raw::arraycopy() does not take >>>>>>>>> src >>>>>>>>> and dst oop operands, but it's called like that. The only reason >>>>>>>>> why >>>>>>>>> this does not blow up (I think) is that because nobody calls it, >>>>>>>>> the >>>>>>>>> compiler doesn't even get there. >>>>>>>>> >>>>>>>>> This change fixes the Access API/impl and adds the relevant >>>>>>>>> calls into >>>>>>>>> it (in C1 and runtime land). C2 uses arraycopy stubs (which >>>>>>>>> cannot be >>>>>>>>> handled here) or calls out to the ArrayKlass::copy_array(), which >>>>>>>>> should >>>>>>>>> be covered with this change. >>>>>>>>> >>>>>>>>> It should be possible to use the same Access API for Java-array <-> >>>>>>>>> native-array bulk transfers, which currently use the rather ugly >>>>>>>>> typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address that in a >>>>>>>>> separate change though. >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ >>>>>>>>> >>>>>>>>> Tests: tier1 ok >>>>>>>>> >>>>>>>>> Please review! >>>>>>>>> Thanks, Roman >>>>>>>>> > From thomas.schatzl at oracle.com Thu Mar 8 13:31:52 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 08 Mar 2018 14:31:52 +0100 Subject: RFR [S][5/7]: 8197573: Remove concurrent cleanup and secondary free list handling In-Reply-To: References: <1520258985.2532.40.camel@oracle.com> <1520262995.2532.99.camel@oracle.com> <1520510384.3121.17.camel@oracle.com> Message-ID: <1520515912.3121.23.camel@oracle.com> Hi, On Thu, 2018-03-08 at 13:28 +0100, Stefan Johansson wrote: > On 2018-03-08 12:59, Thomas Schatzl wrote: > > Hi, > > > > On Mon, 2018-03-05 at 16:16 +0100, Thomas Schatzl wrote: > > > Hi all, > > > > > > can I have reviews for this change that removes the concurrent > > > cleanup phase and the secondary free list handling entirely. It > > > moves leftover work into the Cleanup pause. > > > > > > In the previous change the remembered set scrubbing, which took > > > in cases where it mattered like 95%+ of time, has been removed > > > because it is not required any more. > > > > > > The reasons for removal are: > > > - due to JDK-8180415 *freeing* remembered sets (as opposed to > > > calculating occupancy) is really fast, so this phase has never > > > been very long for a long time. > > > - it adds a lock in the LAB allocation path > > > > > > It also allows us to remove the "mixed_gc_pending" flag in the > > > CollectorState, as there is no further time to wait until G1 can > > > schedule the "last young gc". > > > > > > CR: > > > https://bugs.openjdk.java.net/browse/JDK-8197573 > > > Webrev: > > > http://cr.openjdk.java.net/~tschatzl/8197573/webrev/index.html > > > Testing: > > > hs-tier 1-3, .... > > > > > > (I consider this an "S" changeset because it is mostly removal of > > > code). > > > > > > > Some ripple-through after latest changes to JDK-8180415 requires an > > update of the webrevs: > > http://cr.openjdk.java.net/~tschatzl/8197573/webrev.0_to_1 (diff) > > http://cr.openjdk.java.net/~tschatzl/8197573/webrev.1 (full) > > Looks lovely, just some minor things: > src/hotspot/share/gc/g1/concurrentMarkThread.cpp > 436 log_info(gc, marking)("Concurrent Mark Abort"); > > This line was removed, and it looks like will never log a concurrent > mark abort now. Looks like this could be added to > G1ConcurrentMark::concurrent_cycle_end() where we trace a potential > cm failure. Nice catch! > --- > src/hotspot/share/gc/g1/g1FullGCPrepareTask.cpp > 106 _g1h->free_humongous_region(hr, &dummy_free_list); > > Should be safe to call free_humongous_region with locked = true > here, since this will be the only thread working on that region. Fixed. > > 115 _g1h->card_table()->clear(MemRegion(hr->bottom(), hr->end())); > > Pre-existing, but since you have added the clear_cardtable() function > we could use it here as well, seems to be the only place where the > old "scheme" is used. Done. > --- > > > I also found some more opportunity to do minor cleanup after > > Stefan's > > last remark about log messages... > > Nice, and you also restructured the code exactly like I planned to > comment :) :) New webrevs: http://cr.openjdk.java.net/~tschatzl/8197573/webrev.1_to_2/ (diff) http://cr.openjdk.java.net/~tschatzl/8197573/webrev.2/ (full) Thanks, Thomas From thomas.schatzl at oracle.com Thu Mar 8 13:33:33 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 08 Mar 2018 14:33:33 +0100 Subject: RFR (S): 8199326: Remove G1 gc time stamp logic Message-ID: <1520516013.3121.24.camel@oracle.com> Hi, with the rebuild remsets concurrently changes the last user of the gc time stamps (create live data) has gone away. This change removes all of the corresponding code I could find. Could you review it? CR: https://bugs.openjdk.java.net/browse/JDK-8199326 Webrev: http://cr.openjdk.java.net/~tschatzl/8197932/webrev/ Testing: hs-tier1 Thanks, Thomas From stefan.johansson at oracle.com Thu Mar 8 15:18:19 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 8 Mar 2018 16:18:19 +0100 Subject: RFR (S): 8199326: Remove G1 gc time stamp logic In-Reply-To: <1520516013.3121.24.camel@oracle.com> References: <1520516013.3121.24.camel@oracle.com> Message-ID: <7b19df2a-cac8-41d7-c3e9-d2cf893f9d85@oracle.com> Hi Thomas, On 2018-03-08 14:33, Thomas Schatzl wrote: > Hi, > > with the rebuild remsets concurrently changes the last user of the gc > time stamps (create live data) has gone away. > > This change removes all of the corresponding code I could find. > > Could you review it? > > CR: > https://bugs.openjdk.java.net/browse/JDK-8199326 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8197932/webrev/ Looks like you pasted the wrong link, and I couldn't find the correct one either. Stefan > Testing: > hs-tier1 > > Thanks, > Thomas > From thomas.schatzl at oracle.com Thu Mar 8 15:46:20 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 08 Mar 2018 16:46:20 +0100 Subject: RFR (S): 8199326: Remove G1 gc time stamp logic In-Reply-To: <7b19df2a-cac8-41d7-c3e9-d2cf893f9d85@oracle.com> References: <1520516013.3121.24.camel@oracle.com> <7b19df2a-cac8-41d7-c3e9-d2cf893f9d85@oracle.com> Message-ID: <1520523980.3121.29.camel@oracle.com> On Thu, 2018-03-08 at 16:18 +0100, Stefan Johansson wrote: > Hi Thomas, > > On 2018-03-08 14:33, Thomas Schatzl wrote: > > Hi, > > > > with the rebuild remsets concurrently changes the last user of > > the gc > > time stamps (create live data) has gone away. > > > > This change removes all of the corresponding code I could find. > > > > Could you review it? > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8199326 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8197932/webrev/ > > Looks like you pasted the wrong link, and I couldn't find the > correct > one either. Sorry. The correct link is: http://cr.openjdk.java.net/~tschatzl/8199326/webrev Thomas From rkennke at redhat.com Thu Mar 8 20:12:16 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 8 Mar 2018 21:12:16 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: <5AA136E9.9080100@oracle.com> References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> <5A9E8F76.1050809@oracle.com> <83de65bd-5d3f-78df-02d2-eb4c53aa0c62@oracle.com> <5A9E95D8.6080602@oracle.com> <2cc9a407-aba8-209a-7ef7-4ab26b88fef6@redhat.com> <6d8435af-5bec-da70-fa8a-60d708937f44@redhat.com> <5A9EBD6E.3070005@oracle.com> <2612309c-868b-6094-5944-e9de54daf8d2@redhat.com> <5AA136E9.9080100@oracle.com> Message-ID: <3034e0b7-9894-80e6-afd4-171e31ad5836@redhat.com> Am 08.03.2018 um 14:13 schrieb Erik ?sterlund: > Hi Roman, > > On 2018-03-08 11:25, Roman Kennke wrote: >> Wouldn't that lead to mis-aligned accesses when you try to copy an array >> of, e.g., bytes, but shoehorn it to pointer-size? Or am I >> misunderstanding? > > No it would not. The element type is void, not void*. Because the > element type is type erased. And that would map to > Copy::conjoint_memory_atomic, which is precisely what typeArrayKlass.cpp > maps such an arraycopy to today. Assuming elements are aligned to their > natural size (which they are), Copy::conjoint_memory_atomic is > guaranteed to be at least as atomic as you need your elements to be, up > to an element size of 8. Ah, ok. Alright, so here comes rev.02: Differential: http://cr.openjdk.java.net/~rkennke/8198445/webrev.02.diff/ Full: http://cr.openjdk.java.net/~rkennke/8198445/webrev.02/ Still passes tier1 tests. Ok now? Cheers, Roman > >> An idea would be to copy equal-sized elements using the corresponding >> Copy::conjoint_XXX_atomic() calls, e.g. char and short would use the >> short version, boolean and byte would use the byte version (or is >> boolean sometimes int?!), float and int would use the int version and >> long and double the long version. WDYT? > > I'm not sure I see what the advantage of that would be. > > Thanks, > /Erik > >> Roman >> >> >>> My spontaneous idea was to have an overload for type erased void* that >>> when called with ARRAYCOPY_ATOMIC maps to Copy::conjoint_memory_atomic >>> which AFAIK is conservatively atomic regardless of what the precise >>> element type is. >>> >>> But if you have better ideas, I am open for suggestions. >>> >>> Thanks, >>> /Erik >>> >>> On 2018-03-06 17:03, Roman Kennke wrote: >>>> Am 06.03.2018 um 14:35 schrieb Roman Kennke: >>>>> Makes me wonder: why attempt to be smart in c1_Runtime1.cpp, when >>>>> we can >>>>> just as well call typeArrayOop::copy_array() and have it do the right >>>>> thing? Or go even further and also do it for oop-arraycopy? >>>> Something like: >>>> >>>> http://cr.openjdk.java.net/~rkennke/8198445/8198445-1.patch >>>> >>>> This wouldn't compile because of bunch of missing >>>> arraycopy_conjoint_atomic defintions for extra types like jfloat, >>>> jdouble, jboolean, etc, which in turn would be missing the same >>>> Copy::conjoint_jFluffs_atomic() which drags in a bunch of platform >>>> specific stuff... and my question before I go there is: do we want all >>>> that? Or can you think of a better way to solve it? >>>> >>>> Roman >>>> >>>>> Roman >>>>> >>>>>> The ARRAYCOPY_ATOMIC decorator makes the arraycopy atomic over the >>>>>> size >>>>>> of the passed in elements. >>>>>> In this case, it looks like the address has been type erased to >>>>>> void*, >>>>>> and hence lost what the element size was. There is currently no >>>>>> overload >>>>>> accepted for type erased element - only accurate elements {jbyte, >>>>>> jshort, jint, jlong}. >>>>>> >>>>>> So it looks like an overload must be added to accept type erased >>>>>> void* >>>>>> elements and make that call conjoint_memory_atomic when the >>>>>> ARRAYCOPY_ATOMIC decorator is passed in. >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>>>> On 2018-03-06 13:56, David Holmes wrote: >>>>>>> On 6/03/2018 10:54 PM, Erik ?sterlund wrote: >>>>>>>> Hi David, >>>>>>>> >>>>>>>> It is atomic with the ARRAYCOPY_ATOMIC decorator. If that >>>>>>>> comment is >>>>>>>> correct (I do not know if it is), then the ARRAYCOPY_ATOMIC >>>>>>>> decorator >>>>>>>> should probably be used here. >>>>>>> If that code implements a Java array copy then yes it is required to >>>>>>> be 32-bit atomic. Do you need the decorator to get 32-bit atomicity? >>>>>>> >>>>>>> David >>>>>>> >>>>>>>> Thanks, >>>>>>>> /Erik >>>>>>>> >>>>>>>> On 2018-03-06 13:48, David Holmes wrote: >>>>>>>>> Hi Roman, >>>>>>>>> >>>>>>>>> Not a review as I'm not familiar enough with the Access API, >>>>>>>>> but in >>>>>>>>> src/hotspot/share/c1/c1_Runtime1.cpp the comments above the >>>>>>>>> changed >>>>>>>>> code need updating - probably deleting. I assume the Access API >>>>>>>>> arraycopy is atomic? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> >>>>>>>>> On 6/03/2018 9:56 PM, Roman Kennke wrote: >>>>>>>>>> Currently, the Access API is only used for oop-arraycopy, but >>>>>>>>>> not for >>>>>>>>>> primitive arraycopy. GCs might want to intercept this too, e.g. >>>>>>>>>> resolve >>>>>>>>>> src and dst arrays. >>>>>>>>>> >>>>>>>>>> There *is* an implementation of primitive arraycopy in the >>>>>>>>>> Access API, >>>>>>>>>> but it doesn't even compile, because Raw::arraycopy() does not >>>>>>>>>> take >>>>>>>>>> src >>>>>>>>>> and dst oop operands, but it's called like that. The only reason >>>>>>>>>> why >>>>>>>>>> this does not blow up (I think) is that because nobody calls it, >>>>>>>>>> the >>>>>>>>>> compiler doesn't even get there. >>>>>>>>>> >>>>>>>>>> This change fixes the Access API/impl and adds the relevant >>>>>>>>>> calls into >>>>>>>>>> it (in C1 and runtime land). C2 uses arraycopy stubs (which >>>>>>>>>> cannot be >>>>>>>>>> handled here) or calls out to the ArrayKlass::copy_array(), which >>>>>>>>>> should >>>>>>>>>> be covered with this change. >>>>>>>>>> >>>>>>>>>> It should be possible to use the same Access API for >>>>>>>>>> Java-array <-> >>>>>>>>>> native-array bulk transfers, which currently use the rather ugly >>>>>>>>>> typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address that >>>>>>>>>> in a >>>>>>>>>> separate change though. >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ >>>>>>>>>> >>>>>>>>>> Tests: tier1 ok >>>>>>>>>> >>>>>>>>>> Please review! >>>>>>>>>> Thanks, Roman >>>>>>>>>> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Fri Mar 9 08:41:08 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 09 Mar 2018 09:41:08 +0100 Subject: PING: RFR: JDK-8153333: [REDO] STW phases at Concurrent GC should count in PerfCounter In-Reply-To: References: <2f4e0901-1602-6276-e7fd-84e168e7b317@gmail.com> <3210723e-c5f9-4277-a97a-be61e10e6b3d@oracle.com> <89a1a161-874d-4743-e52b-7ab202fd8976@gmail.com> <9799a2ad-6727-cae9-312b-fd9de13c8203@oracle.com> <9183dfd3-522f-fd73-a681-5ea957f1d717@gmail.com> Message-ID: <1520584868.3872.6.camel@oracle.com> Hi Yasumasa, On Wed, 2018-03-07 at 21:18 +0900, Yasumasa Suenaga wrote: > PING: Could you review it? > > http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.08/ > > JBS: https://bugs.openjdk.java.net/browse/JDK-8153333 > CSR: https://bugs.openjdk.java.net/browse/JDK-8196862 > > This change has passed Mach5 on submit repo. > Also it has passed hotspot/jtreg/:hotspot_serviceability and > jdk/:jdk_tools jtreg tests. > > We need one more reviewer. - one really minor issue I think: in the description in JstatGcCauseResults.java, the descriptions of the new concurrent collections, between "GCT" and "Total Garbage collection time." there should probably be one space less to align with the "main" gc phase times. - in gcCapacityOutput1.awk, gcNewCapacityOutput1.awk, there are some additional newlines (line 10/11). No need for re-review from me, and for the change in JstatGcCauseResults.java I am not completely sure my suggestion is good. Stefan Johansson already mentioned he will sponsor. Thanks, Thomas From jini.george at oracle.com Fri Mar 9 09:29:02 2018 From: jini.george at oracle.com (Jini George) Date: Fri, 9 Mar 2018 14:59:02 +0530 Subject: RFR: JDK-8175312: SA: clhsdb: Provide an improved heap summary for 'universe' for G1GC In-Reply-To: References: <38d71740-0b66-3ce8-26ed-a0f2b9f9e91c@oracle.com> <5e8c582e-b32f-daf7-0e0c-1e6606ceaf3a@oracle.com> Message-ID: Here is the revised webrev: http://cr.openjdk.java.net/~jgeorge/8175312/webrev.02/ I have made modifications to have the 'universe' command display details like: hsdb> universe Heap Parameters: garbage-first heap [0x0000000725200000, 0x00000007c0000000] region size 1024K G1 Heap: regions = 2478 capacity = 2598371328 (2478.0MB) used = 5242880 (5.0MB) free = 2593128448 (2473.0MB) 0.20177562550443906% used G1 Young Generation: Eden Space: regions = 5 capacity = 8388608 (8.0MB) used = 5242880 (5.0MB) free = 3145728 (3.0MB) 62.5% used Survivor Space: regions = 0 capacity = 0 (0.0MB) used = 0 (0.0MB) free = 0 (0.0MB) 0.0% used G1 Old Generation: regions = 0 capacity = 155189248 (148.0MB) used = 0 (0.0MB) free = 155189248 (148.0MB) 0.0% used I did not add the metaspace details since that did not seem to be in line with the 'universe' output for other GCs. I have added a new command "g1regiondetails" to display the region details, and have modified the tests accordingly. hsdb> g1regiondetails Region Details: Region: 0x0000000725200000,0x0000000725200000,0x0000000725300000:Free Region: 0x0000000725300000,0x0000000725300000,0x0000000725400000:Free Region: 0x0000000725400000,0x0000000725400000,0x0000000725500000:Free Region: 0x0000000725500000,0x0000000725500000,0x0000000725600000:Free Region: 0x0000000725600000,0x0000000725600000,0x0000000725700000:Free Region: 0x0000000725700000,0x0000000725700000,0x0000000725800000:Free ... Thanks, Jini. On 2/28/2018 12:56 PM, Jini George wrote: > Thank you very much, Stefan. My answers inline. > > On 2/27/2018 3:30 PM, Stefan Johansson wrote: >> Hi Jini, > >>>> JIRA ID:https://bugs.openjdk.java.net/browse/JDK-8175312 >>>> Webrev: >>>> http://cr.openjdk.java.net/~jgeorge/8175312/webrev.00/index.html >>>> >> It looks like a file is missing, did you forget to add it to the >> changeset? > > Indeed, I had missed that! I added the missing file in the following > webrev: > > http://cr.openjdk.java.net/~jgeorge/8175312/webrev.01/ > >> --- >> open/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/g1/G1CollectedHeap.java:36: >> error: cannot find symbol >> import sun.jvm.hotspot.gc.shared.PrintRegionClosure; >> --- >> >> Otherwise the change looks good, but I would like to see the output >> live. For a big heap this will print a lot of data, just wondering if >> the universe command is the correct choice for this kind of output. I >> like having the possibility to print all regions, so I want the change >> but maybe it should be a different command and 'universe' just prints >> a little more than before. Something like our logging heap-summary at >> shutdown: >> garbage-first heap?? total 16384K, used 3072K [0x00000000ff000000, >> 0x0000000100000000) >> ??region size 1024K, 4 young (4096K), 0 survivors (0K) >> Metaspace?????? used 6731K, capacity 6825K, committed 7040K, reserved >> 1056768K >> ??class space??? used 559K, capacity 594K, committed 640K, reserved >> 1048576K > > Ok, will add this, and could probably have the region details displayed > under a new command called "g1regiondetails", or some such, and send out > a new webrev. > > Thanks, > Jini. > >> >> Thanks, >> Stefan >>>> Modifications have been made to display the regions like: >>>> >>>> ... >>>> Region: 0x00000005c5400000,0x00000005c5600000,0x00000005c5600000:Old >>>> Region: 0x00000005c5600000,0x00000005c5800000,0x00000005c5800000:Old >>>> Region: 0x00000005c5800000,0x00000005c5a00000,0x00000005c5a00000:Old >>>> Region: 0x00000005c5a00000,0x00000005c5c00000,0x00000005c5c00000:Old >>>> Region: 0x00000005c5c00000,0x00000005c5c00000,0x00000005c5e00000:Free >>>> Region: 0x00000005c5e00000,0x00000005c5e00000,0x00000005c6000000:Free >>>> Region: 0x00000005c6000000,0x00000005c6200000,0x00000005c6200000:Old >>>> ... >>>> >>>> The jtreg test at this point does not include any testing for the >>>> display of archived or pinned regions. The testing for this will be >>>> added once JDK-8174994 is resolved. >>>> >>>> The SA tests pass with jprt and Mach5. >>>> >>>> Thanks, >>>> Jini. >> From yasuenag at gmail.com Fri Mar 9 12:12:42 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Fri, 9 Mar 2018 21:12:42 +0900 Subject: PING: RFR: JDK-8153333: [REDO] STW phases at Concurrent GC should count in PerfCounter In-Reply-To: <1520584868.3872.6.camel@oracle.com> References: <2f4e0901-1602-6276-e7fd-84e168e7b317@gmail.com> <3210723e-c5f9-4277-a97a-be61e10e6b3d@oracle.com> <89a1a161-874d-4743-e52b-7ab202fd8976@gmail.com> <9799a2ad-6727-cae9-312b-fd9de13c8203@oracle.com> <9183dfd3-522f-fd73-a681-5ea957f1d717@gmail.com> <1520584868.3872.6.camel@oracle.com> Message-ID: Hi Thomas, > - one really minor issue I think: in the description in > JstatGcCauseResults.java, the descriptions of the new concurrent > collections, between "GCT" and "Total Garbage collection time." there > should probably be one space less to align with the "main" gc phase > times. > > - in gcCapacityOutput1.awk, gcNewCapacityOutput1.awk, there are some > additional newlines (line 10/11). Thanks! I will fix them. Yasumasa On 2018/03/09 17:41, Thomas Schatzl wrote: > Hi Yasumasa, > > On Wed, 2018-03-07 at 21:18 +0900, Yasumasa Suenaga wrote: >> PING: Could you review it? >> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.08/ >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8153333 >> CSR: https://bugs.openjdk.java.net/browse/JDK-8196862 >> >> This change has passed Mach5 on submit repo. >> Also it has passed hotspot/jtreg/:hotspot_serviceability and >> jdk/:jdk_tools jtreg tests. >> >> We need one more reviewer. > > - one really minor issue I think: in the description in > JstatGcCauseResults.java, the descriptions of the new concurrent > collections, between "GCT" and "Total Garbage collection time." there > should probably be one space less to align with the "main" gc phase > times. > > - in gcCapacityOutput1.awk, gcNewCapacityOutput1.awk, there are some > additional newlines (line 10/11). > > No need for re-review from me, and for the change in > JstatGcCauseResults.java I am not completely sure my suggestion is > good. > > Stefan Johansson already mentioned he will sponsor. > > Thanks, > Thomas > From erik.osterlund at oracle.com Fri Mar 9 14:36:27 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 9 Mar 2018 15:36:27 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: <3034e0b7-9894-80e6-afd4-171e31ad5836@redhat.com> References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> <5A9E8F76.1050809@oracle.com> <83de65bd-5d3f-78df-02d2-eb4c53aa0c62@oracle.com> <5A9E95D8.6080602@oracle.com> <2cc9a407-aba8-209a-7ef7-4ab26b88fef6@redhat.com> <6d8435af-5bec-da70-fa8a-60d708937f44@redhat.com> <5A9EBD6E.3070005@oracle.com> <2612309c-868b-6094-5944-e9de54daf8d2@redhat.com> <5AA136E9.9080100@oracle.com> <3034e0b7-9894-80e6-afd4-171e31ad5836@redhat.com> Message-ID: <5AA29BEB.3020402@oracle.com> Hi Roman, I would love to avoid using the switch to enumerate all different Java types you could possibly have. How about this? (was easier to cook something together than explain in detail what I had in mind) Webrev: http://cr.openjdk.java.net/~eosterlund/8198445/webrev.00/ The idea is to let Access recognize arraycopy with element type "void", and then perform a conservative byte arraycopy, that when ARRAYCOPY_ATOMIC uses the natural atomicity of the element alignment by using Copy::conjoint_memory_atomic(). Thanks, /Erik On 2018-03-08 21:12, Roman Kennke wrote: > Am 08.03.2018 um 14:13 schrieb Erik ?sterlund: >> Hi Roman, >> >> On 2018-03-08 11:25, Roman Kennke wrote: >>> Wouldn't that lead to mis-aligned accesses when you try to copy an array >>> of, e.g., bytes, but shoehorn it to pointer-size? Or am I >>> misunderstanding? >> No it would not. The element type is void, not void*. Because the >> element type is type erased. And that would map to >> Copy::conjoint_memory_atomic, which is precisely what typeArrayKlass.cpp >> maps such an arraycopy to today. Assuming elements are aligned to their >> natural size (which they are), Copy::conjoint_memory_atomic is >> guaranteed to be at least as atomic as you need your elements to be, up >> to an element size of 8. > Ah, ok. Alright, so here comes rev.02: > > Differential: > http://cr.openjdk.java.net/~rkennke/8198445/webrev.02.diff/ > Full: > http://cr.openjdk.java.net/~rkennke/8198445/webrev.02/ > > Still passes tier1 tests. > > Ok now? > > Cheers, Roman > > >>> An idea would be to copy equal-sized elements using the corresponding >>> Copy::conjoint_XXX_atomic() calls, e.g. char and short would use the >>> short version, boolean and byte would use the byte version (or is >>> boolean sometimes int?!), float and int would use the int version and >>> long and double the long version. WDYT? >> I'm not sure I see what the advantage of that would be. >> >> Thanks, >> /Erik >> >>> Roman >>> >>> >>>> My spontaneous idea was to have an overload for type erased void* that >>>> when called with ARRAYCOPY_ATOMIC maps to Copy::conjoint_memory_atomic >>>> which AFAIK is conservatively atomic regardless of what the precise >>>> element type is. >>>> >>>> But if you have better ideas, I am open for suggestions. >>>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2018-03-06 17:03, Roman Kennke wrote: >>>>> Am 06.03.2018 um 14:35 schrieb Roman Kennke: >>>>>> Makes me wonder: why attempt to be smart in c1_Runtime1.cpp, when >>>>>> we can >>>>>> just as well call typeArrayOop::copy_array() and have it do the right >>>>>> thing? Or go even further and also do it for oop-arraycopy? >>>>> Something like: >>>>> >>>>> http://cr.openjdk.java.net/~rkennke/8198445/8198445-1.patch >>>>> >>>>> This wouldn't compile because of bunch of missing >>>>> arraycopy_conjoint_atomic defintions for extra types like jfloat, >>>>> jdouble, jboolean, etc, which in turn would be missing the same >>>>> Copy::conjoint_jFluffs_atomic() which drags in a bunch of platform >>>>> specific stuff... and my question before I go there is: do we want all >>>>> that? Or can you think of a better way to solve it? >>>>> >>>>> Roman >>>>> >>>>>> Roman >>>>>> >>>>>>> The ARRAYCOPY_ATOMIC decorator makes the arraycopy atomic over the >>>>>>> size >>>>>>> of the passed in elements. >>>>>>> In this case, it looks like the address has been type erased to >>>>>>> void*, >>>>>>> and hence lost what the element size was. There is currently no >>>>>>> overload >>>>>>> accepted for type erased element - only accurate elements {jbyte, >>>>>>> jshort, jint, jlong}. >>>>>>> >>>>>>> So it looks like an overload must be added to accept type erased >>>>>>> void* >>>>>>> elements and make that call conjoint_memory_atomic when the >>>>>>> ARRAYCOPY_ATOMIC decorator is passed in. >>>>>>> >>>>>>> Thanks, >>>>>>> /Erik >>>>>>> >>>>>>> On 2018-03-06 13:56, David Holmes wrote: >>>>>>>> On 6/03/2018 10:54 PM, Erik ?sterlund wrote: >>>>>>>>> Hi David, >>>>>>>>> >>>>>>>>> It is atomic with the ARRAYCOPY_ATOMIC decorator. If that >>>>>>>>> comment is >>>>>>>>> correct (I do not know if it is), then the ARRAYCOPY_ATOMIC >>>>>>>>> decorator >>>>>>>>> should probably be used here. >>>>>>>> If that code implements a Java array copy then yes it is required to >>>>>>>> be 32-bit atomic. Do you need the decorator to get 32-bit atomicity? >>>>>>>> >>>>>>>> David >>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> /Erik >>>>>>>>> >>>>>>>>> On 2018-03-06 13:48, David Holmes wrote: >>>>>>>>>> Hi Roman, >>>>>>>>>> >>>>>>>>>> Not a review as I'm not familiar enough with the Access API, >>>>>>>>>> but in >>>>>>>>>> src/hotspot/share/c1/c1_Runtime1.cpp the comments above the >>>>>>>>>> changed >>>>>>>>>> code need updating - probably deleting. I assume the Access API >>>>>>>>>> arraycopy is atomic? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> David >>>>>>>>>> >>>>>>>>>> On 6/03/2018 9:56 PM, Roman Kennke wrote: >>>>>>>>>>> Currently, the Access API is only used for oop-arraycopy, but >>>>>>>>>>> not for >>>>>>>>>>> primitive arraycopy. GCs might want to intercept this too, e.g. >>>>>>>>>>> resolve >>>>>>>>>>> src and dst arrays. >>>>>>>>>>> >>>>>>>>>>> There *is* an implementation of primitive arraycopy in the >>>>>>>>>>> Access API, >>>>>>>>>>> but it doesn't even compile, because Raw::arraycopy() does not >>>>>>>>>>> take >>>>>>>>>>> src >>>>>>>>>>> and dst oop operands, but it's called like that. The only reason >>>>>>>>>>> why >>>>>>>>>>> this does not blow up (I think) is that because nobody calls it, >>>>>>>>>>> the >>>>>>>>>>> compiler doesn't even get there. >>>>>>>>>>> >>>>>>>>>>> This change fixes the Access API/impl and adds the relevant >>>>>>>>>>> calls into >>>>>>>>>>> it (in C1 and runtime land). C2 uses arraycopy stubs (which >>>>>>>>>>> cannot be >>>>>>>>>>> handled here) or calls out to the ArrayKlass::copy_array(), which >>>>>>>>>>> should >>>>>>>>>>> be covered with this change. >>>>>>>>>>> >>>>>>>>>>> It should be possible to use the same Access API for >>>>>>>>>>> Java-array <-> >>>>>>>>>>> native-array bulk transfers, which currently use the rather ugly >>>>>>>>>>> typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address that >>>>>>>>>>> in a >>>>>>>>>>> separate change though. >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> Tests: tier1 ok >>>>>>>>>>> >>>>>>>>>>> Please review! >>>>>>>>>>> Thanks, Roman >>>>>>>>>>> > From shade at redhat.com Mon Mar 12 13:12:37 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 12 Mar 2018 14:12:37 +0100 Subject: Epsilon GC JEP updates (was: Re: EpsilonGC and throughput.) In-Reply-To: <1515429904.2381.208.camel@oracle.com> References: <3d50e13e-3324-7058-3c8c-725a52207074@oracle.com> <1162631c-1ebc-f562-f096-d1ac861a42a4@redhat.com> <37071ceb-d21b-cf96-2dea-87da409b2e68@oracle.com> <1513781182.2415.29.camel@oracle.com> <07b29153-f01f-f1b5-18bc-9c38caaaf7eb@redhat.com> <1513805149.2542.112.camel@oracle.com> <304ef832-5e17-b53e-8b31-7c26fcb8c593@redhat.com> <1515429904.2381.208.camel@oracle.com> Message-ID: Hey Thomas, Updated the JEP here: https://bugs.openjdk.java.net/browse/JDK-8174901 On 01/08/2018 05:45 PM, Thomas Schatzl wrote: > I apologize for my somewhat inappropriate words, this has been due to > some frustration; also for the long delay that were due to the winter > holidays. It took me quite some time to recover from this, and I am still slightly bitter. If we want contributions to OpenJDK, then we all have to understand this kind of thing really dissuades people from contributing. I have got quite a few personal "raised eyebrows" replies on this, one person calling the whole thing right away hostile. I can imagine much less thick-skinned contributors just walking away. I know the intent wasn't that, but what matters here is not the intent, but the appearance too (I fell into the same trap before numerous times, and learned to keep it together -- call this thread a karmic justice). > It also helps if the JEP is written in a way to make it interesting for > the community to read it, and respond. The less thinking a reader has > to do to answer whether he is impacted or not, and whether and by how > much it would simplify the life of himself or in general Java users, > the more people will feel urged to get this in (or at least not > deterred). Come to think about it, the public discussion this JEP is getting is positive, and there is no need to urge more people to get it in at this point. Lots of people read it, and surprisingly quite a lot of them tried the prototype, at least for fun, but also for performance work too. > "Motivation > ---------- > JEP text: "Java implementations are well known for a broad choice of > highly configurable GC implementations." > > Potential answer to "Why should this work be done?". Or does the > sentence indicate we need another GC because we already have so many, > and another does not hurt? I am asking this in full seriousness, I > really do not know. Or is this only an introductory sentence without > meaning? This statement underpins there is no single all-purpose GC in OpenJDK. "The variety of available collectors caters for different needs in the end, even if their configurability make their functionality intersect. It is sometimes easier to maintain a separate implementation, rather than piling on another configuration option on the existing GC implementation." > Let's go into these benefits in more detail: > > JEP text: "Performance testing. ..." > > Benefit. Maybe it would be useful to list a few of these performance > artifacts here ("... , e.g. barrier code, concurrent threads"). I added some. We (in Shenandoah development land), and others (in Shenandoah/ZGC/Zing comparison land) have used Epsilon as the ultimate latency baseline. New text captures that bit: "Having a GC that does almost nothing is a useful tool to do differential performance analysis for other, real GCs. Having a no-op GC can help to filter out GC-induced performance artifacts, like GC workers scheduling, GC barriers costs, GC cycles triggered at unfortunate times, locality changes, etc. Moreover, there are latency artifacts that are not GC-induced (e.g. scheduling hiccups, compiler transition hiccups, etc), and removing the GC-induced artifacts help to contrast those. For example, having the no-op GC allows to estimate the natural "background" latency baseline for low-latency GC work." > An alternative could be a developer just nop'ing out the relevant GC > interface section. That is somewhat cumbersome, but for how many users > is this a problem? Spell that out in the appropriate Alternatives > section. Spelled: "The developers might just no-op out the existing GC implementation to get the baseline implementation for testing. The problem with this is inconvenience: the developers would need to make sure such the implementation is still correct, that it provides enough performance to be a good baseline, that it is hooked up into the other runtime facilities (heap dumping, thread stack walking, MXBeans) to amend the differential analysis. The implementations for other platforms would require much more work. Having the ready-to-go no-op implementation in the mainline solves this inconvenience." > JEP text: "Functional testing. For Java code testing, a way to > establish a threshold for allocated memory is useful to assert memory > pressure invariants. Today, we have to pick up the allocation data from > MXBeans, or even resort to parsing GC logs. Having a GC that accepts > only the bounded number of allocations, and fails on heap exhaustion, > simplifies testing." > > Benefit. For regression testing, in how many cases do you think it is > sufficient (or in what circumstances) to get a fail/no-fail answer > only? > This seems to pass work on a failure to the dev, them needing to write > another test that also prints and monitors the memory usage increases > over time anyway. > How much work, given that you already need to monitor memory usage is > the test to fail when heap usage goes above a threshold then? I don't quite believe debugging the test like this would involve tracking the memory allocated so far, because that is not readily actionable. Even if it is, Epsilon prints the messages when n% of the heap was allocated, which actually improves developer's experience, because as a dev I don't need to copy-paste MXBeans blocks anymore. But what is more actionable is the actual heap dump. And this is where Epsilon comes handy: you just set -Xmx1g -XX:HeapDumpOnOutOfMemoryError, and run the test. If the test fails, you get the ready heap dump that tell you what the test had ever allocated to blow the allocation limit, down to every single object. I added the example: "For example, knowing that test should allocate no more than 1 GB of memory, we can configure no-op GC with -Xmx1g, and let it crash with a heap dump if that constraint is violated." > "VM interface testing. For VM development purposes, having a simple GC > helps to understand the absolute minimum required from the VM-GC > interface to have a functional allocator. This serves as proof that the > VM-GC interface is sane, which is important in lieu of JEP 304 > ("Garbage Collector Interface")." > > Benefit. Who are the (main) benefactors for that - probably developers? > For a developer, how much is that benefit if there are already 5 or 6 > implementations of that interface? The benefit is simple: for no-op GC, the implemented GC interface should be in epsilon-neighborhood of zero. It is not right now, by the way, in both native and codegen parts: https://builds.shipilev.net/patch-openjdk-epsilon-jdk/ I added: "For no-op GC, the interface should not have anything implemented, and good interface means Epsilon's BarrierSet would just use no-op barrier implementations from the default implementation". > "Last-drop performance improvements. For ultra-latency-sensitive > applications, where developers are conscious about memory allocations > and know the application memory footprint exactly, or even have > (almost) completely garbage-free applications. In those applications, > GC cycles may be considered an implementation bug that wastes CPU > cycles for no good reason." I split this section in three: "Extremely short lived jobs", "Last-drop latency improvements", and "Last-drop throughput improvements", because it seems more logical that way. > It may be useful to investigate the problem of these power users in > more detail, and see if we could provide a (more?) complete solution > for them. The problem with this is, those power users consider whatever tricks they managed to tame the misbehaving GC their competitive advantage (think HFT), and are not really inclined to share. (Some did not manage, and they moved out of Java, to our disadvantage). The bits and pieces I got are "give us the no-op GC, and we shall figure out how to manage our memory ourselves, thank you very much". And in many cases, from the little glimpses that were shared behind the curtains, those guys seem to really know what they doing. All I'm saying here is that we have no alternative than to take it on faith, which I am willing to do. > "Extremely short lived jobs are one example of this." > > I do not understand the use of Epsilon in such use case. The > alternative I can see would be to restart the VM after every short > lived job (something for the Alternatives section). That seems strange > to me, depending on the definition of a "short lived job", particularly > if nothing survives after execution of that short lived job, a GC will > be extremely fast. This relies on per-supposition that GC is fast, because heap is a graveyard. It is not always the case. I have demonstrated one example: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2017-December/021042.html Added: "A short-lived job might rely on exiting quickly to free the resources (e.g. heap memory). In this case, accepting the GC cycle to futilely clean up the heap is a waste of time, because the heap would be freed on exit anyway. Note that the GC cycle might take a while, because it would depend on the amount of live data in the heap, which can be a lot." > "There are also cases when restarting the JVM -- letting load balancers > figure out failover -- is sometimes a better recovery strategy than > accepting a GC cycle." > > I really can't find a good example where a GC, particularly in the > situation that has been described so far, also for these short-lived > jobs, where a GC (on an almost empty heap) is not at least as fast as a > restart. Again, this relies on per-supposition that GC is fast, because heap is a graveyard. It is, again, not always the case. The real-world systems I know of, the latency of node restart does not matter as much: it is more important to reliably fail the node to let balancer act. In other words, the "global" detect-and-evade latency is more important than "local" restart latency. Accepting the GC cycle makes the availability logic harder: you now have to disambiguate between the normal 100ms execution in the business logic, and the first 100ms of multi-second GC pause. Which probably means timeout to be several-sigma larger than the usual business logic wait time, which prolongs the recovery. Instead, you might just crash, and let balancer figure out where to restart the processing right away. I understand this goes against our own intuition how the systems should be built. Of course, we want our GCs to never push users to come up with these contraptions, but the sad reality is that they are doing that, because the world and GC implementations in this world are not perfect. And you would not find that in our spectrum of well-behaved workloads. Talking to people who maintain large real-world systems can be sobering for understanding what they have to deal with. For example, one customer asked me to come up with this contraption for their high-availability in-memory grid -- they are ready for JVM to crash, and it fact they would like it to crash instead of stalling! https://bugs.openjdk.java.net/browse/JDK-8181143 > It would make for a very good paragraph explaining this use case in the > alternatives section. > > Another problem with these two sentences to me is (and I am by no means > a "FaaS power user") that I believe that waiting for the VM to > crash/shut down to steer the load balancers is not a good strategy. > Maybe you can give some more information about this use case? JEP does not advocate for using this strategy. It just reports what users are doing in the wild. So, discussion around this seems to go outside the scope of the JEP. > In the earlier email I only directly asked for performance numbers > because in order to streamline this discussion, and given that you are > a well-known performance and benchmark guru (afaik you were "R"eviewer > long before me joining) it seemed a logical request. If you can't find > numbers, there is also the reference ("Barriers, Friendlier Still" or > so from Blackburn et al I think) I got that is also mentioned iirc in > the very good Jones GC book. > "Real" newbies I would just ask to perform this test. See here: https://shipilev.net/jvm-anatomy-park/13-intergenerational-barriers/ I considered it a bad taste to link my blog to the JEP, but I can do this anyway. > "Alternatives > ------------ > Again, try to make these alternative review balanced, and in context of > the users the benefit is for. I have rewritten that part with most things we have discussed, and tried to discuss why those alternatives are not exactly better. I might still be missing some salient alternatives, because I ran out of steam... Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From erik.helin at oracle.com Mon Mar 12 13:07:21 2018 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 12 Mar 2018 14:07:21 +0100 Subject: RFR: 8199027: Make protected members private in G1Policy Message-ID: <01235743-3518-3d21-2a6b-8af7d5a7ef98@oracle.com> Hi all, this tiny patch removes the (unnecessary) use of `protected` visibility from the class G1Policy, the members that were protected can just be private. Issue: https://bugs.openjdk.java.net/browse/JDK-8199027 Patch: http://cr.openjdk.java.net/~ehelin/8199027/00/ Testing: - make run-test-tier1 Thanks, Erik From stefan.johansson at oracle.com Mon Mar 12 13:44:19 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 12 Mar 2018 14:44:19 +0100 Subject: RFR: 8199027: Make protected members private in G1Policy In-Reply-To: <01235743-3518-3d21-2a6b-8af7d5a7ef98@oracle.com> References: <01235743-3518-3d21-2a6b-8af7d5a7ef98@oracle.com> Message-ID: <08517c41-ba15-e176-82d2-71b01c08ac34@oracle.com> Looks good, StefanJ On 2018-03-12 14:07, Erik Helin wrote: > Hi all, > > this tiny patch removes the (unnecessary) use of `protected` > visibility from the class G1Policy, the members that were protected > can just be private. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8199027 > > Patch: > http://cr.openjdk.java.net/~ehelin/8199027/00/ > > Testing: > - make run-test-tier1 > > Thanks, > Erik From stefan.johansson at oracle.com Mon Mar 12 15:22:33 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 12 Mar 2018 16:22:33 +0100 Subject: RFR: JDK-8175312: SA: clhsdb: Provide an improved heap summary for 'universe' for G1GC In-Reply-To: References: <38d71740-0b66-3ce8-26ed-a0f2b9f9e91c@oracle.com> <5e8c582e-b32f-daf7-0e0c-1e6606ceaf3a@oracle.com> Message-ID: <59c7ec4f-f9dc-7c25-e7bf-f3fc304b5c60@oracle.com> Hi Jini, This looks good. I'm totally fine with skipping metaspace if that isn't displayed for the other GCs. Cheers, Stefan On 2018-03-09 10:29, Jini George wrote: > Here is the revised webrev: > > http://cr.openjdk.java.net/~jgeorge/8175312/webrev.02/ > > I have made modifications to have the 'universe' command display > details like: > > hsdb> universe > Heap Parameters: > garbage-first heap [0x0000000725200000, 0x00000007c0000000] region > size 1024K > G1 Heap: > ?? regions? = 2478 > ?? capacity = 2598371328 (2478.0MB) > ?? used???? = 5242880 (5.0MB) > ?? free???? = 2593128448 (2473.0MB) > ?? 0.20177562550443906% used > G1 Young Generation: > Eden Space: > ?? regions? = 5 > ?? capacity = 8388608 (8.0MB) > ?? used???? = 5242880 (5.0MB) > ?? free???? = 3145728 (3.0MB) > ?? 62.5% used > Survivor Space: > ?? regions? = 0 > ?? capacity = 0 (0.0MB) > ?? used???? = 0 (0.0MB) > ?? free???? = 0 (0.0MB) > ?? 0.0% used > G1 Old Generation: > ?? regions? = 0 > ?? capacity = 155189248 (148.0MB) > ?? used???? = 0 (0.0MB) > ?? free???? = 155189248 (148.0MB) > ?? 0.0% used > > > I did not add the metaspace details since that did not seem to be in > line with the 'universe' output for other GCs. I have added a new > command "g1regiondetails" to display the region details, and have > modified the tests accordingly. > > hsdb> g1regiondetails > Region Details: > Region: 0x0000000725200000,0x0000000725200000,0x0000000725300000:Free > Region: 0x0000000725300000,0x0000000725300000,0x0000000725400000:Free > Region: 0x0000000725400000,0x0000000725400000,0x0000000725500000:Free > Region: 0x0000000725500000,0x0000000725500000,0x0000000725600000:Free > Region: 0x0000000725600000,0x0000000725600000,0x0000000725700000:Free > Region: 0x0000000725700000,0x0000000725700000,0x0000000725800000:Free > ... > > Thanks, > Jini. > > > On 2/28/2018 12:56 PM, Jini George wrote: >> Thank you very much, Stefan. My answers inline. >> >> On 2/27/2018 3:30 PM, Stefan Johansson wrote: >>> Hi Jini, >> >>>>> JIRA ID:https://bugs.openjdk.java.net/browse/JDK-8175312 >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~jgeorge/8175312/webrev.00/index.html >>>>> >>> It looks like a file is missing, did you forget to add it to the >>> changeset? >> >> Indeed, I had missed that! I added the missing file in the following >> webrev: >> >> http://cr.openjdk.java.net/~jgeorge/8175312/webrev.01/ >> >>> --- >>> open/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/g1/G1CollectedHeap.java:36: >>> error: cannot find symbol >>> import sun.jvm.hotspot.gc.shared.PrintRegionClosure; >>> --- >>> >>> Otherwise the change looks good, but I would like to see the output >>> live. For a big heap this will print a lot of data, just wondering >>> if the universe command is the correct choice for this kind of >>> output. I like having the possibility to print all regions, so I >>> want the change but maybe it should be a different command and >>> 'universe' just prints a little more than before. Something like our >>> logging heap-summary at shutdown: >>> garbage-first heap?? total 16384K, used 3072K [0x00000000ff000000, >>> 0x0000000100000000) >>> ??region size 1024K, 4 young (4096K), 0 survivors (0K) >>> Metaspace?????? used 6731K, capacity 6825K, committed 7040K, >>> reserved 1056768K >>> ??class space??? used 559K, capacity 594K, committed 640K, reserved >>> 1048576K >> >> Ok, will add this, and could probably have the region details >> displayed under a new command called "g1regiondetails", or some such, >> and send out a new webrev. >> >> Thanks, >> Jini. >> >>> >>> Thanks, >>> Stefan >>>>> Modifications have been made to display the regions like: >>>>> >>>>> ... >>>>> Region: 0x00000005c5400000,0x00000005c5600000,0x00000005c5600000:Old >>>>> Region: 0x00000005c5600000,0x00000005c5800000,0x00000005c5800000:Old >>>>> Region: 0x00000005c5800000,0x00000005c5a00000,0x00000005c5a00000:Old >>>>> Region: 0x00000005c5a00000,0x00000005c5c00000,0x00000005c5c00000:Old >>>>> Region: 0x00000005c5c00000,0x00000005c5c00000,0x00000005c5e00000:Free >>>>> Region: 0x00000005c5e00000,0x00000005c5e00000,0x00000005c6000000:Free >>>>> Region: 0x00000005c6000000,0x00000005c6200000,0x00000005c6200000:Old >>>>> ... >>>>> >>>>> The jtreg test at this point does not include any testing for the >>>>> display of archived or pinned regions. The testing for this will >>>>> be added once JDK-8174994 is resolved. >>>>> >>>>> The SA tests pass with jprt and Mach5. >>>>> >>>>> Thanks, >>>>> Jini. >>> From stefan.johansson at oracle.com Mon Mar 12 15:48:30 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 12 Mar 2018 16:48:30 +0100 Subject: RFR (S): 8199326: Remove G1 gc time stamp logic In-Reply-To: <1520523980.3121.29.camel@oracle.com> References: <1520516013.3121.24.camel@oracle.com> <7b19df2a-cac8-41d7-c3e9-d2cf893f9d85@oracle.com> <1520523980.3121.29.camel@oracle.com> Message-ID: Hi Thomas, On 2018-03-08 16:46, Thomas Schatzl wrote: > On Thu, 2018-03-08 at 16:18 +0100, Stefan Johansson wrote: >> Hi Thomas, >> >> On 2018-03-08 14:33, Thomas Schatzl wrote: >>> Hi, >>> >>> with the rebuild remsets concurrently changes the last user of >>> the gc >>> time stamps (create live data) has gone away. >>> >>> This change removes all of the corresponding code I could find. >>> >>> Could you review it? >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8199326 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8197932/webrev/ >> Looks like you pasted the wrong link, and I couldn't find the >> correct >> one either. > Sorry. The correct link is: > http://cr.openjdk.java.net/~tschatzl/8199326/webrev Thanks for doing this cleanup, I found one more reference to the GC timestamp in g1CollectedHeap.hpp: ?296?? // Clear RSets after a compaction. It also resets the GC time stamps. ?297?? void clear_rsets_post_compaction(); You can actually remove those two lines completely if you like. I forgot the declaration when removing this method in the fix for: 8177544: Restructure G1 Full GC code Otherwise the change looks good and if you don't want to remove the declaration in this patch we can create a separate clean up for this. Cheers, Stefan > > Thomas > From jini.george at oracle.com Mon Mar 12 15:52:09 2018 From: jini.george at oracle.com (Jini George) Date: Mon, 12 Mar 2018 21:22:09 +0530 Subject: RFR: JDK-8175312: SA: clhsdb: Provide an improved heap summary for 'universe' for G1GC In-Reply-To: <59c7ec4f-f9dc-7c25-e7bf-f3fc304b5c60@oracle.com> References: <38d71740-0b66-3ce8-26ed-a0f2b9f9e91c@oracle.com> <5e8c582e-b32f-daf7-0e0c-1e6606ceaf3a@oracle.com> <59c7ec4f-f9dc-7c25-e7bf-f3fc304b5c60@oracle.com> Message-ID: Thank you very much, Stefan. Could one more reviewer please take a look at it ? - Jini. On 3/12/2018 8:52 PM, Stefan Johansson wrote: > Hi Jini, > > This looks good. I'm totally fine with skipping metaspace if that isn't > displayed for the other GCs. > > Cheers, > Stefan > > On 2018-03-09 10:29, Jini George wrote: >> Here is the revised webrev: >> >> http://cr.openjdk.java.net/~jgeorge/8175312/webrev.02/ >> >> I have made modifications to have the 'universe' command display >> details like: >> >> hsdb> universe >> Heap Parameters: >> garbage-first heap [0x0000000725200000, 0x00000007c0000000] region >> size 1024K >> G1 Heap: >> ?? regions? = 2478 >> ?? capacity = 2598371328 (2478.0MB) >> ?? used???? = 5242880 (5.0MB) >> ?? free???? = 2593128448 (2473.0MB) >> ?? 0.20177562550443906% used >> G1 Young Generation: >> Eden Space: >> ?? regions? = 5 >> ?? capacity = 8388608 (8.0MB) >> ?? used???? = 5242880 (5.0MB) >> ?? free???? = 3145728 (3.0MB) >> ?? 62.5% used >> Survivor Space: >> ?? regions? = 0 >> ?? capacity = 0 (0.0MB) >> ?? used???? = 0 (0.0MB) >> ?? free???? = 0 (0.0MB) >> ?? 0.0% used >> G1 Old Generation: >> ?? regions? = 0 >> ?? capacity = 155189248 (148.0MB) >> ?? used???? = 0 (0.0MB) >> ?? free???? = 155189248 (148.0MB) >> ?? 0.0% used >> >> >> I did not add the metaspace details since that did not seem to be in >> line with the 'universe' output for other GCs. I have added a new >> command "g1regiondetails" to display the region details, and have >> modified the tests accordingly. >> >> hsdb> g1regiondetails >> Region Details: >> Region: 0x0000000725200000,0x0000000725200000,0x0000000725300000:Free >> Region: 0x0000000725300000,0x0000000725300000,0x0000000725400000:Free >> Region: 0x0000000725400000,0x0000000725400000,0x0000000725500000:Free >> Region: 0x0000000725500000,0x0000000725500000,0x0000000725600000:Free >> Region: 0x0000000725600000,0x0000000725600000,0x0000000725700000:Free >> Region: 0x0000000725700000,0x0000000725700000,0x0000000725800000:Free >> ... >> >> Thanks, >> Jini. >> >> >> On 2/28/2018 12:56 PM, Jini George wrote: >>> Thank you very much, Stefan. My answers inline. >>> >>> On 2/27/2018 3:30 PM, Stefan Johansson wrote: >>>> Hi Jini, >>> >>>>>> JIRA ID:https://bugs.openjdk.java.net/browse/JDK-8175312 >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~jgeorge/8175312/webrev.00/index.html >>>>>> >>>> It looks like a file is missing, did you forget to add it to the >>>> changeset? >>> >>> Indeed, I had missed that! I added the missing file in the following >>> webrev: >>> >>> http://cr.openjdk.java.net/~jgeorge/8175312/webrev.01/ >>> >>>> --- >>>> open/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/g1/G1CollectedHeap.java:36: >>>> error: cannot find symbol >>>> import sun.jvm.hotspot.gc.shared.PrintRegionClosure; >>>> --- >>>> >>>> Otherwise the change looks good, but I would like to see the output >>>> live. For a big heap this will print a lot of data, just wondering >>>> if the universe command is the correct choice for this kind of >>>> output. I like having the possibility to print all regions, so I >>>> want the change but maybe it should be a different command and >>>> 'universe' just prints a little more than before. Something like our >>>> logging heap-summary at shutdown: >>>> garbage-first heap?? total 16384K, used 3072K [0x00000000ff000000, >>>> 0x0000000100000000) >>>> ??region size 1024K, 4 young (4096K), 0 survivors (0K) >>>> Metaspace?????? used 6731K, capacity 6825K, committed 7040K, >>>> reserved 1056768K >>>> ??class space??? used 559K, capacity 594K, committed 640K, reserved >>>> 1048576K >>> >>> Ok, will add this, and could probably have the region details >>> displayed under a new command called "g1regiondetails", or some such, >>> and send out a new webrev. >>> >>> Thanks, >>> Jini. >>> >>>> >>>> Thanks, >>>> Stefan >>>>>> Modifications have been made to display the regions like: >>>>>> >>>>>> ... >>>>>> Region: 0x00000005c5400000,0x00000005c5600000,0x00000005c5600000:Old >>>>>> Region: 0x00000005c5600000,0x00000005c5800000,0x00000005c5800000:Old >>>>>> Region: 0x00000005c5800000,0x00000005c5a00000,0x00000005c5a00000:Old >>>>>> Region: 0x00000005c5a00000,0x00000005c5c00000,0x00000005c5c00000:Old >>>>>> Region: 0x00000005c5c00000,0x00000005c5c00000,0x00000005c5e00000:Free >>>>>> Region: 0x00000005c5e00000,0x00000005c5e00000,0x00000005c6000000:Free >>>>>> Region: 0x00000005c6000000,0x00000005c6200000,0x00000005c6200000:Old >>>>>> ... >>>>>> >>>>>> The jtreg test at this point does not include any testing for the >>>>>> display of archived or pinned regions. The testing for this will >>>>>> be added once JDK-8174994 is resolved. >>>>>> >>>>>> The SA tests pass with jprt and Mach5. >>>>>> >>>>>> Thanks, >>>>>> Jini. >>>> > From yumin.qi at gmail.com Tue Mar 13 00:54:44 2018 From: yumin.qi at gmail.com (yumin qi) Date: Mon, 12 Mar 2018 17:54:44 -0700 Subject: RFR: JDK-8175312: SA: clhsdb: Provide an improved heap summary for 'universe' for G1GC In-Reply-To: References: <38d71740-0b66-3ce8-26ed-a0f2b9f9e91c@oracle.com> <5e8c582e-b32f-daf7-0e0c-1e6606ceaf3a@oracle.com> <59c7ec4f-f9dc-7c25-e7bf-f3fc304b5c60@oracle.com> Message-ID: Jini, Looks good. One minor comment: + public void printG1HeapSummary(G1CollectedHeap heap) {+ G1CollectedHeap g1h = (G1CollectedHeap) heap; 'heap' has been cast to 'G1CollectedHeap' at call site, so seems no need to convert here again. Thanks Yumin On Mon, Mar 12, 2018 at 8:52 AM, Jini George wrote: > Thank you very much, Stefan. Could one more reviewer please take a look at > it ? > > - Jini. > > > On 3/12/2018 8:52 PM, Stefan Johansson wrote: > >> Hi Jini, >> >> This looks good. I'm totally fine with skipping metaspace if that isn't >> displayed for the other GCs. >> >> Cheers, >> Stefan >> >> On 2018-03-09 10:29, Jini George wrote: >> >>> Here is the revised webrev: >>> >>> http://cr.openjdk.java.net/~jgeorge/8175312/webrev.02/ >>> >>> I have made modifications to have the 'universe' command display details >>> like: >>> >>> hsdb> universe >>> Heap Parameters: >>> garbage-first heap [0x0000000725200000, 0x00000007c0000000] region size >>> 1024K >>> G1 Heap: >>> regions = 2478 >>> capacity = 2598371328 (2478.0MB) >>> used = 5242880 (5.0MB) >>> free = 2593128448 (2473.0MB) >>> 0.20177562550443906% used >>> G1 Young Generation: >>> Eden Space: >>> regions = 5 >>> capacity = 8388608 (8.0MB) >>> used = 5242880 (5.0MB) >>> free = 3145728 (3.0MB) >>> 62.5% used >>> Survivor Space: >>> regions = 0 >>> capacity = 0 (0.0MB) >>> used = 0 (0.0MB) >>> free = 0 (0.0MB) >>> 0.0% used >>> G1 Old Generation: >>> regions = 0 >>> capacity = 155189248 (148.0MB) >>> used = 0 (0.0MB) >>> free = 155189248 (148.0MB) >>> 0.0% used >>> >>> >>> I did not add the metaspace details since that did not seem to be in >>> line with the 'universe' output for other GCs. I have added a new command >>> "g1regiondetails" to display the region details, and have modified the >>> tests accordingly. >>> >>> hsdb> g1regiondetails >>> Region Details: >>> Region: 0x0000000725200000,0x0000000725200000,0x0000000725300000:Free >>> Region: 0x0000000725300000,0x0000000725300000,0x0000000725400000:Free >>> Region: 0x0000000725400000,0x0000000725400000,0x0000000725500000:Free >>> Region: 0x0000000725500000,0x0000000725500000,0x0000000725600000:Free >>> Region: 0x0000000725600000,0x0000000725600000,0x0000000725700000:Free >>> Region: 0x0000000725700000,0x0000000725700000,0x0000000725800000:Free >>> ... >>> >>> Thanks, >>> Jini. >>> >>> >>> On 2/28/2018 12:56 PM, Jini George wrote: >>> >>>> Thank you very much, Stefan. My answers inline. >>>> >>>> On 2/27/2018 3:30 PM, Stefan Johansson wrote: >>>> >>>>> Hi Jini, >>>>> >>>> >>>> JIRA ID:https://bugs.openjdk.java.net/browse/JDK-8175312 >>>>>>> Webrev: http://cr.openjdk.java.net/~jgeorge/8175312/webrev.00/index. >>>>>>> html >>>>>>> >>>>>>> It looks like a file is missing, did you forget to add it to the >>>>> changeset? >>>>> >>>> >>>> Indeed, I had missed that! I added the missing file in the following >>>> webrev: >>>> >>>> http://cr.openjdk.java.net/~jgeorge/8175312/webrev.01/ >>>> >>>> --- >>>>> open/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/g1/G1CollectedHeap.java:36: >>>>> error: cannot find symbol >>>>> import sun.jvm.hotspot.gc.shared.PrintRegionClosure; >>>>> --- >>>>> >>>>> Otherwise the change looks good, but I would like to see the output >>>>> live. For a big heap this will print a lot of data, just wondering if the >>>>> universe command is the correct choice for this kind of output. I like >>>>> having the possibility to print all regions, so I want the change but maybe >>>>> it should be a different command and 'universe' just prints a little more >>>>> than before. Something like our logging heap-summary at shutdown: >>>>> garbage-first heap total 16384K, used 3072K [0x00000000ff000000, >>>>> 0x0000000100000000) >>>>> region size 1024K, 4 young (4096K), 0 survivors (0K) >>>>> Metaspace used 6731K, capacity 6825K, committed 7040K, reserved >>>>> 1056768K >>>>> class space used 559K, capacity 594K, committed 640K, reserved >>>>> 1048576K >>>>> >>>> >>>> Ok, will add this, and could probably have the region details displayed >>>> under a new command called "g1regiondetails", or some such, and send out a >>>> new webrev. >>>> >>>> Thanks, >>>> Jini. >>>> >>>> >>>>> Thanks, >>>>> Stefan >>>>> >>>>>> Modifications have been made to display the regions like: >>>>>>> >>>>>>> ... >>>>>>> Region: 0x00000005c5400000,0x00000005c5600000,0x00000005c5600000:Old >>>>>>> Region: 0x00000005c5600000,0x00000005c5800000,0x00000005c5800000:Old >>>>>>> Region: 0x00000005c5800000,0x00000005c5a00000,0x00000005c5a00000:Old >>>>>>> Region: 0x00000005c5a00000,0x00000005c5c00000,0x00000005c5c00000:Old >>>>>>> Region: 0x00000005c5c00000,0x00000005c5c00000,0x00000005c5e00000: >>>>>>> Free >>>>>>> Region: 0x00000005c5e00000,0x00000005c5e00000,0x00000005c6000000: >>>>>>> Free >>>>>>> Region: 0x00000005c6000000,0x00000005c6200000,0x00000005c6200000:Old >>>>>>> ... >>>>>>> >>>>>>> The jtreg test at this point does not include any testing for the >>>>>>> display of archived or pinned regions. The testing for this will be added >>>>>>> once JDK-8174994 is resolved. >>>>>>> >>>>>>> The SA tests pass with jprt and Mach5. >>>>>>> >>>>>>> Thanks, >>>>>>> Jini. >>>>>>> >>>>>> >>>>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From rkennke at redhat.com Tue Mar 13 10:02:38 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 13 Mar 2018 11:02:38 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: <5AA29BEB.3020402@oracle.com> References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> <5A9E8F76.1050809@oracle.com> <83de65bd-5d3f-78df-02d2-eb4c53aa0c62@oracle.com> <5A9E95D8.6080602@oracle.com> <2cc9a407-aba8-209a-7ef7-4ab26b88fef6@redhat.com> <6d8435af-5bec-da70-fa8a-60d708937f44@redhat.com> <5A9EBD6E.3070005@oracle.com> <2612309c-868b-6094-5944-e9de54daf8d2@redhat.com> <5AA136E9.9080100@oracle.com> <3034e0b7-9894-80e6-afd4-171e31ad5836@redhat.com> <5AA29BEB.3020402@oracle.com> Message-ID: <0032c706-3171-1cee-20b0-baf67b438f31@redhat.com> Hi Erik, thanks for doing this! It looks good to me! I'm putting this up here again, with commit msg fixed: http://cr.openjdk.java.net/~rkennke/8198445/webrev.03/ Can I get (final?) reviews? And then I'd put it through the new submit repo and push it? Thanks, Roman > Hi Roman, > > I would love to avoid using the switch to enumerate all different Java > types you could possibly have. > How about this? (was easier to cook something together than explain in > detail what I had in mind) > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8198445/webrev.00/ > > The idea is to let Access recognize arraycopy with element type "void", > and then perform a conservative byte arraycopy, that when > ARRAYCOPY_ATOMIC uses the natural atomicity of the element alignment by > using Copy::conjoint_memory_atomic(). > > Thanks, > /Erik > > On 2018-03-08 21:12, Roman Kennke wrote: >> Am 08.03.2018 um 14:13 schrieb Erik ?sterlund: >>> Hi Roman, >>> >>> On 2018-03-08 11:25, Roman Kennke wrote: >>>> Wouldn't that lead to mis-aligned accesses when you try to copy an >>>> array >>>> of, e.g., bytes, but shoehorn it to pointer-size? Or am I >>>> misunderstanding? >>> No it would not. The element type is void, not void*. Because the >>> element type is type erased. And that would map to >>> Copy::conjoint_memory_atomic, which is precisely what typeArrayKlass.cpp >>> maps such an arraycopy to today. Assuming elements are aligned to their >>> natural size (which they are), Copy::conjoint_memory_atomic is >>> guaranteed to be at least as atomic as you need your elements to be, up >>> to an element size of 8. >> Ah, ok. Alright, so here comes rev.02: >> >> Differential: >> http://cr.openjdk.java.net/~rkennke/8198445/webrev.02.diff/ >> Full: >> http://cr.openjdk.java.net/~rkennke/8198445/webrev.02/ >> >> Still passes tier1 tests. >> >> Ok now? >> >> Cheers, Roman >> >> >>>> An idea would be to copy equal-sized elements using the corresponding >>>> Copy::conjoint_XXX_atomic() calls, e.g. char and short would use the >>>> short version, boolean and byte would use the byte version (or is >>>> boolean sometimes int?!), float and int would use the int version and >>>> long and double the long version. WDYT? >>> I'm not sure I see what the advantage of that would be. >>> >>> Thanks, >>> /Erik >>> >>>> Roman >>>> >>>> >>>>> My spontaneous idea was to have an overload for type erased void* that >>>>> when called with ARRAYCOPY_ATOMIC maps to Copy::conjoint_memory_atomic >>>>> which AFAIK is conservatively atomic regardless of what the precise >>>>> element type is. >>>>> >>>>> But if you have better ideas, I am open for suggestions. >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>> On 2018-03-06 17:03, Roman Kennke wrote: >>>>>> Am 06.03.2018 um 14:35 schrieb Roman Kennke: >>>>>>> Makes me wonder: why attempt to be smart in c1_Runtime1.cpp, when >>>>>>> we can >>>>>>> just as well call typeArrayOop::copy_array() and have it do the >>>>>>> right >>>>>>> thing? Or go even further and also do it for oop-arraycopy? >>>>>> Something like: >>>>>> >>>>>> http://cr.openjdk.java.net/~rkennke/8198445/8198445-1.patch >>>>>> >>>>>> This wouldn't compile because of bunch of missing >>>>>> arraycopy_conjoint_atomic defintions for extra types like jfloat, >>>>>> jdouble, jboolean, etc, which in turn would be missing the same >>>>>> Copy::conjoint_jFluffs_atomic() which drags in a bunch of platform >>>>>> specific stuff... and my question before I go there is: do we want >>>>>> all >>>>>> that? Or can you think of a better way to solve it? >>>>>> >>>>>> Roman >>>>>> >>>>>>> Roman >>>>>>> >>>>>>>> The ARRAYCOPY_ATOMIC decorator makes the arraycopy atomic over the >>>>>>>> size >>>>>>>> of the passed in elements. >>>>>>>> In this case, it looks like the address has been type erased to >>>>>>>> void*, >>>>>>>> and hence lost what the element size was. There is currently no >>>>>>>> overload >>>>>>>> accepted for type erased element - only accurate elements {jbyte, >>>>>>>> jshort, jint, jlong}. >>>>>>>> >>>>>>>> So it looks like an overload must be added to accept type erased >>>>>>>> void* >>>>>>>> elements and make that call conjoint_memory_atomic when the >>>>>>>> ARRAYCOPY_ATOMIC decorator is passed in. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> /Erik >>>>>>>> >>>>>>>> On 2018-03-06 13:56, David Holmes wrote: >>>>>>>>> On 6/03/2018 10:54 PM, Erik ?sterlund wrote: >>>>>>>>>> Hi David, >>>>>>>>>> >>>>>>>>>> It is atomic with the ARRAYCOPY_ATOMIC decorator. If that >>>>>>>>>> comment is >>>>>>>>>> correct (I do not know if it is), then the ARRAYCOPY_ATOMIC >>>>>>>>>> decorator >>>>>>>>>> should probably be used here. >>>>>>>>> If that code implements a Java array copy then yes it is >>>>>>>>> required to >>>>>>>>> be 32-bit atomic. Do you need the decorator to get 32-bit >>>>>>>>> atomicity? >>>>>>>>> >>>>>>>>> David >>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> /Erik >>>>>>>>>> >>>>>>>>>> On 2018-03-06 13:48, David Holmes wrote: >>>>>>>>>>> Hi Roman, >>>>>>>>>>> >>>>>>>>>>> Not a review as I'm not familiar enough with the Access API, >>>>>>>>>>> but in >>>>>>>>>>> src/hotspot/share/c1/c1_Runtime1.cpp the comments above the >>>>>>>>>>> changed >>>>>>>>>>> code need updating - probably deleting. I assume the Access API >>>>>>>>>>> arraycopy is atomic? >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> David >>>>>>>>>>> >>>>>>>>>>> On 6/03/2018 9:56 PM, Roman Kennke wrote: >>>>>>>>>>>> Currently, the Access API is only used for oop-arraycopy, but >>>>>>>>>>>> not for >>>>>>>>>>>> primitive arraycopy. GCs might want to intercept this too, e.g. >>>>>>>>>>>> resolve >>>>>>>>>>>> src and dst arrays. >>>>>>>>>>>> >>>>>>>>>>>> There *is* an implementation of primitive arraycopy in the >>>>>>>>>>>> Access API, >>>>>>>>>>>> but it doesn't even compile, because Raw::arraycopy() does not >>>>>>>>>>>> take >>>>>>>>>>>> src >>>>>>>>>>>> and dst oop operands, but it's called like that. The only >>>>>>>>>>>> reason >>>>>>>>>>>> why >>>>>>>>>>>> this does not blow up (I think) is that because nobody calls >>>>>>>>>>>> it, >>>>>>>>>>>> the >>>>>>>>>>>> compiler doesn't even get there. >>>>>>>>>>>> >>>>>>>>>>>> This change fixes the Access API/impl and adds the relevant >>>>>>>>>>>> calls into >>>>>>>>>>>> it (in C1 and runtime land). C2 uses arraycopy stubs (which >>>>>>>>>>>> cannot be >>>>>>>>>>>> handled here) or calls out to the ArrayKlass::copy_array(), >>>>>>>>>>>> which >>>>>>>>>>>> should >>>>>>>>>>>> be covered with this change. >>>>>>>>>>>> >>>>>>>>>>>> It should be possible to use the same Access API for >>>>>>>>>>>> Java-array <-> >>>>>>>>>>>> native-array bulk transfers, which currently use the rather >>>>>>>>>>>> ugly >>>>>>>>>>>> typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address that >>>>>>>>>>>> in a >>>>>>>>>>>> separate change though. >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> Tests: tier1 ok >>>>>>>>>>>> >>>>>>>>>>>> Please review! >>>>>>>>>>>> Thanks, Roman >>>>>>>>>>>> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From martin.doerr at sap.com Tue Mar 13 10:33:55 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 13 Mar 2018 10:33:55 +0000 Subject: RFR(S): 8198510: Enable UseDynamicNumberOfGCThreads by default In-Reply-To: <23a06a29-d8c1-3f85-06be-93a8ca5380a7@oracle.com> References: <96da428b596e4a729612b32415d2ed11@sap.com> <23a06a29-d8c1-3f85-06be-93a8ca5380a7@oracle.com> Message-ID: Hi, thank you very much for testing, reviewing and approving the CSR. I've pushed it. Best regards, Martin From: hotspot-gc-dev [mailto:hotspot-gc-dev-bounces at openjdk.java.net] On Behalf Of Stefan Johansson Sent: Mittwoch, 7. M?rz 2018 10:29 To: hotspot-gc-dev at openjdk.java.net Subject: Re: RFR(S): 8198510: Enable UseDynamicNumberOfGCThreads by default Hi Martin, On 2018-03-06 15:37, Doerr, Martin wrote: Hi, I'd like to solve the problem that an unnecessarily high number of GC threads get started in some situations. This wastes resources. The proposal is to enable the existing flag UseDynamicNumberOfGCThreads by default. Bug: https://bugs.openjdk.java.net/browse/JDK-8198510 CSR: https://bugs.openjdk.java.net/browse/JDK-8198547 Webrev: http://cr.openjdk.java.net/~mdoerr/8198510_gc_threads/webrev.01/ Change looks good and I also reviewed the CSR. So feel free to move it to proposed (or finalized if you want to fast-track it). Thanks, Stefan Please review. Thanks and best regards, Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From per.liden at oracle.com Tue Mar 13 10:48:04 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 13 Mar 2018 11:48:04 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: <0032c706-3171-1cee-20b0-baf67b438f31@redhat.com> References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> <5A9E8F76.1050809@oracle.com> <83de65bd-5d3f-78df-02d2-eb4c53aa0c62@oracle.com> <5A9E95D8.6080602@oracle.com> <2cc9a407-aba8-209a-7ef7-4ab26b88fef6@redhat.com> <6d8435af-5bec-da70-fa8a-60d708937f44@redhat.com> <5A9EBD6E.3070005@oracle.com> <2612309c-868b-6094-5944-e9de54daf8d2@redhat.com> <5AA136E9.9080100@oracle.com> <3034e0b7-9894-80e6-afd4-171e31ad5836@redhat.com> <5AA29BEB.3020402@oracle.com> <0032c706-3171-1cee-20b0-baf67b438f31@redhat.com> Message-ID: Looks good to me. (Erik and I discussed removing Runtime1::arraycopy() completely, and let those few platforms that doesn't implement this in C1 fall back to the VM implementation instead, but let's take that as a separate RFE). /Per On 03/13/2018 11:02 AM, Roman Kennke wrote: > Hi Erik, > > thanks for doing this! It looks good to me! > > I'm putting this up here again, with commit msg fixed: > http://cr.openjdk.java.net/~rkennke/8198445/webrev.03/ > > Can I get (final?) reviews? And then I'd put it through the new submit > repo and push it? > > Thanks, Roman > > >> Hi Roman, >> >> I would love to avoid using the switch to enumerate all different Java >> types you could possibly have. >> How about this? (was easier to cook something together than explain in >> detail what I had in mind) >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8198445/webrev.00/ >> >> The idea is to let Access recognize arraycopy with element type "void", >> and then perform a conservative byte arraycopy, that when >> ARRAYCOPY_ATOMIC uses the natural atomicity of the element alignment by >> using Copy::conjoint_memory_atomic(). >> >> Thanks, >> /Erik >> >> On 2018-03-08 21:12, Roman Kennke wrote: >>> Am 08.03.2018 um 14:13 schrieb Erik ?sterlund: >>>> Hi Roman, >>>> >>>> On 2018-03-08 11:25, Roman Kennke wrote: >>>>> Wouldn't that lead to mis-aligned accesses when you try to copy an >>>>> array >>>>> of, e.g., bytes, but shoehorn it to pointer-size? Or am I >>>>> misunderstanding? >>>> No it would not. The element type is void, not void*. Because the >>>> element type is type erased. And that would map to >>>> Copy::conjoint_memory_atomic, which is precisely what typeArrayKlass.cpp >>>> maps such an arraycopy to today. Assuming elements are aligned to their >>>> natural size (which they are), Copy::conjoint_memory_atomic is >>>> guaranteed to be at least as atomic as you need your elements to be, up >>>> to an element size of 8. >>> Ah, ok. Alright, so here comes rev.02: >>> >>> Differential: >>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.02.diff/ >>> Full: >>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.02/ >>> >>> Still passes tier1 tests. >>> >>> Ok now? >>> >>> Cheers, Roman >>> >>> >>>>> An idea would be to copy equal-sized elements using the corresponding >>>>> Copy::conjoint_XXX_atomic() calls, e.g. char and short would use the >>>>> short version, boolean and byte would use the byte version (or is >>>>> boolean sometimes int?!), float and int would use the int version and >>>>> long and double the long version. WDYT? >>>> I'm not sure I see what the advantage of that would be. >>>> >>>> Thanks, >>>> /Erik >>>> >>>>> Roman >>>>> >>>>> >>>>>> My spontaneous idea was to have an overload for type erased void* that >>>>>> when called with ARRAYCOPY_ATOMIC maps to Copy::conjoint_memory_atomic >>>>>> which AFAIK is conservatively atomic regardless of what the precise >>>>>> element type is. >>>>>> >>>>>> But if you have better ideas, I am open for suggestions. >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>>>> On 2018-03-06 17:03, Roman Kennke wrote: >>>>>>> Am 06.03.2018 um 14:35 schrieb Roman Kennke: >>>>>>>> Makes me wonder: why attempt to be smart in c1_Runtime1.cpp, when >>>>>>>> we can >>>>>>>> just as well call typeArrayOop::copy_array() and have it do the >>>>>>>> right >>>>>>>> thing? Or go even further and also do it for oop-arraycopy? >>>>>>> Something like: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~rkennke/8198445/8198445-1.patch >>>>>>> >>>>>>> This wouldn't compile because of bunch of missing >>>>>>> arraycopy_conjoint_atomic defintions for extra types like jfloat, >>>>>>> jdouble, jboolean, etc, which in turn would be missing the same >>>>>>> Copy::conjoint_jFluffs_atomic() which drags in a bunch of platform >>>>>>> specific stuff... and my question before I go there is: do we want >>>>>>> all >>>>>>> that? Or can you think of a better way to solve it? >>>>>>> >>>>>>> Roman >>>>>>> >>>>>>>> Roman >>>>>>>> >>>>>>>>> The ARRAYCOPY_ATOMIC decorator makes the arraycopy atomic over the >>>>>>>>> size >>>>>>>>> of the passed in elements. >>>>>>>>> In this case, it looks like the address has been type erased to >>>>>>>>> void*, >>>>>>>>> and hence lost what the element size was. There is currently no >>>>>>>>> overload >>>>>>>>> accepted for type erased element - only accurate elements {jbyte, >>>>>>>>> jshort, jint, jlong}. >>>>>>>>> >>>>>>>>> So it looks like an overload must be added to accept type erased >>>>>>>>> void* >>>>>>>>> elements and make that call conjoint_memory_atomic when the >>>>>>>>> ARRAYCOPY_ATOMIC decorator is passed in. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> /Erik >>>>>>>>> >>>>>>>>> On 2018-03-06 13:56, David Holmes wrote: >>>>>>>>>> On 6/03/2018 10:54 PM, Erik ?sterlund wrote: >>>>>>>>>>> Hi David, >>>>>>>>>>> >>>>>>>>>>> It is atomic with the ARRAYCOPY_ATOMIC decorator. If that >>>>>>>>>>> comment is >>>>>>>>>>> correct (I do not know if it is), then the ARRAYCOPY_ATOMIC >>>>>>>>>>> decorator >>>>>>>>>>> should probably be used here. >>>>>>>>>> If that code implements a Java array copy then yes it is >>>>>>>>>> required to >>>>>>>>>> be 32-bit atomic. Do you need the decorator to get 32-bit >>>>>>>>>> atomicity? >>>>>>>>>> >>>>>>>>>> David >>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> /Erik >>>>>>>>>>> >>>>>>>>>>> On 2018-03-06 13:48, David Holmes wrote: >>>>>>>>>>>> Hi Roman, >>>>>>>>>>>> >>>>>>>>>>>> Not a review as I'm not familiar enough with the Access API, >>>>>>>>>>>> but in >>>>>>>>>>>> src/hotspot/share/c1/c1_Runtime1.cpp the comments above the >>>>>>>>>>>> changed >>>>>>>>>>>> code need updating - probably deleting. I assume the Access API >>>>>>>>>>>> arraycopy is atomic? >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> David >>>>>>>>>>>> >>>>>>>>>>>> On 6/03/2018 9:56 PM, Roman Kennke wrote: >>>>>>>>>>>>> Currently, the Access API is only used for oop-arraycopy, but >>>>>>>>>>>>> not for >>>>>>>>>>>>> primitive arraycopy. GCs might want to intercept this too, e.g. >>>>>>>>>>>>> resolve >>>>>>>>>>>>> src and dst arrays. >>>>>>>>>>>>> >>>>>>>>>>>>> There *is* an implementation of primitive arraycopy in the >>>>>>>>>>>>> Access API, >>>>>>>>>>>>> but it doesn't even compile, because Raw::arraycopy() does not >>>>>>>>>>>>> take >>>>>>>>>>>>> src >>>>>>>>>>>>> and dst oop operands, but it's called like that. The only >>>>>>>>>>>>> reason >>>>>>>>>>>>> why >>>>>>>>>>>>> this does not blow up (I think) is that because nobody calls >>>>>>>>>>>>> it, >>>>>>>>>>>>> the >>>>>>>>>>>>> compiler doesn't even get there. >>>>>>>>>>>>> >>>>>>>>>>>>> This change fixes the Access API/impl and adds the relevant >>>>>>>>>>>>> calls into >>>>>>>>>>>>> it (in C1 and runtime land). C2 uses arraycopy stubs (which >>>>>>>>>>>>> cannot be >>>>>>>>>>>>> handled here) or calls out to the ArrayKlass::copy_array(), >>>>>>>>>>>>> which >>>>>>>>>>>>> should >>>>>>>>>>>>> be covered with this change. >>>>>>>>>>>>> >>>>>>>>>>>>> It should be possible to use the same Access API for >>>>>>>>>>>>> Java-array <-> >>>>>>>>>>>>> native-array bulk transfers, which currently use the rather >>>>>>>>>>>>> ugly >>>>>>>>>>>>> typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address that >>>>>>>>>>>>> in a >>>>>>>>>>>>> separate change though. >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ >>>>>>>>>>>>> >>>>>>>>>>>>> Tests: tier1 ok >>>>>>>>>>>>> >>>>>>>>>>>>> Please review! >>>>>>>>>>>>> Thanks, Roman >>>>>>>>>>>>> >>> >> > > From erik.osterlund at oracle.com Tue Mar 13 10:52:08 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 13 Mar 2018 11:52:08 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: <0032c706-3171-1cee-20b0-baf67b438f31@redhat.com> References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> <5A9E8F76.1050809@oracle.com> <83de65bd-5d3f-78df-02d2-eb4c53aa0c62@oracle.com> <5A9E95D8.6080602@oracle.com> <2cc9a407-aba8-209a-7ef7-4ab26b88fef6@redhat.com> <6d8435af-5bec-da70-fa8a-60d708937f44@redhat.com> <5A9EBD6E.3070005@oracle.com> <2612309c-868b-6094-5944-e9de54daf8d2@redhat.com> <5AA136E9.9080100@oracle.com> <3034e0b7-9894-80e6-afd4-171e31ad5836@redhat.com> <5AA29BEB.3020402@oracle.com> <0032c706-3171-1cee-20b0-baf67b438f31@redhat.com> Message-ID: Hi Roman, Ideally I'd like to get rid of Runtime1::arraycopy. It seems like a medium slow path of C1 never used by any platforms other than S390 and PPC, which look like the only platforms that lack a generic arraycopy stub from the StubRoutines. I think that S390 and PPC could easily just jump to the C1 arraycopy stub instead and take the slowpath to the normal native arraycopy that enters through the JNI interface instead. If the performance of this C1 arraycopy path on S390 and PPC are crucial, then they should probably implement stubs for it like the other platforms. If the performance of this is not crucial (I suspect this is the case), then it should not harm to take the normal entry through JNI for generic arraycopy using C1. But I think trying to get rid of that path could be a follow-up RFE instead. Looks good. Thanks, /Erik On 2018-03-13 11:02, Roman Kennke wrote: > Hi Erik, > > thanks for doing this! It looks good to me! > > I'm putting this up here again, with commit msg fixed: > http://cr.openjdk.java.net/~rkennke/8198445/webrev.03/ > > Can I get (final?) reviews? And then I'd put it through the new submit > repo and push it? > > Thanks, Roman > > >> Hi Roman, >> >> I would love to avoid using the switch to enumerate all different Java >> types you could possibly have. >> How about this? (was easier to cook something together than explain in >> detail what I had in mind) >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8198445/webrev.00/ >> >> The idea is to let Access recognize arraycopy with element type "void", >> and then perform a conservative byte arraycopy, that when >> ARRAYCOPY_ATOMIC uses the natural atomicity of the element alignment by >> using Copy::conjoint_memory_atomic(). >> >> Thanks, >> /Erik >> >> On 2018-03-08 21:12, Roman Kennke wrote: >>> Am 08.03.2018 um 14:13 schrieb Erik ?sterlund: >>>> Hi Roman, >>>> >>>> On 2018-03-08 11:25, Roman Kennke wrote: >>>>> Wouldn't that lead to mis-aligned accesses when you try to copy an >>>>> array >>>>> of, e.g., bytes, but shoehorn it to pointer-size? Or am I >>>>> misunderstanding? >>>> No it would not. The element type is void, not void*. Because the >>>> element type is type erased. And that would map to >>>> Copy::conjoint_memory_atomic, which is precisely what typeArrayKlass.cpp >>>> maps such an arraycopy to today. Assuming elements are aligned to their >>>> natural size (which they are), Copy::conjoint_memory_atomic is >>>> guaranteed to be at least as atomic as you need your elements to be, up >>>> to an element size of 8. >>> Ah, ok. Alright, so here comes rev.02: >>> >>> Differential: >>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.02.diff/ >>> Full: >>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.02/ >>> >>> Still passes tier1 tests. >>> >>> Ok now? >>> >>> Cheers, Roman >>> >>> >>>>> An idea would be to copy equal-sized elements using the corresponding >>>>> Copy::conjoint_XXX_atomic() calls, e.g. char and short would use the >>>>> short version, boolean and byte would use the byte version (or is >>>>> boolean sometimes int?!), float and int would use the int version and >>>>> long and double the long version. WDYT? >>>> I'm not sure I see what the advantage of that would be. >>>> >>>> Thanks, >>>> /Erik >>>> >>>>> Roman >>>>> >>>>> >>>>>> My spontaneous idea was to have an overload for type erased void* that >>>>>> when called with ARRAYCOPY_ATOMIC maps to Copy::conjoint_memory_atomic >>>>>> which AFAIK is conservatively atomic regardless of what the precise >>>>>> element type is. >>>>>> >>>>>> But if you have better ideas, I am open for suggestions. >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>>>> On 2018-03-06 17:03, Roman Kennke wrote: >>>>>>> Am 06.03.2018 um 14:35 schrieb Roman Kennke: >>>>>>>> Makes me wonder: why attempt to be smart in c1_Runtime1.cpp, when >>>>>>>> we can >>>>>>>> just as well call typeArrayOop::copy_array() and have it do the >>>>>>>> right >>>>>>>> thing? Or go even further and also do it for oop-arraycopy? >>>>>>> Something like: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~rkennke/8198445/8198445-1.patch >>>>>>> >>>>>>> This wouldn't compile because of bunch of missing >>>>>>> arraycopy_conjoint_atomic defintions for extra types like jfloat, >>>>>>> jdouble, jboolean, etc, which in turn would be missing the same >>>>>>> Copy::conjoint_jFluffs_atomic() which drags in a bunch of platform >>>>>>> specific stuff... and my question before I go there is: do we want >>>>>>> all >>>>>>> that? Or can you think of a better way to solve it? >>>>>>> >>>>>>> Roman >>>>>>> >>>>>>>> Roman >>>>>>>> >>>>>>>>> The ARRAYCOPY_ATOMIC decorator makes the arraycopy atomic over the >>>>>>>>> size >>>>>>>>> of the passed in elements. >>>>>>>>> In this case, it looks like the address has been type erased to >>>>>>>>> void*, >>>>>>>>> and hence lost what the element size was. There is currently no >>>>>>>>> overload >>>>>>>>> accepted for type erased element - only accurate elements {jbyte, >>>>>>>>> jshort, jint, jlong}. >>>>>>>>> >>>>>>>>> So it looks like an overload must be added to accept type erased >>>>>>>>> void* >>>>>>>>> elements and make that call conjoint_memory_atomic when the >>>>>>>>> ARRAYCOPY_ATOMIC decorator is passed in. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> /Erik >>>>>>>>> >>>>>>>>> On 2018-03-06 13:56, David Holmes wrote: >>>>>>>>>> On 6/03/2018 10:54 PM, Erik ?sterlund wrote: >>>>>>>>>>> Hi David, >>>>>>>>>>> >>>>>>>>>>> It is atomic with the ARRAYCOPY_ATOMIC decorator. If that >>>>>>>>>>> comment is >>>>>>>>>>> correct (I do not know if it is), then the ARRAYCOPY_ATOMIC >>>>>>>>>>> decorator >>>>>>>>>>> should probably be used here. >>>>>>>>>> If that code implements a Java array copy then yes it is >>>>>>>>>> required to >>>>>>>>>> be 32-bit atomic. Do you need the decorator to get 32-bit >>>>>>>>>> atomicity? >>>>>>>>>> >>>>>>>>>> David >>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> /Erik >>>>>>>>>>> >>>>>>>>>>> On 2018-03-06 13:48, David Holmes wrote: >>>>>>>>>>>> Hi Roman, >>>>>>>>>>>> >>>>>>>>>>>> Not a review as I'm not familiar enough with the Access API, >>>>>>>>>>>> but in >>>>>>>>>>>> src/hotspot/share/c1/c1_Runtime1.cpp the comments above the >>>>>>>>>>>> changed >>>>>>>>>>>> code need updating - probably deleting. I assume the Access API >>>>>>>>>>>> arraycopy is atomic? >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> David >>>>>>>>>>>> >>>>>>>>>>>> On 6/03/2018 9:56 PM, Roman Kennke wrote: >>>>>>>>>>>>> Currently, the Access API is only used for oop-arraycopy, but >>>>>>>>>>>>> not for >>>>>>>>>>>>> primitive arraycopy. GCs might want to intercept this too, e.g. >>>>>>>>>>>>> resolve >>>>>>>>>>>>> src and dst arrays. >>>>>>>>>>>>> >>>>>>>>>>>>> There *is* an implementation of primitive arraycopy in the >>>>>>>>>>>>> Access API, >>>>>>>>>>>>> but it doesn't even compile, because Raw::arraycopy() does not >>>>>>>>>>>>> take >>>>>>>>>>>>> src >>>>>>>>>>>>> and dst oop operands, but it's called like that. The only >>>>>>>>>>>>> reason >>>>>>>>>>>>> why >>>>>>>>>>>>> this does not blow up (I think) is that because nobody calls >>>>>>>>>>>>> it, >>>>>>>>>>>>> the >>>>>>>>>>>>> compiler doesn't even get there. >>>>>>>>>>>>> >>>>>>>>>>>>> This change fixes the Access API/impl and adds the relevant >>>>>>>>>>>>> calls into >>>>>>>>>>>>> it (in C1 and runtime land). C2 uses arraycopy stubs (which >>>>>>>>>>>>> cannot be >>>>>>>>>>>>> handled here) or calls out to the ArrayKlass::copy_array(), >>>>>>>>>>>>> which >>>>>>>>>>>>> should >>>>>>>>>>>>> be covered with this change. >>>>>>>>>>>>> >>>>>>>>>>>>> It should be possible to use the same Access API for >>>>>>>>>>>>> Java-array <-> >>>>>>>>>>>>> native-array bulk transfers, which currently use the rather >>>>>>>>>>>>> ugly >>>>>>>>>>>>> typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address that >>>>>>>>>>>>> in a >>>>>>>>>>>>> separate change though. >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ >>>>>>>>>>>>> >>>>>>>>>>>>> Tests: tier1 ok >>>>>>>>>>>>> >>>>>>>>>>>>> Please review! >>>>>>>>>>>>> Thanks, Roman >>>>>>>>>>>>> > From rkennke at redhat.com Tue Mar 13 11:02:28 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 13 Mar 2018 12:02:28 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> <5A9E8F76.1050809@oracle.com> <83de65bd-5d3f-78df-02d2-eb4c53aa0c62@oracle.com> <5A9E95D8.6080602@oracle.com> <2cc9a407-aba8-209a-7ef7-4ab26b88fef6@redhat.com> <6d8435af-5bec-da70-fa8a-60d708937f44@redhat.com> <5A9EBD6E.3070005@oracle.com> <2612309c-868b-6094-5944-e9de54daf8d2@redhat.com> <5AA136E9.9080100@oracle.com> <3034e0b7-9894-80e6-afd4-171e31ad5836@redhat.com> <5AA29BEB.3020402@oracle.com> <0032c706-3171-1cee-20b0-baf67b438f31@redhat.com> Message-ID: <39a6ae41-9736-ea0a-99ef-af3e32c772b8@redhat.com> Hi Erik & Per, yes, I agree with getting rid of Runtime1::arraycopy(). Let's take it to another RFE. Let's try the newfangled submit repo and push (mostly) my first patch :-) Thanks for reviewing! Cheer, Roman > Hi Roman, > > Ideally I'd like to get rid of Runtime1::arraycopy. It seems like a > medium slow path of C1 never used by any platforms other than S390 and > PPC, which look like the only platforms that lack a generic arraycopy > stub from the StubRoutines. I think that S390 and PPC could easily just > jump to the C1 arraycopy stub instead and take the slowpath to the > normal native arraycopy that enters through the JNI interface instead. > > If the performance of this C1 arraycopy path on S390 and PPC are > crucial, then they should probably implement stubs for it like the other > platforms. If the performance of this is not crucial (I suspect this is > the case), then it should not harm to take the normal entry through JNI > for generic arraycopy using C1. > > But I think trying to get rid of that path could be a follow-up RFE > instead. > > Looks good. > > Thanks, > /Erik > > On 2018-03-13 11:02, Roman Kennke wrote: >> Hi Erik, >> >> thanks for doing this! It looks good to me! >> >> I'm putting this up here again, with commit msg fixed: >> http://cr.openjdk.java.net/~rkennke/8198445/webrev.03/ >> >> Can I get (final?) reviews? And then I'd put it through the new submit >> repo and push it? >> >> Thanks, Roman >> >> >>> Hi Roman, >>> >>> I would love to avoid using the switch to enumerate all different Java >>> types you could possibly have. >>> How about this? (was easier to cook something together than explain in >>> detail what I had in mind) >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8198445/webrev.00/ >>> >>> The idea is to let Access recognize arraycopy with element type "void", >>> and then perform a conservative byte arraycopy, that when >>> ARRAYCOPY_ATOMIC uses the natural atomicity of the element alignment by >>> using Copy::conjoint_memory_atomic(). >>> >>> Thanks, >>> /Erik >>> >>> On 2018-03-08 21:12, Roman Kennke wrote: >>>> Am 08.03.2018 um 14:13 schrieb Erik ?sterlund: >>>>> Hi Roman, >>>>> >>>>> On 2018-03-08 11:25, Roman Kennke wrote: >>>>>> Wouldn't that lead to mis-aligned accesses when you try to copy an >>>>>> array >>>>>> of, e.g., bytes, but shoehorn it to pointer-size? Or am I >>>>>> misunderstanding? >>>>> No it would not. The element type is void, not void*. Because the >>>>> element type is type erased. And that would map to >>>>> Copy::conjoint_memory_atomic, which is precisely what >>>>> typeArrayKlass.cpp >>>>> maps such an arraycopy to today. Assuming elements are aligned to >>>>> their >>>>> natural size (which they are), Copy::conjoint_memory_atomic is >>>>> guaranteed to be at least as atomic as you need your elements to >>>>> be, up >>>>> to an element size of 8. >>>> Ah, ok. Alright, so here comes rev.02: >>>> >>>> Differential: >>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.02.diff/ >>>> Full: >>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.02/ >>>> >>>> Still passes tier1 tests. >>>> >>>> Ok now? >>>> >>>> Cheers, Roman >>>> >>>> >>>>>> An idea would be to copy equal-sized elements using the corresponding >>>>>> Copy::conjoint_XXX_atomic() calls, e.g. char and short would use the >>>>>> short version, boolean and byte would use the byte version (or is >>>>>> boolean sometimes int?!), float and int would use the int version and >>>>>> long and double the long version. WDYT? >>>>> I'm not sure I see what the advantage of that would be. >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>>> Roman >>>>>> >>>>>> >>>>>>> My spontaneous idea was to have an overload for type erased void* >>>>>>> that >>>>>>> when called with ARRAYCOPY_ATOMIC maps to >>>>>>> Copy::conjoint_memory_atomic >>>>>>> which AFAIK is conservatively atomic regardless of what the precise >>>>>>> element type is. >>>>>>> >>>>>>> But if you have better ideas, I am open for suggestions. >>>>>>> >>>>>>> Thanks, >>>>>>> /Erik >>>>>>> >>>>>>> On 2018-03-06 17:03, Roman Kennke wrote: >>>>>>>> Am 06.03.2018 um 14:35 schrieb Roman Kennke: >>>>>>>>> Makes me wonder: why attempt to be smart in c1_Runtime1.cpp, when >>>>>>>>> we can >>>>>>>>> just as well call typeArrayOop::copy_array() and have it do the >>>>>>>>> right >>>>>>>>> thing? Or go even further and also do it for oop-arraycopy? >>>>>>>> Something like: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~rkennke/8198445/8198445-1.patch >>>>>>>> >>>>>>>> This wouldn't compile because of bunch of missing >>>>>>>> arraycopy_conjoint_atomic defintions for extra types like jfloat, >>>>>>>> jdouble, jboolean, etc, which in turn would be missing the same >>>>>>>> Copy::conjoint_jFluffs_atomic() which drags in a bunch of platform >>>>>>>> specific stuff... and my question before I go there is: do we want >>>>>>>> all >>>>>>>> that? Or can you think of a better way to solve it? >>>>>>>> >>>>>>>> Roman >>>>>>>> >>>>>>>>> Roman >>>>>>>>> >>>>>>>>>> The ARRAYCOPY_ATOMIC decorator makes the arraycopy atomic over >>>>>>>>>> the >>>>>>>>>> size >>>>>>>>>> of the passed in elements. >>>>>>>>>> In this case, it looks like the address has been type erased to >>>>>>>>>> void*, >>>>>>>>>> and hence lost what the element size was. There is currently no >>>>>>>>>> overload >>>>>>>>>> accepted for type erased element - only accurate elements {jbyte, >>>>>>>>>> jshort, jint, jlong}. >>>>>>>>>> >>>>>>>>>> So it looks like an overload must be added to accept type erased >>>>>>>>>> void* >>>>>>>>>> elements and make that call conjoint_memory_atomic when the >>>>>>>>>> ARRAYCOPY_ATOMIC decorator is passed in. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> /Erik >>>>>>>>>> >>>>>>>>>> On 2018-03-06 13:56, David Holmes wrote: >>>>>>>>>>> On 6/03/2018 10:54 PM, Erik ?sterlund wrote: >>>>>>>>>>>> Hi David, >>>>>>>>>>>> >>>>>>>>>>>> It is atomic with the ARRAYCOPY_ATOMIC decorator. If that >>>>>>>>>>>> comment is >>>>>>>>>>>> correct (I do not know if it is), then the ARRAYCOPY_ATOMIC >>>>>>>>>>>> decorator >>>>>>>>>>>> should probably be used here. >>>>>>>>>>> If that code implements a Java array copy then yes it is >>>>>>>>>>> required to >>>>>>>>>>> be 32-bit atomic. Do you need the decorator to get 32-bit >>>>>>>>>>> atomicity? >>>>>>>>>>> >>>>>>>>>>> David >>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> /Erik >>>>>>>>>>>> >>>>>>>>>>>> On 2018-03-06 13:48, David Holmes wrote: >>>>>>>>>>>>> Hi Roman, >>>>>>>>>>>>> >>>>>>>>>>>>> Not a review as I'm not familiar enough with the Access API, >>>>>>>>>>>>> but in >>>>>>>>>>>>> src/hotspot/share/c1/c1_Runtime1.cpp the comments above the >>>>>>>>>>>>> changed >>>>>>>>>>>>> code need updating - probably deleting. I assume the Access >>>>>>>>>>>>> API >>>>>>>>>>>>> arraycopy is atomic? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> David >>>>>>>>>>>>> >>>>>>>>>>>>> On 6/03/2018 9:56 PM, Roman Kennke wrote: >>>>>>>>>>>>>> Currently, the Access API is only used for oop-arraycopy, but >>>>>>>>>>>>>> not for >>>>>>>>>>>>>> primitive arraycopy. GCs might want to intercept this too, >>>>>>>>>>>>>> e.g. >>>>>>>>>>>>>> resolve >>>>>>>>>>>>>> src and dst arrays. >>>>>>>>>>>>>> >>>>>>>>>>>>>> There *is* an implementation of primitive arraycopy in the >>>>>>>>>>>>>> Access API, >>>>>>>>>>>>>> but it doesn't even compile, because Raw::arraycopy() does >>>>>>>>>>>>>> not >>>>>>>>>>>>>> take >>>>>>>>>>>>>> src >>>>>>>>>>>>>> and dst oop operands, but it's called like that. The only >>>>>>>>>>>>>> reason >>>>>>>>>>>>>> why >>>>>>>>>>>>>> this does not blow up (I think) is that because nobody calls >>>>>>>>>>>>>> it, >>>>>>>>>>>>>> the >>>>>>>>>>>>>> compiler doesn't even get there. >>>>>>>>>>>>>> >>>>>>>>>>>>>> This change fixes the Access API/impl and adds the relevant >>>>>>>>>>>>>> calls into >>>>>>>>>>>>>> it (in C1 and runtime land). C2 uses arraycopy stubs (which >>>>>>>>>>>>>> cannot be >>>>>>>>>>>>>> handled here) or calls out to the ArrayKlass::copy_array(), >>>>>>>>>>>>>> which >>>>>>>>>>>>>> should >>>>>>>>>>>>>> be covered with this change. >>>>>>>>>>>>>> >>>>>>>>>>>>>> It should be possible to use the same Access API for >>>>>>>>>>>>>> Java-array <-> >>>>>>>>>>>>>> native-array bulk transfers, which currently use the rather >>>>>>>>>>>>>> ugly >>>>>>>>>>>>>> typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address >>>>>>>>>>>>>> that >>>>>>>>>>>>>> in a >>>>>>>>>>>>>> separate change though. >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> Tests: tier1 ok >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please review! >>>>>>>>>>>>>> Thanks, Roman >>>>>>>>>>>>>> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Tue Mar 13 13:18:32 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 13 Mar 2018 14:18:32 +0100 Subject: RFR: 8199027: Make protected members private in G1Policy In-Reply-To: <01235743-3518-3d21-2a6b-8af7d5a7ef98@oracle.com> References: <01235743-3518-3d21-2a6b-8af7d5a7ef98@oracle.com> Message-ID: <1520947112.3100.17.camel@oracle.com> Hi, On Mon, 2018-03-12 at 14:07 +0100, Erik Helin wrote: > Hi all, > > this tiny patch removes the (unnecessary) use of `protected` > visibility > from the class G1Policy, the members that were protected can just be > private. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8199027 > > Patch: > http://cr.openjdk.java.net/~ehelin/8199027/00/ looks good. Thomas From thomas.schatzl at oracle.com Tue Mar 13 13:41:10 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 13 Mar 2018 14:41:10 +0100 Subject: RFR (XXXS): 8199516: Remove dead code overlooked during Full GC work Message-ID: <1520948470.3100.21.camel@oracle.com> Hi all, can I have one review for this trivial change that removes some dead code found during review of JDK-8199326 as remnants of Full GC restructuring? CR: https://bugs.openjdk.java.net/browse/JDK-8199516 Webrev: http://cr.openjdk.java.net/~tschatzl/8199516/webrev/ Testing: local compilation Thanks, Thomas From thomas.schatzl at oracle.com Tue Mar 13 13:42:13 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 13 Mar 2018 14:42:13 +0100 Subject: RFR (S): 8199326: Remove G1 gc time stamp logic In-Reply-To: References: <1520516013.3121.24.camel@oracle.com> <7b19df2a-cac8-41d7-c3e9-d2cf893f9d85@oracle.com> <1520523980.3121.29.camel@oracle.com> Message-ID: <1520948533.3100.22.camel@oracle.com> Hi, On Mon, 2018-03-12 at 16:48 +0100, Stefan Johansson wrote: > Hi Thomas, > [...] > > > > Sorry. The correct link is: > > http://cr.openjdk.java.net/~tschatzl/8199326/webrev > > Thanks for doing this cleanup, I found one more reference to the GC > timestamp in g1CollectedHeap.hpp: > 296 // Clear RSets after a compaction. It also resets the GC > time stamps. > 297 void clear_rsets_post_compaction(); > > You can actually remove those two lines completely if you like. I > forgot the declaration when removing this method in the fix for: > 8177544: Restructure G1 Full GC code > > Otherwise the change looks good and if you don't want to remove the > declaration in this patch we can create a separate clean up for this. > thanks for your review. I posted a webrev for the removal of this code as imho it does not completely fit here. Thanks, Thomas From thomas.schatzl at oracle.com Tue Mar 13 13:52:33 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 13 Mar 2018 14:52:33 +0100 Subject: RFR [L][3/7]: 8197850: Calculate liveness in regions during marking In-Reply-To: <0cd6cca6-cace-4e4f-7f9c-152e20757dfe@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520260902.2532.66.camel@oracle.com> <1520343234.2378.29.camel@oracle.com> <8f7f25ef-f0c6-20f6-1584-b615dba234e1@oracle.com> <1520412521.2346.4.camel@oracle.com> <0cd6cca6-cace-4e4f-7f9c-152e20757dfe@oracle.com> Message-ID: <1520949153.3100.27.camel@oracle.com> Hi Stefan, On Wed, 2018-03-07 at 11:12 +0100, Stefan Johansson wrote: > Thanks Thomas, > > This looks good to me, reviewed. thanks for your review; unfortunately rebasing the changes to latest caused some additional work. In this particular case, contents of g1RegionMarkStatsCache had to be distributed to .cpp and .inline.hpp files (recent .inline.hpp include fixes), and I preemptively removed the use of one VALUE_OBJ_CLASS_SPEC. Webrevs: http://cr.openjdk.java.net/~tschatzl/8197850/webrev.1_to_2/index.html (diff) http://cr.openjdk.java.net/~tschatzl/8197850/webrev.2/index.html (full) Testing: hs-tier 1-5 Thanks, Thomas > > Stefan > > On 2018-03-07 09:48, Thomas Schatzl wrote: > > Hi, > > > > On Tue, 2018-03-06 at 15:03 +0100, Stefan Johansson wrote: > > > On 2018-03-06 14:33, Thomas Schatzl wrote: > > > > On Tue, 2018-03-06 at 13:31 +0100, Stefan Johansson wrote: > > > > > Hi Thomas, > > > > > > > > > > On 2018-03-05 15:41, Thomas Schatzl wrote: > > > > > > Hi all, > > > > > > > > > > > > ... > > > > > > > > > > > > CR: > > > > > > https://bugs.openjdk.java.net/browse/JDK-8197850 > > > > > > Webrev: > > > > > > http://cr.openjdk.java.net/~tschatzl/8197850/webrev/index.h > > > > > > tml > > > > > > > > > > Look good, just some minor things. > > > > > > > > > > src/hotspot/share/gc/g1/g1ConcurrentMark.cpp > > > > > 394 _region_mark_stats(NEW_C_HEAP_ARRAY(G1RegionMarkStats, > > > > > max_regions, mtGC)) > > > > > > > > > > max_regions is passed into the constructor but everywhere > > > > > else in > > > > > G1ConcurrentMark we use g1h->max_regions(). I would consider > > > > > either > > > > > calling g1h->max_regions() in the constructor too, or save > > > > > the > > > > > value > > > > > in a member and use it everywhere. > > > > > > > > Done using option 1), use g1h->max_regions() everywhere. > > > > > > Thanks. > > > > > --- > > > > > src/hotspot/share/gc/g1/g1_globals.hpp > > > > > 262 experimental(size_t, G1RegionMarkStatsCacheSize, > > > > > 1024, > > > > > ... > > > > > 265 range(128, (max_juint / 2) + 1) > > > > > > > > > > Do we need this flag or should we scale it against the number > > > > > of > > > > > regions? If we want to keep it I think we should have a > > > > > somewhat > > > > > more restrictive range. > > > > > > > > I would not want this to scale according to the number of heap > > > > regions, > > > > because that is going to have a significant impact on pause > > > > times > > > > when > > > > all of the thread's caches are evicted (The O(#regions) part). > > > > > > True, just a thought since 1024 is roughly half the number > > > regions > > > we aim for maybe this could be used together with a reasonable > > > upper > > > bound. > > > > > > > The flag has only been added (rather late, admittedly) to fix > > > > potential issues with marking time with really large heaps with > > > > tens of thousands of regions. > > > > Let me do some more measurements, I will get back to you with > > > > them > > > > before giving you a new webrev. > > > > > > Sound good. > > > > I did some measurements with 100k regions, and I think we can just > > remove the flag. There is always logging available (gc+mark+stats) > > that > > shows cache hit ratio to diagnose issues. > > > > > > > --- > > > > > src/hotspot/share/gc/g1/g1RegionMarkStatsCache.hpp > > > > > > > > > > In G1RegionMarkStatsCacheEntry _region_idx is set to 0 when > > > > > clearing. I don't see a problem with it right now, but it > > > > > feels a > > > > > bit wrong since index 0 is a valid region index. Maybe we > > > > > could > > > > > use another cleared marker. > > > > > > > > Unfortunately there is none (one could use something like -1, > > > > but > > > > that one is valid too), and it seems a waste of memory to > > > > reserve > > > > more space for it. > > > > > > > > I see your point that this is a bit ugly, but there should > > > > never be > > > > any issue because of the initialization of the value part of an > > > > entry with zero. It simply has no impact on totals if you add > > > > zero. > > > > > > > > The code could initialize the cache with region indices from 0 > > > > to > > > > cache size - 1, which are valid values given the hash function, > > > > what do you think? > > > > > > > > However that would slow down clearing a little (the compiler > > > > will > > > > just compile it to a memset() in the loop right now). > > > > > > You are correct that -1 is a valid index, but we seldom see heaps > > > with so many regions. I'm fine with leaving it as is, if you > > > dislike > > > using -1 and if it will slow down clearing. > > > > Just a bit. It seems as bad to use -1 or another random value, so I > > kept it. > > > > Webrevs: > > http://cr.openjdk.java.net/~tschatzl/8197850/webrev.0_to_1/index.ht > > ml > > (diff) > > http://cr.openjdk.java.net/~tschatzl/8197850/webrev.1/index.html > > (full) > > > > Testing: > > This change ran, with some changes to 8180415 due to internal > > comments > > (I will post in a few minutes) through hs-tier 1-5 over night. > > > > Thanks, > > Thomas > > From stefan.johansson at oracle.com Tue Mar 13 14:06:00 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 13 Mar 2018 15:06:00 +0100 Subject: RFR (XXXS): 8199516: Remove dead code overlooked during Full GC work In-Reply-To: <1520948470.3100.21.camel@oracle.com> References: <1520948470.3100.21.camel@oracle.com> Message-ID: Looks good, thanks for cleaning this up. Cheers, Stefan On 2018-03-13 14:41, Thomas Schatzl wrote: > Hi all, > > can I have one review for this trivial change that removes some dead > code found during review of JDK-8199326 as remnants of Full GC > restructuring? > > CR: > https://bugs.openjdk.java.net/browse/JDK-8199516 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8199516/webrev/ > Testing: > local compilation > > Thanks, > Thomas > From stefan.johansson at oracle.com Tue Mar 13 14:26:04 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 13 Mar 2018 15:26:04 +0100 Subject: RFR (S): 8199326: Remove G1 gc time stamp logic In-Reply-To: <1520948533.3100.22.camel@oracle.com> References: <1520516013.3121.24.camel@oracle.com> <7b19df2a-cac8-41d7-c3e9-d2cf893f9d85@oracle.com> <1520523980.3121.29.camel@oracle.com> <1520948533.3100.22.camel@oracle.com> Message-ID: <420307e0-ea36-70c8-1c9c-f98f9a732c16@oracle.com> On 2018-03-13 14:42, Thomas Schatzl wrote: > Hi, > > On Mon, 2018-03-12 at 16:48 +0100, Stefan Johansson wrote: >> Hi Thomas, >> [...] >>> Sorry. The correct link is: >>> http://cr.openjdk.java.net/~tschatzl/8199326/webrev >> Thanks for doing this cleanup, I found one more reference to the GC >> timestamp in g1CollectedHeap.hpp: >> 296 // Clear RSets after a compaction. It also resets the GC >> time stamps. >> 297 void clear_rsets_post_compaction(); >> >> You can actually remove those two lines completely if you like. I >> forgot the declaration when removing this method in the fix for: >> 8177544: Restructure G1 Full GC code >> >> Otherwise the change looks good and if you don't want to remove the >> declaration in this patch we can create a separate clean up for this. >> > thanks for your review. I posted a webrev for the removal of this > code as imho it does not completely fit here. I agree, then consider this fix reviewed. Thanks, Stefan > Thanks, > Thomas > From sangheon.kim at oracle.com Tue Mar 13 14:30:32 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Tue, 13 Mar 2018 07:30:32 -0700 Subject: RFR (XXXS): 8199516: Remove dead code overlooked during Full GC work In-Reply-To: <1520948470.3100.21.camel@oracle.com> References: <1520948470.3100.21.camel@oracle.com> Message-ID: <3e6d979c-4bce-f5cf-d465-17d742ee8ad7@oracle.com> Hi Thomas, Looks good to me too. Thanks, Sangheon On 03/13/2018 06:41 AM, Thomas Schatzl wrote: > Hi all, > > can I have one review for this trivial change that removes some dead > code found during review of JDK-8199326 as remnants of Full GC > restructuring? > > CR: > https://bugs.openjdk.java.net/browse/JDK-8199516 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8199516/webrev/ > Testing: > local compilation > > Thanks, > Thomas > From thomas.schatzl at oracle.com Tue Mar 13 14:54:51 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 13 Mar 2018 15:54:51 +0100 Subject: RFR [L][4/7]: 8180415: Rebuild remembered sets during the concurrent cycle In-Reply-To: <1520509760.3121.15.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520262478.2532.91.camel@oracle.com> <1520413176.2346.9.camel@oracle.com> <1f345c49-7793-7925-2c15-281f24d28e74@oracle.com> <1520509760.3121.15.camel@oracle.com> Message-ID: <1520952891.3100.36.camel@oracle.com> Hi all, while working on some follow-up changes and discussing with StefanJ, there some issues came up: - the amount of marked bytes for humongous objects has been calculated incorrectly, regardless of TAMS value it always considered humongous objects as below tams. There is no actual impact except in gc+liveness log messages in that even if the humongous object were allocated during marking, it looked like it had been allocated before marking. This has been found by the sanity check at the end of the rebuilding in later changes where it has been made to work with humongous regions too due to other changes. - StefanJ mentioned that gc+remset+tracking log messages should have a gc id. Added. New webrevs: http://cr.openjdk.java.net/~tschatzl/8180415/webrev.2_to_3 (diff) http://cr.openjdk.java.net/~tschatzl/8180415/webrev.3 (full) Thanks, Thomas On Thu, 2018-03-08 at 12:49 +0100, Thomas Schatzl wrote: > Hi Stefan, > > On Thu, 2018-03-08 at 10:15 +0100, Stefan Johansson wrote: > > Thanks for addressing my initial comments Thomas, > > > > On 2018-03-07 09:59, Thomas Schatzl wrote: > > > Hi all, > > > > > > Stefan had some comments about the HeapRegionType parameter in > > > the G1RemSetTrackingPolicy::update_at_allocate() method - that > > > one > > > is not really required when placing the call to this method > > > correctly and just using the HeapRegion's type directly. > > > > > > Changing this removes another 40 LOC of changes. > > > > > > There has been another bug introduced by me during cleaning up > > > for > > > final review: G1RemSetTrackingPolicy::update_at_free() is empty > > > in > > > this change, which is wrong for this change - in this change we > > > still do not free the remembered sets during the cleanup pause, > > > only in the Concurrent Cleanup phase. When doing this > > > concurrently, > > > an assert triggers when setting the remembered set state to empty > > > outside a safepoint. > > > The fix is to make the remembered set untracked during the > > > Cleanup > > > phase still. > > > > > > New webrevs: > > > http://cr.openjdk.java.net/~tschatzl/8180415/webrev.0_to_1 (diff) > > > http://cr.openjdk.java.net/~tschatzl/8180415/webrev.1 (full) > > > > The change looks good. Just some additional comments: > > src/hotspot/share/gc/g1/g1ConcurrentMark.cpp > > 1255 GCTraceTime(Debug, gc)("Complete Remembered Set > > Tracking"); > > ... > > 1291 GCTraceTime(Debug, gc)("Finalize Concurrent Mark > > Cleanup"); > > > > Please add the "phases" tag to the above logging to be more > > consistent with the remark and other pauses. > > --- > > src/hotspot/share/gc/g1/g1FullGCOopClosures.inline.hpp > > 52 template inline void > > G1AdjustClosure::adjust_pointer(T* > > p) > > > > In this method the comments should be updated since we no longer > > return anything. > > --- > > src/hotspot/share/gc/g1/g1RemSet.cpp > > 757 HeapRegion* start_region = hr- > > >humongous_start_region(); > > > > start_region is only used to get bottom and crate an oop, so maybe > > instead create the oop directly, like: > > oop humongous = oop(hr->humongous_start_region()->bottom()); > > > > 761 // A humongous object is live (with respect to the > > scanning) either > > 762 // a) it is marked on the bitmap as such > > 763 // b) its TARS is larger than nTAMS, i.e. has been > > allocated during marking. > > 764 if (_cm->next_mark_bitmap()->is_marked(start_region- > > > bottom()) || > > > > 765 (top_at_rebuild_start > hr- > > >next_top_at_mark_start())) > > { > > > > Break this check out to get something like: is_live(humongous, > > tars, > > ntams). > > Fixed. > > > There is also the marked_bytes counting in this method that makes > > the > > code a bit harder to follow. I've discussed this with Thomas > > offline > > and he has an idea of how to avoid this. > > As discussed I would like to postpone this for now as it is much > harder > than it looks and warrants its own CR (actually two or three). > > > Also in G1FullCollector::phase3_adjust_pointers() there is a > > GCTraceTime that needs updating, since we no longer rebuild > > remember > > sets. > > All done. > > New webrevs: > http://cr.openjdk.java.net/~tschatzl/8180415/webrev.1_to_2/ (diff) > http://cr.openjdk.java.net/~tschatzl/8180415/webrev.2/ (full) > > Thomas > From thomas.schatzl at oracle.com Tue Mar 13 15:03:21 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 13 Mar 2018 16:03:21 +0100 Subject: RFR [M][7/7]: 8197932: Better split work in rebuild remembered sets phase In-Reply-To: <1520265115.2532.126.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520265115.2532.126.camel@oracle.com> Message-ID: <1520953401.3100.41.camel@oracle.com> Hi all, On Mon, 2018-03-05 at 16:51 +0100, Thomas Schatzl wrote: > Hi all, > > (last one for now :)) (resend because I forgot to change the > subject) > > please have a look at this change that improves TTSP during the > rebuild remembered set phase. > > [...] the assert at the end of the rebuilding has an issue in this change, in particular if in the last iteration, we have yielded, and during that yield point a GC eagerly reclaimed the region, the assert at the end of the rebuilding fails. The correct fix is to update the wrong assert to re-check the TARS that is updated by the eager reclaim. Webrevs: http://cr.openjdk.java.net/~tschatzl/8197932/webrev.0_to_1/ (diff) http://cr.openjdk.java.net/~tschatzl/8197932/webrev.1/ (full) testing: hs-tier1-5 Thanks, Thomas From rkennke at redhat.com Tue Mar 13 15:40:24 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 13 Mar 2018 16:40:24 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: <39a6ae41-9736-ea0a-99ef-af3e32c772b8@redhat.com> References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> <5A9E8F76.1050809@oracle.com> <83de65bd-5d3f-78df-02d2-eb4c53aa0c62@oracle.com> <5A9E95D8.6080602@oracle.com> <2cc9a407-aba8-209a-7ef7-4ab26b88fef6@redhat.com> <6d8435af-5bec-da70-fa8a-60d708937f44@redhat.com> <5A9EBD6E.3070005@oracle.com> <2612309c-868b-6094-5944-e9de54daf8d2@redhat.com> <5AA136E9.9080100@oracle.com> <3034e0b7-9894-80e6-afd4-171e31ad5836@redhat.com> <5AA29BEB.3020402@oracle.com> <0032c706-3171-1cee-20b0-baf67b438f31@redhat.com> <39a6ae41-9736-ea0a-99ef-af3e32c772b8@redhat.com> Message-ID: The test came back with the below. Doesn't strike me as being related to this change. What to do now? Push it anyway? Re-submit the test? (If so, how?) Roman Build Details: 2018-03-13-1330406.roman.source 1 Failed Test Test Tier Platform Keywords Description Task java/lang/invoke/condy/CondyInterfaceWithOverpassMethods.java tier1 windows-x64 bug8186046 othervm testng Error: failed to clean up files after test task Mach5 Tasks Results Summary NA: 0 FAILED: 0 PASSED: 74 UNABLE_TO_RUN: 0 KILLED: 0 EXECUTED_WITH_FAILURE: 1 Test 1 Executed with failure jdk_open_test_jdk_tier1-windows-x64-66 Results: total: 1775, passed: 1774; failed: 1 > Hi Erik & Per, > > yes, I agree with getting rid of Runtime1::arraycopy(). Let's take it to > another RFE. > > Let's try the newfangled submit repo and push (mostly) my first patch :-) > > Thanks for reviewing! > Cheer, Roman > > >> Hi Roman, >> >> Ideally I'd like to get rid of Runtime1::arraycopy. It seems like a >> medium slow path of C1 never used by any platforms other than S390 and >> PPC, which look like the only platforms that lack a generic arraycopy >> stub from the StubRoutines. I think that S390 and PPC could easily just >> jump to the C1 arraycopy stub instead and take the slowpath to the >> normal native arraycopy that enters through the JNI interface instead. >> >> If the performance of this C1 arraycopy path on S390 and PPC are >> crucial, then they should probably implement stubs for it like the other >> platforms. If the performance of this is not crucial (I suspect this is >> the case), then it should not harm to take the normal entry through JNI >> for generic arraycopy using C1. >> >> But I think trying to get rid of that path could be a follow-up RFE >> instead. >> >> Looks good. >> >> Thanks, >> /Erik >> >> On 2018-03-13 11:02, Roman Kennke wrote: >>> Hi Erik, >>> >>> thanks for doing this! It looks good to me! >>> >>> I'm putting this up here again, with commit msg fixed: >>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.03/ >>> >>> Can I get (final?) reviews? And then I'd put it through the new submit >>> repo and push it? >>> >>> Thanks, Roman >>> >>> >>>> Hi Roman, >>>> >>>> I would love to avoid using the switch to enumerate all different Java >>>> types you could possibly have. >>>> How about this? (was easier to cook something together than explain in >>>> detail what I had in mind) >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8198445/webrev.00/ >>>> >>>> The idea is to let Access recognize arraycopy with element type "void", >>>> and then perform a conservative byte arraycopy, that when >>>> ARRAYCOPY_ATOMIC uses the natural atomicity of the element alignment by >>>> using Copy::conjoint_memory_atomic(). >>>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2018-03-08 21:12, Roman Kennke wrote: >>>>> Am 08.03.2018 um 14:13 schrieb Erik ?sterlund: >>>>>> Hi Roman, >>>>>> >>>>>> On 2018-03-08 11:25, Roman Kennke wrote: >>>>>>> Wouldn't that lead to mis-aligned accesses when you try to copy an >>>>>>> array >>>>>>> of, e.g., bytes, but shoehorn it to pointer-size? Or am I >>>>>>> misunderstanding? >>>>>> No it would not. The element type is void, not void*. Because the >>>>>> element type is type erased. And that would map to >>>>>> Copy::conjoint_memory_atomic, which is precisely what >>>>>> typeArrayKlass.cpp >>>>>> maps such an arraycopy to today. Assuming elements are aligned to >>>>>> their >>>>>> natural size (which they are), Copy::conjoint_memory_atomic is >>>>>> guaranteed to be at least as atomic as you need your elements to >>>>>> be, up >>>>>> to an element size of 8. >>>>> Ah, ok. Alright, so here comes rev.02: >>>>> >>>>> Differential: >>>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.02.diff/ >>>>> Full: >>>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.02/ >>>>> >>>>> Still passes tier1 tests. >>>>> >>>>> Ok now? >>>>> >>>>> Cheers, Roman >>>>> >>>>> >>>>>>> An idea would be to copy equal-sized elements using the corresponding >>>>>>> Copy::conjoint_XXX_atomic() calls, e.g. char and short would use the >>>>>>> short version, boolean and byte would use the byte version (or is >>>>>>> boolean sometimes int?!), float and int would use the int version and >>>>>>> long and double the long version. WDYT? >>>>>> I'm not sure I see what the advantage of that would be. >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>>>>> Roman >>>>>>> >>>>>>> >>>>>>>> My spontaneous idea was to have an overload for type erased void* >>>>>>>> that >>>>>>>> when called with ARRAYCOPY_ATOMIC maps to >>>>>>>> Copy::conjoint_memory_atomic >>>>>>>> which AFAIK is conservatively atomic regardless of what the precise >>>>>>>> element type is. >>>>>>>> >>>>>>>> But if you have better ideas, I am open for suggestions. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> /Erik >>>>>>>> >>>>>>>> On 2018-03-06 17:03, Roman Kennke wrote: >>>>>>>>> Am 06.03.2018 um 14:35 schrieb Roman Kennke: >>>>>>>>>> Makes me wonder: why attempt to be smart in c1_Runtime1.cpp, when >>>>>>>>>> we can >>>>>>>>>> just as well call typeArrayOop::copy_array() and have it do the >>>>>>>>>> right >>>>>>>>>> thing? Or go even further and also do it for oop-arraycopy? >>>>>>>>> Something like: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~rkennke/8198445/8198445-1.patch >>>>>>>>> >>>>>>>>> This wouldn't compile because of bunch of missing >>>>>>>>> arraycopy_conjoint_atomic defintions for extra types like jfloat, >>>>>>>>> jdouble, jboolean, etc, which in turn would be missing the same >>>>>>>>> Copy::conjoint_jFluffs_atomic() which drags in a bunch of platform >>>>>>>>> specific stuff... and my question before I go there is: do we want >>>>>>>>> all >>>>>>>>> that? Or can you think of a better way to solve it? >>>>>>>>> >>>>>>>>> Roman >>>>>>>>> >>>>>>>>>> Roman >>>>>>>>>> >>>>>>>>>>> The ARRAYCOPY_ATOMIC decorator makes the arraycopy atomic over >>>>>>>>>>> the >>>>>>>>>>> size >>>>>>>>>>> of the passed in elements. >>>>>>>>>>> In this case, it looks like the address has been type erased to >>>>>>>>>>> void*, >>>>>>>>>>> and hence lost what the element size was. There is currently no >>>>>>>>>>> overload >>>>>>>>>>> accepted for type erased element - only accurate elements {jbyte, >>>>>>>>>>> jshort, jint, jlong}. >>>>>>>>>>> >>>>>>>>>>> So it looks like an overload must be added to accept type erased >>>>>>>>>>> void* >>>>>>>>>>> elements and make that call conjoint_memory_atomic when the >>>>>>>>>>> ARRAYCOPY_ATOMIC decorator is passed in. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> /Erik >>>>>>>>>>> >>>>>>>>>>> On 2018-03-06 13:56, David Holmes wrote: >>>>>>>>>>>> On 6/03/2018 10:54 PM, Erik ?sterlund wrote: >>>>>>>>>>>>> Hi David, >>>>>>>>>>>>> >>>>>>>>>>>>> It is atomic with the ARRAYCOPY_ATOMIC decorator. If that >>>>>>>>>>>>> comment is >>>>>>>>>>>>> correct (I do not know if it is), then the ARRAYCOPY_ATOMIC >>>>>>>>>>>>> decorator >>>>>>>>>>>>> should probably be used here. >>>>>>>>>>>> If that code implements a Java array copy then yes it is >>>>>>>>>>>> required to >>>>>>>>>>>> be 32-bit atomic. Do you need the decorator to get 32-bit >>>>>>>>>>>> atomicity? >>>>>>>>>>>> >>>>>>>>>>>> David >>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> /Erik >>>>>>>>>>>>> >>>>>>>>>>>>> On 2018-03-06 13:48, David Holmes wrote: >>>>>>>>>>>>>> Hi Roman, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Not a review as I'm not familiar enough with the Access API, >>>>>>>>>>>>>> but in >>>>>>>>>>>>>> src/hotspot/share/c1/c1_Runtime1.cpp the comments above the >>>>>>>>>>>>>> changed >>>>>>>>>>>>>> code need updating - probably deleting. I assume the Access >>>>>>>>>>>>>> API >>>>>>>>>>>>>> arraycopy is atomic? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> David >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 6/03/2018 9:56 PM, Roman Kennke wrote: >>>>>>>>>>>>>>> Currently, the Access API is only used for oop-arraycopy, but >>>>>>>>>>>>>>> not for >>>>>>>>>>>>>>> primitive arraycopy. GCs might want to intercept this too, >>>>>>>>>>>>>>> e.g. >>>>>>>>>>>>>>> resolve >>>>>>>>>>>>>>> src and dst arrays. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> There *is* an implementation of primitive arraycopy in the >>>>>>>>>>>>>>> Access API, >>>>>>>>>>>>>>> but it doesn't even compile, because Raw::arraycopy() does >>>>>>>>>>>>>>> not >>>>>>>>>>>>>>> take >>>>>>>>>>>>>>> src >>>>>>>>>>>>>>> and dst oop operands, but it's called like that. The only >>>>>>>>>>>>>>> reason >>>>>>>>>>>>>>> why >>>>>>>>>>>>>>> this does not blow up (I think) is that because nobody calls >>>>>>>>>>>>>>> it, >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> compiler doesn't even get there. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This change fixes the Access API/impl and adds the relevant >>>>>>>>>>>>>>> calls into >>>>>>>>>>>>>>> it (in C1 and runtime land). C2 uses arraycopy stubs (which >>>>>>>>>>>>>>> cannot be >>>>>>>>>>>>>>> handled here) or calls out to the ArrayKlass::copy_array(), >>>>>>>>>>>>>>> which >>>>>>>>>>>>>>> should >>>>>>>>>>>>>>> be covered with this change. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> It should be possible to use the same Access API for >>>>>>>>>>>>>>> Java-array <-> >>>>>>>>>>>>>>> native-array bulk transfers, which currently use the rather >>>>>>>>>>>>>>> ugly >>>>>>>>>>>>>>> typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address >>>>>>>>>>>>>>> that >>>>>>>>>>>>>>> in a >>>>>>>>>>>>>>> separate change though. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Tests: tier1 ok >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please review! >>>>>>>>>>>>>>> Thanks, Roman >>>>>>>>>>>>>>> >>> >> > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Tue Mar 13 15:43:26 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 13 Mar 2018 16:43:26 +0100 Subject: RFR [0/7]: Rebuild remembered sets concurrently In-Reply-To: <1520258985.2532.40.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> Message-ID: <1520955806.3100.42.camel@oracle.com> Hi, after this round of updated due to rebasing, here is a new "complete" webrev for reference: http://cr.openjdk.java.net/~tschatzl/rebuild-all/webrev.1 Thanks, Thomas On Mon, 2018-03-05 at 15:09 +0100, Thomas Schatzl wrote: > Hi all, > > I would like to ask for reviews for a significant change in how G1 > manages its remembered sets by rebuilding its remembered sets > concurrently. > > Currently G1 maintains the remembered sets for all regions all the > time. This has several drawbacks: > > - memory consumption for remembered sets is very high all the time, > not > only due to actually storing these remembered sets, but also due to > additional memory wastage due to fragmentation. > - the mechanism to combat this memory fragmentation (somewhat) has > been > to periodically "scrub" remembered sets, at high costs during the > "Cleanup" pause. > - maintenance of ultimately unused remembered sets decreases overall > throughput. > > With these changes, G1 will only maintain remembered sets that will > ever be considered for evacuation ("candidate collection set > regions") > only during marking and the following mixed gc phase. > > This not only to a very significant degree decreases average memory > consumption (e.g. during the young-only phase mainly remembered sets > for young-only regions are maintained), but also maximum memory > consumption, as there is little fragmentation while remembered sets > are > maintained. > > There is no need for scrubbing the remembered sets any more either, > making the Cleanup pause very fast. This allows us to actually remove > some previously (very small) concurrent phase. > > As a beneficial side-effect we noticed that particularly mixed gc > performance may increase significantly (up to like 40%-50% faster) > due > to improved remembered set memory locality. > > Particularly "big data" workloads where the remembered sets take a > lot > of memory and maintenance cpu resources benefit most, but we have > also > noticed significant throughput improvements when running with G1 in > other cases. > > The only drawback we are aware of is that the delay from start of > marking to the first mixed gc is longer. G1's mechanism to adaptively > start marking seems to be able to manage this change nicely, but some > people that already set a very high specific IHOP manually may > experience full gcs. > > Due to the large benefits we do not intend to provide the original > behavior, although it could be implemented as the decisions which > regions to maintain the remembered sets are encapsulated into a > single > class. > > For more information look at my somewhat recent FOSDEM talk > explaining > this change in more details [1]. > > For further review purposes this change will be split into seven > distinct changesets that will be organized into a thread below this > post, each tagged with a running number. Obviously they are dependent > on each other. > > These are in order: > > 8197569: Refactor eager reclaim for concurrent remembered set > rebuilding > 8197570: Make rules for choosing collection set candidates more > explicit > 8197850: Calculate liveness in regions during marking > 8180415: Rebuild remembered sets during concurrent mark (main change) > 8197573: Remove concurrent cleanup and secondary free list handling > 8197928: Only enqueue deferred cards with references into regions > that > have a tracked remembered set during GC > 8197932: Better split work in rebuild remembered sets phase > > Each of those changesets will have its own webrev, and there will be > notes about each of the changes. However, for reference a webrev > containing all the changes can be found here: > > http://cr.openjdk.java.net/~tschatzl/rebuild-all/webrev/ > > Testing: hs-tier1-5, internal perf tests > > Enjoy! :) > > Thomas > > [1] https://fosdem.org/2018/schedule/event/g1/ > From thomas.schatzl at oracle.com Tue Mar 13 15:45:33 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 13 Mar 2018 16:45:33 +0100 Subject: RFR [1/7]: 8197569: Refactor eager reclaim for concurrent remembered set rebuilding In-Reply-To: References: <1520258985.2532.40.camel@oracle.com> <1520259499.2532.44.camel@oracle.com> <8d58ae9c-1c18-c317-a40c-66241faa9a81@oracle.com> <1520342273.2378.15.camel@oracle.com> Message-ID: <1520955933.3100.43.camel@oracle.com> Hi, On Tue, 2018-03-06 at 14:49 +0100, Stefan Johansson wrote: > > On 2018-03-06 14:17, Thomas Schatzl wrote: > > Hi Stefan, > > > > thanks for looking into this. > > > > On Mon, 2018-03-05 at 16:11 +0100, Stefan Johansson wrote: > > > Thanks for splitting this change up Thomas, > > > > > > On 2018-03-05 15:18, Thomas Schatzl wrote: > > > > Hi all, > > > > > > > > can I have some reviews for the following change that > > > > refactors the[...] > > > > Either way is kind of fine for me right now, so let's see what > > others say? > > I'm fine with this as well. Consider it reviewed :) thanks Stefan and Kim for your reviews. Thomas From jesper.wilhelmsson at oracle.com Tue Mar 13 15:59:38 2018 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Tue, 13 Mar 2018 16:59:38 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> <5A9E8F76.1050809@oracle.com> <83de65bd-5d3f-78df-02d2-eb4c53aa0c62@oracle.com> <5A9E95D8.6080602@oracle.com> <2cc9a407-aba8-209a-7ef7-4ab26b88fef6@redhat.com> <6d8435af-5bec-da70-fa8a-60d708937f44@redhat.com> <5A9EBD6E.3070005@oracle.com> <2612309c-868b-6094-5944-e9de54daf8d2@redhat.com> <5AA136E9.9080100@oracle.com> <3034e0b7-9894-80e6-afd4-171e31ad5836@redhat.com> <5AA29BEB.3020402@oracle.com> <0032c706-3171-1cee-20b0-baf67b438f31@redhat.com> <39a6ae41-9736-ea0a-99ef-af3e32c772b8@redhat.com> Message-ID: <3677DC43-EF68-4746-A155-73C413D445DE@oracle.com> Roman, The first thing to do in this case is to search JBS for any known issue in this test. By searching for the test name for instance you will find this issue: JDK-8199515 https://bugs.openjdk.java.net/issues/?jql=text%20~%20%22java%2Flang%2Finvoke%2Fcondy%2FCondyInterfaceWithOverpassMethods.java%22 Obviously, without more information about your particular failure you can not be 100% sure that this is the same bug, but it's a good pice of information to provide to the Oracle engineer who you ask to look at your failure. If there is an error in your tier1 run you should NOT push unless someone that has investigated the issue gives you a GO. In this case your failure is a known issue that is being worked on, JDK-8199515, and you can go ahead and push provided the change is reviewed and all those other checkboxes are ticked. /Jesper > On 13 Mar 2018, at 16:40, Roman Kennke wrote: > > The test came back with the below. Doesn't strike me as being related to > this change. > > What to do now? Push it anyway? Re-submit the test? (If so, how?) > > Roman > > Build Details: 2018-03-13-1330406.roman.source > 1 Failed Test > Test Tier Platform Keywords Description Task > java/lang/invoke/condy/CondyInterfaceWithOverpassMethods.java tier1 > windows-x64 bug8186046 othervm testng Error: failed to clean up files > after test task > Mach5 Tasks Results Summary > > NA: 0 > FAILED: 0 > PASSED: 74 > UNABLE_TO_RUN: 0 > KILLED: 0 > EXECUTED_WITH_FAILURE: 1 > Test > > 1 Executed with failure > jdk_open_test_jdk_tier1-windows-x64-66 Results: total: 1775, > passed: 1774; failed: 1 > > > >> Hi Erik & Per, >> >> yes, I agree with getting rid of Runtime1::arraycopy(). Let's take it to >> another RFE. >> >> Let's try the newfangled submit repo and push (mostly) my first patch :-) >> >> Thanks for reviewing! >> Cheer, Roman >> >> >>> Hi Roman, >>> >>> Ideally I'd like to get rid of Runtime1::arraycopy. It seems like a >>> medium slow path of C1 never used by any platforms other than S390 and >>> PPC, which look like the only platforms that lack a generic arraycopy >>> stub from the StubRoutines. I think that S390 and PPC could easily just >>> jump to the C1 arraycopy stub instead and take the slowpath to the >>> normal native arraycopy that enters through the JNI interface instead. >>> >>> If the performance of this C1 arraycopy path on S390 and PPC are >>> crucial, then they should probably implement stubs for it like the other >>> platforms. If the performance of this is not crucial (I suspect this is >>> the case), then it should not harm to take the normal entry through JNI >>> for generic arraycopy using C1. >>> >>> But I think trying to get rid of that path could be a follow-up RFE >>> instead. >>> >>> Looks good. >>> >>> Thanks, >>> /Erik >>> >>> On 2018-03-13 11:02, Roman Kennke wrote: >>>> Hi Erik, >>>> >>>> thanks for doing this! It looks good to me! >>>> >>>> I'm putting this up here again, with commit msg fixed: >>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.03/ >>>> >>>> Can I get (final?) reviews? And then I'd put it through the new submit >>>> repo and push it? >>>> >>>> Thanks, Roman >>>> >>>> >>>>> Hi Roman, >>>>> >>>>> I would love to avoid using the switch to enumerate all different Java >>>>> types you could possibly have. >>>>> How about this? (was easier to cook something together than explain in >>>>> detail what I had in mind) >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8198445/webrev.00/ >>>>> >>>>> The idea is to let Access recognize arraycopy with element type "void", >>>>> and then perform a conservative byte arraycopy, that when >>>>> ARRAYCOPY_ATOMIC uses the natural atomicity of the element alignment by >>>>> using Copy::conjoint_memory_atomic(). >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>> On 2018-03-08 21:12, Roman Kennke wrote: >>>>>> Am 08.03.2018 um 14:13 schrieb Erik ?sterlund: >>>>>>> Hi Roman, >>>>>>> >>>>>>> On 2018-03-08 11:25, Roman Kennke wrote: >>>>>>>> Wouldn't that lead to mis-aligned accesses when you try to copy an >>>>>>>> array >>>>>>>> of, e.g., bytes, but shoehorn it to pointer-size? Or am I >>>>>>>> misunderstanding? >>>>>>> No it would not. The element type is void, not void*. Because the >>>>>>> element type is type erased. And that would map to >>>>>>> Copy::conjoint_memory_atomic, which is precisely what >>>>>>> typeArrayKlass.cpp >>>>>>> maps such an arraycopy to today. Assuming elements are aligned to >>>>>>> their >>>>>>> natural size (which they are), Copy::conjoint_memory_atomic is >>>>>>> guaranteed to be at least as atomic as you need your elements to >>>>>>> be, up >>>>>>> to an element size of 8. >>>>>> Ah, ok. Alright, so here comes rev.02: >>>>>> >>>>>> Differential: >>>>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.02.diff/ >>>>>> Full: >>>>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.02/ >>>>>> >>>>>> Still passes tier1 tests. >>>>>> >>>>>> Ok now? >>>>>> >>>>>> Cheers, Roman >>>>>> >>>>>> >>>>>>>> An idea would be to copy equal-sized elements using the corresponding >>>>>>>> Copy::conjoint_XXX_atomic() calls, e.g. char and short would use the >>>>>>>> short version, boolean and byte would use the byte version (or is >>>>>>>> boolean sometimes int?!), float and int would use the int version and >>>>>>>> long and double the long version. WDYT? >>>>>>> I'm not sure I see what the advantage of that would be. >>>>>>> >>>>>>> Thanks, >>>>>>> /Erik >>>>>>> >>>>>>>> Roman >>>>>>>> >>>>>>>> >>>>>>>>> My spontaneous idea was to have an overload for type erased void* >>>>>>>>> that >>>>>>>>> when called with ARRAYCOPY_ATOMIC maps to >>>>>>>>> Copy::conjoint_memory_atomic >>>>>>>>> which AFAIK is conservatively atomic regardless of what the precise >>>>>>>>> element type is. >>>>>>>>> >>>>>>>>> But if you have better ideas, I am open for suggestions. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> /Erik >>>>>>>>> >>>>>>>>> On 2018-03-06 17:03, Roman Kennke wrote: >>>>>>>>>> Am 06.03.2018 um 14:35 schrieb Roman Kennke: >>>>>>>>>>> Makes me wonder: why attempt to be smart in c1_Runtime1.cpp, when >>>>>>>>>>> we can >>>>>>>>>>> just as well call typeArrayOop::copy_array() and have it do the >>>>>>>>>>> right >>>>>>>>>>> thing? Or go even further and also do it for oop-arraycopy? >>>>>>>>>> Something like: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8198445/8198445-1.patch >>>>>>>>>> >>>>>>>>>> This wouldn't compile because of bunch of missing >>>>>>>>>> arraycopy_conjoint_atomic defintions for extra types like jfloat, >>>>>>>>>> jdouble, jboolean, etc, which in turn would be missing the same >>>>>>>>>> Copy::conjoint_jFluffs_atomic() which drags in a bunch of platform >>>>>>>>>> specific stuff... and my question before I go there is: do we want >>>>>>>>>> all >>>>>>>>>> that? Or can you think of a better way to solve it? >>>>>>>>>> >>>>>>>>>> Roman >>>>>>>>>> >>>>>>>>>>> Roman >>>>>>>>>>> >>>>>>>>>>>> The ARRAYCOPY_ATOMIC decorator makes the arraycopy atomic over >>>>>>>>>>>> the >>>>>>>>>>>> size >>>>>>>>>>>> of the passed in elements. >>>>>>>>>>>> In this case, it looks like the address has been type erased to >>>>>>>>>>>> void*, >>>>>>>>>>>> and hence lost what the element size was. There is currently no >>>>>>>>>>>> overload >>>>>>>>>>>> accepted for type erased element - only accurate elements {jbyte, >>>>>>>>>>>> jshort, jint, jlong}. >>>>>>>>>>>> >>>>>>>>>>>> So it looks like an overload must be added to accept type erased >>>>>>>>>>>> void* >>>>>>>>>>>> elements and make that call conjoint_memory_atomic when the >>>>>>>>>>>> ARRAYCOPY_ATOMIC decorator is passed in. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> /Erik >>>>>>>>>>>> >>>>>>>>>>>> On 2018-03-06 13:56, David Holmes wrote: >>>>>>>>>>>>> On 6/03/2018 10:54 PM, Erik ?sterlund wrote: >>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>> >>>>>>>>>>>>>> It is atomic with the ARRAYCOPY_ATOMIC decorator. If that >>>>>>>>>>>>>> comment is >>>>>>>>>>>>>> correct (I do not know if it is), then the ARRAYCOPY_ATOMIC >>>>>>>>>>>>>> decorator >>>>>>>>>>>>>> should probably be used here. >>>>>>>>>>>>> If that code implements a Java array copy then yes it is >>>>>>>>>>>>> required to >>>>>>>>>>>>> be 32-bit atomic. Do you need the decorator to get 32-bit >>>>>>>>>>>>> atomicity? >>>>>>>>>>>>> >>>>>>>>>>>>> David >>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> /Erik >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2018-03-06 13:48, David Holmes wrote: >>>>>>>>>>>>>>> Hi Roman, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Not a review as I'm not familiar enough with the Access API, >>>>>>>>>>>>>>> but in >>>>>>>>>>>>>>> src/hotspot/share/c1/c1_Runtime1.cpp the comments above the >>>>>>>>>>>>>>> changed >>>>>>>>>>>>>>> code need updating - probably deleting. I assume the Access >>>>>>>>>>>>>>> API >>>>>>>>>>>>>>> arraycopy is atomic? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> David >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 6/03/2018 9:56 PM, Roman Kennke wrote: >>>>>>>>>>>>>>>> Currently, the Access API is only used for oop-arraycopy, but >>>>>>>>>>>>>>>> not for >>>>>>>>>>>>>>>> primitive arraycopy. GCs might want to intercept this too, >>>>>>>>>>>>>>>> e.g. >>>>>>>>>>>>>>>> resolve >>>>>>>>>>>>>>>> src and dst arrays. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> There *is* an implementation of primitive arraycopy in the >>>>>>>>>>>>>>>> Access API, >>>>>>>>>>>>>>>> but it doesn't even compile, because Raw::arraycopy() does >>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>> take >>>>>>>>>>>>>>>> src >>>>>>>>>>>>>>>> and dst oop operands, but it's called like that. The only >>>>>>>>>>>>>>>> reason >>>>>>>>>>>>>>>> why >>>>>>>>>>>>>>>> this does not blow up (I think) is that because nobody calls >>>>>>>>>>>>>>>> it, >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>> compiler doesn't even get there. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> This change fixes the Access API/impl and adds the relevant >>>>>>>>>>>>>>>> calls into >>>>>>>>>>>>>>>> it (in C1 and runtime land). C2 uses arraycopy stubs (which >>>>>>>>>>>>>>>> cannot be >>>>>>>>>>>>>>>> handled here) or calls out to the ArrayKlass::copy_array(), >>>>>>>>>>>>>>>> which >>>>>>>>>>>>>>>> should >>>>>>>>>>>>>>>> be covered with this change. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> It should be possible to use the same Access API for >>>>>>>>>>>>>>>> Java-array <-> >>>>>>>>>>>>>>>> native-array bulk transfers, which currently use the rather >>>>>>>>>>>>>>>> ugly >>>>>>>>>>>>>>>> typeArrayOop::XYZ_addr() + memcpy() pattern. I'll address >>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>> in a >>>>>>>>>>>>>>>> separate change though. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/8198445/webrev.00/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Tests: tier1 ok >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Please review! >>>>>>>>>>>>>>>> Thanks, Roman >>>>>>>>>>>>>>>> >>>> >>> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rkennke at redhat.com Tue Mar 13 16:15:50 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 13 Mar 2018 17:15:50 +0100 Subject: RFR: JDK-8198445: Access API for primitive/native arraycopy In-Reply-To: <3677DC43-EF68-4746-A155-73C413D445DE@oracle.com> References: <007e671a-a1ef-c3f0-4181-bb3a41733c7e@redhat.com> <5A9E8F76.1050809@oracle.com> <83de65bd-5d3f-78df-02d2-eb4c53aa0c62@oracle.com> <5A9E95D8.6080602@oracle.com> <2cc9a407-aba8-209a-7ef7-4ab26b88fef6@redhat.com> <6d8435af-5bec-da70-fa8a-60d708937f44@redhat.com> <5A9EBD6E.3070005@oracle.com> <2612309c-868b-6094-5944-e9de54daf8d2@redhat.com> <5AA136E9.9080100@oracle.com> <3034e0b7-9894-80e6-afd4-171e31ad5836@redhat.com> <5AA29BEB.3020402@oracle.com> <0032c706-3171-1cee-20b0-baf67b438f31@redhat.com> <39a6ae41-9736-ea0a-99ef-af3e32c772b8@redhat.com> <3677DC43-EF68-4746-A155-73C413D445DE@oracle.com> Message-ID: Hi Jesper, > The first thing to do in this case is to search JBS for any known issue > in this test. By searching for the test name for instance you will find > this issue: JDK-8199515 > https://bugs.openjdk.java.net/issues/?jql=text%20~%20%22java%2Flang%2Finvoke%2Fcondy%2FCondyInterfaceWithOverpassMethods.java%22 > "java/lang/invoke/condy/CondyInterfaceWithOverpassMethods.java"> > > Obviously, without more information about your particular failure you > can not be 100% sure that this is the same bug, but it's a good pice of > information to provide to the Oracle engineer who you ask to look at > your failure. If there is an error in your tier1 run you should NOT push > unless someone that has investigated the issue gives you a GO. Ok, this is good stuff to know. Thanks! > In this case your failure is a known issue that is being worked on, > JDK-8199515, and you can go ahead and push provided the change is > reviewed and all those other checkboxes are ticked. Yes, all checkboxes are ticked ;-) I just pushed the very first Hotspot change *myself* :-) Yippie! This is great progress, thank you!! Cheers, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Tue Mar 13 20:00:55 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 13 Mar 2018 21:00:55 +0100 Subject: RFR(XS): JDK-8199559: objArrayKlass::oop_iterate() and friends must use base_raw() instead of base() Message-ID: <36ef9c8e-7278-1613-989e-acc929f2d2ed@redhat.com> Currently, objArrayKlass::oop_iterate() and related methods use base() to get the actual starting address of the array. However, base() resolves the oop via barriers. oop_iterate() is used solely by GC code, which requires direct access to the array, no matter which copy, and thus needs to use base_raw() instead. Bug: https://bugs.openjdk.java.net/browse/JDK-8199559 Webrev: http://cr.openjdk.java.net/~rkennke/8199559/webrev.00/ Can I please get a review? Thanks, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From sangheon.kim at oracle.com Tue Mar 13 21:39:21 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Tue, 13 Mar 2018 14:39:21 -0700 Subject: RFR [L][3/7]: 8197850: Calculate liveness in regions during marking In-Reply-To: <1520949153.3100.27.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520260902.2532.66.camel@oracle.com> <1520343234.2378.29.camel@oracle.com> <8f7f25ef-f0c6-20f6-1584-b615dba234e1@oracle.com> <1520412521.2346.4.camel@oracle.com> <0cd6cca6-cace-4e4f-7f9c-152e20757dfe@oracle.com> <1520949153.3100.27.camel@oracle.com> Message-ID: <4c671223-0d99-ef72-682a-402374cdb988@oracle.com> Hi Thomas, On 03/13/2018 06:52 AM, Thomas Schatzl wrote: > Hi Stefan, > > On Wed, 2018-03-07 at 11:12 +0100, Stefan Johansson wrote: >> Thanks Thomas, >> >> This looks good to me, reviewed. > thanks for your review; unfortunately rebasing the changes to latest > caused some additional work. > > In this particular case, contents of g1RegionMarkStatsCache had to be > distributed to .cpp and .inline.hpp files (recent .inline.hpp include > fixes), and I preemptively removed the use of one VALUE_OBJ_CLASS_SPEC. > > Webrevs: > http://cr.openjdk.java.net/~tschatzl/8197850/webrev.1_to_2/index.html > (diff) > http://cr.openjdk.java.net/~tschatzl/8197850/webrev.2/index.html (full) Looks good, I have several minor comments. =============================== src/hotspot/share/gc/g1/g1CardLiveData.cpp 292 assert(!hr->is_old() || marked_bytes == (_cm->liveness(hr->hrm_index()) * HeapWordSize), Don't we need to print assert message for hr->is_old()? =============================== src/hotspot/share/gc/g1/g1CollectedHeap.cpp 4215 // Any worker id is fine here as we are in the VM thread and single-threaded. Would be better to add 'valid'? // Any _valid_ worker id ... =============================== src/hotspot/share/gc/g1/g1ConcurrentMark.cpp 1840 log_debug(gc, stats)("Mark stats cache hits " SIZE_FORMAT " misses " SIZE_FORMAT " ratio %1.3lf", gc+stats is not registered at logPrefix.hpp, so currently GC ID is not printed. I guess this is not intensional? =============================== src/hotspot/share/gc/g1/g1RegionMarkStatsCache.hpp 25 #ifndef SHARE_VM_GC_G1_G1REGIONMARKSTATSCACHE_HPP I'm okay leaving as is but we don't have 'VM' directory anymore. At some point we would want to clean up these stuffs at once? 61 // logical and. This seems incomplete sentence? Thanks, Sangheon > > Testing: > hs-tier 1-5 > > Thanks, > Thomas > > > >> Stefan >> >> On 2018-03-07 09:48, Thomas Schatzl wrote: >>> Hi, >>> >>> On Tue, 2018-03-06 at 15:03 +0100, Stefan Johansson wrote: >>>> On 2018-03-06 14:33, Thomas Schatzl wrote: >>>>> On Tue, 2018-03-06 at 13:31 +0100, Stefan Johansson wrote: >>>>>> Hi Thomas, >>>>>> >>>>>> On 2018-03-05 15:41, Thomas Schatzl wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> ... >>>>>>> >>>>>>> CR: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8197850 >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~tschatzl/8197850/webrev/index.h >>>>>>> tml >>>>>> Look good, just some minor things. >>>>>> >>>>>> src/hotspot/share/gc/g1/g1ConcurrentMark.cpp >>>>>> 394 _region_mark_stats(NEW_C_HEAP_ARRAY(G1RegionMarkStats, >>>>>> max_regions, mtGC)) >>>>>> >>>>>> max_regions is passed into the constructor but everywhere >>>>>> else in >>>>>> G1ConcurrentMark we use g1h->max_regions(). I would consider >>>>>> either >>>>>> calling g1h->max_regions() in the constructor too, or save >>>>>> the >>>>>> value >>>>>> in a member and use it everywhere. >>>>> Done using option 1), use g1h->max_regions() everywhere. >>>> Thanks. >>>>>> --- >>>>>> src/hotspot/share/gc/g1/g1_globals.hpp >>>>>> 262 experimental(size_t, G1RegionMarkStatsCacheSize, >>>>>> 1024, >>>>>> ... >>>>>> 265 range(128, (max_juint / 2) + 1) >>>>>> >>>>>> Do we need this flag or should we scale it against the number >>>>>> of >>>>>> regions? If we want to keep it I think we should have a >>>>>> somewhat >>>>>> more restrictive range. >>>>> I would not want this to scale according to the number of heap >>>>> regions, >>>>> because that is going to have a significant impact on pause >>>>> times >>>>> when >>>>> all of the thread's caches are evicted (The O(#regions) part). >>>> True, just a thought since 1024 is roughly half the number >>>> regions >>>> we aim for maybe this could be used together with a reasonable >>>> upper >>>> bound. >>>> >>>>> The flag has only been added (rather late, admittedly) to fix >>>>> potential issues with marking time with really large heaps with >>>>> tens of thousands of regions. >>>>> Let me do some more measurements, I will get back to you with >>>>> them >>>>> before giving you a new webrev. >>>> Sound good. >>> I did some measurements with 100k regions, and I think we can just >>> remove the flag. There is always logging available (gc+mark+stats) >>> that >>> shows cache hit ratio to diagnose issues. >>> >>>>>> --- >>>>>> src/hotspot/share/gc/g1/g1RegionMarkStatsCache.hpp >>>>>> >>>>>> In G1RegionMarkStatsCacheEntry _region_idx is set to 0 when >>>>>> clearing. I don't see a problem with it right now, but it >>>>>> feels a >>>>>> bit wrong since index 0 is a valid region index. Maybe we >>>>>> could >>>>>> use another cleared marker. >>>>> Unfortunately there is none (one could use something like -1, >>>>> but >>>>> that one is valid too), and it seems a waste of memory to >>>>> reserve >>>>> more space for it. >>>>> >>>>> I see your point that this is a bit ugly, but there should >>>>> never be >>>>> any issue because of the initialization of the value part of an >>>>> entry with zero. It simply has no impact on totals if you add >>>>> zero. >>>>> >>>>> The code could initialize the cache with region indices from 0 >>>>> to >>>>> cache size - 1, which are valid values given the hash function, >>>>> what do you think? >>>>> >>>>> However that would slow down clearing a little (the compiler >>>>> will >>>>> just compile it to a memset() in the loop right now). >>>> You are correct that -1 is a valid index, but we seldom see heaps >>>> with so many regions. I'm fine with leaving it as is, if you >>>> dislike >>>> using -1 and if it will slow down clearing. >>> Just a bit. It seems as bad to use -1 or another random value, so I >>> kept it. >>> >>> Webrevs: >>> http://cr.openjdk.java.net/~tschatzl/8197850/webrev.0_to_1/index.ht >>> ml >>> (diff) >>> http://cr.openjdk.java.net/~tschatzl/8197850/webrev.1/index.html >>> (full) >>> >>> Testing: >>> This change ran, with some changes to 8180415 due to internal >>> comments >>> (I will post in a few minutes) through hs-tier 1-5 over night. >>> >>> Thanks, >>> Thomas >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From sangheon.kim at oracle.com Tue Mar 13 21:40:11 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Tue, 13 Mar 2018 14:40:11 -0700 Subject: RFR (S): 8199326: Remove G1 gc time stamp logic In-Reply-To: <1520523980.3121.29.camel@oracle.com> References: <1520516013.3121.24.camel@oracle.com> <7b19df2a-cac8-41d7-c3e9-d2cf893f9d85@oracle.com> <1520523980.3121.29.camel@oracle.com> Message-ID: <9ef3f6cb-4487-c03a-2d06-da6f5911972a@oracle.com> Hi Thomas, On 03/08/2018 07:46 AM, Thomas Schatzl wrote: > On Thu, 2018-03-08 at 16:18 +0100, Stefan Johansson wrote: >> Hi Thomas, >> >> On 2018-03-08 14:33, Thomas Schatzl wrote: >>> Hi, >>> >>> with the rebuild remsets concurrently changes the last user of >>> the gc >>> time stamps (create live data) has gone away. >>> >>> This change removes all of the corresponding code I could find. >>> >>> Could you review it? >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8199326 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8197932/webrev/ >> Looks like you pasted the wrong link, and I couldn't find the >> correct >> one either. > Sorry. The correct link is: > http://cr.openjdk.java.net/~tschatzl/8199326/webrev This looks good. Thanks, Sangheon > > > Thomas > From thomas.schatzl at oracle.com Wed Mar 14 07:55:21 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 14 Mar 2018 08:55:21 +0100 Subject: RFR [L][3/7]: 8197850: Calculate liveness in regions during marking In-Reply-To: <4c671223-0d99-ef72-682a-402374cdb988@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520260902.2532.66.camel@oracle.com> <1520343234.2378.29.camel@oracle.com> <8f7f25ef-f0c6-20f6-1584-b615dba234e1@oracle.com> <1520412521.2346.4.camel@oracle.com> <0cd6cca6-cace-4e4f-7f9c-152e20757dfe@oracle.com> <1520949153.3100.27.camel@oracle.com> <4c671223-0d99-ef72-682a-402374cdb988@oracle.com> Message-ID: <1521014121.2381.11.camel@oracle.com> Hi, On Tue, 2018-03-13 at 14:39 -0700, sangheon.kim wrote: > Hi Thomas, > > On 03/13/2018 06:52 AM, Thomas Schatzl wrote: > > Hi Stefan, > > > > On Wed, 2018-03-07 at 11:12 +0100, Stefan Johansson wrote: > > > Thanks Thomas, > > > > > > [...] > > In this particular case, contents of g1RegionMarkStatsCache had to > > be distributed to .cpp and .inline.hpp files (recent .inline.hpp > > include fixes), and I preemptively removed the use of one > > VALUE_OBJ_CLASS_SPEC. > > > > Webrevs: > > http://cr.openjdk.java.net/~tschatzl/8197850/webrev.1_to_2/index.ht > > ml > > (diff) > > http://cr.openjdk.java.net/~tschatzl/8197850/webrev.2/index.html > > (full) > Looks good, I have several minor comments. > > =============================== > src/hotspot/share/gc/g1/g1CardLiveData.cpp > 292 assert(!hr->is_old() || marked_bytes == (_cm- > >liveness(hr->hrm_index()) * HeapWordSize), > Don't we need to print assert message for hr->is_old()? This is a pre-existing bug (or feature, depending on your POV) - for humongous objects the regions are considered to be always full (by definition), while obviously if there is only one live object in them, they do not always fill all regions the humongous object is in. Also, the marking attributes all of the humongous object size to the starts region. There is need to distribute that. So the assert does not hold for other type of regions at the moment. This will be fixed with JDK-8178105 as far as my plan goes. > =============================== > src/hotspot/share/gc/g1/g1CollectedHeap.cpp > 4215 // Any worker id is fine here as we are in the VM thread > and single-threaded. > > Would be better to add 'valid'? > // Any valid worker id ... Done. > =============================== > src/hotspot/share/gc/g1/g1ConcurrentMark.cpp > 1840 log_debug(gc, stats)("Mark stats cache hits " SIZE_FORMAT " > misses " SIZE_FORMAT " ratio %1.3lf", > gc+stats is not registered at logPrefix.hpp, so currently GC ID is > not printed. I guess this is not intensional? This is a pre-existing issue. I filed JDK-8199598. > > =============================== > src/hotspot/share/gc/g1/g1RegionMarkStatsCache.hpp > 25 #ifndef SHARE_VM_GC_G1_G1REGIONMARKSTATSCACHE_HPP > I'm okay leaving as is but we don't have 'VM' directory anymore. At > some point we would want to clean up these stuffs at once? Yes. > 61 // logical and. > This seems incomplete sentence? This refers to the logical and ("&") operation and seems correct. New webrevs: http://cr.openjdk.java.net/~tschatzl/8197850/webrev.2_to_3/ (diff) http://cr.openjdk.java.net/~tschatzl/8197850/webrev.3/ (full) Thanks, Thomas From thomas.schatzl at oracle.com Wed Mar 14 07:56:57 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 14 Mar 2018 08:56:57 +0100 Subject: RFR (XXXS): 8199516: Remove dead code overlooked during Full GC work In-Reply-To: <3e6d979c-4bce-f5cf-d465-17d742ee8ad7@oracle.com> References: <1520948470.3100.21.camel@oracle.com> <3e6d979c-4bce-f5cf-d465-17d742ee8ad7@oracle.com> Message-ID: <1521014217.2381.12.camel@oracle.com> Hi Sangheon and Stefan, On Tue, 2018-03-13 at 07:30 -0700, sangheon.kim wrote: > Hi Thomas, > > Looks good to me too. > > Thanks, > Sangheon > thanks for your reviews. Thomas From erik.osterlund at oracle.com Wed Mar 14 10:16:35 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 14 Mar 2018 11:16:35 +0100 Subject: RFR(XS): JDK-8199559: objArrayKlass::oop_iterate() and friends must use base_raw() instead of base() In-Reply-To: <36ef9c8e-7278-1613-989e-acc929f2d2ed@redhat.com> References: <36ef9c8e-7278-1613-989e-acc929f2d2ed@redhat.com> Message-ID: <5AA8F683.2070801@oracle.com> Hi Roman, Looks good. Thanks, /Erik On 2018-03-13 21:00, Roman Kennke wrote: > Currently, objArrayKlass::oop_iterate() and related methods use base() > to get the actual starting address of the array. However, base() > resolves the oop via barriers. oop_iterate() is used solely by GC code, > which requires direct access to the array, no matter which copy, and > thus needs to use base_raw() instead. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8199559 > Webrev: > http://cr.openjdk.java.net/~rkennke/8199559/webrev.00/ > > Can I please get a review? > > Thanks, Roman > From thomas.schatzl at oracle.com Wed Mar 14 10:54:07 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 14 Mar 2018 11:54:07 +0100 Subject: RFR(XS): JDK-8199559: objArrayKlass::oop_iterate() and friends must use base_raw() instead of base() In-Reply-To: <36ef9c8e-7278-1613-989e-acc929f2d2ed@redhat.com> References: <36ef9c8e-7278-1613-989e-acc929f2d2ed@redhat.com> Message-ID: <1521024847.2380.0.camel@oracle.com> Hi, On Tue, 2018-03-13 at 21:00 +0100, Roman Kennke wrote: > Currently, objArrayKlass::oop_iterate() and related methods use > base() > to get the actual starting address of the array. However, base() > resolves the oop via barriers. oop_iterate() is used solely by GC > code, > which requires direct access to the array, no matter which copy, and > thus needs to use base_raw() instead. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8199559 > Webrev: > http://cr.openjdk.java.net/~rkennke/8199559/webrev.00/ > > Can I please get a review? looks good. Thomas From rkennke at redhat.com Wed Mar 14 14:30:33 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 14 Mar 2018 15:30:33 +0100 Subject: RFR: JDK-8199612: Replace remaining uses of G1BarrierSet::enqueue() with appropriate Access API calls Message-ID: We still have 2 places where we call directly to G1BarrierSet::enqueue(), those should use the appropriate Access API instead. Bug: https://bugs.openjdk.java.net/browse/JDK-8199612 Webrev: http://cr.openjdk.java.net/~rkennke/8199612/webrev.00/ Passes: tier1_runtime tests, currently running all of tier1 Please review! Thanks, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From sangheon.kim at oracle.com Wed Mar 14 16:56:20 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Wed, 14 Mar 2018 09:56:20 -0700 Subject: RFR [L][3/7]: 8197850: Calculate liveness in regions during marking In-Reply-To: <1521014121.2381.11.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520260902.2532.66.camel@oracle.com> <1520343234.2378.29.camel@oracle.com> <8f7f25ef-f0c6-20f6-1584-b615dba234e1@oracle.com> <1520412521.2346.4.camel@oracle.com> <0cd6cca6-cace-4e4f-7f9c-152e20757dfe@oracle.com> <1520949153.3100.27.camel@oracle.com> <4c671223-0d99-ef72-682a-402374cdb988@oracle.com> <1521014121.2381.11.camel@oracle.com> Message-ID: Hi Thomas, On 03/14/2018 12:55 AM, Thomas Schatzl wrote: > Hi, > > On Tue, 2018-03-13 at 14:39 -0700, sangheon.kim wrote: >> Hi Thomas, >> >> On 03/13/2018 06:52 AM, Thomas Schatzl wrote: >>> Hi Stefan, >>> >>> On Wed, 2018-03-07 at 11:12 +0100, Stefan Johansson wrote: >>>> Thanks Thomas, >>>> >>>> > [...] >>> In this particular case, contents of g1RegionMarkStatsCache had to >>> be distributed to .cpp and .inline.hpp files (recent .inline.hpp >>> include fixes), and I preemptively removed the use of one >>> VALUE_OBJ_CLASS_SPEC. >>> >>> Webrevs: >>> http://cr.openjdk.java.net/~tschatzl/8197850/webrev.1_to_2/index.ht >>> ml >>> (diff) >>> http://cr.openjdk.java.net/~tschatzl/8197850/webrev.2/index.html >>> (full) >> Looks good, I have several minor comments. >> >> =============================== >> src/hotspot/share/gc/g1/g1CardLiveData.cpp >> 292 assert(!hr->is_old() || marked_bytes == (_cm- >>> liveness(hr->hrm_index()) * HeapWordSize), >> Don't we need to print assert message for hr->is_old()? > This is a pre-existing bug (or feature, depending on your POV) - for > humongous objects the regions are considered to be always full (by > definition), while obviously if there is only one live object in them, > they do not always fill all regions the humongous object is in. > > Also, the marking attributes all of the humongous object size to the > starts region. There is need to distribute that. > > So the assert does not hold for other type of regions at the moment. > > This will be fixed with JDK-8178105 as far as my plan goes. Okay, thanks. > >> =============================== >> src/hotspot/share/gc/g1/g1CollectedHeap.cpp >> 4215 // Any worker id is fine here as we are in the VM thread >> and single-threaded. >> >> Would be better to add 'valid'? >> // Any valid worker id ... > Done. OK. > >> =============================== >> src/hotspot/share/gc/g1/g1ConcurrentMark.cpp >> 1840 log_debug(gc, stats)("Mark stats cache hits " SIZE_FORMAT " >> misses " SIZE_FORMAT " ratio %1.3lf", >> gc+stats is not registered at logPrefix.hpp, so currently GC ID is >> not printed. I guess this is not intensional? > This is a pre-existing issue. I filed JDK-8199598. Thanks for filing a new one. > >> =============================== >> src/hotspot/share/gc/g1/g1RegionMarkStatsCache.hpp >> 25 #ifndef SHARE_VM_GC_G1_G1REGIONMARKSTATSCACHE_HPP >> I'm okay leaving as is but we don't have 'VM' directory anymore. At >> some point we would want to clean up these stuffs at once? > Yes. OK > >> 61 // logical and. >> This seems incomplete sentence? > This refers to the logical and ("&") operation and seems correct. My bad, sorry. > > New webrevs: > http://cr.openjdk.java.net/~tschatzl/8197850/webrev.2_to_3/ (diff) > http://cr.openjdk.java.net/~tschatzl/8197850/webrev.3/ (full) Looks good to me. Thanks, Sangheon > > Thanks, > Thomas > From rkennke at redhat.com Wed Mar 14 18:13:51 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 14 Mar 2018 19:13:51 +0100 Subject: RFR: JDK-8199620: Support for JNI object pinning Message-ID: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> Currently, the Get/Release*Critical() family of functions use the GCLocker protocol to ensure that no JNI critical arrays are in use when a GC pause is entered. Some GCs may instead want to use object pinning and guarantee that the object does not move. For example, this is easy to do with region-based GCs (G1, Shenandoah, ZGC) by simply not including regions with pinned objects in the collection set. The implementation/API that I'm proposing is fairly simple: add two methods oop pin_object(oop) and void unpin_object(oop) to CollectedHeap, and call them from JNI's Get/Release*Critical methods. This approach has been working perfectly fine since a long time in Shenandoah. Bug: https://bugs.openjdk.java.net/browse/JDK-8199620 Webrev: http://cr.openjdk.java.net/~rkennke/8199620/webrev.00/ What do you think? Thanks, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed Mar 14 18:51:17 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 14 Mar 2018 19:51:17 +0100 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> Message-ID: <413a2cd0-295f-e179-ff79-84f135b05b05@redhat.com> Am 14.03.2018 um 19:13 schrieb Roman Kennke: > Currently, the Get/Release*Critical() family of functions use the > GCLocker protocol to ensure that no JNI critical arrays are in use when > a GC pause is entered. > > Some GCs may instead want to use object pinning and guarantee that the > object does not move. For example, this is easy to do with region-based > GCs (G1, Shenandoah, ZGC) by simply not including regions with pinned > objects in the collection set. > > The implementation/API that I'm proposing is fairly simple: add two > methods oop pin_object(oop) and void unpin_object(oop) to CollectedHeap, > and call them from JNI's Get/Release*Critical methods. > > This approach has been working perfectly fine since a long time in > Shenandoah. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8199620 > Webrev: > http://cr.openjdk.java.net/~rkennke/8199620/webrev.00/ And here is the correct patch: http://cr.openjdk.java.net/~rkennke/8199620/webrev.01/ Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed Mar 14 20:34:12 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 14 Mar 2018 21:34:12 +0100 Subject: RFR: JDK-8199612: Replace remaining uses of G1BarrierSet::enqueue() with appropriate Access API calls In-Reply-To: References: Message-ID: <08ea65a6-e682-ec80-f006-e974aec2e12b@redhat.com> Am 14.03.2018 um 15:30 schrieb Roman Kennke: > We still have 2 places where we call directly to > G1BarrierSet::enqueue(), those should use the appropriate Access API > instead. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8199612 > Webrev: > http://cr.openjdk.java.net/~rkennke/8199612/webrev.00/ > > Passes: tier1_runtime tests, currently running all of tier1 > > Please review! > > Thanks, Roman > Apparently this is already being worked on elsewhere. I'm withdrawing this RFR and bug-report. Thanks, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From Derek.White at cavium.com Wed Mar 14 22:22:07 2018 From: Derek.White at cavium.com (White, Derek) Date: Wed, 14 Mar 2018 22:22:07 +0000 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: <413a2cd0-295f-e179-ff79-84f135b05b05@redhat.com> References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> <413a2cd0-295f-e179-ff79-84f135b05b05@redhat.com> Message-ID: Hi Roman, Interesting idea. Is the intention to NOT call GCLocker::lock_critical() in the case when the object can be pinned? I can see how this allows pinning a region, but I don't see in this webrev where we?re getting any benefit. Thanks! - Derek > -----Original Message----- > From: hotspot-gc-dev [mailto:hotspot-gc-dev-bounces at openjdk.java.net] > On Behalf Of Roman Kennke > Sent: Wednesday, March 14, 2018 2:51 PM > To: hotspot-gc-dev at openjdk.java.net > Subject: Re: RFR: JDK-8199620: Support for JNI object pinning > > Am 14.03.2018 um 19:13 schrieb Roman Kennke: > > Currently, the Get/Release*Critical() family of functions use the > > GCLocker protocol to ensure that no JNI critical arrays are in use > > when a GC pause is entered. > > > > Some GCs may instead want to use object pinning and guarantee that the > > object does not move. For example, this is easy to do with > > region-based GCs (G1, Shenandoah, ZGC) by simply not including regions > > with pinned objects in the collection set. > > > > The implementation/API that I'm proposing is fairly simple: add two > > methods oop pin_object(oop) and void unpin_object(oop) to > > CollectedHeap, and call them from JNI's Get/Release*Critical methods. > > > > This approach has been working perfectly fine since a long time in > > Shenandoah. > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8199620 > > Webrev: > > http://cr.openjdk.java.net/~rkennke/8199620/webrev.00/ > > And here is the correct patch: > > http://cr.openjdk.java.net/~rkennke/8199620/webrev.01/ > > Roman From jini.george at oracle.com Thu Mar 15 04:21:13 2018 From: jini.george at oracle.com (Jini George) Date: Thu, 15 Mar 2018 09:51:13 +0530 Subject: RFR: JDK-8175312: SA: clhsdb: Provide an improved heap summary for 'universe' for G1GC In-Reply-To: References: <38d71740-0b66-3ce8-26ed-a0f2b9f9e91c@oracle.com> <5e8c582e-b32f-daf7-0e0c-1e6606ceaf3a@oracle.com> <59c7ec4f-f9dc-7c25-e7bf-f3fc304b5c60@oracle.com> Message-ID: Thank you very much, Yumin. The modified webrev (after incorporating your comment) is at: http://cr.openjdk.java.net/~jgeorge/8175312/webrev.03/index.html Thanks, Jini. On 3/13/2018 6:24 AM, yumin qi wrote: > Jini, > > ? Looks good. One minor comment: > > + public void printG1HeapSummary(G1CollectedHeap heap) { > + G1CollectedHeap g1h = (G1CollectedHeap) heap; > > ?'heap' has been cast to 'G1CollectedHeap' at call site, so seems no > need to convert here again. > > Thanks > Yumin > > On Mon, Mar 12, 2018 at 8:52 AM, Jini George > wrote: > > Thank you very much, Stefan. Could one more reviewer please take a > look at it ? > > - Jini. > > > On 3/12/2018 8:52 PM, Stefan Johansson wrote: > > Hi Jini, > > This looks good. I'm totally fine with skipping metaspace if > that isn't displayed for the other GCs. > > Cheers, > Stefan > > On 2018-03-09 10:29, Jini George wrote: > > Here is the revised webrev: > > http://cr.openjdk.java.net/~jgeorge/8175312/webrev.02/ > > > I have made modifications to have the 'universe' command > display details like: > > hsdb> universe > Heap Parameters: > garbage-first heap [0x0000000725200000, 0x00000007c0000000] > region size 1024K > G1 Heap: > ?? regions? = 2478 > ?? capacity = 2598371328 (2478.0MB) > ?? used???? = 5242880 (5.0MB) > ?? free???? = 2593128448 (2473.0MB) > ?? 0.20177562550443906% used > G1 Young Generation: > Eden Space: > ?? regions? = 5 > ?? capacity = 8388608 (8.0MB) > ?? used???? = 5242880 (5.0MB) > ?? free???? = 3145728 (3.0MB) > ?? 62.5% used > Survivor Space: > ?? regions? = 0 > ?? capacity = 0 (0.0MB) > ?? used???? = 0 (0.0MB) > ?? free???? = 0 (0.0MB) > ?? 0.0% used > G1 Old Generation: > ?? regions? = 0 > ?? capacity = 155189248 (148.0MB) > ?? used???? = 0 (0.0MB) > ?? free???? = 155189248 (148.0MB) > ?? 0.0% used > > > I did not add the metaspace details since that did not seem > to be in line with the 'universe' output for other GCs. I > have added a new command "g1regiondetails" to display the > region details, and have modified the tests accordingly. > > hsdb> g1regiondetails > Region Details: > Region: > 0x0000000725200000,0x0000000725200000,0x0000000725300000:Free > Region: > 0x0000000725300000,0x0000000725300000,0x0000000725400000:Free > Region: > 0x0000000725400000,0x0000000725400000,0x0000000725500000:Free > Region: > 0x0000000725500000,0x0000000725500000,0x0000000725600000:Free > Region: > 0x0000000725600000,0x0000000725600000,0x0000000725700000:Free > Region: > 0x0000000725700000,0x0000000725700000,0x0000000725800000:Free > ... > > Thanks, > Jini. > > > On 2/28/2018 12:56 PM, Jini George wrote: > > Thank you very much, Stefan. My answers inline. > > On 2/27/2018 3:30 PM, Stefan Johansson wrote: > > Hi Jini, > > > JIRA > ID:https://bugs.openjdk.java.net/browse/JDK-8175312 > > Webrev: > http://cr.openjdk.java.net/~jgeorge/8175312/webrev.00/index.html > > > It looks like a file is missing, did you forget to > add it to the changeset? > > > Indeed, I had missed that! I added the missing file in > the following webrev: > > http://cr.openjdk.java.net/~jgeorge/8175312/webrev.01/ > > > --- > open/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/g1/G1CollectedHeap.java:36: > error: cannot find symbol > import sun.jvm.hotspot.gc.shared.PrintRegionClosure; > --- > > Otherwise the change looks good, but I would like to > see the output live. For a big heap this will print > a lot of data, just wondering if the universe > command is the correct choice for this kind of > output. I like having the possibility to print all > regions, so I want the change but maybe it should be > a different command and 'universe' just prints a > little more than before. Something like our logging > heap-summary at shutdown: > garbage-first heap?? total 16384K, used 3072K > [0x00000000ff000000, 0x0000000100000000) > ??region size 1024K, 4 young (4096K), 0 survivors (0K) > Metaspace?????? used 6731K, capacity 6825K, > committed 7040K, reserved 1056768K > ??class space??? used 559K, capacity 594K, > committed 640K, reserved 1048576K > > > Ok, will add this, and could probably have the region > details displayed under a new command called > "g1regiondetails", or some such, and send out a new webrev. > > Thanks, > Jini. > > > Thanks, > Stefan > > Modifications have been made to display the > regions like: > > ... > Region: > 0x00000005c5400000,0x00000005c5600000,0x00000005c5600000:Old > Region: > 0x00000005c5600000,0x00000005c5800000,0x00000005c5800000:Old > Region: > 0x00000005c5800000,0x00000005c5a00000,0x00000005c5a00000:Old > Region: > 0x00000005c5a00000,0x00000005c5c00000,0x00000005c5c00000:Old > Region: > 0x00000005c5c00000,0x00000005c5c00000,0x00000005c5e00000:Free > Region: > 0x00000005c5e00000,0x00000005c5e00000,0x00000005c6000000:Free > Region: > 0x00000005c6000000,0x00000005c6200000,0x00000005c6200000:Old > ... > > The jtreg test at this point does not > include any testing for the display of > archived or pinned regions. The testing for > this will be added once JDK-8174994 is resolved. > > The SA tests pass with jprt and Mach5. > > Thanks, > Jini. > > > > From rkennke at redhat.com Thu Mar 15 10:20:32 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 15 Mar 2018 11:20:32 +0100 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> <413a2cd0-295f-e179-ff79-84f135b05b05@redhat.com> Message-ID: <531380d5-df59-b235-0b72-0748b1ce9d5d@redhat.com> Hi Derek, I did not provide an implementation for G1 because I am not in the position to decide for G1 to use pinning or GCLocker. Nor do I know well G1 enough that I could be confident to get it right. The implementation should be fairly simple though: in Shenandoah we simply set a flag in the heap region containing the object, and when we get to build the collection set, skip the regions with the pinned flag set. The JNI code should still call GCLocker::lock_critical() and unlock_critical(). As long as the GC does not participate in the protocol, this amounts to counting up and down. It *is* possible to use both mechanisms in one GC even. For a while, in Shenandoah, we considered to use pinning for concurrent phases, but use the GCLocker protocol in full-GC, but we opted against it, because the coordination between GCLocker, control thread, GC workers and mutators seemed to error/deadlock-prone, and, TBH, critical arrays are not very common and don't seem worth the hassle. I hope that clarifies the idea a bit? Roman > Interesting idea. > > Is the intention to NOT call GCLocker::lock_critical() in the case when the object can be pinned? > > I can see how this allows pinning a region, but I don't see in this webrev where we?re getting any benefit. > > Thanks! > > - Derek > >> -----Original Message----- >> From: hotspot-gc-dev [mailto:hotspot-gc-dev-bounces at openjdk.java.net] >> On Behalf Of Roman Kennke >> Sent: Wednesday, March 14, 2018 2:51 PM >> To: hotspot-gc-dev at openjdk.java.net >> Subject: Re: RFR: JDK-8199620: Support for JNI object pinning >> >> Am 14.03.2018 um 19:13 schrieb Roman Kennke: >>> Currently, the Get/Release*Critical() family of functions use the >>> GCLocker protocol to ensure that no JNI critical arrays are in use >>> when a GC pause is entered. >>> >>> Some GCs may instead want to use object pinning and guarantee that the >>> object does not move. For example, this is easy to do with >>> region-based GCs (G1, Shenandoah, ZGC) by simply not including regions >>> with pinned objects in the collection set. >>> >>> The implementation/API that I'm proposing is fairly simple: add two >>> methods oop pin_object(oop) and void unpin_object(oop) to >>> CollectedHeap, and call them from JNI's Get/Release*Critical methods. >>> >>> This approach has been working perfectly fine since a long time in >>> Shenandoah. >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8199620 >>> Webrev: >>> http://cr.openjdk.java.net/~rkennke/8199620/webrev.00/ >> >> And here is the correct patch: >> >> http://cr.openjdk.java.net/~rkennke/8199620/webrev.01/ >> >> Roman > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Thu Mar 15 12:50:33 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 15 Mar 2018 13:50:33 +0100 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: <413a2cd0-295f-e179-ff79-84f135b05b05@redhat.com> References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> <413a2cd0-295f-e179-ff79-84f135b05b05@redhat.com> Message-ID: <75a17e6e-a8dd-3d79-26d8-7ae4eda08ce9@oracle.com> Hi Roman, On 03/14/2018 07:51 PM, Roman Kennke wrote: > Am 14.03.2018 um 19:13 schrieb Roman Kennke: >> Currently, the Get/Release*Critical() family of functions use the >> GCLocker protocol to ensure that no JNI critical arrays are in use when >> a GC pause is entered. >> >> Some GCs may instead want to use object pinning and guarantee that the >> object does not move. For example, this is easy to do with region-based >> GCs (G1, Shenandoah, ZGC) by simply not including regions with pinned >> objects in the collection set. >> >> The implementation/API that I'm proposing is fairly simple: add two >> methods oop pin_object(oop) and void unpin_object(oop) to CollectedHeap, >> and call them from JNI's Get/Release*Critical methods. >> >> This approach has been working perfectly fine since a long time in >> Shenandoah. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8199620 >> Webrev: >> http://cr.openjdk.java.net/~rkennke/8199620/webrev.00/ > > And here is the correct patch: > > http://cr.openjdk.java.net/~rkennke/8199620/webrev.01/ It seems your patch both grabs the GCLocker _and_ pins the object? I would have assumed that the default implementation of pin_objct() would be the one calling the GCLocker if pinning isn't supported by the GC. Also, it seems that this patch doesn't take "critical native" functions into account, i.e. those special functions which grabs the GCLocker "lazily" when a safepoint happens. See SafepointSynchronize::check_for_lazy_critical_native(). cheers, Per > > Roman > From stefan.johansson at oracle.com Thu Mar 15 13:05:01 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 15 Mar 2018 14:05:01 +0100 Subject: RFR: 8199674: Improve G1 Full GC array marking Message-ID: <64527c58-0876-47c8-cdf8-09dadfb08bf9@oracle.com> Hi, Please review this change to improve the Full GC marking code. Links JBS: https://bugs.openjdk.java.net/browse/JDK-8199674 Webrev: http://cr.openjdk.java.net/~sjohanss/8199674/00/ Summary Currently when marking an object array it is split up to avoid one worker doing all the work. The code iterates over a chunk of the array and then push a task for the rest of the array to the marking stack. The problem with this is that if the iteration it self don't find any new pointers to follow, the next thing to do is to handle the next chunk in the array. This make work-stealing problematic and the performance far from optimal. There are many schemes that can be used to improve the situation and we might want to dig into this more at some point. The fix I propose here is simple and easy but gives good improvements to the micro benchmarks I've tested it on. The fix is to push the task for the rest of the array onto the mark stack before doing the iteration of the current chunk. Testing Functional testing through mach5 and local performance testing. Created two small JMH benchmarks that create a big object array, one fills it with object the other one is completely empty. Baseline: Benchmark????????????????? Mode? Cnt??? Score??? Error? Units SystemGCOneBigEmpty.bench??? ss?? 25?? 179.740 ??? 8.497? ms/op SystemGCOneBigFull.bench???? ss?? 25? 5220.095 ? 192.680? ms/op With patch applied: Benchmark????????????????? Mode? Cnt??? Score???? Error? Units SystemGCOneBigEmpty.bench??? ss?? 25?? 111.854 ?? 7.596? ms/op SystemGCOneBigFull.bench???? ss?? 25? 1713.544 ? 36.956? ms/op These benchmarks measure the whole time of the Full GC and not only the marking times but the improvement is still pretty clear. Thanks, Stefan From shade at redhat.com Thu Mar 15 13:11:11 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 15 Mar 2018 14:11:11 +0100 Subject: RFR: 8199674: Improve G1 Full GC array marking In-Reply-To: <64527c58-0876-47c8-cdf8-09dadfb08bf9@oracle.com> References: <64527c58-0876-47c8-cdf8-09dadfb08bf9@oracle.com> Message-ID: <2cb74c0a-76bb-eca0-b0ef-8d41a6676965@redhat.com> On 03/15/2018 02:05 PM, Stefan Johansson wrote: > JBS: https://bugs.openjdk.java.net/browse/JDK-8199674 > Webrev: http://cr.openjdk.java.net/~sjohanss/8199674/00/ Yes, of course. It is silly to do this in other order :) I'll add a new line in-between here, because those blocks are logically separate: 115 array->oop_iterate_range(mark_closure(), beg_index, end_index); 116 if (VerifyDuringGC) { -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Thu Mar 15 13:13:24 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 15 Mar 2018 14:13:24 +0100 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: <75a17e6e-a8dd-3d79-26d8-7ae4eda08ce9@oracle.com> References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> <413a2cd0-295f-e179-ff79-84f135b05b05@redhat.com> <75a17e6e-a8dd-3d79-26d8-7ae4eda08ce9@oracle.com> Message-ID: Am 15.03.2018 um 13:50 schrieb Per Liden: > Hi Roman, > > On 03/14/2018 07:51 PM, Roman Kennke wrote: >> Am 14.03.2018 um 19:13 schrieb Roman Kennke: >>> Currently, the Get/Release*Critical() family of functions use the >>> GCLocker protocol to ensure that no JNI critical arrays are in use when >>> a GC pause is entered. >>> >>> Some GCs may instead want to use object pinning and guarantee that the >>> object does not move. For example, this is easy to do with region-based >>> GCs (G1, Shenandoah, ZGC) by simply not including regions with pinned >>> objects in the collection set. >>> >>> The implementation/API that I'm proposing is fairly simple: add two >>> methods oop pin_object(oop) and void unpin_object(oop) to CollectedHeap, >>> and call them from JNI's Get/Release*Critical methods. >>> >>> This approach has been working perfectly fine since a long time in >>> Shenandoah. >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8199620 >>> Webrev: >>> http://cr.openjdk.java.net/~rkennke/8199620/webrev.00/ >> >> And here is the correct patch: >> >> http://cr.openjdk.java.net/~rkennke/8199620/webrev.01/ > > It seems your patch both grabs the GCLocker _and_ pins the object? I > would have assumed that the default implementation of pin_objct() would > be the one calling the GCLocker if pinning isn't supported by the GC. As I explained to Derek, this is kindof intentional. Calling into GCLocker::lock_critical() and GCLocker::unlock_critical() does nothing in the way of waiting or blocking or otherwise harmful, as long as the GC does not participate in the protocol. This way a GC could infact use both mechanisms, e.g. using pinning and ignore GCLocker for the pauses around concurrent phases, and ignore pinning and use GCLocker for full-GC pauses. > Also, it seems that this patch doesn't take "critical native" functions > into account, i.e. those special functions which grabs the GCLocker > "lazily" when a safepoint happens. See > SafepointSynchronize::check_for_lazy_critical_native(). I am not sure that I'd have to do anything there? What's this supposed to do? Prevent to reach a GC safepoint as long as such a function is in progress? With what purpose? Can you give me an example? Thanks, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Thu Mar 15 13:17:01 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 15 Mar 2018 14:17:01 +0100 Subject: RFR: 8199674: Improve G1 Full GC array marking In-Reply-To: <64527c58-0876-47c8-cdf8-09dadfb08bf9@oracle.com> References: <64527c58-0876-47c8-cdf8-09dadfb08bf9@oracle.com> Message-ID: <1521119821.2448.27.camel@oracle.com> Hi, On Thu, 2018-03-15 at 14:05 +0100, Stefan Johansson wrote: > Hi, > > Please review this change to improve the Full GC marking code. > [...] > The fix is to push the task for the > rest of the array onto the mark stack before doing the iteration of > the current chunk. > looks good. Please add the suggested newline before pushing. Thanks, Thomas From stefan.johansson at oracle.com Thu Mar 15 13:26:33 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 15 Mar 2018 14:26:33 +0100 Subject: RFR: 8199674: Improve G1 Full GC array marking In-Reply-To: <1521119821.2448.27.camel@oracle.com> References: <64527c58-0876-47c8-cdf8-09dadfb08bf9@oracle.com> <1521119821.2448.27.camel@oracle.com> Message-ID: <12e5c3ad-1a45-ea9b-96cb-3587043c0bcc@oracle.com> Thanks Thomas and Aleksey, I will add the newline and push tomorrow. Cheers, Stefan On 2018-03-15 14:17, Thomas Schatzl wrote: > Hi, > > On Thu, 2018-03-15 at 14:05 +0100, Stefan Johansson wrote: >> Hi, >> >> Please review this change to improve the Full GC marking code. >> > [...] >> The fix is to push the task for the >> rest of the array onto the mark stack before doing the iteration of >> the current chunk. >> > looks good. Please add the suggested newline before pushing. > > Thanks, > Thomas > > From rkennke at redhat.com Thu Mar 15 13:31:17 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 15 Mar 2018 14:31:17 +0100 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: <75a17e6e-a8dd-3d79-26d8-7ae4eda08ce9@oracle.com> References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> <413a2cd0-295f-e179-ff79-84f135b05b05@redhat.com> <75a17e6e-a8dd-3d79-26d8-7ae4eda08ce9@oracle.com> Message-ID: <92582a69-5b07-e844-5a5f-036679e90ff1@redhat.com> Am 15.03.2018 um 13:50 schrieb Per Liden: > Hi Roman, > > On 03/14/2018 07:51 PM, Roman Kennke wrote: >> Am 14.03.2018 um 19:13 schrieb Roman Kennke: >>> Currently, the Get/Release*Critical() family of functions use the >>> GCLocker protocol to ensure that no JNI critical arrays are in use when >>> a GC pause is entered. >>> >>> Some GCs may instead want to use object pinning and guarantee that the >>> object does not move. For example, this is easy to do with region-based >>> GCs (G1, Shenandoah, ZGC) by simply not including regions with pinned >>> objects in the collection set. >>> >>> The implementation/API that I'm proposing is fairly simple: add two >>> methods oop pin_object(oop) and void unpin_object(oop) to CollectedHeap, >>> and call them from JNI's Get/Release*Critical methods. >>> >>> This approach has been working perfectly fine since a long time in >>> Shenandoah. >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8199620 >>> Webrev: >>> http://cr.openjdk.java.net/~rkennke/8199620/webrev.00/ >> >> And here is the correct patch: >> >> http://cr.openjdk.java.net/~rkennke/8199620/webrev.01/ > > It seems your patch both grabs the GCLocker _and_ pins the object? I > would have assumed that the default implementation of pin_objct() would > be the one calling the GCLocker if pinning isn't supported by the GC. > > Also, it seems that this patch doesn't take "critical native" functions > into account, i.e. those special functions which grabs the GCLocker > "lazily" when a safepoint happens. See > SafepointSynchronize::check_for_lazy_critical_native(). > I researched a little, and this seems only be used in: src/jdk.crypto.ucrypto/solaris/native/libj2ucrypto/nativeCrypto.c src/jdk.crypto.ucrypto/solaris/native/libj2ucrypto/nativeCryptoMD.c and it is an optional internal feature, to be turned off by -CriticalJNINatives and only used in Solaris. Opting to ignore it must be safe, or not? Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From Derek.White at cavium.com Thu Mar 15 15:38:56 2018 From: Derek.White at cavium.com (White, Derek) Date: Thu, 15 Mar 2018 15:38:56 +0000 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: <531380d5-df59-b235-0b72-0748b1ce9d5d@redhat.com> References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> <413a2cd0-295f-e179-ff79-84f135b05b05@redhat.com> <531380d5-df59-b235-0b72-0748b1ce9d5d@redhat.com> Message-ID: Hi Roman, Thanks for clearing that up. I didn't consider a GC opting out of the GCLocker protocol. [BTW, asking as a GC enthusiast, no longer a GC professional] Is there a danger that the API for these two options of freezing an object (pinning vs. gc lock-out) being too disjoint and asymmetrical? The GCLocker::lock_critical() API knows the thread requesting the lock, but nothing about the object (address, size, type, etc) while the pinning API knows all of that. You could imagine schemes that would take object information into account when deciding to lock-out or to pin. For example, pinning an object that fully occupies a region has no drawbacks, while pinning tiny objects might end up protecting a lot of garbage (and these are cases that would have had short lock-out durations anyway). I don't see how the proposed API could support this case. But if you consider handling this case over-engineering for the problem at hand, I understand. - Derek > -----Original Message----- > From: Roman Kennke [mailto:rkennke at redhat.com] > Sent: Thursday, March 15, 2018 6:21 AM > To: White, Derek ; hotspot-gc- > dev at openjdk.java.net > Subject: Re: RFR: JDK-8199620: Support for JNI object pinning > > Hi Derek, > > I did not provide an implementation for G1 because I am not in the position > to decide for G1 to use pinning or GCLocker. Nor do I know well > G1 enough that I could be confident to get it right. > > The implementation should be fairly simple though: in Shenandoah we > simply set a flag in the heap region containing the object, and when we get > to build the collection set, skip the regions with the pinned flag set. > > The JNI code should still call GCLocker::lock_critical() and unlock_critical(). As > long as the GC does not participate in the protocol, this amounts to counting > up and down. > > It *is* possible to use both mechanisms in one GC even. For a while, in > Shenandoah, we considered to use pinning for concurrent phases, but use > the GCLocker protocol in full-GC, but we opted against it, because the > coordination between GCLocker, control thread, GC workers and mutators > seemed to error/deadlock-prone, and, TBH, critical arrays are not very > common and don't seem worth the hassle. > > I hope that clarifies the idea a bit? > > Roman > > > > > Interesting idea. > > > > Is the intention to NOT call GCLocker::lock_critical() in the case when the > object can be pinned? > > > > I can see how this allows pinning a region, but I don't see in this webrev > where we?re getting any benefit. > > > > Thanks! > > > > - Derek > > > >> -----Original Message----- > >> From: hotspot-gc-dev [mailto:hotspot-gc-dev-bounces at openjdk.java.net] > >> On Behalf Of Roman Kennke > >> Sent: Wednesday, March 14, 2018 2:51 PM > >> To: hotspot-gc-dev at openjdk.java.net > >> Subject: Re: RFR: JDK-8199620: Support for JNI object pinning > >> > >> Am 14.03.2018 um 19:13 schrieb Roman Kennke: > >>> Currently, the Get/Release*Critical() family of functions use the > >>> GCLocker protocol to ensure that no JNI critical arrays are in use > >>> when a GC pause is entered. > >>> > >>> Some GCs may instead want to use object pinning and guarantee that > >>> the object does not move. For example, this is easy to do with > >>> region-based GCs (G1, Shenandoah, ZGC) by simply not including > >>> regions with pinned objects in the collection set. > >>> > >>> The implementation/API that I'm proposing is fairly simple: add two > >>> methods oop pin_object(oop) and void unpin_object(oop) to > >>> CollectedHeap, and call them from JNI's Get/Release*Critical methods. > >>> > >>> This approach has been working perfectly fine since a long time in > >>> Shenandoah. > >>> > >>> Bug: > >>> https://bugs.openjdk.java.net/browse/JDK-8199620 > >>> Webrev: > >>> http://cr.openjdk.java.net/~rkennke/8199620/webrev.00/ > >> > >> And here is the correct patch: > >> > >> http://cr.openjdk.java.net/~rkennke/8199620/webrev.01/ > >> > >> Roman > > > From rkennke at redhat.com Thu Mar 15 16:57:43 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 15 Mar 2018 17:57:43 +0100 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> <413a2cd0-295f-e179-ff79-84f135b05b05@redhat.com> <531380d5-df59-b235-0b72-0748b1ce9d5d@redhat.com> Message-ID: <35D6A087-024B-4883-82CE-A269A3D04200@redhat.com> The calls into GCLocker in jni.cpp are only the JNI part of the protocol. The GC must still do its part to initiate the actual wait-and-retry when no threads are in critical region. It is conceivable with the proposed API that the GC sees a small object in a region of otherwise garbage and elects to participate in GCLocker instead of pinning the object / region, on a case-by-case basis. What I'm not sure about is support for CriticalJNINatives... This does not seem easily possible without GCLocker. But I would consider that optional. Thanks, Roman Am 15. M?rz 2018 16:38:56 MEZ schrieb "White, Derek" : >Hi Roman, > >Thanks for clearing that up. I didn't consider a GC opting out of the >GCLocker protocol. > >[BTW, asking as a GC enthusiast, no longer a GC professional] > >Is there a danger that the API for these two options of freezing an >object (pinning vs. gc lock-out) being too disjoint and asymmetrical? > >The GCLocker::lock_critical() API knows the thread requesting the lock, >but nothing about the object (address, size, type, etc) while the >pinning API knows all of that. > >You could imagine schemes that would take object information into >account when deciding to lock-out or to pin. For example, pinning an >object that fully occupies a region has no drawbacks, while pinning >tiny objects might end up protecting a lot of garbage (and these are >cases that would have had short lock-out durations anyway). > >I don't see how the proposed API could support this case. But if you >consider handling this case over-engineering for the problem at hand, I >understand. > > - Derek > >> -----Original Message----- >> From: Roman Kennke [mailto:rkennke at redhat.com] >> Sent: Thursday, March 15, 2018 6:21 AM >> To: White, Derek ; hotspot-gc- >> dev at openjdk.java.net >> Subject: Re: RFR: JDK-8199620: Support for JNI object pinning >> >> Hi Derek, >> >> I did not provide an implementation for G1 because I am not in the >position >> to decide for G1 to use pinning or GCLocker. Nor do I know well >> G1 enough that I could be confident to get it right. >> >> The implementation should be fairly simple though: in Shenandoah we >> simply set a flag in the heap region containing the object, and when >we get >> to build the collection set, skip the regions with the pinned flag >set. >> >> The JNI code should still call GCLocker::lock_critical() and >unlock_critical(). As >> long as the GC does not participate in the protocol, this amounts to >counting >> up and down. >> >> It *is* possible to use both mechanisms in one GC even. For a while, >in >> Shenandoah, we considered to use pinning for concurrent phases, but >use >> the GCLocker protocol in full-GC, but we opted against it, because >the >> coordination between GCLocker, control thread, GC workers and >mutators >> seemed to error/deadlock-prone, and, TBH, critical arrays are not >very >> common and don't seem worth the hassle. >> >> I hope that clarifies the idea a bit? >> >> Roman >> >> >> >> > Interesting idea. >> > >> > Is the intention to NOT call GCLocker::lock_critical() in the case >when the >> object can be pinned? >> > >> > I can see how this allows pinning a region, but I don't see in this >webrev >> where we?re getting any benefit. >> > >> > Thanks! >> > >> > - Derek >> > >> >> -----Original Message----- >> >> From: hotspot-gc-dev >[mailto:hotspot-gc-dev-bounces at openjdk.java.net] >> >> On Behalf Of Roman Kennke >> >> Sent: Wednesday, March 14, 2018 2:51 PM >> >> To: hotspot-gc-dev at openjdk.java.net >> >> Subject: Re: RFR: JDK-8199620: Support for JNI object pinning >> >> >> >> Am 14.03.2018 um 19:13 schrieb Roman Kennke: >> >>> Currently, the Get/Release*Critical() family of functions use the >> >>> GCLocker protocol to ensure that no JNI critical arrays are in >use >> >>> when a GC pause is entered. >> >>> >> >>> Some GCs may instead want to use object pinning and guarantee >that >> >>> the object does not move. For example, this is easy to do with >> >>> region-based GCs (G1, Shenandoah, ZGC) by simply not including >> >>> regions with pinned objects in the collection set. >> >>> >> >>> The implementation/API that I'm proposing is fairly simple: add >two >> >>> methods oop pin_object(oop) and void unpin_object(oop) to >> >>> CollectedHeap, and call them from JNI's Get/Release*Critical >methods. >> >>> >> >>> This approach has been working perfectly fine since a long time >in >> >>> Shenandoah. >> >>> >> >>> Bug: >> >>> https://bugs.openjdk.java.net/browse/JDK-8199620 >> >>> Webrev: >> >>> http://cr.openjdk.java.net/~rkennke/8199620/webrev.00/ >> >> >> >> And here is the correct patch: >> >> >> >> http://cr.openjdk.java.net/~rkennke/8199620/webrev.01/ >> >> >> >> Roman >> > >> -- Diese Nachricht wurde von meinem Android-Ger?t mit K-9 Mail gesendet. From irogers at google.com Thu Mar 15 17:49:53 2018 From: irogers at google.com (Ian Rogers) Date: Thu, 15 Mar 2018 17:49:53 +0000 Subject: GetPrimitiveArrayCritical vs GetByteArrayRegion: 140x slow-down using -Xcheck:jni and java.util.zip.DeflaterOutputStream In-Reply-To: <1521102348.2448.25.camel@oracle.com> References: <22bbd632-47c1-7702-bd65-6a51f0bbad02@oracle.com> <9b5ff318-af41-03c4-efdc-b41b97cf6e61@oracle.com> <5A9D8C60.3050505@oracle.com> <5A9D8F33.5060500@oracle.com> <5A9DAC84.906@oracle.com> <5A9DAE87.8020801@oracle.com> <1521102348.2448.25.camel@oracle.com> Message-ID: +hotspot-gc-dev On Thu, Mar 15, 2018 at 1:25 AM Thomas Schatzl wrote: > Hi, > > On Thu, 2018-03-15 at 01:00 +0000, Ian Rogers wrote: > > An old data point on how large a critical region should be comes from > > java.nio.Bits. In JDK 9 the code migrated into unsafe, but in JDK 8 > > the copies within a critical region were bound at most copying 1MB: > > http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/ > > native/java/nio/Bits.c#l88 This is inconsistent with Deflater and > > ObjectOutputStream which both allow unlimited arrays and thereby > > critical region sizes. > > > > In JDK 9 the copies starve the garbage collector in nio Bits too as > > there is no 1MB limit to the copy sizes: > > http://hg.openjdk.java.net/jdk/jdk/rev/f70e100d3195 > > which came from: > > https://bugs.openjdk.java.net/browse/JDK-8149596 > > > > Perhaps this is a regression not demonstrated due to the testing > > challenge. > > [...] > > It doesn't seem unreasonable to have the loops for the copies occur > > in 1MB chunks but JDK-8149596 lost this and so I'm confused on what > > the HotSpot stand point is. > > Please file a bug (seems to be a core-libs/java.nio regression?), > preferably with some kind of regression test. Also file enhancements (I > would guess) for the other cases allowing unlimited arrays. > I don't have perms to file bugs there's some catch-22 scenario in getting the permissions. Happy to have a bug filed or to file were that not an issue. Happy to create a test case but can't see any others for TTSP issues. This feels like a potential use case for jmh, perhaps run the benchmark well having a separate thread run GC bench. Should there be a bug to add, in debug mode, a TTSP watcher thread whose job it is to bring "random" threads into safepoints and report on tardy ones? Should there be a bug to warn on being in a JNI critical for more than just a short period? Seems like there should be a bug on Unsafe.copyMemory and Unsafe.copySwapMemory having TTSP issues. Seems like there should be a bug on all uses of critical that don't chunk their critical region work based on some bound (like 1MB chunks for nio Bits)? How are these bounds set? A past reference that I've lost is in having the bound be the equivalent of 65535 bytecodes due to the expectation of GC work at least once in a method or on a loop backedge - I thought this was in a spec somewhere but now I can't find it. The bytecode size feels as arbitrary as 1MB, a time period would be better but that can depend on the kind of GC you want as delays with concurrent GC mean more than non-concurrent. Clearly the chunk size shouldn't just be 0, but this appears to currently be the norm in the JDK. The original reason for coming here was a 140x slow down in -Xcheck:jni in Deflater.deflate There are a few options there that its useful to enumerate: 1) rewrite in Java but there are correctness and open source related issues 2) remove underflow/overflow protection from critical arrays (revert JDK-6311046 or perhaps bound protection to arrays of a particular small size) - this removes checking and doesn't deal with TTSP 3) add a critical array slice API to JNI so that copies with -Xcheck:jni aren't unbounded (martinrb@ proposed this) - keeps checking but doesn't deal with TTSP 4) rewrite primitive array criticals with GetArrayRegion as O(n) beats the "silent killer" TTSP (effectively deprecate the critical APIs) In general (ie not just the deflate case) I think (1) is the most preferable. (2) and (3) both have TTSP issues. (4) isn't great performance wise, which motivates more use of approach (1), but I think deprecating criticals may just be the easiest and sanest way forward. I think that discussion is worth having on an e-mail thread rather than a bug. > Long TTSP is a performance bug as any other. > > > In a way criticals are better than unsafe as they may > > pin the memory and not starve GC, which shenandoah does. > > (Region based) Object pinning has its own share of problems: > > - only (relatively) easily implemented in region based collectors > > - may slow down pause a bit in presence of pinned regions/objects (for > non-concurrent copying collectors) > > - excessive use of pinning may cause OOME and VM exit probably earlier > than the gc locker. GC locker seems to provide a more gradual > degradation. E.g. pinning regions typically makes these regions > unavailable for allocation. > I.e. you still should not use it for many, very long living objects. > Of course this somewhat depends on the sophistication of the > implementation. > > I think region based pinning would be a good addition to other > collectors than Shenandoah too. It has been on our minds for a long > time, but there are so many other more important issues :), so of > course we are eager to see contributions in this area. ;) > > If you are interested on working on this, please ping us on hotspot-gc- > dev for implementation ideas to get you jump-started. > > Thanks, > Thomas > I'd rather deprecate criticals than build upon the complexity, but I'm very glad this is a concern. Thanks, Ian -------------- next part -------------- An HTML attachment was scrubbed... URL: From coleen.phillimore at oracle.com Thu Mar 15 23:30:30 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 15 Mar 2018 19:30:30 -0400 Subject: RFR (XXS) 8199716: Unused AdjustKlassClosure in psParallelCompact.hpp Message-ID: <6a8f8ff2-a95d-6d98-e52a-ec6c2cd112cd@oracle.com> open webrev at http://cr.openjdk.java.net/~coleenp/8199716.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8199716 Tested with mach5 tier1. thanks, Coleen From stefan.karlsson at oracle.com Fri Mar 16 06:25:25 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 16 Mar 2018 07:25:25 +0100 Subject: RFR (XXS) 8199716: Unused AdjustKlassClosure in psParallelCompact.hpp In-Reply-To: <6a8f8ff2-a95d-6d98-e52a-ec6c2cd112cd@oracle.com> References: <6a8f8ff2-a95d-6d98-e52a-ec6c2cd112cd@oracle.com> Message-ID: <836165f1-0149-eb5c-0a8c-f6e2c7e2ce69@oracle.com> Looks good. StefanK On 2018-03-16 00:30, coleen.phillimore at oracle.com wrote: > open webrev at http://cr.openjdk.java.net/~coleenp/8199716.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8199716 > > Tested with mach5 tier1. > thanks, > Coleen From thomas.schatzl at oracle.com Fri Mar 16 07:56:30 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 16 Mar 2018 08:56:30 +0100 Subject: RFR (XXS) 8199716: Unused AdjustKlassClosure in psParallelCompact.hpp In-Reply-To: <6a8f8ff2-a95d-6d98-e52a-ec6c2cd112cd@oracle.com> References: <6a8f8ff2-a95d-6d98-e52a-ec6c2cd112cd@oracle.com> Message-ID: <1521186990.2391.0.camel@oracle.com> Hi, On Thu, 2018-03-15 at 19:30 -0400, coleen.phillimore at oracle.com wrote: > open webrev at http://cr.openjdk.java.net/~coleenp/8199716.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8199716 > > Tested with mach5 tier1. looks good. Thomas From coleen.phillimore at oracle.com Fri Mar 16 12:00:59 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 16 Mar 2018 08:00:59 -0400 Subject: RFR (XXS) 8199716: Unused AdjustKlassClosure in psParallelCompact.hpp In-Reply-To: <836165f1-0149-eb5c-0a8c-f6e2c7e2ce69@oracle.com> References: <6a8f8ff2-a95d-6d98-e52a-ec6c2cd112cd@oracle.com> <836165f1-0149-eb5c-0a8c-f6e2c7e2ce69@oracle.com> Message-ID: Thanks, I think this is trivial. Coleen On 3/16/18 2:25 AM, Stefan Karlsson wrote: > Looks good. > > StefanK > > On 2018-03-16 00:30, coleen.phillimore at oracle.com wrote: >> open webrev at http://cr.openjdk.java.net/~coleenp/8199716.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8199716 >> >> Tested with mach5 tier1. >> thanks, >> Coleen > > From coleen.phillimore at oracle.com Fri Mar 16 12:33:11 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 16 Mar 2018 08:33:11 -0400 Subject: RFR (XXS) 8199716: Unused AdjustKlassClosure in psParallelCompact.hpp In-Reply-To: <1521186990.2391.0.camel@oracle.com> References: <6a8f8ff2-a95d-6d98-e52a-ec6c2cd112cd@oracle.com> <1521186990.2391.0.camel@oracle.com> Message-ID: <321ed2b1-ca6c-b864-eebc-3ba981ee5080@oracle.com> Thanks! (I just pushed it as trivial) Coleen On 3/16/18 3:56 AM, Thomas Schatzl wrote: > Hi, > > On Thu, 2018-03-15 at 19:30 -0400, coleen.phillimore at oracle.com wrote: >> open webrev at http://cr.openjdk.java.net/~coleenp/8199716.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8199716 >> >> Tested with mach5 tier1. > looks good. > > Thomas From irogers at google.com Fri Mar 16 17:19:58 2018 From: irogers at google.com (Ian Rogers) Date: Fri, 16 Mar 2018 17:19:58 +0000 Subject: GetPrimitiveArrayCritical vs GetByteArrayRegion: 140x slow-down using -Xcheck:jni and java.util.zip.DeflaterOutputStream In-Reply-To: <23639144-5217-4A0F-930C-EF24B4976544@oracle.com> References: <22bbd632-47c1-7702-bd65-6a51f0bbad02@oracle.com> <9b5ff318-af41-03c4-efdc-b41b97cf6e61@oracle.com> <5A9D8C60.3050505@oracle.com> <5A9D8F33.5060500@oracle.com> <5A9DAC84.906@oracle.com> <5A9DAE87.8020801@oracle.com> <1521102348.2448.25.camel@oracle.com> <23639144-5217-4A0F-930C-EF24B4976544@oracle.com> Message-ID: Thanks Paul, very interesting. On Fri, Mar 16, 2018 at 9:21 AM Paul Sandoz wrote: > Hi Ian, Thomas, > > Some background on the bulk copying for byte buffers after talking with > Mikael who worked on these changes a few years ago. > > Please correct the following if needed as our memory is hazy :-) > > IIRC at the time we felt this was a reasonable thing to do because C2 did > not strip mine the loops for the bulk copying of large Java arrays i.e. the > issue was there anyway for more common cases. However, i believe that may > no longer be the so in some cases after Roland implemented loop strip > mining in C2 [1]. So we should go back and revisit/check the current > support in buffers and Java arrays (System.arraycopy etc). > The C2 issue is a well known TTSP issue :-) Its great that there is a strip mining optimization, revisiting the bulk copies would be great! > (This is also something we need to consider if we modify buffers to > support capacities larger than Integer.MAX_VALUE. Also connects with > Project Panama.) > > If Thomas has not done so or does not plan to i can log an issue for you. > That'd be great. I wonder if identifying more TTSP issues should also be a bug. Its interesting to observe that overlooking TTSP in C2 motivated the Unsafe.copyMemory change permitting a fresh TTSP issue. If TTSP is a 1st class issue then maybe we can deprecate JNI critical regions to support that effort :-) Thanks, Ian > Paul. > > > [1] https://bugs.openjdk.java.net/browse/JDK-8186027 > > On Mar 15, 2018, at 10:49 AM, Ian Rogers wrote: > > +hotspot-gc-dev > > On Thu, Mar 15, 2018 at 1:25 AM Thomas Schatzl > wrote: > > Hi, > > On Thu, 2018-03-15 at 01:00 +0000, Ian Rogers wrote: > > An old data point on how large a critical region should be comes from > java.nio.Bits. In JDK 9 the code migrated into unsafe, but in JDK 8 > the copies within a critical region were bound at most copying 1MB: > http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/ > native/java/nio/Bits.c#l88 This is inconsistent with Deflater and > ObjectOutputStream which both allow unlimited arrays and thereby > critical region sizes. > > In JDK 9 the copies starve the garbage collector in nio Bits too as > there is no 1MB limit to the copy sizes: > http://hg.openjdk.java.net/jdk/jdk/rev/f70e100d3195 > which came from: > https://bugs.openjdk.java.net/browse/JDK-8149596 > > Perhaps this is a regression not demonstrated due to the testing > challenge. > [...] > It doesn't seem unreasonable to have the loops for the copies occur > in 1MB chunks but JDK-8149596 lost this and so I'm confused on what > the HotSpot stand point is. > > > Please file a bug (seems to be a core-libs/java.nio regression?), > preferably with some kind of regression test. Also file enhancements (I > would guess) for the other cases allowing unlimited arrays. > > > I don't have perms to file bugs there's some catch-22 scenario in getting > the permissions. Happy to have a bug filed or to file were that not an > issue. Happy to create a test case but can't see any others for TTSP > issues. This feels like a potential use case for jmh, perhaps run the > benchmark well having a separate thread run GC bench. > > Should there be a bug to add, in debug mode, a TTSP watcher thread whose > job it is to bring "random" threads into safepoints and report on tardy > ones? > Should there be a bug to warn on being in a JNI critical for more than just > a short period? > Seems like there should be a bug on Unsafe.copyMemory and > Unsafe.copySwapMemory having TTSP issues. > Seems like there should be a bug on all uses of critical that don't chunk > their critical region work based on some bound (like 1MB chunks for nio > Bits)? How are these bounds set? A past reference that I've lost is in > having the bound be the equivalent of 65535 bytecodes due to the > expectation of GC work at least once in a method or on a loop backedge - I > thought this was in a spec somewhere but now I can't find it. The bytecode > size feels as arbitrary as 1MB, a time period would be better but that can > depend on the kind of GC you want as delays with concurrent GC mean more > than non-concurrent. Clearly the chunk size shouldn't just be 0, but this > appears to currently be the norm in the JDK. > > The original reason for coming here was a 140x slow down in -Xcheck:jni in > Deflater.deflate There are a few options there that its useful to > enumerate: > 1) rewrite in Java but there are correctness and open source related issues > 2) remove underflow/overflow protection from critical arrays (revert > JDK-6311046 > or perhaps bound protection to arrays of a particular small size) - this > removes checking and doesn't deal with TTSP > 3) add a critical array slice API to JNI so that copies with -Xcheck:jni > aren't unbounded (martinrb@ proposed this) - keeps checking but doesn't > deal with TTSP > 4) rewrite primitive array criticals with GetArrayRegion as O(n) beats the > "silent killer" TTSP (effectively deprecate the critical APIs) > > In general (ie not just the deflate case) I think (1) is the most > preferable. (2) and (3) both have TTSP issues. (4) isn't great performance > wise, which motivates more use of approach (1), but I think deprecating > criticals may just be the easiest and sanest way forward. I think that > discussion is worth having on an e-mail thread rather than a bug. > > > Long TTSP is a performance bug as any other. > > In a way criticals are better than unsafe as they may > pin the memory and not starve GC, which shenandoah does. > > > (Region based) Object pinning has its own share of problems: > > - only (relatively) easily implemented in region based collectors > > - may slow down pause a bit in presence of pinned regions/objects (for > non-concurrent copying collectors) > > - excessive use of pinning may cause OOME and VM exit probably earlier > than the gc locker. GC locker seems to provide a more gradual > degradation. E.g. pinning regions typically makes these regions > unavailable for allocation. > I.e. you still should not use it for many, very long living objects. > Of course this somewhat depends on the sophistication of the > implementation. > > I think region based pinning would be a good addition to other > collectors than Shenandoah too. It has been on our minds for a long > time, but there are so many other more important issues :), so of > course we are eager to see contributions in this area. ;) > > If you are interested on working on this, please ping us on hotspot-gc- > dev for implementation ideas to get you jump-started. > > Thanks, > Thomas > > > I'd rather deprecate criticals than build upon the complexity, but I'm very > glad this is a concern. > > Thanks, > Ian > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Mon Mar 19 09:28:07 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 19 Mar 2018 10:28:07 +0100 Subject: GetPrimitiveArrayCritical vs GetByteArrayRegion: 140x slow-down using -Xcheck:jni and java.util.zip.DeflaterOutputStream In-Reply-To: References: <22bbd632-47c1-7702-bd65-6a51f0bbad02@oracle.com> <9b5ff318-af41-03c4-efdc-b41b97cf6e61@oracle.com> <5A9D8C60.3050505@oracle.com> <5A9D8F33.5060500@oracle.com> <5A9DAC84.906@oracle.com> <5A9DAE87.8020801@oracle.com> <1521102348.2448.25.camel@oracle.com> <23639144-5217-4A0F-930C-EF24B4976544@oracle.com> Message-ID: <1521451687.2323.5.camel@oracle.com> Hi, On Fri, 2018-03-16 at 17:19 +0000, Ian Rogers wrote: > Thanks Paul, very interesting. > > On Fri, Mar 16, 2018 at 9:21 AM Paul Sandoz > wrote: > > Hi Ian, Thomas, > > > > [...] > > (This is also something we need to consider if we modify buffers to > > support capacities larger than Integer.MAX_VALUE. Also connects > > with Project Panama.) > > > > If Thomas has not done so or does not plan to i can log an issue > > for you. > > > > That'd be great. I wonder if identifying more TTSP issues should also > be a bug. Its interesting to observe that overlooking TTSP in C2 > motivated the Unsafe.copyMemory change permitting a fresh TTSP issue. > If TTSP is a 1st class issue then maybe we can deprecate JNI critical > regions to support that effort :-) Please log an issue. I am still a bit unsure what and how many issues should be filed. @Ian: at bugreports.oracle.com everyone may file bug reports without the need for an account. It will take some time until they show up in Jira due to vetting, but if you have a good case, and can e.g. link to the mailing list, this should be painless. Thanks, Thomas From rkennke at redhat.com Mon Mar 19 10:40:30 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 19 Mar 2018 11:40:30 +0100 Subject: RFR: JDK-8199735: Mark word updates need to use Access API Message-ID: <7bd73e32-a26b-2504-f069-b74807ae5e53@redhat.com> Currently, the mark word is accessed directly in oopDesc::mark() set_mark() and a bunch of other accessors. Those need to use the Access API instead because GC might want to employ barriers on those accesses, pretty much like every other field in oopDesc. Notice that this is *not* about accessing the bits and fields inside the markOop, but about accessing the header itself. http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.01/ Testing: build fastdebug/release with and without PCH, passed tier1 fastdebug/release. Can I please get reviews? Thanks, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Mon Mar 19 11:07:08 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 19 Mar 2018 12:07:08 +0100 Subject: RFR: JDK-8199735: Mark word updates need to use Access API In-Reply-To: <7bd73e32-a26b-2504-f069-b74807ae5e53@redhat.com> References: <7bd73e32-a26b-2504-f069-b74807ae5e53@redhat.com> Message-ID: <859ac657-78c9-c7ab-0ec0-eddda331001b@redhat.com> Am 19.03.2018 um 11:40 schrieb Roman Kennke: > Currently, the mark word is accessed directly in oopDesc::mark() > set_mark() and a bunch of other accessors. Those need to use the Access > API instead because GC might want to employ barriers on those accesses, > pretty much like every other field in oopDesc. > > Notice that this is *not* about accessing the bits and fields inside the > markOop, but about accessing the header itself. > > http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.01/ > > Testing: build fastdebug/release with and without PCH, passed tier1 > fastdebug/release. > > Can I please get reviews? > > Thanks, Roman > Just when I sent it, I realized that this is dropping the volatile from the mark word access. Fixed here: http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.02/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Mon Mar 19 12:03:49 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 19 Mar 2018 13:03:49 +0100 Subject: RFR [M][7/7]: 8197932: Better split work in rebuild remembered sets phase In-Reply-To: <1520953401.3100.41.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520265115.2532.126.camel@oracle.com> <1520953401.3100.41.camel@oracle.com> Message-ID: <1521461029.2323.11.camel@oracle.com> Hi all, Stefan proposed to use some iterator over the work area that internally handles the boundary at the TAMS wrt to liveness when scanning. This makes the change a bit more compact, performance does not seem to differ, so why not... :) Webrevs: http://cr.openjdk.java.net/~tschatzl/8197932/webrev.1_to_2/ (diff) http://cr.openjdk.java.net/~tschatzl/8197932/webrev.2/ (full) testing: hs-tier1-5 Thanks, Thomas From rkennke at redhat.com Mon Mar 19 14:23:10 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 19 Mar 2018 15:23:10 +0100 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> Message-ID: Am 14.03.2018 um 19:13 schrieb Roman Kennke: > Currently, the Get/Release*Critical() family of functions use the > GCLocker protocol to ensure that no JNI critical arrays are in use when > a GC pause is entered. > > Some GCs may instead want to use object pinning and guarantee that the > object does not move. For example, this is easy to do with region-based > GCs (G1, Shenandoah, ZGC) by simply not including regions with pinned > objects in the collection set. > > The implementation/API that I'm proposing is fairly simple: add two > methods oop pin_object(oop) and void unpin_object(oop) to CollectedHeap, > and call them from JNI's Get/Release*Critical methods. > > This approach has been working perfectly fine since a long time in > Shenandoah. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8199620 > Webrev: > http://cr.openjdk.java.net/~rkennke/8199620/webrev.00/ > > Ping? Any takers for this? Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From stefan.johansson at oracle.com Mon Mar 19 15:39:48 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 19 Mar 2018 16:39:48 +0100 Subject: RFR [M][7/7]: 8197932: Better split work in rebuild remembered sets phase In-Reply-To: <1521461029.2323.11.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520265115.2532.126.camel@oracle.com> <1520953401.3100.41.camel@oracle.com> <1521461029.2323.11.camel@oracle.com> Message-ID: <4110ec65-c66f-eca0-3b7a-6580f0edf00d@oracle.com> Thanks Thomas, On 2018-03-19 13:03, Thomas Schatzl wrote: > Hi all, > > Stefan proposed to use some iterator over the work area that > internally handles the boundary at the TAMS wrt to liveness when > scanning. > > This makes the change a bit more compact, performance does not seem to > differ, so why not... :) I really like this approach :) A few additional small things: src/hotspot/share/gc/g1/g1RemSet.cpp 787???????? if (!is_live(_current)) { I would like to invert the if-statement in the LiveObjIterator constructor and move the call to move_if_below_ntams() into the if-statement updating _current, something like this: if (is_live(_current)) { ? // Non-objArrays were scanned by the previous part of that region. ? if (_current < mr.start() && !oop(_current)->is_objArray()) { ????? _current += oop(_current)->size(); ????? // We might have positioned _current on a non-live object. Reposition to the next ?? ?? // live one if needed. ????? move_if_below_ntams(); ??? } ? } } else { ? ... } 926 #ifdef ASSERT 927?????? // In the final iteration of the loop the region might have been eagerly reclaimed. 928?????? // Simply filter out those regions. We can not just use region type because there 929?????? // might have already been new allocations into these regions. 930?????? HeapWord* const top_at_rebuild_start = _cm->top_at_rebuild_start(region_idx); 931?????? assert(!hr->is_old() || What do you think about making this assert into a guarantee or use DEBUG_ONLY() to avoid the #ifdef ASSERT? Also maybe adding something to the comment explaining that it is the check on top_at_rebuild_start that filters out eagerly reclaimed regions. --- src/hotspot/share/gc/g1/g1_globals.hpp 259?? experimental(size_t, G1RebuildRemSetChunkSize, 256 * K, Did you test to see if changing the G1RebuildRemSetChunkSize flag had any impact on performance, otherwise maybe we could make it a constant instead. --- Otherwise I think this looks great. Thanks, Stefan > > Webrevs: > http://cr.openjdk.java.net/~tschatzl/8197932/webrev.1_to_2/ (diff) > http://cr.openjdk.java.net/~tschatzl/8197932/webrev.2/ (full) > testing: > hs-tier1-5 > > Thanks, > Thomas > From erik.osterlund at oracle.com Mon Mar 19 16:04:26 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 19 Mar 2018 17:04:26 +0100 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> Message-ID: <5AAFDF8A.1080309@oracle.com> Hi Roman, Would there be a problem for you to move the GCLocker stuff into the default implementation of object pinning? I have seen code in the wild (not just Solaris) outside of hotspot that exploits critical native. I would feel less worried if a given GC is not forced to choose between completely ignoring the GC locking protocol (and hence breaking such applications), or only locking. Thanks, /Erik On 2018-03-19 15:23, Roman Kennke wrote: > Am 14.03.2018 um 19:13 schrieb Roman Kennke: >> Currently, the Get/Release*Critical() family of functions use the >> GCLocker protocol to ensure that no JNI critical arrays are in use when >> a GC pause is entered. >> >> Some GCs may instead want to use object pinning and guarantee that the >> object does not move. For example, this is easy to do with region-based >> GCs (G1, Shenandoah, ZGC) by simply not including regions with pinned >> objects in the collection set. >> >> The implementation/API that I'm proposing is fairly simple: add two >> methods oop pin_object(oop) and void unpin_object(oop) to CollectedHeap, >> and call them from JNI's Get/Release*Critical methods. >> >> This approach has been working perfectly fine since a long time in >> Shenandoah. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8199620 >> Webrev: >> http://cr.openjdk.java.net/~rkennke/8199620/webrev.00/ >> >> > Ping? Any takers for this? > > Roman > From paul.sandoz at oracle.com Fri Mar 16 16:21:32 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 16 Mar 2018 09:21:32 -0700 Subject: GetPrimitiveArrayCritical vs GetByteArrayRegion: 140x slow-down using -Xcheck:jni and java.util.zip.DeflaterOutputStream In-Reply-To: References: <22bbd632-47c1-7702-bd65-6a51f0bbad02@oracle.com> <9b5ff318-af41-03c4-efdc-b41b97cf6e61@oracle.com> <5A9D8C60.3050505@oracle.com> <5A9D8F33.5060500@oracle.com> <5A9DAC84.906@oracle.com> <5A9DAE87.8020801@oracle.com> <1521102348.2448.25.camel@oracle.com> Message-ID: <23639144-5217-4A0F-930C-EF24B4976544@oracle.com> Hi Ian, Thomas, Some background on the bulk copying for byte buffers after talking with Mikael who worked on these changes a few years ago. Please correct the following if needed as our memory is hazy :-) IIRC at the time we felt this was a reasonable thing to do because C2 did not strip mine the loops for the bulk copying of large Java arrays i.e. the issue was there anyway for more common cases. However, i believe that may no longer be the so in some cases after Roland implemented loop strip mining in C2 [1]. So we should go back and revisit/check the current support in buffers and Java arrays (System.arraycopy etc). (This is also something we need to consider if we modify buffers to support capacities larger than Integer.MAX_VALUE. Also connects with Project Panama.) If Thomas has not done so or does not plan to i can log an issue for you. Paul. [1] https://bugs.openjdk.java.net/browse/JDK-8186027 > On Mar 15, 2018, at 10:49 AM, Ian Rogers wrote: > > +hotspot-gc-dev > > On Thu, Mar 15, 2018 at 1:25 AM Thomas Schatzl > wrote: > >> Hi, >> >> On Thu, 2018-03-15 at 01:00 +0000, Ian Rogers wrote: >>> An old data point on how large a critical region should be comes from >>> java.nio.Bits. In JDK 9 the code migrated into unsafe, but in JDK 8 >>> the copies within a critical region were bound at most copying 1MB: >>> http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/ >>> native/java/nio/Bits.c#l88 This is inconsistent with Deflater and >>> ObjectOutputStream which both allow unlimited arrays and thereby >>> critical region sizes. >>> >>> In JDK 9 the copies starve the garbage collector in nio Bits too as >>> there is no 1MB limit to the copy sizes: >>> http://hg.openjdk.java.net/jdk/jdk/rev/f70e100d3195 >>> which came from: >>> https://bugs.openjdk.java.net/browse/JDK-8149596 >>> >>> Perhaps this is a regression not demonstrated due to the testing >>> challenge. >>> [...] >>> It doesn't seem unreasonable to have the loops for the copies occur >>> in 1MB chunks but JDK-8149596 lost this and so I'm confused on what >>> the HotSpot stand point is. >> >> Please file a bug (seems to be a core-libs/java.nio regression?), >> preferably with some kind of regression test. Also file enhancements (I >> would guess) for the other cases allowing unlimited arrays. >> > > I don't have perms to file bugs there's some catch-22 scenario in getting > the permissions. Happy to have a bug filed or to file were that not an > issue. Happy to create a test case but can't see any others for TTSP > issues. This feels like a potential use case for jmh, perhaps run the > benchmark well having a separate thread run GC bench. > > Should there be a bug to add, in debug mode, a TTSP watcher thread whose > job it is to bring "random" threads into safepoints and report on tardy > ones? > Should there be a bug to warn on being in a JNI critical for more than just > a short period? > Seems like there should be a bug on Unsafe.copyMemory and > Unsafe.copySwapMemory having TTSP issues. > Seems like there should be a bug on all uses of critical that don't chunk > their critical region work based on some bound (like 1MB chunks for nio > Bits)? How are these bounds set? A past reference that I've lost is in > having the bound be the equivalent of 65535 bytecodes due to the > expectation of GC work at least once in a method or on a loop backedge - I > thought this was in a spec somewhere but now I can't find it. The bytecode > size feels as arbitrary as 1MB, a time period would be better but that can > depend on the kind of GC you want as delays with concurrent GC mean more > than non-concurrent. Clearly the chunk size shouldn't just be 0, but this > appears to currently be the norm in the JDK. > > The original reason for coming here was a 140x slow down in -Xcheck:jni in > Deflater.deflate There are a few options there that its useful to enumerate: > 1) rewrite in Java but there are correctness and open source related issues > 2) remove underflow/overflow protection from critical arrays (revert > JDK-6311046 > or perhaps bound protection to arrays of a particular small size) - this > removes checking and doesn't deal with TTSP > 3) add a critical array slice API to JNI so that copies with -Xcheck:jni > aren't unbounded (martinrb@ proposed this) - keeps checking but doesn't > deal with TTSP > 4) rewrite primitive array criticals with GetArrayRegion as O(n) beats the > "silent killer" TTSP (effectively deprecate the critical APIs) > > In general (ie not just the deflate case) I think (1) is the most > preferable. (2) and (3) both have TTSP issues. (4) isn't great performance > wise, which motivates more use of approach (1), but I think deprecating > criticals may just be the easiest and sanest way forward. I think that > discussion is worth having on an e-mail thread rather than a bug. > > >> Long TTSP is a performance bug as any other. >> >>> In a way criticals are better than unsafe as they may >>> pin the memory and not starve GC, which shenandoah does. >> >> (Region based) Object pinning has its own share of problems: >> >> - only (relatively) easily implemented in region based collectors >> >> - may slow down pause a bit in presence of pinned regions/objects (for >> non-concurrent copying collectors) >> >> - excessive use of pinning may cause OOME and VM exit probably earlier >> than the gc locker. GC locker seems to provide a more gradual >> degradation. E.g. pinning regions typically makes these regions >> unavailable for allocation. >> I.e. you still should not use it for many, very long living objects. >> Of course this somewhat depends on the sophistication of the >> implementation. >> >> I think region based pinning would be a good addition to other >> collectors than Shenandoah too. It has been on our minds for a long >> time, but there are so many other more important issues :), so of >> course we are eager to see contributions in this area. ;) >> >> If you are interested on working on this, please ping us on hotspot-gc- >> dev for implementation ideas to get you jump-started. >> >> Thanks, >> Thomas >> > > I'd rather deprecate criticals than build upon the complexity, but I'm very > glad this is a concern. > > Thanks, > Ian -------------- next part -------------- An HTML attachment was scrubbed... URL: From rkennke at redhat.com Mon Mar 19 17:07:50 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 19 Mar 2018 18:07:50 +0100 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: <5AAFDF8A.1080309@oracle.com> References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> <5AAFDF8A.1080309@oracle.com> Message-ID: Hi Erik, thanks for reviewing! I can do that, but a GC would get the opportunity to do both, or one or the other in any case. As it stands, the JNI side of the GCLocker protocol is always executed. If a GC decides to participate in GCLocker protocol, by calling GCLocker::check_active_before_gc() and related API, then will get JNI Critical function support, otherwise GCLocker is basically counting up and down counters. If it instead wishes to use pinning and ignore JNI critical functions, then it only needs to override the pinning methods and not participate in GCLocker, and everything would be fine. However, I don't mind either way, so here's the alternative: Diff: http://cr.openjdk.java.net/~rkennke/8199620/webrev.02.diff/ Full: http://cr.openjdk.java.net/~rkennke/8199620/webrev.02/ Better? Thanks, Roman > Would there be a problem for you to move the GCLocker stuff into the > default implementation of object pinning? > I have seen code in the wild (not just Solaris) outside of hotspot that > exploits critical native. I would feel less worried if a given GC is not > forced to choose between completely ignoring the GC locking protocol > (and hence breaking such applications), or only locking. > > Thanks, > /Erik > > On 2018-03-19 15:23, Roman Kennke wrote: >> Am 14.03.2018 um 19:13 schrieb Roman Kennke: >>> Currently, the Get/Release*Critical() family of functions use the >>> GCLocker protocol to ensure that no JNI critical arrays are in use when >>> a GC pause is entered. >>> >>> Some GCs may instead want to use object pinning and guarantee that the >>> object does not move. For example, this is easy to do with region-based >>> GCs (G1, Shenandoah, ZGC) by simply not including regions with pinned >>> objects in the collection set. >>> >>> The implementation/API that I'm proposing is fairly simple: add two >>> methods oop pin_object(oop) and void unpin_object(oop) to CollectedHeap, >>> and call them from JNI's Get/Release*Critical methods. >>> >>> This approach has been working perfectly fine since a long time in >>> Shenandoah. >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8199620 >>> Webrev: >>> http://cr.openjdk.java.net/~rkennke/8199620/webrev.00/ >>> >>> >> Ping? Any takers for this? >> >> Roman >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Mon Mar 19 18:32:19 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 19 Mar 2018 19:32:19 +0100 Subject: RFR: JDK-8199781: Don't use naked == for comparing oops Message-ID: GCs might need to handle object comparison specially, for example, when a concurrent GC allows both from- and to-space copies of an object to be in use. In this case, object equality cannot be expressed by direct address comparsion, but must also take potential different versions of an object into account. This adds a method oopDesc::equals(oop o1, oop o2), which goes through the Access API to compare objects. Default impl does == It is based on our finds from Shenandoah. We do, in fact, have some extra code in oopsHierarchy.hpp to detect naked oop==oop (with CheckUnhandledOops enabled), but it requires that code that does really want naked == (e.g. GCs) to use another method oopDesc::unsafe_equals() instead, and code that wants to use == and assert that the comparison is safe (e.g. both objects are already in to-space) use oopDesc::safe_equals(). Not sure if you want that? Also, I'd have preferred to mark equals() as inline in oop.hpp, but that pulls in a rats nest of dependencies problems. I hope the way I did it is acceptable? There's one little thing in growableArray.hpp: in order to do the right thing for GrowableArray I needed to provide a special templated comparison function, with a specialized version for oop. It also overloads the == operator in Handle to do the right thing. Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8199781/webrev.00/ Testing: built fastdebug/release, tier1_runtime, tier1_gc Please review! Thanks, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Tue Mar 20 08:57:42 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 20 Mar 2018 09:57:42 +0100 Subject: RFR: JDK-8199801: Finer grained primitive arrays bulk access barriers Message-ID: <1c2c48f2-26cc-bc2b-24f4-36108a9f9947@redhat.com> Currently, for any bulk access on primitive arrays (e.g. *_at_addr() family of methods, arraycopy, and the like), we do Access::resolve() on the base arrayOop (via arrayOopDesc::base() ). This is safe and does the right thing, but it is overly broad. In a garbage collector like Shenandoah, Access::resolve() would have to do a write-barrier, which is a copy-on-write. This could lead to double-copying of potentially large arrays, i.e. resolving the src array would copy it from from-space to to-space, and then it needs to be copied again from src to dst. This adds unnecessary latency to Java threads. Most of the accesses via _at_addr() are read-accesses, and they could use a weaker read-barrier, which only chases the forwarding pointer. This change splits Access<>::resolve() into resolve_for_read() and resolve_for_write(). This is then used in all the relevant places. In particular: - typeArrayOop::*_at_addr() accessors now differentiate between get_*_at_addr() and put_*_at_addr(), with the get_* accessors returning a const T* to prevent/discourage writing to them. This percolates a little bit through the code base, but in a positive sense (i.e. can use all the utf8 methods with const arrays now). - The part in c1_Runtime1.cpp will have to be rebased/eliminated once Erik's patch goes in. - arrayOopDesc::base_raw() and objArrayOopDesc::base_raw() go away, and (remaining) uses replaced by ::base(). - The arraycopy stuff now calls obj_at_addr_raw() instead of obj_at_addr(). The reason for this is that the actual objects are better resolved in HeapAccess<>::arraycopy() methods, and the pointer passed into it must really be within the array that's also passed in, and not in another copy. - objArrayOopDesc::atomic_compare_exchange_oop() now uses oop_atomic_cmpxchg_at() ... the existing version does not allow to resolve the target object. (this is only related in that it eliminates a last remaining use of obj_at_addr()). - oopDesc::field_addr() is only used in one place (DependencyContext) which should be refactored to use proper oopDesc accessors, and then field_addr() can be eliminated too. For now it uses resolve_for_write(), b/c DependencyContext may write to the field. - I noticed a bunch of memcpy() around the _at_addr() calls (e.g. in jni.cpp).. I wonder if they should be replaced by corresponding Copy:: functions instead? - jvmtiTagMap.cpp has an odd one: - // Don't use char_at_addr(0) if length is 0 - value = (jchar*) s_value->base(T_CHAR); + value = s_value->get_char_at_addr(0); In my mind, base(T_CHAR) should be equivalent to get_char_at_addr(0), both return the address of the first element. I don't see the point. I can see why we are not going into possible latin1 inflation for length==0. - I refactored some stuff in heapDumper to accept const void* instead of void* in write_raw(). Tests: build fastdebug/release (with+w/o PCH), passes hotspot/tier1 tests (fastdebug+release) Bug: https://bugs.openjdk.java.net/browse/JDK-8199801 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8199801/webrev.00/ What do you think? Can I please get reviews? Thanks, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Tue Mar 20 09:11:05 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 20 Mar 2018 10:11:05 +0100 Subject: RFR: 8199850: Move parsing of VerifyGCType to G1 Message-ID: <5d3f62d2-f4c7-affa-a21c-be3330262af9@oracle.com> When JDK-8191821 introduced the VerifyGCType option for G1 I didn't quite see the need for this to affect the GCArguments class. I'm proposing we move this logic into G1HeapVerifier, which I believe is a better place for this. Bug: https://bugs.openjdk.java.net/browse/JDK-8199850 Webrev: http://cr.openjdk.java.net/~pliden/8199850/webrev.0 Testing: Used gc/g1/TestVerifyGCType.java to verify that nothing broke /Per From erik.osterlund at oracle.com Tue Mar 20 10:10:35 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 20 Mar 2018 11:10:35 +0100 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> <5AAFDF8A.1080309@oracle.com> Message-ID: <5AB0DE1B.5040200@oracle.com> Hi Roman, Looks good. Thanks, /Erik On 2018-03-19 18:07, Roman Kennke wrote: > Hi Erik, > > thanks for reviewing! > > I can do that, but a GC would get the opportunity to do both, or one or > the other in any case. As it stands, the JNI side of the GCLocker > protocol is always executed. If a GC decides to participate in GCLocker > protocol, by calling GCLocker::check_active_before_gc() and related API, > then will get JNI Critical function support, otherwise GCLocker is > basically counting up and down counters. > > If it instead wishes to use pinning and ignore JNI critical functions, > then it only needs to override the pinning methods and not participate > in GCLocker, and everything would be fine. > > However, I don't mind either way, so here's the alternative: > Diff: > http://cr.openjdk.java.net/~rkennke/8199620/webrev.02.diff/ > Full: > http://cr.openjdk.java.net/~rkennke/8199620/webrev.02/ > > Better? > > Thanks, > Roman > >> Would there be a problem for you to move the GCLocker stuff into the >> default implementation of object pinning? >> I have seen code in the wild (not just Solaris) outside of hotspot that >> exploits critical native. I would feel less worried if a given GC is not >> forced to choose between completely ignoring the GC locking protocol >> (and hence breaking such applications), or only locking. >> >> Thanks, >> /Erik >> >> On 2018-03-19 15:23, Roman Kennke wrote: >>> Am 14.03.2018 um 19:13 schrieb Roman Kennke: >>>> Currently, the Get/Release*Critical() family of functions use the >>>> GCLocker protocol to ensure that no JNI critical arrays are in use when >>>> a GC pause is entered. >>>> >>>> Some GCs may instead want to use object pinning and guarantee that the >>>> object does not move. For example, this is easy to do with region-based >>>> GCs (G1, Shenandoah, ZGC) by simply not including regions with pinned >>>> objects in the collection set. >>>> >>>> The implementation/API that I'm proposing is fairly simple: add two >>>> methods oop pin_object(oop) and void unpin_object(oop) to CollectedHeap, >>>> and call them from JNI's Get/Release*Critical methods. >>>> >>>> This approach has been working perfectly fine since a long time in >>>> Shenandoah. >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8199620 >>>> Webrev: >>>> http://cr.openjdk.java.net/~rkennke/8199620/webrev.00/ >>>> >>>> >>> Ping? Any takers for this? >>> >>> Roman >>> > From thomas.schatzl at oracle.com Tue Mar 20 10:28:18 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 20 Mar 2018 11:28:18 +0100 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> <5AAFDF8A.1080309@oracle.com> Message-ID: <1521541698.3121.6.camel@oracle.com> Hi, On Mon, 2018-03-19 at 18:07 +0100, Roman Kennke wrote: > Hi Erik, > > thanks for reviewing! > > I can do that, but a GC would get the opportunity to do both, or one > or the other in any case. As it stands, the JNI side of the GCLocker > protocol is always executed. If a GC decides to participate in > GCLocker protocol, by calling GCLocker::check_active_before_gc() and > related API, then will get JNI Critical function support, otherwise > GCLocker is basically counting up and down counters. > > If it instead wishes to use pinning and ignore JNI critical > functions, then it only needs to override the pinning methods and not > participate in GCLocker, and everything would be fine. > > However, I don't mind either way, so here's the alternative: > Diff: > http://cr.openjdk.java.net/~rkennke/8199620/webrev.02.diff/ > Full: > http://cr.openjdk.java.net/~rkennke/8199620/webrev.02/ > > Better? I also prefer this version. Looks good. Thanks, Thomas From per.liden at oracle.com Tue Mar 20 10:32:43 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 20 Mar 2018 11:32:43 +0100 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> <5AAFDF8A.1080309@oracle.com> Message-ID: <30509316-0d9e-40c0-deec-8767990c070d@oracle.com> Hi Roman, This looks much better, thanks for fixing. Reviewed. Btw, as Erik mentioned, there are JavaCritical_ libraries out there on many other platforms (I've seen performance critical stuff like OpenGL library bindings using this... anyway, Shenandoah can opt-out from that) cheers, /Per On 03/19/2018 06:07 PM, Roman Kennke wrote: > Hi Erik, > > thanks for reviewing! > > I can do that, but a GC would get the opportunity to do both, or one or > the other in any case. As it stands, the JNI side of the GCLocker > protocol is always executed. If a GC decides to participate in GCLocker > protocol, by calling GCLocker::check_active_before_gc() and related API, > then will get JNI Critical function support, otherwise GCLocker is > basically counting up and down counters. > > If it instead wishes to use pinning and ignore JNI critical functions, > then it only needs to override the pinning methods and not participate > in GCLocker, and everything would be fine. > > However, I don't mind either way, so here's the alternative: > Diff: > http://cr.openjdk.java.net/~rkennke/8199620/webrev.02.diff/ > Full: > http://cr.openjdk.java.net/~rkennke/8199620/webrev.02/ > > Better? > > Thanks, > Roman > >> Would there be a problem for you to move the GCLocker stuff into the >> default implementation of object pinning? >> I have seen code in the wild (not just Solaris) outside of hotspot that >> exploits critical native. I would feel less worried if a given GC is not >> forced to choose between completely ignoring the GC locking protocol >> (and hence breaking such applications), or only locking. >> >> Thanks, >> /Erik >> >> On 2018-03-19 15:23, Roman Kennke wrote: >>> Am 14.03.2018 um 19:13 schrieb Roman Kennke: >>>> Currently, the Get/Release*Critical() family of functions use the >>>> GCLocker protocol to ensure that no JNI critical arrays are in use when >>>> a GC pause is entered. >>>> >>>> Some GCs may instead want to use object pinning and guarantee that the >>>> object does not move. For example, this is easy to do with region-based >>>> GCs (G1, Shenandoah, ZGC) by simply not including regions with pinned >>>> objects in the collection set. >>>> >>>> The implementation/API that I'm proposing is fairly simple: add two >>>> methods oop pin_object(oop) and void unpin_object(oop) to CollectedHeap, >>>> and call them from JNI's Get/Release*Critical methods. >>>> >>>> This approach has been working perfectly fine since a long time in >>>> Shenandoah. >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8199620 >>>> Webrev: >>>> http://cr.openjdk.java.net/~rkennke/8199620/webrev.00/ >>>> >>>> >>> Ping? Any takers for this? >>> >>> Roman >>> >> > > From Derek.White at cavium.com Tue Mar 20 15:37:52 2018 From: Derek.White at cavium.com (White, Derek) Date: Tue, 20 Mar 2018 15:37:52 +0000 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: <30509316-0d9e-40c0-deec-8767990c070d@oracle.com> References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> <5AAFDF8A.1080309@oracle.com> <30509316-0d9e-40c0-deec-8767990c070d@oracle.com> Message-ID: Hi Roman, Thanks for the new version. Having a centralized decision point in pin_object() should be clearer and open up other options (like only pinning huge arrays). - Derek > -----Original Message----- > From: hotspot-gc-dev [mailto:hotspot-gc-dev-bounces at openjdk.java.net] > On Behalf Of Per Liden > Sent: Tuesday, March 20, 2018 6:33 AM > To: Roman Kennke ; Erik ?sterlund > ; hotspot-gc-dev at openjdk.java.net > Subject: Re: RFR: JDK-8199620: Support for JNI object pinning > > Hi Roman, > > This looks much better, thanks for fixing. Reviewed. > > Btw, as Erik mentioned, there are JavaCritical_ libraries out there on many > other platforms (I've seen performance critical stuff like OpenGL library > bindings using this... anyway, Shenandoah can opt-out from that) > > cheers, > /Per > > On 03/19/2018 06:07 PM, Roman Kennke wrote: > > Hi Erik, > > > > thanks for reviewing! > > > > I can do that, but a GC would get the opportunity to do both, or one > > or the other in any case. As it stands, the JNI side of the GCLocker > > protocol is always executed. If a GC decides to participate in > > GCLocker protocol, by calling GCLocker::check_active_before_gc() and > > related API, then will get JNI Critical function support, otherwise > > GCLocker is basically counting up and down counters. > > > > If it instead wishes to use pinning and ignore JNI critical functions, > > then it only needs to override the pinning methods and not participate > > in GCLocker, and everything would be fine. > > > > However, I don't mind either way, so here's the alternative: > > Diff: > > http://cr.openjdk.java.net/~rkennke/8199620/webrev.02.diff/ > > Full: > > http://cr.openjdk.java.net/~rkennke/8199620/webrev.02/ > > > > Better? > > > > Thanks, > > Roman > > > >> Would there be a problem for you to move the GCLocker stuff into the > >> default implementation of object pinning? > >> I have seen code in the wild (not just Solaris) outside of hotspot > >> that exploits critical native. I would feel less worried if a given > >> GC is not forced to choose between completely ignoring the GC locking > >> protocol (and hence breaking such applications), or only locking. > >> > >> Thanks, > >> /Erik > >> > >> On 2018-03-19 15:23, Roman Kennke wrote: > >>> Am 14.03.2018 um 19:13 schrieb Roman Kennke: > >>>> Currently, the Get/Release*Critical() family of functions use the > >>>> GCLocker protocol to ensure that no JNI critical arrays are in use > >>>> when a GC pause is entered. > >>>> > >>>> Some GCs may instead want to use object pinning and guarantee that > >>>> the object does not move. For example, this is easy to do with > >>>> region-based GCs (G1, Shenandoah, ZGC) by simply not including > >>>> regions with pinned objects in the collection set. > >>>> > >>>> The implementation/API that I'm proposing is fairly simple: add two > >>>> methods oop pin_object(oop) and void unpin_object(oop) to > >>>> CollectedHeap, and call them from JNI's Get/Release*Critical methods. > >>>> > >>>> This approach has been working perfectly fine since a long time in > >>>> Shenandoah. > >>>> > >>>> Bug: > >>>> https://bugs.openjdk.java.net/browse/JDK-8199620 > >>>> Webrev: > >>>> http://cr.openjdk.java.net/~rkennke/8199620/webrev.00/ > >>>> > >>>> > >>> Ping? Any takers for this? > >>> > >>> Roman > >>> > >> > > > > From stefan.karlsson at oracle.com Tue Mar 20 16:06:16 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 20 Mar 2018 17:06:16 +0100 Subject: RFR: JDK-8199781: Don't use naked == for comparing oops In-Reply-To: References: Message-ID: <74ca6651-49aa-2b77-d7e9-a588fb2b8417@oracle.com> Hi Roman, On 2018-03-19 19:32, Roman Kennke wrote: > GCs might need to handle object comparison specially, for example, when > a concurrent GC allows both from- and to-space copies of an object to be > in use. In this case, object equality cannot be expressed by direct > address comparsion, but must also take potential different versions of > an object into account. > > This adds a method oopDesc::equals(oop o1, oop o2), which goes through > the Access API to compare objects. Default impl does == > > It is based on our finds from Shenandoah. We do, in fact, have some > extra code in oopsHierarchy.hpp to detect naked oop==oop (with > CheckUnhandledOops enabled), but it requires that code that does really > want naked == (e.g. GCs) to use another method oopDesc::unsafe_equals() > instead, and code that wants to use == and assert that the comparison is > safe (e.g. both objects are already in to-space) use > oopDesc::safe_equals(). Not sure if you want that? > > Also, I'd have preferred to mark equals() as inline in oop.hpp, but that > pulls in a rats nest of dependencies problems. I hope the way I did it > is acceptable? We have actively been working on cleaning up our include dependencies, so this should be dealt with properly before this patch can be accepted. All files that use oopDesc::equals need to include oop.inline.hpp. The usages of oopDesc::equals in .hpp needs to move to either .cpp files or .inline.hpp files. See the files section on this page: https://wiki.openjdk.java.net/display/HotSpot/StyleGuide Currently, you need to include access.inline.hpp to use the Access API. I wish this wasn't the case, and that we somehow hide the GC internal dependencies, so that the call sites didn't have to bring all those dependencies. Erik ? used to have a facility to generate the runtime dispatched functions out-of-line. Maybe if we build upon this, we could get rid of the need to include access.inline.hpp when using the Access API, and oopDesc::equals could be placed in oopDesc::equals. However, we don't have this in place (yet) so we need to play along with the set rules for .inline.hpp files. Thanks, StefanK > > There's one little thing in growableArray.hpp: in order to do the right > thing for GrowableArray I needed to provide a special templated > comparison function, with a specialized version for oop. > > It also overloads the == operator in Handle to do the right thing. > > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8199781/webrev.00/ > > Testing: built fastdebug/release, tier1_runtime, tier1_gc > > Please review! > > Thanks, Roman > From rkennke at redhat.com Tue Mar 20 16:18:57 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 20 Mar 2018 17:18:57 +0100 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> <5AAFDF8A.1080309@oracle.com> <30509316-0d9e-40c0-deec-8767990c070d@oracle.com> Message-ID: <3f1f969e-868e-9850-7adb-6fbd92497a89@redhat.com> FWIW, I also filed: https://bugs.openjdk.java.net/browse/JDK-8199868 I think it may be possible to also support JNI critical functions via pinning. We'd have to pin/unpin all the arguments to such functions. However, it takes a bit of wrangling to get those arguments to the GC. I'll look at this at some point in the future. Thanks, Roman > Hi Roman, > > Thanks for the new version. Having a centralized decision point in pin_object() should be clearer and open up other options (like only pinning huge arrays). > > - Derek > >> -----Original Message----- >> From: hotspot-gc-dev [mailto:hotspot-gc-dev-bounces at openjdk.java.net] >> On Behalf Of Per Liden >> Sent: Tuesday, March 20, 2018 6:33 AM >> To: Roman Kennke ; Erik ?sterlund >> ; hotspot-gc-dev at openjdk.java.net >> Subject: Re: RFR: JDK-8199620: Support for JNI object pinning >> >> Hi Roman, >> >> This looks much better, thanks for fixing. Reviewed. >> >> Btw, as Erik mentioned, there are JavaCritical_ libraries out there on many >> other platforms (I've seen performance critical stuff like OpenGL library >> bindings using this... anyway, Shenandoah can opt-out from that) >> >> cheers, >> /Per >> >> On 03/19/2018 06:07 PM, Roman Kennke wrote: >>> Hi Erik, >>> >>> thanks for reviewing! >>> >>> I can do that, but a GC would get the opportunity to do both, or one >>> or the other in any case. As it stands, the JNI side of the GCLocker >>> protocol is always executed. If a GC decides to participate in >>> GCLocker protocol, by calling GCLocker::check_active_before_gc() and >>> related API, then will get JNI Critical function support, otherwise >>> GCLocker is basically counting up and down counters. >>> >>> If it instead wishes to use pinning and ignore JNI critical functions, >>> then it only needs to override the pinning methods and not participate >>> in GCLocker, and everything would be fine. >>> >>> However, I don't mind either way, so here's the alternative: >>> Diff: >>> http://cr.openjdk.java.net/~rkennke/8199620/webrev.02.diff/ >>> Full: >>> http://cr.openjdk.java.net/~rkennke/8199620/webrev.02/ >>> >>> Better? >>> >>> Thanks, >>> Roman >>> >>>> Would there be a problem for you to move the GCLocker stuff into the >>>> default implementation of object pinning? >>>> I have seen code in the wild (not just Solaris) outside of hotspot >>>> that exploits critical native. I would feel less worried if a given >>>> GC is not forced to choose between completely ignoring the GC locking >>>> protocol (and hence breaking such applications), or only locking. >>>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2018-03-19 15:23, Roman Kennke wrote: >>>>> Am 14.03.2018 um 19:13 schrieb Roman Kennke: >>>>>> Currently, the Get/Release*Critical() family of functions use the >>>>>> GCLocker protocol to ensure that no JNI critical arrays are in use >>>>>> when a GC pause is entered. >>>>>> >>>>>> Some GCs may instead want to use object pinning and guarantee that >>>>>> the object does not move. For example, this is easy to do with >>>>>> region-based GCs (G1, Shenandoah, ZGC) by simply not including >>>>>> regions with pinned objects in the collection set. >>>>>> >>>>>> The implementation/API that I'm proposing is fairly simple: add two >>>>>> methods oop pin_object(oop) and void unpin_object(oop) to >>>>>> CollectedHeap, and call them from JNI's Get/Release*Critical methods. >>>>>> >>>>>> This approach has been working perfectly fine since a long time in >>>>>> Shenandoah. >>>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8199620 >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~rkennke/8199620/webrev.00/ >>>>>> >>>>>> >>>>> Ping? Any takers for this? >>>>> >>>>> Roman >>>>> >>>> >>> >>> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Tue Mar 20 20:59:09 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 20 Mar 2018 21:59:09 +0100 Subject: URGENT: RFR: 8199886: gc/cslocker/TestCSLocker.java crashes Message-ID: <8ee95fd0-8b45-2610-19bf-e1d372d2b237@oracle.com> JDK-8199620 "Support for JNI object pinning" introduced a bug where a naked oop was passed across a safepoint (a call to GCLocker::lock_critical). This caused GCLocker-related tests in both hs-tier1-2 to fail. This patch stores the oop in a Handle before crossing the safepoint. Bug: https://bugs.openjdk.java.net/browse/JDK-8199886 Webrev: http://cr.openjdk.java.net/~pliden/8199886/webrev.0 Testing: Ran affected jtreg tests locally. hs-tier1-2 currently running in mach5 /Per From rkennke at redhat.com Tue Mar 20 21:02:02 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 20 Mar 2018 22:02:02 +0100 Subject: URGENT: RFR: 8199886: gc/cslocker/TestCSLocker.java crashes In-Reply-To: <8ee95fd0-8b45-2610-19bf-e1d372d2b237@oracle.com> References: <8ee95fd0-8b45-2610-19bf-e1d372d2b237@oracle.com> Message-ID: <84e8aba9-1f8a-9730-d294-ca4f77f51652@redhat.com> Am 20.03.2018 um 21:59 schrieb Per Liden: > JDK-8199620 "Support for JNI object pinning" introduced a bug where a > naked oop was passed across a safepoint (a call to > GCLocker::lock_critical). This caused GCLocker-related tests in both > hs-tier1-2 to fail. This patch stores the oop in a Handle before > crossing the safepoint. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8199886 > Webrev: http://cr.openjdk.java.net/~pliden/8199886/webrev.0 > > Testing: Ran affected jtreg tests locally. hs-tier1-2 currently running > in mach5 > I was not aware that lock_critical() might cause a safepoint. I've been chasing the bug for some hours. Thanks for fixing it. The patch looks good. Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From kim.barrett at oracle.com Tue Mar 20 21:02:15 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 20 Mar 2018 17:02:15 -0400 Subject: URGENT: RFR: 8199886: gc/cslocker/TestCSLocker.java crashes In-Reply-To: <8ee95fd0-8b45-2610-19bf-e1d372d2b237@oracle.com> References: <8ee95fd0-8b45-2610-19bf-e1d372d2b237@oracle.com> Message-ID: <580277C3-3A6C-492E-84B9-09493C4BFF88@oracle.com> > On Mar 20, 2018, at 4:59 PM, Per Liden wrote: > > JDK-8199620 "Support for JNI object pinning" introduced a bug where a naked oop was passed across a safepoint (a call to GCLocker::lock_critical). This caused GCLocker-related tests in both hs-tier1-2 to fail. This patch stores the oop in a Handle before crossing the safepoint. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8199886 > Webrev: http://cr.openjdk.java.net/~pliden/8199886/webrev.0 > > Testing: Ran affected jtreg tests locally. hs-tier1-2 currently running in mach5 > > /Per Looks good. From per.liden at oracle.com Tue Mar 20 21:12:56 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 20 Mar 2018 22:12:56 +0100 Subject: URGENT: RFR: 8199886: gc/cslocker/TestCSLocker.java crashes In-Reply-To: <580277C3-3A6C-492E-84B9-09493C4BFF88@oracle.com> References: <8ee95fd0-8b45-2610-19bf-e1d372d2b237@oracle.com> <580277C3-3A6C-492E-84B9-09493C4BFF88@oracle.com> Message-ID: Thanks for reviewing Roman and Kim. Awaiting mach5 hs-trier{1,2} results before pushing. /Per On 03/20/2018 10:02 PM, Kim Barrett wrote: >> On Mar 20, 2018, at 4:59 PM, Per Liden wrote: >> >> JDK-8199620 "Support for JNI object pinning" introduced a bug where a naked oop was passed across a safepoint (a call to GCLocker::lock_critical). This caused GCLocker-related tests in both hs-tier1-2 to fail. This patch stores the oop in a Handle before crossing the safepoint. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8199886 >> Webrev: http://cr.openjdk.java.net/~pliden/8199886/webrev.0 >> >> Testing: Ran affected jtreg tests locally. hs-tier1-2 currently running in mach5 >> >> /Per > > Looks good. > From irogers at google.com Wed Mar 21 02:07:30 2018 From: irogers at google.com (Ian Rogers) Date: Wed, 21 Mar 2018 02:07:30 +0000 Subject: GetPrimitiveArrayCritical vs GetByteArrayRegion: 140x slow-down using -Xcheck:jni and java.util.zip.DeflaterOutputStream In-Reply-To: <1521451687.2323.5.camel@oracle.com> References: <22bbd632-47c1-7702-bd65-6a51f0bbad02@oracle.com> <9b5ff318-af41-03c4-efdc-b41b97cf6e61@oracle.com> <5A9D8C60.3050505@oracle.com> <5A9D8F33.5060500@oracle.com> <5A9DAC84.906@oracle.com> <5A9DAE87.8020801@oracle.com> <1521102348.2448.25.camel@oracle.com> <23639144-5217-4A0F-930C-EF24B4976544@oracle.com> <1521451687.2323.5.camel@oracle.com> Message-ID: Thanks, via bugreport.java.com I filed bugs/RFEs with ids 9053047, 9053048, 9053049, 9053050 and 9053051. Ian On Mon, Mar 19, 2018 at 2:28 AM Thomas Schatzl wrote: > Hi, > > On Fri, 2018-03-16 at 17:19 +0000, Ian Rogers wrote: > > Thanks Paul, very interesting. > > > > On Fri, Mar 16, 2018 at 9:21 AM Paul Sandoz > > wrote: > > > Hi Ian, Thomas, > > > > > > [...] > > > (This is also something we need to consider if we modify buffers to > > > support capacities larger than Integer.MAX_VALUE. Also connects > > > with Project Panama.) > > > > > > If Thomas has not done so or does not plan to i can log an issue > > > for you. > > > > > > > That'd be great. I wonder if identifying more TTSP issues should also > > be a bug. Its interesting to observe that overlooking TTSP in C2 > > motivated the Unsafe.copyMemory change permitting a fresh TTSP issue. > > If TTSP is a 1st class issue then maybe we can deprecate JNI critical > > regions to support that effort :-) > > Please log an issue. I am still a bit unsure what and how many issues > should be filed. > > @Ian: at bugreports.oracle.com everyone may file bug reports without > the need for an account. > It will take some time until they show up in Jira due to vetting, but > if you have a good case, and can e.g. link to the mailing list, this > should be painless. > > Thanks, > Thomas > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From per.liden at oracle.com Wed Mar 21 08:09:32 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 21 Mar 2018 09:09:32 +0100 Subject: RFR: 8199918: Shorten names of CollectedHeap::Name members Message-ID: <185a147f-87a7-7ef3-9b51-6411e84d7a2e@oracle.com> I propose that we shorten the member names in the CollectedHeap::Name enum. This is a pure cosmetic change. Essentially naming the GCs, rather than some of the CollectedHeap childen. From: SerialHeap ParallelScavengeHeap G1CollectedHeap CMSHeap To: Serial Parallel CMS G1 This is a preparatory cleanup for JDK-8199925 and JDK-8199927. Bug: https://bugs.openjdk.java.net/browse/JDK-8199918 Webrev: http://cr.openjdk.java.net/~pliden/8199918/webrev.0 /Per From per.liden at oracle.com Wed Mar 21 08:12:51 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 21 Mar 2018 09:12:51 +0100 Subject: RFR: 8199925: Break out GC selection logic from GCArguments to GCSelector Message-ID: In an attempt to make WhiteBox more GC agnostic, and in turn make it easier to plugin new GC without touching a lot of non-GC code, this patch breaks out the GC selection logic from GCArguments to GCSelector. There are two reasons why I think this makes sense: 1) The GC selection logic is self-contained and fairly unrelated to the rest of the flags processing done by GCArguments and its GC-specific children. 2) Parts of the GC selection logic is needed by WhiteBox to answer questions about which GC are supported, selected, etc. A follow up patch (JDK-8199927) will change WhiteBox (WB_CurrentGC, WB_AllSupportedGC, WB_GCSelectedByErgo), to use the GCSelector. Bug: https://bugs.openjdk.java.net/browse/JDK-8199925 Webrev: http://cr.openjdk.java.net/~pliden/8199925/webrev.0 /Per From per.liden at oracle.com Wed Mar 21 08:14:42 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 21 Mar 2018 09:14:42 +0100 Subject: RFR: 8199927: Make WhiteBox more GC agnostic Message-ID: <8e0b6cdb-c1d4-37f9-1170-b5596051865d@oracle.com> Make WhiteBox more GC agnostic by having it use the GCSelector to answer questions about which GCs are supported, selected, etc. Going forward, this will make it easier to plug in new GC without having to touch and adjust the WhiteBox code. Bug: https://bugs.openjdk.java.net/browse/JDK-8199927 Webrev: http://cr.openjdk.java.net/~pliden/8199927/webrev.0 /Per From rkennke at redhat.com Wed Mar 21 09:15:00 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 21 Mar 2018 10:15:00 +0100 Subject: RFR: 8199925: Break out GC selection logic from GCArguments to GCSelector In-Reply-To: References: Message-ID: <35136ca4-cc0c-d977-8351-523a52a39c7a@redhat.com> Am 21.03.2018 um 09:12 schrieb Per Liden: > In an attempt to make WhiteBox more GC agnostic, and in turn make it > easier to plugin new GC without touching a lot of non-GC code, this > patch breaks out the GC selection logic from GCArguments to GCSelector. > There are two reasons why I think this makes sense: > 1) The GC selection logic is self-contained and fairly unrelated to the > rest of the flags processing done by GCArguments and its GC-specific > children. > 2) Parts of the GC selection logic is needed by WhiteBox to answer > questions about which GC are supported, selected, etc. > > A follow up patch (JDK-8199927) will change WhiteBox (WB_CurrentGC, > WB_AllSupportedGC, WB_GCSelectedByErgo), to use the GCSelector. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8199925 > Webrev: http://cr.openjdk.java.net/~pliden/8199925/webrev.0 > > /Per My original proposal for the GCArguments did have a separate GCFactory. :-) In generally I like your change. I'd probably go even further and let GCSelector initialize and return an instance of GCArguments. This way we wouldn't end up with two places that have knowledge of all possible GCs, but would only have one place (GCSelector) where one had to hook up for adding a new GC. What do you think? Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Wed Mar 21 09:19:54 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 21 Mar 2018 10:19:54 +0100 Subject: RFR: 8199918: Shorten names of CollectedHeap::Name members In-Reply-To: <185a147f-87a7-7ef3-9b51-6411e84d7a2e@oracle.com> References: <185a147f-87a7-7ef3-9b51-6411e84d7a2e@oracle.com> Message-ID: <0494181c-db6f-261b-e3d5-f3575c2baa47@redhat.com> On 03/21/2018 09:09 AM, Per Liden wrote: > I propose that we shorten the member names in the CollectedHeap::Name enum. This is a pure cosmetic > change. Essentially naming the GCs, rather than some of the CollectedHeap childen. > > From: > SerialHeap > ParallelScavengeHeap > G1CollectedHeap > CMSHeap > > To: > Serial > Parallel > CMS > G1 > > This is a preparatory cleanup for JDK-8199925 and JDK-8199927. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8199918 > Webrev: http://cr.openjdk.java.net/~pliden/8199918/webrev.0 I like it very much. Some assert messages are now outdated: 135 assert(heap->kind() == CollectedHeap::CMS, "Not a CMSHeap"); 2485 assert(heap->kind() == CollectedHeap::G1, "Not a G1CollectedHeap"); 625 assert(heap->kind() == CollectedHeap::Parallel, "Not a ParallelScavengeHeap"); 1235 assert(heap->kind() == CollectedHeap::Serial || 1236 heap->kind() == CollectedHeap::CMS, "Not a GenCollectedHeap"); -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Wed Mar 21 10:33:04 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 21 Mar 2018 11:33:04 +0100 Subject: RFR: 8199925: Break out GC selection logic from GCArguments to GCSelector In-Reply-To: <35136ca4-cc0c-d977-8351-523a52a39c7a@redhat.com> References: <35136ca4-cc0c-d977-8351-523a52a39c7a@redhat.com> Message-ID: <15700f36-bf1b-b8a7-4954-8501b5ed7626@oracle.com> Hi Roman, On 03/21/2018 10:15 AM, Roman Kennke wrote: > Am 21.03.2018 um 09:12 schrieb Per Liden: >> In an attempt to make WhiteBox more GC agnostic, and in turn make it >> easier to plugin new GC without touching a lot of non-GC code, this >> patch breaks out the GC selection logic from GCArguments to GCSelector. >> There are two reasons why I think this makes sense: >> 1) The GC selection logic is self-contained and fairly unrelated to the >> rest of the flags processing done by GCArguments and its GC-specific >> children. >> 2) Parts of the GC selection logic is needed by WhiteBox to answer >> questions about which GC are supported, selected, etc. >> >> A follow up patch (JDK-8199927) will change WhiteBox (WB_CurrentGC, >> WB_AllSupportedGC, WB_GCSelectedByErgo), to use the GCSelector. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8199925 >> Webrev: http://cr.openjdk.java.net/~pliden/8199925/webrev.0 >> >> /Per > > My original proposal for the GCArguments did have a separate GCFactory. :-) > > In generally I like your change. > > I'd probably go even further and let GCSelector initialize and return an > instance of GCArguments. This way we wouldn't end up with two places > that have knowledge of all possible GCs, but would only have one place > (GCSelector) where one had to hook up for adding a new GC. > > What do you think? My intention here was that GCSelector would be a pure translator of Use*GC flags to CollectedHeap::Name, i.e. a small and well defined component, which knows nothing about GCArguments. The purpose of this was to clean out the WhiteBox stuff. On the other hand, I see what you mean, and too would like to reach a state where we have one place in the code that "knows about all the GCs". Let me think about it some more and maybe prototype an alternative, and we'll see if that gets us to a better place. /Per > > Roman > From per.liden at oracle.com Wed Mar 21 13:11:52 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 21 Mar 2018 14:11:52 +0100 Subject: RFR: 8199925: Break out GC selection logic from GCArguments to GCSelector In-Reply-To: <15700f36-bf1b-b8a7-4954-8501b5ed7626@oracle.com> References: <35136ca4-cc0c-d977-8351-523a52a39c7a@redhat.com> <15700f36-bf1b-b8a7-4954-8501b5ed7626@oracle.com> Message-ID: Hi, Here's an updated webrev. I renamed GCSelector to GCConfig and it's now also the goto place for getting hold of the GCArguments instance for the currently selected GC. http://cr.openjdk.java.net/~pliden/8199925/webrev.1 Btw, this is built on top of JDK-8199850 (Move parsing of VerifyGCType to G1) and JDK-8199918 (Shorten names of CollectedHeap::Name members), and the GCConfig::is_gc_* functions will eventually be used by JDK-8199927 (Make WhiteBox more GC agnostic). /Per On 03/21/2018 11:33 AM, Per Liden wrote: > Hi Roman, > > On 03/21/2018 10:15 AM, Roman Kennke wrote: >> Am 21.03.2018 um 09:12 schrieb Per Liden: >>> In an attempt to make WhiteBox more GC agnostic, and in turn make it >>> easier to plugin new GC without touching a lot of non-GC code, this >>> patch breaks out the GC selection logic from GCArguments to GCSelector. >>> There are two reasons why I think this makes sense: >>> 1) The GC selection logic is self-contained and fairly unrelated to the >>> rest of the flags processing done by GCArguments and its GC-specific >>> children. >>> 2) Parts of the GC selection logic is needed by WhiteBox to answer >>> questions about which GC are supported, selected, etc. >>> >>> A follow up patch (JDK-8199927) will change WhiteBox (WB_CurrentGC, >>> WB_AllSupportedGC, WB_GCSelectedByErgo), to use the GCSelector. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8199925 >>> Webrev: http://cr.openjdk.java.net/~pliden/8199925/webrev.0 >>> >>> /Per >> >> My original proposal for the GCArguments did have a separate >> GCFactory. :-) >> >> In generally I like your change. >> >> I'd probably go even further and let GCSelector initialize and return an >> instance of GCArguments. This way we wouldn't end up with two places >> that have knowledge of all possible GCs, but would only have one place >> (GCSelector) where one had to hook up for adding a new GC. >> >> What do you think? > > My intention here was that GCSelector would be a pure translator of > Use*GC flags to CollectedHeap::Name, i.e. a small and well defined > component, which knows nothing about GCArguments. The purpose of this > was to clean out the WhiteBox stuff. > > On the other hand, I see what you mean, and too would like to reach a > state where we have one place in the code that "knows about all the GCs". > > Let me think about it some more and maybe prototype an alternative, and > we'll see if that gets us to a better place. > > /Per > >> >> Roman >> From per.liden at oracle.com Wed Mar 21 13:57:17 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 21 Mar 2018 14:57:17 +0100 Subject: RFR: 8199918: Shorten names of CollectedHeap::Name members In-Reply-To: <0494181c-db6f-261b-e3d5-f3575c2baa47@redhat.com> References: <185a147f-87a7-7ef3-9b51-6411e84d7a2e@oracle.com> <0494181c-db6f-261b-e3d5-f3575c2baa47@redhat.com> Message-ID: Thanks for reviewing. Updated webrev: http://cr.openjdk.java.net/~pliden/8199918/webrev.0vs1 http://cr.openjdk.java.net/~pliden/8199918/webrev.1 /Per On 03/21/2018 10:19 AM, Aleksey Shipilev wrote: > On 03/21/2018 09:09 AM, Per Liden wrote: >> I propose that we shorten the member names in the CollectedHeap::Name enum. This is a pure cosmetic >> change. Essentially naming the GCs, rather than some of the CollectedHeap childen. >> >> From: >> SerialHeap >> ParallelScavengeHeap >> G1CollectedHeap >> CMSHeap >> >> To: >> Serial >> Parallel >> CMS >> G1 >> >> This is a preparatory cleanup for JDK-8199925 and JDK-8199927. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8199918 >> Webrev: http://cr.openjdk.java.net/~pliden/8199918/webrev.0 > > I like it very much. > > Some assert messages are now outdated: > > 135 assert(heap->kind() == CollectedHeap::CMS, "Not a CMSHeap"); > > 2485 assert(heap->kind() == CollectedHeap::G1, "Not a G1CollectedHeap"); > > 625 assert(heap->kind() == CollectedHeap::Parallel, "Not a ParallelScavengeHeap"); > > 1235 assert(heap->kind() == CollectedHeap::Serial || > 1236 heap->kind() == CollectedHeap::CMS, "Not a GenCollectedHeap"); > > > -Aleksey > > From shade at redhat.com Wed Mar 21 14:00:45 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 21 Mar 2018 15:00:45 +0100 Subject: RFR: 8199918: Shorten names of CollectedHeap::Name members In-Reply-To: References: <185a147f-87a7-7ef3-9b51-6411e84d7a2e@oracle.com> <0494181c-db6f-261b-e3d5-f3575c2baa47@redhat.com> Message-ID: On 03/21/2018 02:57 PM, Per Liden wrote: > Thanks for reviewing. > > Updated webrev: > > http://cr.openjdk.java.net/~pliden/8199918/webrev.0vs1 > http://cr.openjdk.java.net/~pliden/8199918/webrev.1 Looks good. Should it be "Invalid kind", not "Invalid name"? Maybe "kind()" is tripping me off, and it clashes with "name()" that returns char*. It is a minor nit, your call if you want to keep it this way. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From erik.osterlund at oracle.com Wed Mar 21 14:20:22 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 21 Mar 2018 15:20:22 +0100 Subject: RFR: 8199918: Shorten names of CollectedHeap::Name members In-Reply-To: References: <185a147f-87a7-7ef3-9b51-6411e84d7a2e@oracle.com> <0494181c-db6f-261b-e3d5-f3575c2baa47@redhat.com> Message-ID: <5AB26A26.6050903@oracle.com> Hi Per, Looks good. Thanks, /Erik On 2018-03-21 14:57, Per Liden wrote: > Thanks for reviewing. > > Updated webrev: > > http://cr.openjdk.java.net/~pliden/8199918/webrev.0vs1 > http://cr.openjdk.java.net/~pliden/8199918/webrev.1 > > /Per > > On 03/21/2018 10:19 AM, Aleksey Shipilev wrote: >> On 03/21/2018 09:09 AM, Per Liden wrote: >>> I propose that we shorten the member names in the >>> CollectedHeap::Name enum. This is a pure cosmetic >>> change. Essentially naming the GCs, rather than some of the >>> CollectedHeap childen. >>> >>> From: >>> SerialHeap >>> ParallelScavengeHeap >>> G1CollectedHeap >>> CMSHeap >>> >>> To: >>> Serial >>> Parallel >>> CMS >>> G1 >>> >>> This is a preparatory cleanup for JDK-8199925 and JDK-8199927. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8199918 >>> Webrev: http://cr.openjdk.java.net/~pliden/8199918/webrev.0 >> >> I like it very much. >> >> Some assert messages are now outdated: >> >> 135 assert(heap->kind() == CollectedHeap::CMS, "Not a CMSHeap"); >> >> 2485 assert(heap->kind() == CollectedHeap::G1, "Not a >> G1CollectedHeap"); >> >> 625 assert(heap->kind() == CollectedHeap::Parallel, "Not a >> ParallelScavengeHeap"); >> >> 1235 assert(heap->kind() == CollectedHeap::Serial || >> 1236 heap->kind() == CollectedHeap::CMS, "Not a >> GenCollectedHeap"); >> >> >> -Aleksey >> >> From per.liden at oracle.com Wed Mar 21 14:18:42 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 21 Mar 2018 15:18:42 +0100 Subject: RFR: 8199918: Shorten names of CollectedHeap::Name members In-Reply-To: References: <185a147f-87a7-7ef3-9b51-6411e84d7a2e@oracle.com> <0494181c-db6f-261b-e3d5-f3575c2baa47@redhat.com> Message-ID: <970b2a51-db69-0b69-484b-38f6e721602b@oracle.com> On 03/21/2018 03:00 PM, Aleksey Shipilev wrote: > On 03/21/2018 02:57 PM, Per Liden wrote: >> Thanks for reviewing. >> >> Updated webrev: >> >> http://cr.openjdk.java.net/~pliden/8199918/webrev.0vs1 >> http://cr.openjdk.java.net/~pliden/8199918/webrev.1 > > Looks good. Thanks! > > Should it be "Invalid kind", not "Invalid name"? Maybe "kind()" is tripping me off, and it clashes > with "name()" that returns char*. It is a minor nit, your call if you want to keep it this way. Yeah, it's a bit messy (for some historic reasons), both a kind and name would be "right" here. The enum type returned by CollectedHeap::kind() is CollectedHeap::Name. I'd like to change the type name to Kind, but the "enum Name + kind()" pattern is in at least 4 other GC-related classes, so for inconsistency we should change them all in that case. /Per > > -Aleksey > > > > From per.liden at oracle.com Wed Mar 21 14:20:26 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 21 Mar 2018 15:20:26 +0100 Subject: RFR: 8199918: Shorten names of CollectedHeap::Name members In-Reply-To: <5AB26A26.6050903@oracle.com> References: <185a147f-87a7-7ef3-9b51-6411e84d7a2e@oracle.com> <0494181c-db6f-261b-e3d5-f3575c2baa47@redhat.com> <5AB26A26.6050903@oracle.com> Message-ID: <421b74ca-d7e8-0c5f-8b3b-3427b322ef46@oracle.com> Thanks Erik for reviewing! /Per On 03/21/2018 03:20 PM, Erik ?sterlund wrote: > Hi Per, > > Looks good. > > Thanks, > /Erik > > On 2018-03-21 14:57, Per Liden wrote: >> Thanks for reviewing. >> >> Updated webrev: >> >> http://cr.openjdk.java.net/~pliden/8199918/webrev.0vs1 >> http://cr.openjdk.java.net/~pliden/8199918/webrev.1 >> >> /Per >> >> On 03/21/2018 10:19 AM, Aleksey Shipilev wrote: >>> On 03/21/2018 09:09 AM, Per Liden wrote: >>>> I propose that we shorten the member names in the >>>> CollectedHeap::Name enum. This is a pure cosmetic >>>> change. Essentially naming the GCs, rather than some of the >>>> CollectedHeap childen. >>>> >>>> From: >>>> SerialHeap >>>> ParallelScavengeHeap >>>> G1CollectedHeap >>>> CMSHeap >>>> >>>> To: >>>> Serial >>>> Parallel >>>> CMS >>>> G1 >>>> >>>> This is a preparatory cleanup for JDK-8199925 and JDK-8199927. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8199918 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8199918/webrev.0 >>> >>> I like it very much. >>> >>> Some assert messages are now outdated: >>> >>> ? 135?? assert(heap->kind() == CollectedHeap::CMS, "Not a CMSHeap"); >>> >>> 2485?? assert(heap->kind() == CollectedHeap::G1, "Not a >>> G1CollectedHeap"); >>> >>> ? 625?? assert(heap->kind() == CollectedHeap::Parallel, "Not a >>> ParallelScavengeHeap"); >>> >>> 1235?? assert(heap->kind() == CollectedHeap::Serial || >>> 1236????????? heap->kind() == CollectedHeap::CMS, "Not a >>> GenCollectedHeap"); >>> >>> >>> -Aleksey >>> >>> > From erik.osterlund at oracle.com Wed Mar 21 14:37:54 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 21 Mar 2018 15:37:54 +0100 Subject: RFR: 8199925: Break out GC selection logic from GCArguments to GCSelector In-Reply-To: References: <35136ca4-cc0c-d977-8351-523a52a39c7a@redhat.com> <15700f36-bf1b-b8a7-4954-8501b5ed7626@oracle.com> Message-ID: <5AB26E42.7040702@oracle.com> Hi Per, I like this. Looks good. Thanks, /Erik On 2018-03-21 14:11, Per Liden wrote: > Hi, > > Here's an updated webrev. I renamed GCSelector to GCConfig and it's > now also the goto place for getting hold of the GCArguments instance > for the currently selected GC. > > http://cr.openjdk.java.net/~pliden/8199925/webrev.1 > > Btw, this is built on top of JDK-8199850 (Move parsing of VerifyGCType > to G1) and JDK-8199918 (Shorten names of CollectedHeap::Name members), > and the GCConfig::is_gc_* functions will eventually be used by > JDK-8199927 (Make WhiteBox more GC agnostic). > > /Per > > On 03/21/2018 11:33 AM, Per Liden wrote: >> Hi Roman, >> >> On 03/21/2018 10:15 AM, Roman Kennke wrote: >>> Am 21.03.2018 um 09:12 schrieb Per Liden: >>>> In an attempt to make WhiteBox more GC agnostic, and in turn make it >>>> easier to plugin new GC without touching a lot of non-GC code, this >>>> patch breaks out the GC selection logic from GCArguments to >>>> GCSelector. >>>> There are two reasons why I think this makes sense: >>>> 1) The GC selection logic is self-contained and fairly unrelated to >>>> the >>>> rest of the flags processing done by GCArguments and its GC-specific >>>> children. >>>> 2) Parts of the GC selection logic is needed by WhiteBox to answer >>>> questions about which GC are supported, selected, etc. >>>> >>>> A follow up patch (JDK-8199927) will change WhiteBox (WB_CurrentGC, >>>> WB_AllSupportedGC, WB_GCSelectedByErgo), to use the GCSelector. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8199925 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8199925/webrev.0 >>>> >>>> /Per >>> >>> My original proposal for the GCArguments did have a separate >>> GCFactory. :-) >>> >>> In generally I like your change. >>> >>> I'd probably go even further and let GCSelector initialize and >>> return an >>> instance of GCArguments. This way we wouldn't end up with two places >>> that have knowledge of all possible GCs, but would only have one place >>> (GCSelector) where one had to hook up for adding a new GC. >>> >>> What do you think? >> >> My intention here was that GCSelector would be a pure translator of >> Use*GC flags to CollectedHeap::Name, i.e. a small and well defined >> component, which knows nothing about GCArguments. The purpose of this >> was to clean out the WhiteBox stuff. >> >> On the other hand, I see what you mean, and too would like to reach a >> state where we have one place in the code that "knows about all the >> GCs". >> >> Let me think about it some more and maybe prototype an alternative, >> and we'll see if that gets us to a better place. >> >> /Per >> >>> >>> Roman >>> From per.liden at oracle.com Wed Mar 21 14:50:31 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 21 Mar 2018 15:50:31 +0100 Subject: RFR: 8199927: Make WhiteBox more GC agnostic In-Reply-To: <8e0b6cdb-c1d4-37f9-1170-b5596051865d@oracle.com> References: <8e0b6cdb-c1d4-37f9-1170-b5596051865d@oracle.com> Message-ID: <95ce0e01-69ba-b7fa-bace-2e56a7a8d3ea@oracle.com> Updated webrev to reflect that GCSelector changed name to GCConfig in the patch for JDK-8199925. http://cr.openjdk.java.net/~pliden/8199927/webrev.1 /Per On 03/21/2018 09:14 AM, Per Liden wrote: > Make WhiteBox more GC agnostic by having it use the GCSelector to answer > questions about which GCs are supported, selected, etc. > > Going forward, this will make it easier to plug in new GC without having > to touch and adjust the WhiteBox code. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8199927 > Webrev: http://cr.openjdk.java.net/~pliden/8199927/webrev.0 > > /Per From shade at redhat.com Wed Mar 21 15:03:59 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 21 Mar 2018 16:03:59 +0100 Subject: RFR: 8199927: Make WhiteBox more GC agnostic In-Reply-To: <95ce0e01-69ba-b7fa-bace-2e56a7a8d3ea@oracle.com> References: <8e0b6cdb-c1d4-37f9-1170-b5596051865d@oracle.com> <95ce0e01-69ba-b7fa-bace-2e56a7a8d3ea@oracle.com> Message-ID: <70192223-16e2-c19b-9a78-d7c8e96c605f@redhat.com> On 03/21/2018 03:50 PM, Per Liden wrote: > Updated webrev to reflect that GCSelector changed name to GCConfig in the patch for JDK-8199925. > > http://cr.openjdk.java.net/~pliden/8199927/webrev.1 Looks good to me. This is a little dizzy, because I would have expected gc.isSelected() would tell me exactly what GC was being selected: isAcceptable = gc.isSupported() && (gc.isSelected() || GC.isSelectedErgonomically()); ...so it should be just: for (GC gc : GC.values()) { map.put("vm.gc." + gc.name(), "" + gc.isSelected()); } ? Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed Mar 21 15:07:12 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 21 Mar 2018 16:07:12 +0100 Subject: RFR: 8199925: Break out GC selection logic from GCArguments to GCSelector In-Reply-To: References: <35136ca4-cc0c-d977-8351-523a52a39c7a@redhat.com> <15700f36-bf1b-b8a7-4954-8501b5ed7626@oracle.com> Message-ID: <0C591305-5A1E-4E13-8411-C58E2A218623@redhat.com> Looks good! Thank you! Am 21. M?rz 2018 14:11:52 MEZ schrieb Per Liden : >Hi, > >Here's an updated webrev. I renamed GCSelector to GCConfig and it's now > >also the goto place for getting hold of the GCArguments instance for >the >currently selected GC. > >http://cr.openjdk.java.net/~pliden/8199925/webrev.1 > >Btw, this is built on top of JDK-8199850 (Move parsing of VerifyGCType >to G1) and JDK-8199918 (Shorten names of CollectedHeap::Name members), >and the GCConfig::is_gc_* functions will eventually be used by >JDK-8199927 (Make WhiteBox more GC agnostic). > >/Per > >On 03/21/2018 11:33 AM, Per Liden wrote: >> Hi Roman, >> >> On 03/21/2018 10:15 AM, Roman Kennke wrote: >>> Am 21.03.2018 um 09:12 schrieb Per Liden: >>>> In an attempt to make WhiteBox more GC agnostic, and in turn make >it >>>> easier to plugin new GC without touching a lot of non-GC code, this >>>> patch breaks out the GC selection logic from GCArguments to >GCSelector. >>>> There are two reasons why I think this makes sense: >>>> 1) The GC selection logic is self-contained and fairly unrelated to >the >>>> rest of the flags processing done by GCArguments and its >GC-specific >>>> children. >>>> 2) Parts of the GC selection logic is needed by WhiteBox to answer >>>> questions about which GC are supported, selected, etc. >>>> >>>> A follow up patch (JDK-8199927) will change WhiteBox (WB_CurrentGC, >>>> WB_AllSupportedGC, WB_GCSelectedByErgo), to use the GCSelector. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8199925 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8199925/webrev.0 >>>> >>>> /Per >>> >>> My original proposal for the GCArguments did have a separate >>> GCFactory. :-) >>> >>> In generally I like your change. >>> >>> I'd probably go even further and let GCSelector initialize and >return an >>> instance of GCArguments. This way we wouldn't end up with two places >>> that have knowledge of all possible GCs, but would only have one >place >>> (GCSelector) where one had to hook up for adding a new GC. >>> >>> What do you think? >> >> My intention here was that GCSelector would be a pure translator of >> Use*GC flags to CollectedHeap::Name, i.e. a small and well defined >> component, which knows nothing about GCArguments. The purpose of this > >> was to clean out the WhiteBox stuff. >> >> On the other hand, I see what you mean, and too would like to reach a > >> state where we have one place in the code that "knows about all the >GCs". >> >> Let me think about it some more and maybe prototype an alternative, >and >> we'll see if that gets us to a better place. >> >> /Per >> >>> >>> Roman >>> -- Diese Nachricht wurde von meinem Android-Ger?t mit K-9 Mail gesendet. From rkennke at redhat.com Wed Mar 21 16:58:01 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 21 Mar 2018 17:58:01 +0100 Subject: RFR: JDK-8199781: Don't use naked == for comparing oops In-Reply-To: <74ca6651-49aa-2b77-d7e9-a588fb2b8417@oracle.com> References: <74ca6651-49aa-2b77-d7e9-a588fb2b8417@oracle.com> Message-ID: <096620eb-e252-9689-c157-098314ac9119@redhat.com> Am 20.03.2018 um 17:06 schrieb Stefan Karlsson: > Hi Roman, > > On 2018-03-19 19:32, Roman Kennke wrote: >> GCs might need to handle object comparison specially, for example, when >> a concurrent GC allows both from- and to-space copies of an object to be >> in use. In this case, object equality cannot be expressed by direct >> address comparsion, but must also take potential different versions of >> an object into account. >> >> This adds a method oopDesc::equals(oop o1, oop o2), which goes through >> the Access API to compare objects. Default impl does == >> >> It is based on our finds from Shenandoah. We do, in fact, have some >> extra code in oopsHierarchy.hpp to detect naked oop==oop (with >> CheckUnhandledOops enabled), but it requires that code that does really >> want naked == (e.g. GCs) to use another method oopDesc::unsafe_equals() >> instead, and code that wants to use == and assert that the comparison is >> safe (e.g. both objects are already in to-space) use >> oopDesc::safe_equals(). Not sure if you want that? >> >> Also, I'd have preferred to mark equals() as inline in oop.hpp, but that >> pulls in a rats nest of dependencies problems. I hope the way I did it >> is acceptable? > > > We have actively been working on cleaning up our include dependencies, > so this should be dealt with properly before this patch can be accepted. > All files that use oopDesc::equals need to include oop.inline.hpp. The > usages of oopDesc::equals in .hpp needs to move to either .cpp files or > .inline.hpp files. > > See the files section on this page: > https://wiki.openjdk.java.net/display/HotSpot/StyleGuide > The problem here is that it involves oop.hpp and handles.hpp which are included very broadly all over the code base, and that my original attempt was to make equals inline. The obvious way out is to avoid that and make oopDesc::equals and the Handle::== operators non-inline and put them in the regular .cpp files. Looking at all the uses of equals, I don't see much that looks like it would benefit from inlining anyway. I believe that inline is over-used in OpenJDK too. It might be better to start out with regular methods and *if* they show up in profiles, consider to inline them. Would that be acceptable? Differential: http://cr.openjdk.java.net/~rkennke/JDK-8199781/webrev.01.diff Full: http://cr.openjdk.java.net/~rkennke/JDK-8199781/webrev.01 Give it a little while to upload, it's lots of stuff, and my connection is tiny ;-) Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From irogers at google.com Wed Mar 21 17:05:01 2018 From: irogers at google.com (Ian Rogers) Date: Wed, 21 Mar 2018 17:05:01 +0000 Subject: GetPrimitiveArrayCritical vs GetByteArrayRegion: 140x slow-down using -Xcheck:jni and java.util.zip.DeflaterOutputStream In-Reply-To: References: <22bbd632-47c1-7702-bd65-6a51f0bbad02@oracle.com> <9b5ff318-af41-03c4-efdc-b41b97cf6e61@oracle.com> <5A9D8C60.3050505@oracle.com> <5A9D8F33.5060500@oracle.com> <5A9DAC84.906@oracle.com> <5A9DAE87.8020801@oracle.com> <1521102348.2448.25.camel@oracle.com> <23639144-5217-4A0F-930C-EF24B4976544@oracle.com> <1521451687.2323.5.camel@oracle.com> Message-ID: Thanks for accepting these as bugs/RFEs: "140x slow-down using -Xcheck:jni and java.util.zip.DeflaterOutputStream" https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8199920 "Deprecate JNI critical APIs" https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8199919 "Report tardy threads as potential TTSP issues" https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8199921 "With -Xcheck:jni warn when a JNI critical region is "too long"" https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8199922 "Reduce size of JNI critical regions in JDK (for example via loop strip mining)" https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8199916 Ian On Tue, Mar 20, 2018 at 7:07 PM Ian Rogers wrote: > Thanks, via bugreport.java.com I filed bugs/RFEs with ids 9053047, > 9053048, 9053049, 9053050 and 9053051. > Ian > > On Mon, Mar 19, 2018 at 2:28 AM Thomas Schatzl > wrote: > >> Hi, >> >> On Fri, 2018-03-16 at 17:19 +0000, Ian Rogers wrote: >> > Thanks Paul, very interesting. >> > >> > On Fri, Mar 16, 2018 at 9:21 AM Paul Sandoz >> > wrote: >> > > Hi Ian, Thomas, >> > > >> > > [...] >> > > (This is also something we need to consider if we modify buffers to >> > > support capacities larger than Integer.MAX_VALUE. Also connects >> > > with Project Panama.) >> > > >> > > If Thomas has not done so or does not plan to i can log an issue >> > > for you. >> > > >> > >> > That'd be great. I wonder if identifying more TTSP issues should also >> > be a bug. Its interesting to observe that overlooking TTSP in C2 >> > motivated the Unsafe.copyMemory change permitting a fresh TTSP issue. >> > If TTSP is a 1st class issue then maybe we can deprecate JNI critical >> > regions to support that effort :-) >> >> Please log an issue. I am still a bit unsure what and how many issues >> should be filed. >> >> @Ian: at bugreports.oracle.com everyone may file bug reports without >> the need for an account. >> It will take some time until they show up in Jira due to vetting, but >> if you have a good case, and can e.g. link to the mailing list, this >> should be painless. >> >> Thanks, >> Thomas >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From rkennke at redhat.com Wed Mar 21 19:21:20 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 21 Mar 2018 20:21:20 +0100 Subject: RFR: JDK-8199735: Mark word updates need to use Access API In-Reply-To: <859ac657-78c9-c7ab-0ec0-eddda331001b@redhat.com> References: <7bd73e32-a26b-2504-f069-b74807ae5e53@redhat.com> <859ac657-78c9-c7ab-0ec0-eddda331001b@redhat.com> Message-ID: Am 19.03.2018 um 12:07 schrieb Roman Kennke: > Am 19.03.2018 um 11:40 schrieb Roman Kennke: >> Currently, the mark word is accessed directly in oopDesc::mark() >> set_mark() and a bunch of other accessors. Those need to use the Access >> API instead because GC might want to employ barriers on those accesses, >> pretty much like every other field in oopDesc. >> >> Notice that this is *not* about accessing the bits and fields inside the >> markOop, but about accessing the header itself. >> >> http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.01/ >> >> Testing: build fastdebug/release with and without PCH, passed tier1 >> fastdebug/release. >> >> Can I please get reviews? >> >> Thanks, Roman >> > > Just when I sent it, I realized that this is dropping the volatile from > the mark word access. Fixed here: > > > http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.02/ > Ping? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From sangheon.kim at oracle.com Thu Mar 22 04:47:15 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Wed, 21 Mar 2018 21:47:15 -0700 Subject: RFR [L][4/7]: 8180415: Rebuild remembered sets during the concurrent cycle In-Reply-To: <1520952891.3100.36.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520262478.2532.91.camel@oracle.com> <1520413176.2346.9.camel@oracle.com> <1f345c49-7793-7925-2c15-281f24d28e74@oracle.com> <1520509760.3121.15.camel@oracle.com> <1520952891.3100.36.camel@oracle.com> Message-ID: Hi Thomas, On 03/13/2018 07:54 AM, Thomas Schatzl wrote: > Hi all, > > while working on some follow-up changes and discussing with StefanJ, > there some issues came up: > > - the amount of marked bytes for humongous objects has been calculated > incorrectly, regardless of TAMS value it always considered humongous > objects as below tams. > There is no actual impact except in gc+liveness log messages in that > even if the humongous object were allocated during marking, it looked > like it had been allocated before marking. > > This has been found by the sanity check at the end of the rebuilding in > later changes where it has been made to work with humongous regions too > due to other changes. > > - StefanJ mentioned that gc+remset+tracking log messages should have a > gc id. Added. > > New webrevs: > http://cr.openjdk.java.net/~tschatzl/8180415/webrev.2_to_3 (diff) > http://cr.openjdk.java.net/~tschatzl/8180415/webrev.3 (full) webrev.3 looks good to me. I have some minor comments. =========================================================== src/hotspot/share/gc/g1/g1AllocRegion.cpp - Need to revert copyright update as there's no change. =========================================================== src/hotspot/share/gc/g1/g1ConcurrentMark.cpp 1045 class G1UpdateRemSetTrackingAfterRebuild : public HeapRegionClosure { - Couldn't have a constructor to pass G1CollectedHeap same as G1UpdateRemSetTrackingBeforeRebuild and use it from do_heap_region()? =========================================================== src/hotspot/share/gc/g1/g1ConcurrentMark.hpp ?143?? static const size_t RegionMarkStatsCacheSize = 1024; ?643?? static const uint RegionMarkStatsCacheSize = 1024; - There are 2 RegionMarkStatsCacheSize and only the latter one seems to be used. Or am I missing something? ?623?? // Rebuilds the remembered sets for chosen regions concurrently and in parallel to the application. - Please ignore if I'm wrong. :) I guess you meant, processes with MT and concurrently with the application. Something like '... chosen regions in parallel and concurrently with the application.'? =========================================================== src/hotspot/share/gc/g1/g1FullCollector.cpp - Copyright year update =========================================================== /src/hotspot/share/gc/g1/g1OopClosures.inline.hpp 145?? HeapRegion* to = _g1->heap_region_containing(obj); - Getting HeapRegionRemSet and using it seems better for 147,148,149 lines. =========================================================== src/hotspot/share/gc/g1/g1RemSet.cpp ?764???????? log_debug(gc,remset)("Humongous obj region %u marked %d start " PTR_FORMAT " region start " PTR_FORMAT " tams " PTR_FORMAT " tars " PTR_FORMAT, ?842?????????????????????????????????????? "bot " PTR_FORMAT " ntams " PTR_FORMAT " TARS " PTR_FORMAT, - Uppercase for tams and tars used in log message? ?888? uint num_workers = workers->active_workers(); - Need one more space before 'uint'. =========================================================== src/hotspot/share/gc/g1/heapRegionRemSet.hpp ?135?? bool add_reference(OopOrNarrowOopStar from, uint tid); - It would be helpful to mention when it returns true/false. I guess returns true when we add a reference. - I couldn't see returning true in add_reference(). Am I missing something? I know we are not using the return value so no problem so far. Or heapRegionRemSet.cpp:line 432 should return true? ?432?? return false; (heapRegionRemSet.cpp) =========================================================== src/hotspot/share/logging/logPrefix.hpp - Copyright year update =========================================================== test/hotspot/jtreg/gc/concurrent_phase_control/TestConcurrentPhaseControlG1Basics.java ?? 2? * Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved. - Should be '2017, 2018' =========================================================== src/hotspot/share/gc/g1/g1RemSetTrackingPolicy.cpp ?36 bool G1RemSetTrackingPolicy::needs_scan_for_rebuild(HeapRegion* r) const{ - Need a space between const and { ? 85???? if ((total_live_bytes > 0) && ? 86?????? (is_interesting_humongous_region(r) || CollectionSetChooser::region_occupancy_low_enough_for_evac(total_live_bytes)) && ? 87?????? !r->rem_set()->is_tracked()) { - Maybe I'm wrong but Line 86 and 87 seem to need more spaces to align with line 85. =========================================================== src/hotspot/share/gc/g1/g1RemSetTrackingPolicy.hpp 52?? // pause. Called at safepoint in the remark pause. - cleanup pause Thanks, Sangheon > > Thanks, > Thomas > > On Thu, 2018-03-08 at 12:49 +0100, Thomas Schatzl wrote: >> Hi Stefan, >> >> On Thu, 2018-03-08 at 10:15 +0100, Stefan Johansson wrote: >>> Thanks for addressing my initial comments Thomas, >>> >>> On 2018-03-07 09:59, Thomas Schatzl wrote: >>>> Hi all, >>>> >>>> Stefan had some comments about the HeapRegionType parameter in >>>> the G1RemSetTrackingPolicy::update_at_allocate() method - that >>>> one >>>> is not really required when placing the call to this method >>>> correctly and just using the HeapRegion's type directly. >>>> >>>> Changing this removes another 40 LOC of changes. >>>> >>>> There has been another bug introduced by me during cleaning up >>>> for >>>> final review: G1RemSetTrackingPolicy::update_at_free() is empty >>>> in >>>> this change, which is wrong for this change - in this change we >>>> still do not free the remembered sets during the cleanup pause, >>>> only in the Concurrent Cleanup phase. When doing this >>>> concurrently, >>>> an assert triggers when setting the remembered set state to empty >>>> outside a safepoint. >>>> The fix is to make the remembered set untracked during the >>>> Cleanup >>>> phase still. >>>> >>>> New webrevs: >>>> http://cr.openjdk.java.net/~tschatzl/8180415/webrev.0_to_1 (diff) >>>> http://cr.openjdk.java.net/~tschatzl/8180415/webrev.1 (full) >>> The change looks good. Just some additional comments: >>> src/hotspot/share/gc/g1/g1ConcurrentMark.cpp >>> 1255 GCTraceTime(Debug, gc)("Complete Remembered Set >>> Tracking"); >>> ... >>> 1291 GCTraceTime(Debug, gc)("Finalize Concurrent Mark >>> Cleanup"); >>> >>> Please add the "phases" tag to the above logging to be more >>> consistent with the remark and other pauses. >>> --- >>> src/hotspot/share/gc/g1/g1FullGCOopClosures.inline.hpp >>> 52 template inline void >>> G1AdjustClosure::adjust_pointer(T* >>> p) >>> >>> In this method the comments should be updated since we no longer >>> return anything. >>> --- >>> src/hotspot/share/gc/g1/g1RemSet.cpp >>> 757 HeapRegion* start_region = hr- >>>> humongous_start_region(); >>> start_region is only used to get bottom and crate an oop, so maybe >>> instead create the oop directly, like: >>> oop humongous = oop(hr->humongous_start_region()->bottom()); >>> >>> 761 // A humongous object is live (with respect to the >>> scanning) either >>> 762 // a) it is marked on the bitmap as such >>> 763 // b) its TARS is larger than nTAMS, i.e. has been >>> allocated during marking. >>> 764 if (_cm->next_mark_bitmap()->is_marked(start_region- >>>> bottom()) || >>> 765 (top_at_rebuild_start > hr- >>>> next_top_at_mark_start())) >>> { >>> >>> Break this check out to get something like: is_live(humongous, >>> tars, >>> ntams). >> Fixed. >> >>> There is also the marked_bytes counting in this method that makes >>> the >>> code a bit harder to follow. I've discussed this with Thomas >>> offline >>> and he has an idea of how to avoid this. >> As discussed I would like to postpone this for now as it is much >> harder >> than it looks and warrants its own CR (actually two or three). >> >>> Also in G1FullCollector::phase3_adjust_pointers() there is a >>> GCTraceTime that needs updating, since we no longer rebuild >>> remember >>> sets. >> All done. >> >> New webrevs: >> http://cr.openjdk.java.net/~tschatzl/8180415/webrev.1_to_2/ (diff) >> http://cr.openjdk.java.net/~tschatzl/8180415/webrev.2/ (full) >> >> Thomas >> From per.liden at oracle.com Thu Mar 22 09:04:42 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 22 Mar 2018 10:04:42 +0100 Subject: RFR: 8199927: Make WhiteBox more GC agnostic In-Reply-To: <70192223-16e2-c19b-9a78-d7c8e96c605f@redhat.com> References: <8e0b6cdb-c1d4-37f9-1170-b5596051865d@oracle.com> <95ce0e01-69ba-b7fa-bace-2e56a7a8d3ea@oracle.com> <70192223-16e2-c19b-9a78-d7c8e96c605f@redhat.com> Message-ID: <46525ef2-1fbe-6461-c417-b4e3ec08f710@oracle.com> On 03/21/2018 04:03 PM, Aleksey Shipilev wrote: > On 03/21/2018 03:50 PM, Per Liden wrote: >> Updated webrev to reflect that GCSelector changed name to GCConfig in the patch for JDK-8199925. >> >> http://cr.openjdk.java.net/~pliden/8199927/webrev.1 > > Looks good to me. Thanks for reviewing! > > This is a little dizzy, because I would have expected gc.isSelected() would tell me exactly what GC > was being selected: > > isAcceptable = gc.isSupported() && (gc.isSelected() || GC.isSelectedErgonomically()); > > ...so it should be just: > > for (GC gc : GC.values()) { > map.put("vm.gc." + gc.name(), "" + gc.isSelected()); > } > The way VMProps works and how the properties are set up can indeed make you a little dizzy. For example, vm.gc.Parallel=true doesn't necessarily mean that ParallelGC was "selected", just that it's an "acceptable" configuration. A test running with an explicit -XX:UseG1GC flag will get: vm.gc.Serial=false vm.gc.Parallel=false vm.gc.CMS=false vm.gc.G1=true And a test running with no explicit GC flag (which selects G1 ergonomically), will get: vm.gc.Serial=true vm.gc.Parallel=true vm.gc.CMS=true vm.gc.G1=true I believe your suggested cleanup would change that behavior. cheers, Per > ? > > Thanks, > -Aleksey > From shade at redhat.com Thu Mar 22 09:08:12 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 22 Mar 2018 10:08:12 +0100 Subject: RFR: 8199927: Make WhiteBox more GC agnostic In-Reply-To: <46525ef2-1fbe-6461-c417-b4e3ec08f710@oracle.com> References: <8e0b6cdb-c1d4-37f9-1170-b5596051865d@oracle.com> <95ce0e01-69ba-b7fa-bace-2e56a7a8d3ea@oracle.com> <70192223-16e2-c19b-9a78-d7c8e96c605f@redhat.com> <46525ef2-1fbe-6461-c417-b4e3ec08f710@oracle.com> Message-ID: <546d4ea5-3590-88dc-c7ec-8296e73c6e14@redhat.com> On 03/22/2018 10:04 AM, Per Liden wrote: > On 03/21/2018 04:03 PM, Aleksey Shipilev wrote: >> This is a little dizzy, because I would have expected gc.isSelected() would tell me exactly what GC >> was being selected: >> >> ? isAcceptable = gc.isSupported() && (gc.isSelected() || GC.isSelectedErgonomically()); >> >> ...so it should be just: >> >> ? for (GC gc : GC.values()) { >> ??? map.put("vm.gc." + gc.name(), "" + gc.isSelected()); >> ? } >> > > The way VMProps works and how the properties are set up can indeed make you a little dizzy. For > example, vm.gc.Parallel=true doesn't necessarily mean that ParallelGC was "selected", just that it's > an "acceptable" configuration. > > A test running with an explicit -XX:UseG1GC flag will get: > vm.gc.Serial=false > vm.gc.Parallel=false > vm.gc.CMS=false > vm.gc.G1=true > > And a test running with no explicit GC flag (which selects G1 ergonomically), will get: > vm.gc.Serial=true > vm.gc.Parallel=true > vm.gc.CMS=true > vm.gc.G1=true Ewww, messy. I naively thought vm.gc.* filters for jtregs are filtering based on what GC was actually running. But this is another thing to follow-up on. The patch itself looks good then. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Thu Mar 22 10:21:53 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 22 Mar 2018 11:21:53 +0100 Subject: RFR: 8199927: Make WhiteBox more GC agnostic In-Reply-To: <546d4ea5-3590-88dc-c7ec-8296e73c6e14@redhat.com> References: <8e0b6cdb-c1d4-37f9-1170-b5596051865d@oracle.com> <95ce0e01-69ba-b7fa-bace-2e56a7a8d3ea@oracle.com> <70192223-16e2-c19b-9a78-d7c8e96c605f@redhat.com> <46525ef2-1fbe-6461-c417-b4e3ec08f710@oracle.com> <546d4ea5-3590-88dc-c7ec-8296e73c6e14@redhat.com> Message-ID: <99d7a779-ae87-eea5-f702-46111cf311e0@oracle.com> On 03/22/2018 10:08 AM, Aleksey Shipilev wrote: > On 03/22/2018 10:04 AM, Per Liden wrote: >> On 03/21/2018 04:03 PM, Aleksey Shipilev wrote: >>> This is a little dizzy, because I would have expected gc.isSelected() would tell me exactly what GC >>> was being selected: >>> >>> ? isAcceptable = gc.isSupported() && (gc.isSelected() || GC.isSelectedErgonomically()); >>> >>> ...so it should be just: >>> >>> ? for (GC gc : GC.values()) { >>> ??? map.put("vm.gc." + gc.name(), "" + gc.isSelected()); >>> ? } >>> >> >> The way VMProps works and how the properties are set up can indeed make you a little dizzy. For >> example, vm.gc.Parallel=true doesn't necessarily mean that ParallelGC was "selected", just that it's >> an "acceptable" configuration. >> >> A test running with an explicit -XX:UseG1GC flag will get: >> vm.gc.Serial=false >> vm.gc.Parallel=false >> vm.gc.CMS=false >> vm.gc.G1=true >> >> And a test running with no explicit GC flag (which selects G1 ergonomically), will get: >> vm.gc.Serial=true >> vm.gc.Parallel=true >> vm.gc.CMS=true >> vm.gc.G1=true > > Ewww, messy. I naively thought vm.gc.* filters for jtregs are filtering based on what GC was > actually running. But this is another thing to follow-up on. The patch itself looks good then. To add to the confusion, there's also a vm.gc property, which only reflects what was explicitly specified on the command line. I don't quite see why it was done like this. I kind of hope we can clean that up some day. A single vm.gc property should be enough as far as I can tell, but maybe I'm missing something. Anyways, thanks for reviewing! /Per > > -Aleksey > From thomas.schatzl at oracle.com Thu Mar 22 10:37:29 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 22 Mar 2018 11:37:29 +0100 Subject: RFR [L][4/7]: 8180415: Rebuild remembered sets during the concurrent cycle In-Reply-To: References: <1520258985.2532.40.camel@oracle.com> <1520262478.2532.91.camel@oracle.com> <1520413176.2346.9.camel@oracle.com> <1f345c49-7793-7925-2c15-281f24d28e74@oracle.com> <1520509760.3121.15.camel@oracle.com> <1520952891.3100.36.camel@oracle.com> Message-ID: <1521715049.2448.10.camel@oracle.com> Hi Sangheon, thanks for your thorough review. On Wed, 2018-03-21 at 21:47 -0700, sangheon.kim wrote: > Hi Thomas, > > On 03/13/2018 07:54 AM, Thomas Schatzl wrote: > > Hi all, > > [...] > > > > New webrevs: > > http://cr.openjdk.java.net/~tschatzl/8180415/webrev.2_to_3 (diff) > > http://cr.openjdk.java.net/~tschatzl/8180415/webrev.3 (full) > > webrev.3 looks good to me. > I have some minor comments. > > =========================================================== > src/hotspot/share/gc/g1/g1AllocRegion.cpp > - Need to revert copyright update as there's no change. > > =========================================================== > src/hotspot/share/gc/g1/g1ConcurrentMark.cpp > 1045 class G1UpdateRemSetTrackingAfterRebuild : public > HeapRegionClosure { > - Couldn't have a constructor to pass G1CollectedHeap same as > G1UpdateRemSetTrackingBeforeRebuild and use it from do_heap_region()? Fixed. I will do a cleanup pass to make every method use _g1h in g1ConcurrentMark.cpp soon. > =========================================================== > src/hotspot/share/gc/g1/g1ConcurrentMark.hpp > 143 static const size_t RegionMarkStatsCacheSize = 1024; > 643 static const uint RegionMarkStatsCacheSize = 1024; > - There are 2 RegionMarkStatsCacheSize and only the latter one seems > to be used. Or am I missing something? > > 623 // Rebuilds the remembered sets for chosen regions > concurrently > and in parallel to the application. > - Please ignore if I'm wrong. :) I guess you meant, processes with > MT > and concurrently with the application. Something like '... chosen > regions in parallel and concurrently with the application.'? Fixed, thanks! > > =========================================================== > src/hotspot/share/gc/g1/g1FullCollector.cpp > - Copyright year update > > =========================================================== > /src/hotspot/share/gc/g1/g1OopClosures.inline.hpp > 145 HeapRegion* to = _g1->heap_region_containing(obj); > - Getting HeapRegionRemSet and using it seems better for 147,148,149 > lines. > > =========================================================== > src/hotspot/share/gc/g1/g1RemSet.cpp > 764 log_debug(gc,remset)("Humongous obj region %u marked > %d > start " PTR_FORMAT " region start " PTR_FORMAT " tams " PTR_FORMAT " > tars " PTR_FORMAT, > 842 "bot " PTR_FORMAT " ntams > " > PTR_FORMAT " TARS " PTR_FORMAT, > - Uppercase for tams and tars used in log message? > > 888 uint num_workers = workers->active_workers(); > - Need one more space before 'uint'. > > =========================================================== > src/hotspot/share/gc/g1/heapRegionRemSet.hpp > 135 bool add_reference(OopOrNarrowOopStar from, uint tid); > - It would be helpful to mention when it returns true/false. I guess > returns true when we add a reference. > - I couldn't see returning true in add_reference(). Am I missing > something? I know we are not using the return value so no problem so > far. Or heapRegionRemSet.cpp:line 432 should return true? > 432 return false; (heapRegionRemSet.cpp) > The return value is not used (any more), I removed the return value. > =========================================================== > src/hotspot/share/logging/logPrefix.hpp > - Copyright year update > > =========================================================== > test/hotspot/jtreg/gc/concurrent_phase_control/TestConcurrentPhaseCon > trolG1Basics.java > 2 * Copyright (c) 2018, Oracle and/or its affiliates. All > rights reserved. > - Should be '2017, 2018' > > =========================================================== > src/hotspot/share/gc/g1/g1RemSetTrackingPolicy.cpp > 36 bool G1RemSetTrackingPolicy::needs_scan_for_rebuild(HeapRegion* > r) > const{ > - Need a space between const and { > > 85 if ((total_live_bytes > 0) && > 86 (is_interesting_humongous_region(r) || > CollectionSetChooser::region_occupancy_low_enough_for_evac(total_live > _bytes)) > && > 87 !r->rem_set()->is_tracked()) { > - Maybe I'm wrong but Line 86 and 87 seem to need more spaces to > align > with line 85. > > =========================================================== > src/hotspot/share/gc/g1/g1RemSetTrackingPolicy.hpp > 52 // pause. Called at safepoint in the remark pause. > - cleanup pause > All done. http://cr.openjdk.java.net/~tschatzl/8180415/webrev.3_to_4 (diff) http://cr.openjdk.java.net/~tschatzl/8180415/webrev.4 (full) In addition to that I added the following minor changes: - renamed the "CLEAR_REMEMBERED_SET" phase in the concurrent gc phase manager to "REBUILD_REMEMBERED_Set" as this matches the code better. - several fixes to later re-enable the TestVerifyGCType (JDK-8193067: gc/g1/TestVerifyGCType.java still unstable) later. That test parses the log messages done during verification, and expects them to have a "During GC" prefix. - there is one minor I think pre-existing issue fixed, in g1Policy.cpp::decide_on_conc_mark_initiation(): it is possible that a user initiated concurrent mark request can happen right after the Cleanup pause before the "last gc before mixed gc". During that time there is a collection set, and that one gets messed up in the following marking (the evacuation efficiency of the regions may change). This triggers some (benign) assert in the collection set chooser. We need to clear any existing collection set before initiating concurrent mark. Thanks, Thomas From thomas.schatzl at oracle.com Thu Mar 22 10:46:02 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 22 Mar 2018 11:46:02 +0100 Subject: RFR [M][7/7]: 8197932: Better split work in rebuild remembered sets phase In-Reply-To: <4110ec65-c66f-eca0-3b7a-6580f0edf00d@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520265115.2532.126.camel@oracle.com> <1520953401.3100.41.camel@oracle.com> <1521461029.2323.11.camel@oracle.com> <4110ec65-c66f-eca0-3b7a-6580f0edf00d@oracle.com> Message-ID: <1521715562.2448.15.camel@oracle.com> Hi Stefan, thanks for your review, and sorry for the late reply... On Mon, 2018-03-19 at 16:39 +0100, Stefan Johansson wrote: > Thanks Thomas, > > On 2018-03-19 13:03, Thomas Schatzl wrote: > > Hi all, > > > > Stefan proposed to use some iterator over the work area that > > internally handles the boundary at the TAMS wrt to liveness when > > scanning. > > > > This makes the change a bit more compact, performance does not seem > > to differ, so why not... :) > > I really like this approach :) A few additional small things: > src/hotspot/share/gc/g1/g1RemSet.cpp > 787 if (!is_live(_current)) { > > I would like to invert the if-statement in the LiveObjIterator > constructor and move the call to move_if_below_ntams() into the > if-statement updating _current, something like this: > if (is_live(_current)) { > // Non-objArrays were scanned by the previous part of that region. > if (_current < mr.start() && !oop(_current)->is_objArray()) { > _current += oop(_current)->size(); > // We might have positioned _current on a non-live object. > Reposition to the next > // live one if needed. > move_if_below_ntams(); > } > } > } else { > ... > } > > 926 #ifdef ASSERT > 927 // In the final iteration of the loop the region might > have been eagerly reclaimed. > 928 // Simply filter out those regions. We can not just use > region > type because there > 929 // might have already been new allocations into these > regions. > 930 HeapWord* const top_at_rebuild_start = > _cm->top_at_rebuild_start(region_idx); > 931 assert(!hr->is_old() || > > What do you think about making this assert into a guarantee or use > DEBUG_ONLY() to avoid the #ifdef ASSERT? Also maybe adding something > to the comment explaining that it is the check on > top_at_rebuild_start that filters out eagerly reclaimed regions. Done. > --- > src/hotspot/share/gc/g1/g1_globals.hpp > 259 experimental(size_t, G1RebuildRemSetChunkSize, 256 * K, > > Did you test to see if changing the G1RebuildRemSetChunkSize flag > had any impact on performance, otherwise maybe we could make it a > constant instead. The problem is not so much increasing the value, but decreasing it for very slow systems. Decreasing it further adds some overhead. The other point is that since we track the time a chunk takes, it seems strange to not have any knobs to deal with problems in that area. New webrev: http://cr.openjdk.java.net/~tschatzl/8197932/webrev.2_to_3 (diff) http://cr.openjdk.java.net/~tschatzl/8197932/webrev.3 (full) In this webrev I also did some somewhat messy looking renaming of nTAMS to TAMS (removing the n) and passing the (n)TAMS around in conjunction with the bitmap. This will make a follow-up change a lot easier to read. Thanks, Thomas From stefan.johansson at oracle.com Thu Mar 22 13:18:58 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 22 Mar 2018 14:18:58 +0100 Subject: RFR: 8199850: Move parsing of VerifyGCType to G1 In-Reply-To: <5d3f62d2-f4c7-affa-a21c-be3330262af9@oracle.com> References: <5d3f62d2-f4c7-affa-a21c-be3330262af9@oracle.com> Message-ID: Seems reasonable =) StefanJ On 2018-03-20 10:11, Per Liden wrote: > When JDK-8191821 introduced the VerifyGCType option for G1 I didn't > quite see the need for this to affect the GCArguments class. I'm > proposing we move this logic into G1HeapVerifier, which I believe is a > better place for this. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8199850 > Webrev: http://cr.openjdk.java.net/~pliden/8199850/webrev.0 > > Testing: Used gc/g1/TestVerifyGCType.java to verify that nothing broke > > /Per From erik.osterlund at oracle.com Thu Mar 22 13:32:02 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 22 Mar 2018 14:32:02 +0100 Subject: RFR: JDK-8199735: Mark word updates need to use Access API In-Reply-To: References: <7bd73e32-a26b-2504-f069-b74807ae5e53@redhat.com> <859ac657-78c9-c7ab-0ec0-eddda331001b@redhat.com> Message-ID: <5AB3B052.4000304@oracle.com> Hi Roman, A few comments: 1) Please make sure all files referencing these accessors that have been moved to the oop.inline.hpp file now #include the oop.inline.hpp file. I found that there are quite a few missing by looking for references to oopDesc::mark() with rtags. 2) I think the mark_addr() function should now be called mark_addr_raw() for consistency. 3) I wonder if there should be _raw() accessors that the GCs use instead. It feels like the GC should not go through the Access API to acquire mark words. Thanks, /Erik On 2018-03-21 20:21, Roman Kennke wrote: > Am 19.03.2018 um 12:07 schrieb Roman Kennke: >> Am 19.03.2018 um 11:40 schrieb Roman Kennke: >>> Currently, the mark word is accessed directly in oopDesc::mark() >>> set_mark() and a bunch of other accessors. Those need to use the Access >>> API instead because GC might want to employ barriers on those accesses, >>> pretty much like every other field in oopDesc. >>> >>> Notice that this is *not* about accessing the bits and fields inside the >>> markOop, but about accessing the header itself. >>> >>> http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.01/ >>> >>> Testing: build fastdebug/release with and without PCH, passed tier1 >>> fastdebug/release. >>> >>> Can I please get reviews? >>> >>> Thanks, Roman >>> >> Just when I sent it, I realized that this is dropping the volatile from >> the mark word access. Fixed here: >> >> >> http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.02/ >> > Ping? > From erik.osterlund at oracle.com Thu Mar 22 14:19:55 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 22 Mar 2018 15:19:55 +0100 Subject: RFR: 8200113: Make Access load proxys smarter Message-ID: <5AB3BB8B.8090309@oracle.com> Hi, Access returns the result of loads through load proxy objects that implicity convert themselves to a template inferred type. This is a metaprogramming technique used to infer return types in C++. However, I have heard requests that it would be great if it could be a bit smarter and do more than just be assigned to a type. Example use cases that do not work today without workarounds: oop val = ...; narrowOop narrow = 0u; oop *oop_val = &val; narrowOop *narrow_val = &narrow; HeapWord *heap_word_val = reinterpret_cast(oop_val); if (val == HeapAccess<>::oop_load_at(val, 16)) {} if (HeapAccess<>::oop_load_at(val, 16) == val) {} if (val != HeapAccess<>::oop_load_at(val, 16)) {} if (HeapAccess<>::oop_load_at(val, 16) != val) {} if (HeapAccess<>::oop_load(oop_val) != val) {} if (HeapAccess<>::oop_load(heap_word_val) != val) {} if (RawAccess<>::oop_load(narrow_val) != narrow) {} if (HeapAccess<>::oop_load(oop_val) == val) {} if (HeapAccess<>::oop_load(heap_word_val) == val) {} if (RawAccess<>::oop_load(narrow_val) == narrow) {} if (val != HeapAccess<>::oop_load(oop_val)) {} if (val != HeapAccess<>::oop_load(heap_word_val)) {} if (narrow != RawAccess<>::oop_load(narrow_val)) {} if (val == HeapAccess<>::oop_load(oop_val)) {} if (val == HeapAccess<>::oop_load(heap_word_val)) {} if (narrow == RawAccess<>::oop_load(narrow_val)) {} if ((oop)HeapAccess<>::oop_load(oop_val) == NULL) {} oop tmp = true ? HeapAccess<>::oop_load(oop_val) : val; Here is a patch that solves this: http://cr.openjdk.java.net/~eosterlund/8200113/webrev.00/ ...and here is the bug ID: https://bugs.openjdk.java.net/browse/JDK-8200113 Thanks, /Erik From stefan.karlsson at oracle.com Thu Mar 22 15:11:12 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 22 Mar 2018 16:11:12 +0100 Subject: RFR: 8200113: Make Access load proxys smarter In-Reply-To: <5AB3BB8B.8090309@oracle.com> References: <5AB3BB8B.8090309@oracle.com> Message-ID: On 2018-03-22 15:19, Erik ?sterlund wrote: > Hi, > > Access returns the result of loads through load proxy objects that > implicity convert themselves to a template inferred type. This is a > metaprogramming technique used to infer return types in C++. > > However, I have heard requests that it would be great if it could be a > bit smarter and do more than just be assigned to a type. > > Example use cases that do not work today without workarounds: > > ? oop val = ...; > ? narrowOop narrow = 0u; > ? oop *oop_val = &val; > ? narrowOop *narrow_val = &narrow; > ? HeapWord *heap_word_val = reinterpret_cast(oop_val); > > ? if (val == HeapAccess<>::oop_load_at(val, 16)) {} > ? if (HeapAccess<>::oop_load_at(val, 16) == val) {} > ? if (val != HeapAccess<>::oop_load_at(val, 16)) {} > ? if (HeapAccess<>::oop_load_at(val, 16) != val) {} > > ? if (HeapAccess<>::oop_load(oop_val) != val) {} > ? if (HeapAccess<>::oop_load(heap_word_val) != val) {} > ? if (RawAccess<>::oop_load(narrow_val) != narrow) {} > > ? if (HeapAccess<>::oop_load(oop_val) == val) {} > ? if (HeapAccess<>::oop_load(heap_word_val) == val) {} > ? if (RawAccess<>::oop_load(narrow_val) == narrow) {} > > ? if (val != HeapAccess<>::oop_load(oop_val)) {} > ? if (val != HeapAccess<>::oop_load(heap_word_val)) {} > ? if (narrow != RawAccess<>::oop_load(narrow_val)) {} > > ? if (val == HeapAccess<>::oop_load(oop_val)) {} > ? if (val == HeapAccess<>::oop_load(heap_word_val)) {} > ? if (narrow == RawAccess<>::oop_load(narrow_val)) {} > > ? if ((oop)HeapAccess<>::oop_load(oop_val) == NULL) {} > > ? oop tmp = true ? HeapAccess<>::oop_load(oop_val) : val; > > Here is a patch that solves this: > http://cr.openjdk.java.net/~eosterlund/8200113/webrev.00/ This looks good to me. Thanks, StefanK > > ...and here is the bug ID: > https://bugs.openjdk.java.net/browse/JDK-8200113 > > Thanks, > /Erik > From erik.osterlund at oracle.com Thu Mar 22 15:42:57 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 22 Mar 2018 16:42:57 +0100 Subject: RFR: 8200113: Make Access load proxys smarter In-Reply-To: References: <5AB3BB8B.8090309@oracle.com> Message-ID: <5AB3CF01.5090008@oracle.com> Hi Stefan, Thanks for the review. /Erik On 2018-03-22 16:11, Stefan Karlsson wrote: > > > On 2018-03-22 15:19, Erik ?sterlund wrote: >> Hi, >> >> Access returns the result of loads through load proxy objects that >> implicity convert themselves to a template inferred type. This is a >> metaprogramming technique used to infer return types in C++. >> >> However, I have heard requests that it would be great if it could be >> a bit smarter and do more than just be assigned to a type. >> >> Example use cases that do not work today without workarounds: >> >> oop val = ...; >> narrowOop narrow = 0u; >> oop *oop_val = &val; >> narrowOop *narrow_val = &narrow; >> HeapWord *heap_word_val = reinterpret_cast(oop_val); >> >> if (val == HeapAccess<>::oop_load_at(val, 16)) {} >> if (HeapAccess<>::oop_load_at(val, 16) == val) {} >> if (val != HeapAccess<>::oop_load_at(val, 16)) {} >> if (HeapAccess<>::oop_load_at(val, 16) != val) {} >> >> if (HeapAccess<>::oop_load(oop_val) != val) {} >> if (HeapAccess<>::oop_load(heap_word_val) != val) {} >> if (RawAccess<>::oop_load(narrow_val) != narrow) {} >> >> if (HeapAccess<>::oop_load(oop_val) == val) {} >> if (HeapAccess<>::oop_load(heap_word_val) == val) {} >> if (RawAccess<>::oop_load(narrow_val) == narrow) {} >> >> if (val != HeapAccess<>::oop_load(oop_val)) {} >> if (val != HeapAccess<>::oop_load(heap_word_val)) {} >> if (narrow != RawAccess<>::oop_load(narrow_val)) {} >> >> if (val == HeapAccess<>::oop_load(oop_val)) {} >> if (val == HeapAccess<>::oop_load(heap_word_val)) {} >> if (narrow == RawAccess<>::oop_load(narrow_val)) {} >> >> if ((oop)HeapAccess<>::oop_load(oop_val) == NULL) {} >> >> oop tmp = true ? HeapAccess<>::oop_load(oop_val) : val; >> >> Here is a patch that solves this: >> http://cr.openjdk.java.net/~eosterlund/8200113/webrev.00/ > > This looks good to me. > > Thanks, > StefanK > >> >> ...and here is the bug ID: >> https://bugs.openjdk.java.net/browse/JDK-8200113 >> >> Thanks, >> /Erik >> From stefan.johansson at oracle.com Thu Mar 22 15:42:46 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 22 Mar 2018 16:42:46 +0100 Subject: RFR/RFC: 8196071: Change G1 Full GC heap and thread sizing ergonomics Message-ID: <7497d3fb-044d-35f2-f02f-70ea79f0e2aa@oracle.com> Hi, Please review or comment on this change to let the G1 Full GC calculate the number of worker threads. Links JBS: https://bugs.openjdk.java.net/browse/JDK-8196071 Webrev: http://cr.openjdk.java.net/~sjohanss/8196071/00/ Summary After making UseDynamicNumberOfGCThreads true by default, having a sane thread sizing model for the G1 Full GC is important. Before this change the Full GC would use the same number of threads as the previous collection, and if being the first just use 1 thread. The proposed change now have the Full GC consider two metrics to get a good number of worker threads. The first metric is to try to avoid to much wasted space. On average each worker used by the parallel full GC will cause half a region of unused memory. To make sure we don't use too many workers we use G1HeapWastePercent to limit the number of workers. The other metric is the one used in AdaptiveSizePolicy, it looks at the flag HeapSizePerGCWorker and limit the number by this. In most cases this is more restrictive than using G1HeapWastePercent, but for small heaps setting a large G1HeapRegionSize, G1HeapWastePercent will come into play. This change also changes the default value of HeapSizePerGCWorker to 32M instead of 64M, this can be reverted if anyone has a good argument to leave it as is. My testing shows that for DaCapo, SpecJVM98 and GCBasher (gc intensive stress test) with small heaps we get better pause times with the flag set to 32M. There might be other arguments, such as not using to much resources on small systems that might also be important so please let me know if you have reasons why the flag should be left unchanged. Testing Functional testing through Mach5 and manual performance testing with small heaps and different values for HeapSizePerGCThread. Thanks, Stefan From erik.osterlund at oracle.com Thu Mar 22 16:03:58 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 22 Mar 2018 17:03:58 +0100 Subject: RFR: 8200113: Make Access load proxys smarter In-Reply-To: <5AB3BB8B.8090309@oracle.com> References: <5AB3BB8B.8090309@oracle.com> Message-ID: <5AB3D3EE.1070209@oracle.com> Looping in hotspot-dev. /Erik On 2018-03-22 15:19, Erik ?sterlund wrote: > Hi, > > Access returns the result of loads through load proxy objects that > implicity convert themselves to a template inferred type. This is a > metaprogramming technique used to infer return types in C++. > > However, I have heard requests that it would be great if it could be a > bit smarter and do more than just be assigned to a type. > > Example use cases that do not work today without workarounds: > > oop val = ...; > narrowOop narrow = 0u; > oop *oop_val = &val; > narrowOop *narrow_val = &narrow; > HeapWord *heap_word_val = reinterpret_cast(oop_val); > > if (val == HeapAccess<>::oop_load_at(val, 16)) {} > if (HeapAccess<>::oop_load_at(val, 16) == val) {} > if (val != HeapAccess<>::oop_load_at(val, 16)) {} > if (HeapAccess<>::oop_load_at(val, 16) != val) {} > > if (HeapAccess<>::oop_load(oop_val) != val) {} > if (HeapAccess<>::oop_load(heap_word_val) != val) {} > if (RawAccess<>::oop_load(narrow_val) != narrow) {} > > if (HeapAccess<>::oop_load(oop_val) == val) {} > if (HeapAccess<>::oop_load(heap_word_val) == val) {} > if (RawAccess<>::oop_load(narrow_val) == narrow) {} > > if (val != HeapAccess<>::oop_load(oop_val)) {} > if (val != HeapAccess<>::oop_load(heap_word_val)) {} > if (narrow != RawAccess<>::oop_load(narrow_val)) {} > > if (val == HeapAccess<>::oop_load(oop_val)) {} > if (val == HeapAccess<>::oop_load(heap_word_val)) {} > if (narrow == RawAccess<>::oop_load(narrow_val)) {} > > if ((oop)HeapAccess<>::oop_load(oop_val) == NULL) {} > > oop tmp = true ? HeapAccess<>::oop_load(oop_val) : val; > > Here is a patch that solves this: > http://cr.openjdk.java.net/~eosterlund/8200113/webrev.00/ > > ...and here is the bug ID: > https://bugs.openjdk.java.net/browse/JDK-8200113 > > Thanks, > /Erik > From thomas.schatzl at oracle.com Thu Mar 22 16:20:59 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 22 Mar 2018 17:20:59 +0100 Subject: RFR: 8199850: Move parsing of VerifyGCType to G1 In-Reply-To: <5d3f62d2-f4c7-affa-a21c-be3330262af9@oracle.com> References: <5d3f62d2-f4c7-affa-a21c-be3330262af9@oracle.com> Message-ID: <1521735659.2264.5.camel@oracle.com> Hi, On Tue, 2018-03-20 at 10:11 +0100, Per Liden wrote: > When JDK-8191821 introduced the VerifyGCType option for G1 I didn't > quite see the need for this to affect the GCArguments class. I'm > proposing we move this logic into G1HeapVerifier, which I believe is > a better place for this. I would actually prefer this parsing to be located and called by the G1Arguments class. I really dislike the (existing) G1 habit of every class doing its own arguments parsing. It's much more convenient to have one place to look at compared to first searching where it's actually parsed. I do agree that since it is not supported by other collectors (yet?) the shared code may not be the perfect place for it. However if others disagree, keep it this way. > Bug: https://bugs.openjdk.java.net/browse/JDK-8199850 > Webrev: http://cr.openjdk.java.net/~pliden/8199850/webrev.0 > > Testing: Used gc/g1/TestVerifyGCType.java to verify that nothing > broke (I assume you executed it directly as it is currently problemlisted) Thanks, Thomas From sangheon.kim at oracle.com Thu Mar 22 17:23:35 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Thu, 22 Mar 2018 10:23:35 -0700 Subject: RFR [L][4/7]: 8180415: Rebuild remembered sets during the concurrent cycle In-Reply-To: <1521715049.2448.10.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520262478.2532.91.camel@oracle.com> <1520413176.2346.9.camel@oracle.com> <1f345c49-7793-7925-2c15-281f24d28e74@oracle.com> <1520509760.3121.15.camel@oracle.com> <1520952891.3100.36.camel@oracle.com> <1521715049.2448.10.camel@oracle.com> Message-ID: <60ec86ca-8f7b-9e62-b352-8fcd24b1051b@oracle.com> Hi Thomas, On 03/22/2018 03:37 AM, Thomas Schatzl wrote: > Hi Sangheon, > > thanks for your thorough review. > > On Wed, 2018-03-21 at 21:47 -0700, sangheon.kim wrote: >> Hi Thomas, >> >> On 03/13/2018 07:54 AM, Thomas Schatzl wrote: >>> Hi all, >>> [...] >>> >>> New webrevs: >>> http://cr.openjdk.java.net/~tschatzl/8180415/webrev.2_to_3 (diff) >>> http://cr.openjdk.java.net/~tschatzl/8180415/webrev.3 (full) >> webrev.3 looks good to me. >> I have some minor comments. >> >> =========================================================== >> src/hotspot/share/gc/g1/g1AllocRegion.cpp >> - Need to revert copyright update as there's no change. >> >> =========================================================== >> src/hotspot/share/gc/g1/g1ConcurrentMark.cpp >> 1045 class G1UpdateRemSetTrackingAfterRebuild : public >> HeapRegionClosure { >> - Couldn't have a constructor to pass G1CollectedHeap same as >> G1UpdateRemSetTrackingBeforeRebuild and use it from do_heap_region()? > Fixed. I will do a cleanup pass to make every method use _g1h in > g1ConcurrentMark.cpp soon. > >> =========================================================== >> src/hotspot/share/gc/g1/g1ConcurrentMark.hpp >> 143 static const size_t RegionMarkStatsCacheSize = 1024; >> 643 static const uint RegionMarkStatsCacheSize = 1024; >> - There are 2 RegionMarkStatsCacheSize and only the latter one seems >> to be used. Or am I missing something? >> >> 623 // Rebuilds the remembered sets for chosen regions >> concurrently >> and in parallel to the application. >> - Please ignore if I'm wrong. :) I guess you meant, processes with >> MT >> and concurrently with the application. Something like '... chosen >> regions in parallel and concurrently with the application.'? > Fixed, thanks! > >> =========================================================== >> src/hotspot/share/gc/g1/g1FullCollector.cpp >> - Copyright year update >> >> =========================================================== >> /src/hotspot/share/gc/g1/g1OopClosures.inline.hpp >> 145 HeapRegion* to = _g1->heap_region_containing(obj); >> - Getting HeapRegionRemSet and using it seems better for 147,148,149 >> lines. >> >> =========================================================== >> src/hotspot/share/gc/g1/g1RemSet.cpp >> 764 log_debug(gc,remset)("Humongous obj region %u marked >> %d >> start " PTR_FORMAT " region start " PTR_FORMAT " tams " PTR_FORMAT " >> tars " PTR_FORMAT, >> 842 "bot " PTR_FORMAT " ntams >> " >> PTR_FORMAT " TARS " PTR_FORMAT, >> - Uppercase for tams and tars used in log message? >> >> 888 uint num_workers = workers->active_workers(); >> - Need one more space before 'uint'. >> >> =========================================================== >> src/hotspot/share/gc/g1/heapRegionRemSet.hpp >> 135 bool add_reference(OopOrNarrowOopStar from, uint tid); >> - It would be helpful to mention when it returns true/false. I guess >> returns true when we add a reference. >> - I couldn't see returning true in add_reference(). Am I missing >> something? I know we are not using the return value so no problem so >> far. Or heapRegionRemSet.cpp:line 432 should return true? >> 432 return false; (heapRegionRemSet.cpp) >> > The return value is not used (any more), I removed the return value. > >> =========================================================== >> src/hotspot/share/logging/logPrefix.hpp >> - Copyright year update >> >> =========================================================== >> test/hotspot/jtreg/gc/concurrent_phase_control/TestConcurrentPhaseCon >> trolG1Basics.java >> 2 * Copyright (c) 2018, Oracle and/or its affiliates. All >> rights reserved. >> - Should be '2017, 2018' >> >> =========================================================== >> src/hotspot/share/gc/g1/g1RemSetTrackingPolicy.cpp >> 36 bool G1RemSetTrackingPolicy::needs_scan_for_rebuild(HeapRegion* >> r) >> const{ >> - Need a space between const and { >> >> 85 if ((total_live_bytes > 0) && >> 86 (is_interesting_humongous_region(r) || >> CollectionSetChooser::region_occupancy_low_enough_for_evac(total_live >> _bytes)) >> && >> 87 !r->rem_set()->is_tracked()) { >> - Maybe I'm wrong but Line 86 and 87 seem to need more spaces to >> align >> with line 85. >> >> =========================================================== >> src/hotspot/share/gc/g1/g1RemSetTrackingPolicy.hpp >> 52 // pause. Called at safepoint in the remark pause. >> - cleanup pause >> > All done. > > http://cr.openjdk.java.net/~tschatzl/8180415/webrev.3_to_4 (diff) > http://cr.openjdk.java.net/~tschatzl/8180415/webrev.4 (full) > > In addition to that I added the following minor changes: > - renamed the "CLEAR_REMEMBERED_SET" phase in the concurrent gc phase > manager to "REBUILD_REMEMBERED_Set" as this matches the code better. > > - several fixes to later re-enable the TestVerifyGCType (JDK-8193067: > gc/g1/TestVerifyGCType.java still unstable) later. That test parses the > log messages done during verification, and expects them to have a > "During GC" prefix. > > - there is one minor I think pre-existing issue fixed, in > g1Policy.cpp::decide_on_conc_mark_initiation(): it is possible that a > user initiated concurrent mark request can happen right after the > Cleanup pause before the "last gc before mixed gc". During that time > there is a collection set, and that one gets messed up in the following > marking (the evacuation efficiency of the regions may change). This > triggers some (benign) assert in the collection set chooser. > We need to clear any existing collection set before initiating > concurrent mark. Webrev.4 looks good to me. Thank you for fixing all! Sangheon > > Thanks, > Thomas > > From per.liden at oracle.com Fri Mar 23 08:05:41 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 23 Mar 2018 09:05:41 +0100 Subject: RFR: 8200113: Make Access load proxys smarter In-Reply-To: <5AB3D3EE.1070209@oracle.com> References: <5AB3BB8B.8090309@oracle.com> <5AB3D3EE.1070209@oracle.com> Message-ID: <159c0462-ad12-c2be-ee7a-8fb255d20861@oracle.com> Looks good! /Per On 03/22/2018 05:03 PM, Erik ?sterlund wrote: > Looping in hotspot-dev. > > /Erik > > On 2018-03-22 15:19, Erik ?sterlund wrote: >> Hi, >> >> Access returns the result of loads through load proxy objects that >> implicity convert themselves to a template inferred type. This is a >> metaprogramming technique used to infer return types in C++. >> >> However, I have heard requests that it would be great if it could be a >> bit smarter and do more than just be assigned to a type. >> >> Example use cases that do not work today without workarounds: >> >> ? oop val = ...; >> ? narrowOop narrow = 0u; >> ? oop *oop_val = &val; >> ? narrowOop *narrow_val = &narrow; >> ? HeapWord *heap_word_val = reinterpret_cast(oop_val); >> >> ? if (val == HeapAccess<>::oop_load_at(val, 16)) {} >> ? if (HeapAccess<>::oop_load_at(val, 16) == val) {} >> ? if (val != HeapAccess<>::oop_load_at(val, 16)) {} >> ? if (HeapAccess<>::oop_load_at(val, 16) != val) {} >> >> ? if (HeapAccess<>::oop_load(oop_val) != val) {} >> ? if (HeapAccess<>::oop_load(heap_word_val) != val) {} >> ? if (RawAccess<>::oop_load(narrow_val) != narrow) {} >> >> ? if (HeapAccess<>::oop_load(oop_val) == val) {} >> ? if (HeapAccess<>::oop_load(heap_word_val) == val) {} >> ? if (RawAccess<>::oop_load(narrow_val) == narrow) {} >> >> ? if (val != HeapAccess<>::oop_load(oop_val)) {} >> ? if (val != HeapAccess<>::oop_load(heap_word_val)) {} >> ? if (narrow != RawAccess<>::oop_load(narrow_val)) {} >> >> ? if (val == HeapAccess<>::oop_load(oop_val)) {} >> ? if (val == HeapAccess<>::oop_load(heap_word_val)) {} >> ? if (narrow == RawAccess<>::oop_load(narrow_val)) {} >> >> ? if ((oop)HeapAccess<>::oop_load(oop_val) == NULL) {} >> >> ? oop tmp = true ? HeapAccess<>::oop_load(oop_val) : val; >> >> Here is a patch that solves this: >> http://cr.openjdk.java.net/~eosterlund/8200113/webrev.00/ >> >> ...and here is the bug ID: >> https://bugs.openjdk.java.net/browse/JDK-8200113 >> >> Thanks, >> /Erik >> > From erik.osterlund at oracle.com Fri Mar 23 08:29:17 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 23 Mar 2018 09:29:17 +0100 Subject: RFR: 8200113: Make Access load proxys smarter In-Reply-To: <159c0462-ad12-c2be-ee7a-8fb255d20861@oracle.com> References: <5AB3BB8B.8090309@oracle.com> <5AB3D3EE.1070209@oracle.com> <159c0462-ad12-c2be-ee7a-8fb255d20861@oracle.com> Message-ID: <326c449f-a513-5961-945b-79cb337f520c@oracle.com> Hi Per, Thanks for the review. /Erik On 2018-03-23 09:05, Per Liden wrote: > Looks good! > > /Per > > On 03/22/2018 05:03 PM, Erik ?sterlund wrote: >> Looping in hotspot-dev. >> >> /Erik >> >> On 2018-03-22 15:19, Erik ?sterlund wrote: >>> Hi, >>> >>> Access returns the result of loads through load proxy objects that >>> implicity convert themselves to a template inferred type. This is a >>> metaprogramming technique used to infer return types in C++. >>> >>> However, I have heard requests that it would be great if it could be >>> a bit smarter and do more than just be assigned to a type. >>> >>> Example use cases that do not work today without workarounds: >>> >>> ? oop val = ...; >>> ? narrowOop narrow = 0u; >>> ? oop *oop_val = &val; >>> ? narrowOop *narrow_val = &narrow; >>> ? HeapWord *heap_word_val = reinterpret_cast(oop_val); >>> >>> ? if (val == HeapAccess<>::oop_load_at(val, 16)) {} >>> ? if (HeapAccess<>::oop_load_at(val, 16) == val) {} >>> ? if (val != HeapAccess<>::oop_load_at(val, 16)) {} >>> ? if (HeapAccess<>::oop_load_at(val, 16) != val) {} >>> >>> ? if (HeapAccess<>::oop_load(oop_val) != val) {} >>> ? if (HeapAccess<>::oop_load(heap_word_val) != val) {} >>> ? if (RawAccess<>::oop_load(narrow_val) != narrow) {} >>> >>> ? if (HeapAccess<>::oop_load(oop_val) == val) {} >>> ? if (HeapAccess<>::oop_load(heap_word_val) == val) {} >>> ? if (RawAccess<>::oop_load(narrow_val) == narrow) {} >>> >>> ? if (val != HeapAccess<>::oop_load(oop_val)) {} >>> ? if (val != HeapAccess<>::oop_load(heap_word_val)) {} >>> ? if (narrow != RawAccess<>::oop_load(narrow_val)) {} >>> >>> ? if (val == HeapAccess<>::oop_load(oop_val)) {} >>> ? if (val == HeapAccess<>::oop_load(heap_word_val)) {} >>> ? if (narrow == RawAccess<>::oop_load(narrow_val)) {} >>> >>> ? if ((oop)HeapAccess<>::oop_load(oop_val) == NULL) {} >>> >>> ? oop tmp = true ? HeapAccess<>::oop_load(oop_val) : val; >>> >>> Here is a patch that solves this: >>> http://cr.openjdk.java.net/~eosterlund/8200113/webrev.00/ >>> >>> ...and here is the bug ID: >>> https://bugs.openjdk.java.net/browse/JDK-8200113 >>> >>> Thanks, >>> /Erik >>> >> From thomas.schatzl at oracle.com Fri Mar 23 08:39:17 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 23 Mar 2018 09:39:17 +0100 Subject: RFR (S): 8196485: FromCardCache default card index can cause crashes In-Reply-To: <1520499556.3121.8.camel@oracle.com> References: <1520499556.3121.8.camel@oracle.com> Message-ID: <1521794357.3106.0.camel@oracle.com> Ping? On Thu, 2018-03-08 at 09:59 +0100, Thomas Schatzl wrote: > Hi all, > > can I have reviews for the following change that fixes a potential > crash due to missing remembered set entries on huge (2TB+) heaps, or > heaps that cross the 2TB address boundary (set via HeapBaseAddress)? > > The reason is that in the FromCardCache that is used to filter out > repeated additions of the same card G1 uses 32 bits to represent that > card, with a default value of 2^32-1. > > So if the first card we add to the remembered set has the index -1, > G1 assumes that we already added that value and wrongly bails out. > > 2TB+ because 2^32 + 2^9 (card size) = 2^41 which is 2TB. > > Note that some assertions in that code already catch this issue. > > The fix is to extend the size of a FromCardCacheEntry to 64 bits, > which allows 2^73 byte sized heaps - which hopefully is sufficient > for some time :) > > On my machine the passing test takes 8s (in debug mode), so I added > it to the regular tests run everytime. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8196485 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8196485/webrev/ > Testing: > hs-tier1+2, new jtreg test asserting without the patch, running > through with the patch. > > Thanks, > Thomas From thomas.schatzl at oracle.com Fri Mar 23 08:39:43 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 23 Mar 2018 09:39:43 +0100 Subject: RFR (S): 8199326: Remove G1 gc time stamp logic In-Reply-To: <9ef3f6cb-4487-c03a-2d06-da6f5911972a@oracle.com> References: <1520516013.3121.24.camel@oracle.com> <7b19df2a-cac8-41d7-c3e9-d2cf893f9d85@oracle.com> <1520523980.3121.29.camel@oracle.com> <9ef3f6cb-4487-c03a-2d06-da6f5911972a@oracle.com> Message-ID: <1521794383.3106.1.camel@oracle.com> Hi Sangheon, Stefan, thanks for your reviews. Thomas On Tue, 2018-03-13 at 14:40 -0700, sangheon.kim wrote: > Hi Thomas, > > On 03/08/2018 07:46 AM, Thomas Schatzl wrote: > > On Thu, 2018-03-08 at 16:18 +0100, Stefan Johansson wrote: > > > Hi Thomas, > > > > > > On 2018-03-08 14:33, Thomas Schatzl wrote: > > > > Hi, > > > > > > > > with the rebuild remsets concurrently changes the last user > > > > of > > > > the gc > > > > time stamps (create live data) has gone away. > > > > > > > > This change removes all of the corresponding code I could find. > > > > > > > > Could you review it? > > > > > > > > CR: > > > > https://bugs.openjdk.java.net/browse/JDK-8199326 > > > > Webrev: > > > > http://cr.openjdk.java.net/~tschatzl/8197932/webrev/ > > > > > > Looks like you pasted the wrong link, and I couldn't find the > > > correct > > > one either. > > > > Sorry. The correct link is: > > http://cr.openjdk.java.net/~tschatzl/8199326/webrev > > This looks good. > > Thanks, > Sangheon > > > > > > > > Thomas > > > > From shade at redhat.com Fri Mar 23 08:45:42 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 23 Mar 2018 09:45:42 +0100 Subject: RFR (S): 8196485: FromCardCache default card index can cause crashes In-Reply-To: <1520499556.3121.8.camel@oracle.com> References: <1520499556.3121.8.camel@oracle.com> Message-ID: <7187c820-0e40-6187-67a8-9f3bb56500ca@redhat.com> On 03/08/2018 09:59 AM, Thomas Schatzl wrote: > CR: > https://bugs.openjdk.java.net/browse/JDK-8196485 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8196485/webrev/ Ha! Ouch. Looks good, except: *) I feel uneasy about assigning negative values to unsigned vars. Intended to be UINTPTR_MAX? static const uintptr_t InvalidCard = -1; *) "heap region" here, probably: 134 // of the given heapregion. Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From stefan.johansson at oracle.com Fri Mar 23 08:50:37 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Fri, 23 Mar 2018 09:50:37 +0100 Subject: RFR [L][4/7]: 8180415: Rebuild remembered sets during the concurrent cycle In-Reply-To: <1521715049.2448.10.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520262478.2532.91.camel@oracle.com> <1520413176.2346.9.camel@oracle.com> <1f345c49-7793-7925-2c15-281f24d28e74@oracle.com> <1520509760.3121.15.camel@oracle.com> <1520952891.3100.36.camel@oracle.com> <1521715049.2448.10.camel@oracle.com> Message-ID: <93d3e31e-bebf-fa64-4560-e133a71f203a@oracle.com> On 2018-03-22 11:37, Thomas Schatzl wrote: > ... > All done. > > http://cr.openjdk.java.net/~tschatzl/8180415/webrev.3_to_4 (diff) > http://cr.openjdk.java.net/~tschatzl/8180415/webrev.4 (full) Looks good to me too, just one very minor thing. The second comment in g1RemSet.cpp still calls it nTAMS, is that on purpose? No need for a re-review, just update it if you want. Thanks, Stefan > In addition to that I added the following minor changes: > - renamed the "CLEAR_REMEMBERED_SET" phase in the concurrent gc phase > manager to "REBUILD_REMEMBERED_Set" as this matches the code better. > > - several fixes to later re-enable the TestVerifyGCType (JDK-8193067: > gc/g1/TestVerifyGCType.java still unstable) later. That test parses the > log messages done during verification, and expects them to have a > "During GC" prefix. > > - there is one minor I think pre-existing issue fixed, in > g1Policy.cpp::decide_on_conc_mark_initiation(): it is possible that a > user initiated concurrent mark request can happen right after the > Cleanup pause before the "last gc before mixed gc". During that time > there is a collection set, and that one gets messed up in the following > marking (the evacuation efficiency of the regions may change). This > triggers some (benign) assert in the collection set chooser. > We need to clear any existing collection set before initiating > concurrent mark. > > Thanks, > Thomas > > From stefan.johansson at oracle.com Fri Mar 23 08:52:50 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Fri, 23 Mar 2018 09:52:50 +0100 Subject: RFR [M][7/7]: 8197932: Better split work in rebuild remembered sets phase In-Reply-To: <1521715562.2448.15.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520265115.2532.126.camel@oracle.com> <1520953401.3100.41.camel@oracle.com> <1521461029.2323.11.camel@oracle.com> <4110ec65-c66f-eca0-3b7a-6580f0edf00d@oracle.com> <1521715562.2448.15.camel@oracle.com> Message-ID: On 2018-03-22 11:46, Thomas Schatzl wrote: > ... > The problem is not so much increasing the value, but decreasing it for > very slow systems. Decreasing it further adds some overhead. > > The other point is that since we track the time a chunk takes, it seems > strange to not have any knobs to deal with problems in that area. > > New webrev: > http://cr.openjdk.java.net/~tschatzl/8197932/webrev.2_to_3 (diff) > http://cr.openjdk.java.net/~tschatzl/8197932/webrev.3 (full) Looks good! Thanks, Stefan > In this webrev I also did some somewhat messy looking renaming of nTAMS > to TAMS (removing the n) and passing the (n)TAMS around in conjunction > with the bitmap. > This will make a follow-up change a lot easier to read. > > Thanks, > Thomas > From leo.korinth at oracle.com Fri Mar 23 09:05:53 2018 From: leo.korinth at oracle.com (Leo Korinth) Date: Fri, 23 Mar 2018 10:05:53 +0100 Subject: RFR: 8196865: G1 log for active workers is wrong Message-ID: <01296880-b79c-3abb-55ac-cf39dc45a3ed@oracle.com> Hi, Updating a local variable after recalculating number of active workers (fixes bad logging). I need a sponsor for the change. Bug: https://bugs.openjdk.java.net/browse/JDK-8196865 Webrev: http://cr.openjdk.java.net/~lkorinth/8196865/00/ Testing: - hs-tier1, hs-tier2 (on linux-x64) Also, manual testing: Before: [0.500s][trace][gc,task] GC(0) GC Thread: using 1 out of 23 workers [0.500s][info ][gc,task] GC(0) Using 6 workers of 23 for evacuation After: [0.511s][trace][gc,task] GC(0) GC Thread: using 1 out of 23 workers [0.511s][info ][gc,task] GC(0) Using 1 workers of 23 for evacuation Thanks, Leo From thomas.schatzl at oracle.com Fri Mar 23 09:17:20 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 23 Mar 2018 10:17:20 +0100 Subject: RFR: 8196865: G1 log for active workers is wrong In-Reply-To: <01296880-b79c-3abb-55ac-cf39dc45a3ed@oracle.com> References: <01296880-b79c-3abb-55ac-cf39dc45a3ed@oracle.com> Message-ID: <1521796640.14740.2.camel@oracle.com> On Fri, 2018-03-23 at 10:05 +0100, Leo Korinth wrote: > Hi, > > Updating a local variable after recalculating number of active > workers > (fixes bad logging). I need a sponsor for the change. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8196865 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/8196865/00/ looks good. I can sponsor. Please send me a complete patch. Thanks, Thomas From leo.korinth at oracle.com Fri Mar 23 09:23:50 2018 From: leo.korinth at oracle.com (Leo Korinth) Date: Fri, 23 Mar 2018 10:23:50 +0100 Subject: RFR: 8196865: G1 log for active workers is wrong In-Reply-To: <1521796640.14740.2.camel@oracle.com> References: <01296880-b79c-3abb-55ac-cf39dc45a3ed@oracle.com> <1521796640.14740.2.camel@oracle.com> Message-ID: On 23/03/18 10:17, Thomas Schatzl wrote: > On Fri, 2018-03-23 at 10:05 +0100, Leo Korinth wrote: >> Hi, >> >> Updating a local variable after recalculating number of active >> workers >> (fixes bad logging). I need a sponsor for the change. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8196865 >> >> Webrev: >> http://cr.openjdk.java.net/~lkorinth/8196865/00/ > > looks good. I can sponsor. Please send me a complete patch. Thanks Thomas! /Leo > > Thanks, > Thomas > From per.liden at oracle.com Fri Mar 23 09:27:16 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 23 Mar 2018 10:27:16 +0100 Subject: RFR: 8199850: Move parsing of VerifyGCType to G1 In-Reply-To: <1521735659.2264.5.camel@oracle.com> References: <5d3f62d2-f4c7-affa-a21c-be3330262af9@oracle.com> <1521735659.2264.5.camel@oracle.com> Message-ID: <3da59cea-4040-1446-a591-85240404b25f@oracle.com> Hi Thomas, On 03/22/2018 05:20 PM, Thomas Schatzl wrote: > Hi, > > On Tue, 2018-03-20 at 10:11 +0100, Per Liden wrote: >> When JDK-8191821 introduced the VerifyGCType option for G1 I didn't >> quite see the need for this to affect the GCArguments class. I'm >> proposing we move this logic into G1HeapVerifier, which I believe is >> a better place for this. > > I would actually prefer this parsing to be located and called by the > G1Arguments class. I really dislike the (existing) G1 habit of every > class doing its own arguments parsing. > > It's much more convenient to have one place to look at compared to > first searching where it's actually parsed. > > I do agree that since it is not supported by other collectors (yet?) > the shared code may not be the perfect place for it. > > However if others disagree, keep it this way. No problem. Here's an updated webrev, which moves the arguments parsing logic to G1Arguments. http://cr.openjdk.java.net/~pliden/8199850/webrev.1 /Per > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8199850 >> Webrev: http://cr.openjdk.java.net/~pliden/8199850/webrev.0 >> >> Testing: Used gc/g1/TestVerifyGCType.java to verify that nothing >> broke > > (I assume you executed it directly as it is currently problemlisted) > > Thanks, > Thomas > From stefan.johansson at oracle.com Fri Mar 23 09:38:25 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Fri, 23 Mar 2018 10:38:25 +0100 Subject: RFR: 8199850: Move parsing of VerifyGCType to G1 In-Reply-To: <3da59cea-4040-1446-a591-85240404b25f@oracle.com> References: <5d3f62d2-f4c7-affa-a21c-be3330262af9@oracle.com> <1521735659.2264.5.camel@oracle.com> <3da59cea-4040-1446-a591-85240404b25f@oracle.com> Message-ID: <9328b828-5b55-52d9-8dda-bbf026d035f2@oracle.com> On 2018-03-23 10:27, Per Liden wrote: > Hi Thomas, > > On 03/22/2018 05:20 PM, Thomas Schatzl wrote: >> Hi, >> >> On Tue, 2018-03-20 at 10:11 +0100, Per Liden wrote: >>> When JDK-8191821 introduced the VerifyGCType option for G1 I didn't >>> quite see the need for this to affect the GCArguments class. I'm >>> proposing we move this logic into G1HeapVerifier, which I believe is >>> a better place for this. >> >> I would actually prefer this parsing to be located and called by the >> G1Arguments class. I really dislike the (existing) G1 habit of every >> class doing its own arguments parsing. >> >> It's much more convenient to have one place to look at compared to >> first searching where it's actually parsed. >> >> I do agree that since it is not supported by other collectors (yet?) >> the shared code may not be the perfect place for it. >> >> However if others disagree, keep it this way. > > No problem. Here's an updated webrev, which moves the arguments > parsing logic to G1Arguments. > > http://cr.openjdk.java.net/~pliden/8199850/webrev.1 > Looks better :) Thanks, Stefan > /Per > >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8199850 >>> Webrev: http://cr.openjdk.java.net/~pliden/8199850/webrev.0 >>> >>> Testing: Used gc/g1/TestVerifyGCType.java to verify that nothing >>> broke >> >> (I assume you executed it directly as it is currently problemlisted) >> >> Thanks, >> ?? Thomas >> From stefan.johansson at oracle.com Fri Mar 23 09:39:38 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Fri, 23 Mar 2018 10:39:38 +0100 Subject: RFR: 8196865: G1 log for active workers is wrong In-Reply-To: <01296880-b79c-3abb-55ac-cf39dc45a3ed@oracle.com> References: <01296880-b79c-3abb-55ac-cf39dc45a3ed@oracle.com> Message-ID: <2ba0543d-f54a-7b29-d4f0-a8432c450710@oracle.com> Looks good! Thanks for taking care of this, Stefan On 2018-03-23 10:05, Leo Korinth wrote: > Hi, > > Updating a local variable after recalculating number of active workers > (fixes bad logging). I need a sponsor for the change. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8196865 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/8196865/00/ > > Testing: > - hs-tier1, hs-tier2 (on linux-x64) > > Also, manual testing: > > Before: > [0.500s][trace][gc,task] GC(0) GC Thread: using 1 out of 23 workers > [0.500s][info ][gc,task] GC(0) Using 6 workers of 23 for evacuation > > After: > [0.511s][trace][gc,task] GC(0) GC Thread: using 1 out of 23 workers > [0.511s][info ][gc,task] GC(0) Using 1 workers of 23 for evacuation > > Thanks, > Leo From leo.korinth at oracle.com Fri Mar 23 09:47:05 2018 From: leo.korinth at oracle.com (Leo Korinth) Date: Fri, 23 Mar 2018 10:47:05 +0100 Subject: RFR: 8196865: G1 log for active workers is wrong In-Reply-To: <2ba0543d-f54a-7b29-d4f0-a8432c450710@oracle.com> References: <01296880-b79c-3abb-55ac-cf39dc45a3ed@oracle.com> <2ba0543d-f54a-7b29-d4f0-a8432c450710@oracle.com> Message-ID: <0d09277e-e288-b5cd-a189-7e3a05babb69@oracle.com> On 23/03/18 10:39, Stefan Johansson wrote: > Looks good! > > Thanks for taking care of this, > Stefan Thanks for the review, Leo > > On 2018-03-23 10:05, Leo Korinth wrote: >> Hi, >> >> Updating a local variable after recalculating number of active workers >> (fixes bad logging). I need a sponsor for the change. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8196865 >> >> Webrev: >> http://cr.openjdk.java.net/~lkorinth/8196865/00/ >> >> Testing: >> - hs-tier1, hs-tier2 (on linux-x64) >> >> Also, manual testing: >> >> Before: >> [0.500s][trace][gc,task] GC(0) GC Thread: using 1 out of 23 workers >> [0.500s][info ][gc,task] GC(0) Using 6 workers of 23 for evacuation >> >> After: >> [0.511s][trace][gc,task] GC(0) GC Thread: using 1 out of 23 workers >> [0.511s][info ][gc,task] GC(0) Using 1 workers of 23 for evacuation >> >> Thanks, >> Leo > From thomas.schatzl at oracle.com Fri Mar 23 10:31:05 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 23 Mar 2018 11:31:05 +0100 Subject: RFR: 8199850: Move parsing of VerifyGCType to G1 In-Reply-To: <3da59cea-4040-1446-a591-85240404b25f@oracle.com> References: <5d3f62d2-f4c7-affa-a21c-be3330262af9@oracle.com> <1521735659.2264.5.camel@oracle.com> <3da59cea-4040-1446-a591-85240404b25f@oracle.com> Message-ID: <1521801065.14740.3.camel@oracle.com> Hi, On Fri, 2018-03-23 at 10:27 +0100, Per Liden wrote: > Hi Thomas, > > On 03/22/2018 05:20 PM, Thomas Schatzl wrote: > > Hi, > > > > On Tue, 2018-03-20 at 10:11 +0100, Per Liden wrote: > > > When JDK-8191821 introduced the VerifyGCType option for G1 I > > > didn't > > > quite see the need for this to affect the GCArguments class. I'm > > > proposing we move this logic into G1HeapVerifier, which I believe > > > is > > > a better place for this. > > > > I would actually prefer this parsing to be located and called by > > the > > G1Arguments class. I really dislike the (existing) G1 habit of > > every > > class doing its own arguments parsing. > > > > It's much more convenient to have one place to look at compared to > > first searching where it's actually parsed. > > > > I do agree that since it is not supported by other collectors > > (yet?) > > the shared code may not be the perfect place for it. > > > > However if others disagree, keep it this way. > > No problem. Here's an updated webrev, which moves the arguments > parsing > logic to G1Arguments. > > http://cr.openjdk.java.net/~pliden/8199850/webrev.1 > looks good. Thomas From per.liden at oracle.com Fri Mar 23 10:33:05 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 23 Mar 2018 11:33:05 +0100 Subject: RFR: 8199850: Move parsing of VerifyGCType to G1 In-Reply-To: <1521801065.14740.3.camel@oracle.com> References: <5d3f62d2-f4c7-affa-a21c-be3330262af9@oracle.com> <1521735659.2264.5.camel@oracle.com> <3da59cea-4040-1446-a591-85240404b25f@oracle.com> <1521801065.14740.3.camel@oracle.com> Message-ID: <20217c01-6f58-1eb4-2847-740fe4fb932b@oracle.com> Thanks for reviewing Stefan and Thomas! /Per On 03/23/2018 11:31 AM, Thomas Schatzl wrote: > Hi, > > On Fri, 2018-03-23 at 10:27 +0100, Per Liden wrote: >> Hi Thomas, >> >> On 03/22/2018 05:20 PM, Thomas Schatzl wrote: >>> Hi, >>> >>> On Tue, 2018-03-20 at 10:11 +0100, Per Liden wrote: >>>> When JDK-8191821 introduced the VerifyGCType option for G1 I >>>> didn't >>>> quite see the need for this to affect the GCArguments class. I'm >>>> proposing we move this logic into G1HeapVerifier, which I believe >>>> is >>>> a better place for this. >>> >>> I would actually prefer this parsing to be located and called by >>> the >>> G1Arguments class. I really dislike the (existing) G1 habit of >>> every >>> class doing its own arguments parsing. >>> >>> It's much more convenient to have one place to look at compared to >>> first searching where it's actually parsed. >>> >>> I do agree that since it is not supported by other collectors >>> (yet?) >>> the shared code may not be the perfect place for it. >>> >>> However if others disagree, keep it this way. >> >> No problem. Here's an updated webrev, which moves the arguments >> parsing >> logic to G1Arguments. >> >> http://cr.openjdk.java.net/~pliden/8199850/webrev.1 >> > > looks good. > > Thomas > From per.liden at oracle.com Fri Mar 23 10:57:03 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 23 Mar 2018 11:57:03 +0100 Subject: RFR: 8199925: Break out GC selection logic from GCArguments to GCSelector In-Reply-To: <0C591305-5A1E-4E13-8411-C58E2A218623@redhat.com> References: <35136ca4-cc0c-d977-8351-523a52a39c7a@redhat.com> <15700f36-bf1b-b8a7-4954-8501b5ed7626@oracle.com> <0C591305-5A1E-4E13-8411-C58E2A218623@redhat.com> Message-ID: <1e84e63d-988f-d40f-4719-bfa636aafb3f@oracle.com> Thanks Roman and Erik for reviewing! /Per On 03/21/2018 04:07 PM, Roman Kennke wrote: > Looks good! Thank you! > > Am 21. M?rz 2018 14:11:52 MEZ schrieb Per Liden : >> Hi, >> >> Here's an updated webrev. I renamed GCSelector to GCConfig and it's now >> >> also the goto place for getting hold of the GCArguments instance for >> the >> currently selected GC. >> >> http://cr.openjdk.java.net/~pliden/8199925/webrev.1 >> >> Btw, this is built on top of JDK-8199850 (Move parsing of VerifyGCType >> to G1) and JDK-8199918 (Shorten names of CollectedHeap::Name members), >> and the GCConfig::is_gc_* functions will eventually be used by >> JDK-8199927 (Make WhiteBox more GC agnostic). >> >> /Per >> >> On 03/21/2018 11:33 AM, Per Liden wrote: >>> Hi Roman, >>> >>> On 03/21/2018 10:15 AM, Roman Kennke wrote: >>>> Am 21.03.2018 um 09:12 schrieb Per Liden: >>>>> In an attempt to make WhiteBox more GC agnostic, and in turn make >> it >>>>> easier to plugin new GC without touching a lot of non-GC code, this >>>>> patch breaks out the GC selection logic from GCArguments to >> GCSelector. >>>>> There are two reasons why I think this makes sense: >>>>> 1) The GC selection logic is self-contained and fairly unrelated to >> the >>>>> rest of the flags processing done by GCArguments and its >> GC-specific >>>>> children. >>>>> 2) Parts of the GC selection logic is needed by WhiteBox to answer >>>>> questions about which GC are supported, selected, etc. >>>>> >>>>> A follow up patch (JDK-8199927) will change WhiteBox (WB_CurrentGC, >>>>> WB_AllSupportedGC, WB_GCSelectedByErgo), to use the GCSelector. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8199925 >>>>> Webrev: http://cr.openjdk.java.net/~pliden/8199925/webrev.0 >>>>> >>>>> /Per >>>> >>>> My original proposal for the GCArguments did have a separate >>>> GCFactory. :-) >>>> >>>> In generally I like your change. >>>> >>>> I'd probably go even further and let GCSelector initialize and >> return an >>>> instance of GCArguments. This way we wouldn't end up with two places >>>> that have knowledge of all possible GCs, but would only have one >> place >>>> (GCSelector) where one had to hook up for adding a new GC. >>>> >>>> What do you think? >>> >>> My intention here was that GCSelector would be a pure translator of >>> Use*GC flags to CollectedHeap::Name, i.e. a small and well defined >>> component, which knows nothing about GCArguments. The purpose of this >> >>> was to clean out the WhiteBox stuff. >>> >>> On the other hand, I see what you mean, and too would like to reach a >> >>> state where we have one place in the code that "knows about all the >> GCs". >>> >>> Let me think about it some more and maybe prototype an alternative, >> and >>> we'll see if that gets us to a better place. >>> >>> /Per >>> >>>> >>>> Roman >>>> > From per.liden at oracle.com Fri Mar 23 12:58:24 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 23 Mar 2018 13:58:24 +0100 Subject: RFR: 8199850: Move parsing of VerifyGCType to G1 In-Reply-To: <20217c01-6f58-1eb4-2847-740fe4fb932b@oracle.com> References: <5d3f62d2-f4c7-affa-a21c-be3330262af9@oracle.com> <1521735659.2264.5.camel@oracle.com> <3da59cea-4040-1446-a591-85240404b25f@oracle.com> <1521801065.14740.3.camel@oracle.com> <20217c01-6f58-1eb4-2847-740fe4fb932b@oracle.com> Message-ID: <9ec6fc3c-1d89-4079-e198-ff7edeeda681@oracle.com> Sorry, I noticed that I had missed the gtest, which also needed an adjustments: Diff: http://cr.openjdk.java.net/~pliden/8199850/webrev.1vs2 Full: http://cr.openjdk.java.net/~pliden/8199850/webrev.2 Testing: Passed hg-tier1 /Per On 03/23/2018 11:33 AM, Per Liden wrote: > Thanks for reviewing Stefan and Thomas! > > /Per > > On 03/23/2018 11:31 AM, Thomas Schatzl wrote: >> Hi, >> >> On Fri, 2018-03-23 at 10:27 +0100, Per Liden wrote: >>> Hi Thomas, >>> >>> On 03/22/2018 05:20 PM, Thomas Schatzl wrote: >>>> Hi, >>>> >>>> On Tue, 2018-03-20 at 10:11 +0100, Per Liden wrote: >>>>> When JDK-8191821 introduced the VerifyGCType option for G1 I >>>>> didn't >>>>> quite see the need for this to affect the GCArguments class. I'm >>>>> proposing we move this logic into G1HeapVerifier, which I believe >>>>> is >>>>> a better place for this. >>>> >>>> I would actually prefer this parsing to be located and called by >>>> the >>>> G1Arguments class. I really dislike the (existing) G1 habit of >>>> every >>>> class doing its own arguments parsing. >>>> >>>> It's much more convenient to have one place to look at compared to >>>> first searching where it's actually parsed. >>>> >>>> I do agree that since it is not supported by other collectors >>>> (yet?) >>>> the shared code may not be the perfect place for it. >>>> >>>> However if others disagree, keep it this way. >>> >>> No problem. Here's an updated webrev, which moves the arguments >>> parsing >>> logic to G1Arguments. >>> >>> http://cr.openjdk.java.net/~pliden/8199850/webrev.1 >>> >> >> ?? looks good. >> >> Thomas >> From stefan.johansson at oracle.com Fri Mar 23 13:32:10 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Fri, 23 Mar 2018 14:32:10 +0100 Subject: RFR: 8200169: Flatten G1Allocator class hierarchy Message-ID: <50c3c960-8bed-29d4-edf0-e7009d969c0a@oracle.com> Hi, Please review this change to remove some unnecessary class hierarchies in G1. Links JBS: https://bugs.openjdk.java.net/browse/JDK-8200169 Webrev: http://cr.openjdk.java.net/~sjohanss/8200169/00/ Summary After the removal of several extension points and some other cleanups in G1, there is no longer any need for the two class hierarchies in g1Allocator.hpp. These can be flattened to only consist of two concrete classes instead. Testing * Built both locally and through mach5. * hs-tier1 through mach5 Thanks, Stefan From coleen.phillimore at oracle.com Fri Mar 23 14:29:17 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 23 Mar 2018 10:29:17 -0400 Subject: RFR (S) 8199007: ClassLoaderDataGraph::unload_list_contains() is wrong Message-ID: <714f00e1-6938-e78e-d485-b3efd7297db4@oracle.com> Summary: The function isn't needed any longer. I believe the function existed from an earlier day when Metaspace::contains walked the CLDG, so you needed a function to walk the unloaded list as well. Tested with Per's GC test suite and mach5 tier1-5 (until it failed in infra for some platforms), and my own local testing like: jtreg -exclude:open/test/hotspot/jtreg/ProblemList.txt open/test/hotspot/jtreg:hotspot_all_no_apps (and the closed jtreg tests too). open webrev at http://cr.openjdk.java.net/~coleenp/8199007.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8199007 Thanks, Coleen From coleen.phillimore at oracle.com Fri Mar 23 14:52:25 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 23 Mar 2018 10:52:25 -0400 Subject: RFR (S) 8199007: ClassLoaderDataGraph::unload_list_contains() is wrong In-Reply-To: <714f00e1-6938-e78e-d485-b3efd7297db4@oracle.com> References: <714f00e1-6938-e78e-d485-b3efd7297db4@oracle.com> Message-ID: Can you cc me directly since I'm not on this mailing list? thanks, Coleen On 3/23/18 10:29 AM, coleen.phillimore at oracle.com wrote: > Summary: The function isn't needed any longer. > > I believe the function existed from an earlier day when > Metaspace::contains walked the CLDG, so you needed a function to walk > the unloaded list as well. > > Tested with Per's GC test suite and mach5 tier1-5 (until it failed in > infra for some platforms), and my own local testing like: > > jtreg -exclude:open/test/hotspot/jtreg/ProblemList.txt > open/test/hotspot/jtreg:hotspot_all_no_apps > > (and the closed jtreg tests too). > > open webrev at http://cr.openjdk.java.net/~coleenp/8199007.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8199007 > > Thanks, > Coleen From thomas.stuefe at gmail.com Fri Mar 23 15:02:59 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 23 Mar 2018 15:02:59 +0000 Subject: RFR (S) 8199007: ClassLoaderDataGraph::unload_list_contains() is wrong In-Reply-To: <714f00e1-6938-e78e-d485-b3efd7297db4@oracle.com> References: <714f00e1-6938-e78e-d485-b3efd7297db4@oracle.com> Message-ID: Hi Coleen, This looks good. Thanks, Thomas On Fri 23. Mar 2018 at 15:29, wrote: > Summary: The function isn't needed any longer. > > I believe the function existed from an earlier day when > Metaspace::contains walked the CLDG, so you needed a function to walk > the unloaded list as well. > > Tested with Per's GC test suite and mach5 tier1-5 (until it failed in > infra for some platforms), and my own local testing like: > > jtreg -exclude:open/test/hotspot/jtreg/ProblemList.txt > open/test/hotspot/jtreg:hotspot_all_no_apps > > (and the closed jtreg tests too). > > open webrev at http://cr.openjdk.java.net/~coleenp/8199007.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8199007 > > Thanks, > Coleen > -------------- next part -------------- An HTML attachment was scrubbed... URL: From coleen.phillimore at oracle.com Fri Mar 23 15:07:19 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 23 Mar 2018 11:07:19 -0400 Subject: RFR (S) 8199007: ClassLoaderDataGraph::unload_list_contains() is wrong In-Reply-To: <714f00e1-6938-e78e-d485-b3efd7297db4@oracle.com> References: <714f00e1-6938-e78e-d485-b3efd7297db4@oracle.com> Message-ID: <2972bc5f-6ad2-ec04-7241-a5c4a1a33b2a@oracle.com> On 3/23/18 10:29 AM, coleen.phillimore at oracle.com wrote: > Summary: The function isn't needed any longer. > > I believe the function existed from an earlier day when > Metaspace::contains walked the CLDG, so you needed a function to walk > the unloaded list as well. > > Tested with Per's GC test suite and mach5 tier1-5 (until it failed in > infra for some platforms), and my own local testing like: > > jtreg -exclude:open/test/hotspot/jtreg/ProblemList.txt > open/test/hotspot/jtreg:hotspot_all_no_apps I also ran GCBasher and runThese with -XX:+VerifyBeforeGC -XX:+VerifyDuringGC -XX:+VerifyAfterGC. thanks, Coleen > > (and the closed jtreg tests too). > > open webrev at http://cr.openjdk.java.net/~coleenp/8199007.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8199007 > > Thanks, > Coleen From coleen.phillimore at oracle.com Fri Mar 23 15:07:55 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 23 Mar 2018 11:07:55 -0400 Subject: RFR (S) 8199007: ClassLoaderDataGraph::unload_list_contains() is wrong In-Reply-To: References: <714f00e1-6938-e78e-d485-b3efd7297db4@oracle.com> Message-ID: <14834ed7-69f9-a576-5c7a-138af1d64fa7@oracle.com> Thanks Thomas. Coleen On 3/23/18 11:02 AM, Thomas St?fe wrote: > Hi Coleen, > > This looks good. > > Thanks, Thomas > > > On Fri 23. Mar 2018 at 15:29, > wrote: > > Summary: The function isn't needed any longer. > > I believe the function existed from an earlier day when > Metaspace::contains walked the CLDG, so you needed a function to walk > the unloaded list as well. > > Tested with Per's GC test suite and mach5 tier1-5 (until it failed in > infra for some platforms), and my own local testing like: > > jtreg -exclude:open/test/hotspot/jtreg/ProblemList.txt > open/test/hotspot/jtreg:hotspot_all_no_apps > > (and the closed jtreg tests too). > > open webrev at > http://cr.openjdk.java.net/~coleenp/8199007.01/webrev > > bug link https://bugs.openjdk.java.net/browse/JDK-8199007 > > Thanks, > Coleen > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Fri Mar 23 16:02:08 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 23 Mar 2018 17:02:08 +0100 Subject: RFR: 8200169: Flatten G1Allocator class hierarchy In-Reply-To: <50c3c960-8bed-29d4-edf0-e7009d969c0a@oracle.com> References: <50c3c960-8bed-29d4-edf0-e7009d969c0a@oracle.com> Message-ID: <1521820928.4147.8.camel@oracle.com> Hi, On Fri, 2018-03-23 at 14:32 +0100, Stefan Johansson wrote: > Hi, > > Please review this change to remove some unnecessary class > hierarchies > in G1. > > Links > JBS: https://bugs.openjdk.java.net/browse/JDK-8200169 > Webrev: http://cr.openjdk.java.net/~sjohanss/8200169/00/ > > Summary > After the removal of several extension points and some other cleanups > in G1, there is no longer any need for the two class hierarchies in > g1Allocator.hpp. These can be flattened to only consist of two > concrete classes instead. looks good to me. Thomas From kim.barrett at oracle.com Fri Mar 23 20:13:40 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 23 Mar 2018 16:13:40 -0400 Subject: RFR: 8199925: Break out GC selection logic from GCArguments to GCSelector In-Reply-To: References: <35136ca4-cc0c-d977-8351-523a52a39c7a@redhat.com> <15700f36-bf1b-b8a7-4954-8501b5ed7626@oracle.com> Message-ID: <19DADD7C-0231-4323-A0C6-AB6B8B025AC0@oracle.com> > On Mar 21, 2018, at 9:11 AM, Per Liden wrote: > > Hi, > > Here's an updated webrev. I renamed GCSelector to GCConfig and it's now also the goto place for getting hold of the GCArguments instance for the currently selected GC. > > http://cr.openjdk.java.net/~pliden/8199925/webrev.1 > > Btw, this is built on top of JDK-8199850 (Move parsing of VerifyGCType to G1) and JDK-8199918 (Shorten names of CollectedHeap::Name members), and the GCConfig::is_gc_* functions will eventually be used by JDK-8199927 (Make WhiteBox more GC agnostic). I like this change, but don't have time right now to review it in detail, so don?t count me as a reviewer. I do have a question though. Why are there a bunch of private static members in GCConfig, rather just some more file-scoped statics in gcConfig.cpp? From leonid.mesnik at oracle.com Fri Mar 23 23:31:11 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Fri, 23 Mar 2018 16:31:11 -0700 Subject: RFR(XS): 8200187: Exclude 3 long-running tests from tier1 Message-ID: Hi Could you please review following fix which exclude following tests from tier1 testing: serviceability/sa/ClhsdbScanOops.java serviceability/sa/TestHeapDumpForLargeArray.java gc/g1/ihop/TestIHOPErgo.java Each of them takes more then 5 minutes to complete and significantly increase overall time to complete tier1. Please let me know if there are any reasons to run these tests in tier1 despite on their execution time. webrev:http://cr.openjdk.java.net/~lmesnik/8200187/webrev.00/ bug: https://bugs.openjdk.java.net/browse/JDK-8200187 Leonid -------------- next part -------------- An HTML attachment was scrubbed... URL: From rkennke at redhat.com Sun Mar 25 21:03:27 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sun, 25 Mar 2018 23:03:27 +0200 Subject: RFR: JDK-8199735: Mark word updates need to use Access API In-Reply-To: <5AB3B052.4000304@oracle.com> References: <7bd73e32-a26b-2504-f069-b74807ae5e53@redhat.com> <859ac657-78c9-c7ab-0ec0-eddda331001b@redhat.com> <5AB3B052.4000304@oracle.com> Message-ID: <63199799-0194-a1d4-9c89-d95251c7aa7d@redhat.com> Hi Erik, Thanks for having a look. > A few comments: > > 1) Please make sure all files referencing these accessors that have been > moved to the oop.inline.hpp file now #include the oop.inline.hpp file. I > found that there are quite a few missing by looking for references to > oopDesc::mark() with rtags. > 2) I think the mark_addr() function should now be called mark_addr_raw() > for consistency. > 3) I wonder if there should be _raw() accessors that the GCs use > instead. It feels like the GC should not go through the Access API to > acquire mark words. You are of course right in all 3 points. Unfortunately, that causes massive ripples through the code base ;-) So here is what I did: I added _raw() variants for stuff that is used by GC. I temporarily renamed all relevant methods, e.g. mark() -> mark_fluff() to make the compiler help me. Then for each occurance of any such call decided whether it should call the usual method, or the raw variant (as a general rule, everything under src/hotspot/share/gc uses the _raw() variants). Also, I added #include "oop.inline.hpp" for every such occurance. Then I renamed all _fluff() methods and calls back to usual. Did this exercise with and without PCH, with and without zero. Checked with grep if I might have missed some. Notes: bytecodeInterpreter.cpp needed some additional fixes to use cas_set_mark() instead of the Atomic::cmpxchg(). There are a bunch of methods in oop.hpp that are only used by GC, like is_locked() etc. I haven't changed them to _raw(). This passes tier1 tests release+fastdebug, and as mentioned above builds with and without PCH and zero. Differential: http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.03.diff/ Full: http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.03/ Better now? Best regards, Roman > > Thanks, > /Erik > > > On 2018-03-21 20:21, Roman Kennke wrote: >> Am 19.03.2018 um 12:07 schrieb Roman Kennke: >>> Am 19.03.2018 um 11:40 schrieb Roman Kennke: >>>> Currently, the mark word is accessed directly in oopDesc::mark() >>>> set_mark() and a bunch of other accessors. Those need to use the Access >>>> API instead because GC might want to employ barriers on those accesses, >>>> pretty much like every other field in oopDesc. >>>> >>>> Notice that this is *not* about accessing the bits and fields inside >>>> the >>>> markOop, but about accessing the header itself. >>>> >>>> http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.01/ >>>> >>>> Testing: build fastdebug/release with and without PCH, passed tier1 >>>> fastdebug/release. >>>> >>>> Can I please get reviews? >>>> >>>> Thanks, Roman >>>> >>> Just when I sent it, I realized that this is dropping the volatile from >>> the mark word access. Fixed here: >>> >>> >>> http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.02/ >>> >> Ping? >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From david.holmes at oracle.com Sun Mar 25 22:48:35 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 26 Mar 2018 08:48:35 +1000 Subject: RFR(XS): 8200187: Exclude 3 long-running tests from tier1 In-Reply-To: References: Message-ID: Hi Leonid, On 24/03/2018 9:31 AM, Leonid Mesnik wrote: > Hi > > Could you please review following fix which exclude following tests from > tier1 testing: > serviceability/sa/ClhsdbScanOops.java > serviceability/sa/TestHeapDumpForLargeArray.java > gc/g1/ihop/TestIHOPErgo.java > > Each of them takes more then 5 minutes to complete and significantly > increase overall time to complete tier1. I'd need to see a much more detailed analysis of all the tests run, the order they run and the execution times to ascertain what impact this actually has on overall test execution time. But assuming 5 minutes is too long for tier1, this seems okay. But the tests must still be run in some other tier, and I'm not clear where that would be now? I would expect them to move to tier 3 perhaps, depending on what the time criteria for tier 3 is. Thanks, David > Please let me know if there are any reasons to run these tests in tier1 > despite on their execution time. > > webrev:http://cr.openjdk.java.net/~lmesnik/8200187/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8200187 > > Leonid From per.liden at oracle.com Mon Mar 26 06:33:00 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 26 Mar 2018 08:33:00 +0200 Subject: RFR: 8199925: Break out GC selection logic from GCArguments to GCSelector In-Reply-To: <19DADD7C-0231-4323-A0C6-AB6B8B025AC0@oracle.com> References: <35136ca4-cc0c-d977-8351-523a52a39c7a@redhat.com> <15700f36-bf1b-b8a7-4954-8501b5ed7626@oracle.com> <19DADD7C-0231-4323-A0C6-AB6B8B025AC0@oracle.com> Message-ID: <949b36d9-9f93-ee1c-4d77-4af820cbf80b@oracle.com> Hi Kim, On 03/23/2018 09:13 PM, Kim Barrett wrote: >> On Mar 21, 2018, at 9:11 AM, Per Liden wrote: >> >> Hi, >> >> Here's an updated webrev. I renamed GCSelector to GCConfig and it's now also the goto place for getting hold of the GCArguments instance for the currently selected GC. >> >> http://cr.openjdk.java.net/~pliden/8199925/webrev.1 >> >> Btw, this is built on top of JDK-8199850 (Move parsing of VerifyGCType to G1) and JDK-8199918 (Shorten names of CollectedHeap::Name members), and the GCConfig::is_gc_* functions will eventually be used by JDK-8199927 (Make WhiteBox more GC agnostic). > > I like this change, but don't have time right now to review it in detail, so don?t count me as a reviewer. > > I do have a question though. Why are there a bunch of private static members in GCConfig, rather > just some more file-scoped statics in gcConfig.cpp? I tend to prefer this approach for a few of reasons. First and maybe foremost, they get nicely name spaced. Being private members also makes it clear that they are part of this class' internal logic, and are not intended to be used by other classes. These functions also have public "siblings", in the form of the other is_gc_*, so I think it makes sense to group them in the same class. /Per From per.liden at oracle.com Mon Mar 26 06:40:12 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 26 Mar 2018 08:40:12 +0200 Subject: RFR: 8199925: Break out GC selection logic from GCArguments to GCSelector In-Reply-To: <1e84e63d-988f-d40f-4719-bfa636aafb3f@oracle.com> References: <35136ca4-cc0c-d977-8351-523a52a39c7a@redhat.com> <15700f36-bf1b-b8a7-4954-8501b5ed7626@oracle.com> <0C591305-5A1E-4E13-8411-C58E2A218623@redhat.com> <1e84e63d-988f-d40f-4719-bfa636aafb3f@oracle.com> Message-ID: Hi again, It turned out that the Solaris Studio compiler didn't like structs with reference members without a user-defined constructor (all other compilers happily swallows this), so I had to add one. Updated webrev: Diff: http://cr.openjdk.java.net/~pliden/8199925/webrev.1vs2 Full: http://cr.openjdk.java.net/~pliden/8199925/webrev.2 /Per On 03/23/2018 11:57 AM, Per Liden wrote: > Thanks Roman and Erik for reviewing! > > /Per > > On 03/21/2018 04:07 PM, Roman Kennke wrote: >> Looks good! Thank you! >> >> Am 21. M?rz 2018 14:11:52 MEZ schrieb Per Liden : >>> Hi, >>> >>> Here's an updated webrev. I renamed GCSelector to GCConfig and it's now >>> >>> also the goto place for getting hold of the GCArguments instance for >>> the >>> currently selected GC. >>> >>> http://cr.openjdk.java.net/~pliden/8199925/webrev.1 >>> >>> Btw, this is built on top of JDK-8199850 (Move parsing of VerifyGCType >>> to G1) and JDK-8199918 (Shorten names of CollectedHeap::Name members), >>> and the GCConfig::is_gc_* functions will eventually be used by >>> JDK-8199927 (Make WhiteBox more GC agnostic). >>> >>> /Per >>> >>> On 03/21/2018 11:33 AM, Per Liden wrote: >>>> Hi Roman, >>>> >>>> On 03/21/2018 10:15 AM, Roman Kennke wrote: >>>>> Am 21.03.2018 um 09:12 schrieb Per Liden: >>>>>> In an attempt to make WhiteBox more GC agnostic, and in turn make >>> it >>>>>> easier to plugin new GC without touching a lot of non-GC code, this >>>>>> patch breaks out the GC selection logic from GCArguments to >>> GCSelector. >>>>>> There are two reasons why I think this makes sense: >>>>>> 1) The GC selection logic is self-contained and fairly unrelated to >>> the >>>>>> rest of the flags processing done by GCArguments and its >>> GC-specific >>>>>> children. >>>>>> 2) Parts of the GC selection logic is needed by WhiteBox to answer >>>>>> questions about which GC are supported, selected, etc. >>>>>> >>>>>> A follow up patch (JDK-8199927) will change WhiteBox (WB_CurrentGC, >>>>>> WB_AllSupportedGC, WB_GCSelectedByErgo), to use the GCSelector. >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8199925 >>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8199925/webrev.0 >>>>>> >>>>>> /Per >>>>> >>>>> My original proposal for the GCArguments did have a separate >>>>> GCFactory. :-) >>>>> >>>>> In generally I like your change. >>>>> >>>>> I'd probably go even further and let GCSelector initialize and >>> return an >>>>> instance of GCArguments. This way we wouldn't end up with two places >>>>> that have knowledge of all possible GCs, but would only have one >>> place >>>>> (GCSelector) where one had to hook up for adding a new GC. >>>>> >>>>> What do you think? >>>> >>>> My intention here was that GCSelector would be a pure translator of >>>> Use*GC flags to CollectedHeap::Name, i.e. a small and well defined >>>> component, which knows nothing about GCArguments. The purpose of this >>> >>>> was to clean out the WhiteBox stuff. >>>> >>>> On the other hand, I see what you mean, and too would like to reach a >>> >>>> state where we have one place in the code that "knows about all the >>> GCs". >>>> >>>> Let me think about it some more and maybe prototype an alternative, >>> and >>>> we'll see if that gets us to a better place. >>>> >>>> /Per >>>> >>>>> >>>>> Roman >>>>> >> From erik.osterlund at oracle.com Mon Mar 26 07:10:25 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Mon, 26 Mar 2018 09:10:25 +0200 Subject: RFR: 8199925: Break out GC selection logic from GCArguments to GCSelector In-Reply-To: References: <35136ca4-cc0c-d977-8351-523a52a39c7a@redhat.com> <15700f36-bf1b-b8a7-4954-8501b5ed7626@oracle.com> <0C591305-5A1E-4E13-8411-C58E2A218623@redhat.com> <1e84e63d-988f-d40f-4719-bfa636aafb3f@oracle.com> Message-ID: <7E650A64-50B4-432D-ACAC-0B605147E808@oracle.com> Hi Per, Looks good. Thanks, /Erik > On 26 Mar 2018, at 08:40, Per Liden wrote: > > Hi again, > > It turned out that the Solaris Studio compiler didn't like structs with reference members without a user-defined constructor (all other compilers happily swallows this), so I had to add one. > > Updated webrev: > > Diff: http://cr.openjdk.java.net/~pliden/8199925/webrev.1vs2 > Full: http://cr.openjdk.java.net/~pliden/8199925/webrev.2 > > /Per > >> On 03/23/2018 11:57 AM, Per Liden wrote: >> Thanks Roman and Erik for reviewing! >> /Per >>> On 03/21/2018 04:07 PM, Roman Kennke wrote: >>> Looks good! Thank you! >>> >>> Am 21. M?rz 2018 14:11:52 MEZ schrieb Per Liden : >>>> Hi, >>>> >>>> Here's an updated webrev. I renamed GCSelector to GCConfig and it's now >>>> >>>> also the goto place for getting hold of the GCArguments instance for >>>> the >>>> currently selected GC. >>>> >>>> http://cr.openjdk.java.net/~pliden/8199925/webrev.1 >>>> >>>> Btw, this is built on top of JDK-8199850 (Move parsing of VerifyGCType >>>> to G1) and JDK-8199918 (Shorten names of CollectedHeap::Name members), >>>> and the GCConfig::is_gc_* functions will eventually be used by >>>> JDK-8199927 (Make WhiteBox more GC agnostic). >>>> >>>> /Per >>>> >>>>> On 03/21/2018 11:33 AM, Per Liden wrote: >>>>> Hi Roman, >>>>> >>>>>> On 03/21/2018 10:15 AM, Roman Kennke wrote: >>>>>>> Am 21.03.2018 um 09:12 schrieb Per Liden: >>>>>>> In an attempt to make WhiteBox more GC agnostic, and in turn make >>>> it >>>>>>> easier to plugin new GC without touching a lot of non-GC code, this >>>>>>> patch breaks out the GC selection logic from GCArguments to >>>> GCSelector. >>>>>>> There are two reasons why I think this makes sense: >>>>>>> 1) The GC selection logic is self-contained and fairly unrelated to >>>> the >>>>>>> rest of the flags processing done by GCArguments and its >>>> GC-specific >>>>>>> children. >>>>>>> 2) Parts of the GC selection logic is needed by WhiteBox to answer >>>>>>> questions about which GC are supported, selected, etc. >>>>>>> >>>>>>> A follow up patch (JDK-8199927) will change WhiteBox (WB_CurrentGC, >>>>>>> WB_AllSupportedGC, WB_GCSelectedByErgo), to use the GCSelector. >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8199925 >>>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8199925/webrev.0 >>>>>>> >>>>>>> /Per >>>>>> >>>>>> My original proposal for the GCArguments did have a separate >>>>>> GCFactory. :-) >>>>>> >>>>>> In generally I like your change. >>>>>> >>>>>> I'd probably go even further and let GCSelector initialize and >>>> return an >>>>>> instance of GCArguments. This way we wouldn't end up with two places >>>>>> that have knowledge of all possible GCs, but would only have one >>>> place >>>>>> (GCSelector) where one had to hook up for adding a new GC. >>>>>> >>>>>> What do you think? >>>>> >>>>> My intention here was that GCSelector would be a pure translator of >>>>> Use*GC flags to CollectedHeap::Name, i.e. a small and well defined >>>>> component, which knows nothing about GCArguments. The purpose of this >>>> >>>>> was to clean out the WhiteBox stuff. >>>>> >>>>> On the other hand, I see what you mean, and too would like to reach a >>>> >>>>> state where we have one place in the code that "knows about all the >>>> GCs". >>>>> >>>>> Let me think about it some more and maybe prototype an alternative, >>>> and >>>>> we'll see if that gets us to a better place. >>>>> >>>>> /Per >>>>> >>>>>> >>>>>> Roman >>>>>> >>> From stefan.johansson at oracle.com Mon Mar 26 08:27:09 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 26 Mar 2018 10:27:09 +0200 Subject: RFR: 8200169: Flatten G1Allocator class hierarchy In-Reply-To: <1521820928.4147.8.camel@oracle.com> References: <50c3c960-8bed-29d4-edf0-e7009d969c0a@oracle.com> <1521820928.4147.8.camel@oracle.com> Message-ID: <241196aa-4313-fe29-b199-edd4ba5d7640@oracle.com> Thanks for the review Thomas, Stefan On 2018-03-23 17:02, Thomas Schatzl wrote: > Hi, > > On Fri, 2018-03-23 at 14:32 +0100, Stefan Johansson wrote: >> Hi, >> >> Please review this change to remove some unnecessary class >> hierarchies >> in G1. >> >> Links >> JBS: https://bugs.openjdk.java.net/browse/JDK-8200169 >> Webrev: http://cr.openjdk.java.net/~sjohanss/8200169/00/ >> >> Summary >> After the removal of several extension points and some other cleanups >> in G1, there is no longer any need for the two class hierarchies in >> g1Allocator.hpp. These can be flattened to only consist of two >> concrete classes instead. > looks good to me. > > Thomas > From thomas.schatzl at oracle.com Mon Mar 26 08:47:26 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 26 Mar 2018 10:47:26 +0200 Subject: RFR [L][4/7]: 8180415: Rebuild remembered sets during the concurrent cycle In-Reply-To: <93d3e31e-bebf-fa64-4560-e133a71f203a@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520262478.2532.91.camel@oracle.com> <1520413176.2346.9.camel@oracle.com> <1f345c49-7793-7925-2c15-281f24d28e74@oracle.com> <1520509760.3121.15.camel@oracle.com> <1520952891.3100.36.camel@oracle.com> <1521715049.2448.10.camel@oracle.com> <93d3e31e-bebf-fa64-4560-e133a71f203a@oracle.com> Message-ID: <1522054046.2471.2.camel@oracle.com> Hi Stefan, On Fri, 2018-03-23 at 09:50 +0100, Stefan Johansson wrote: > > On 2018-03-22 11:37, Thomas Schatzl wrote: > > ... > > All done. > > > > http://cr.openjdk.java.net/~tschatzl/8180415/webrev.3_to_4 (diff) > > http://cr.openjdk.java.net/~tschatzl/8180415/webrev.4 (full) > > Looks good to me too, just one very minor thing. The second comment > in g1RemSet.cpp still calls it nTAMS, is that on purpose? No need for > a re-review, just update it if you want. thanks for your review. Note that the renaming from ntams -> tams has been done only later in 8197932. Thanks, Thomas From thomas.schatzl at oracle.com Mon Mar 26 08:47:51 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 26 Mar 2018 10:47:51 +0200 Subject: RFR [L][4/7]: 8180415: Rebuild remembered sets during the concurrent cycle In-Reply-To: <60ec86ca-8f7b-9e62-b352-8fcd24b1051b@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520262478.2532.91.camel@oracle.com> <1520413176.2346.9.camel@oracle.com> <1f345c49-7793-7925-2c15-281f24d28e74@oracle.com> <1520509760.3121.15.camel@oracle.com> <1520952891.3100.36.camel@oracle.com> <1521715049.2448.10.camel@oracle.com> <60ec86ca-8f7b-9e62-b352-8fcd24b1051b@oracle.com> Message-ID: <1522054071.2471.3.camel@oracle.com> Hi, On Thu, 2018-03-22 at 10:23 -0700, sangheon.kim wrote: > Hi Thomas, > > On 03/22/2018 03:37 AM, Thomas Schatzl wrote: > > Hi Sangheon, > > > > thanks for your thorough review. > > [...] > > marking (the evacuation efficiency of the regions may change). This > > triggers some (benign) assert in the collection set chooser. > > We need to clear any existing collection set before initiating > > concurrent mark. > > Webrev.4 looks good to me. > Thank you for fixing all! > thanks for your review. Thomas From thomas.schatzl at oracle.com Mon Mar 26 08:55:21 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 26 Mar 2018 10:55:21 +0200 Subject: RFR: 8199850: Move parsing of VerifyGCType to G1 In-Reply-To: <9ec6fc3c-1d89-4079-e198-ff7edeeda681@oracle.com> References: <5d3f62d2-f4c7-affa-a21c-be3330262af9@oracle.com> <1521735659.2264.5.camel@oracle.com> <3da59cea-4040-1446-a591-85240404b25f@oracle.com> <1521801065.14740.3.camel@oracle.com> <20217c01-6f58-1eb4-2847-740fe4fb932b@oracle.com> <9ec6fc3c-1d89-4079-e198-ff7edeeda681@oracle.com> Message-ID: <1522054521.2471.5.camel@oracle.com> Hi, On Fri, 2018-03-23 at 13:58 +0100, Per Liden wrote: > Sorry, I noticed that I had missed the gtest, which also needed an > adjustments: > > Diff: http://cr.openjdk.java.net/~pliden/8199850/webrev.1vs2 > Full: http://cr.openjdk.java.net/~pliden/8199850/webrev.2 > > Testing: Passed hg-tier1 still good. Thomas From per.liden at oracle.com Mon Mar 26 08:55:47 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 26 Mar 2018 10:55:47 +0200 Subject: RFR: 8199925: Break out GC selection logic from GCArguments to GCSelector In-Reply-To: <7E650A64-50B4-432D-ACAC-0B605147E808@oracle.com> References: <35136ca4-cc0c-d977-8351-523a52a39c7a@redhat.com> <15700f36-bf1b-b8a7-4954-8501b5ed7626@oracle.com> <0C591305-5A1E-4E13-8411-C58E2A218623@redhat.com> <1e84e63d-988f-d40f-4719-bfa636aafb3f@oracle.com> <7E650A64-50B4-432D-ACAC-0B605147E808@oracle.com> Message-ID: <7e98db22-a5f6-5c1b-48eb-43ba155ea7bc@oracle.com> Thanks Erik! /Per On 03/26/2018 09:10 AM, Erik Osterlund wrote: > Hi Per, > > Looks good. > > Thanks, > /Erik > >> On 26 Mar 2018, at 08:40, Per Liden wrote: >> >> Hi again, >> >> It turned out that the Solaris Studio compiler didn't like structs with reference members without a user-defined constructor (all other compilers happily swallows this), so I had to add one. >> >> Updated webrev: >> >> Diff: http://cr.openjdk.java.net/~pliden/8199925/webrev.1vs2 >> Full: http://cr.openjdk.java.net/~pliden/8199925/webrev.2 >> >> /Per >> >>> On 03/23/2018 11:57 AM, Per Liden wrote: >>> Thanks Roman and Erik for reviewing! >>> /Per >>>> On 03/21/2018 04:07 PM, Roman Kennke wrote: >>>> Looks good! Thank you! >>>> >>>> Am 21. M?rz 2018 14:11:52 MEZ schrieb Per Liden : >>>>> Hi, >>>>> >>>>> Here's an updated webrev. I renamed GCSelector to GCConfig and it's now >>>>> >>>>> also the goto place for getting hold of the GCArguments instance for >>>>> the >>>>> currently selected GC. >>>>> >>>>> http://cr.openjdk.java.net/~pliden/8199925/webrev.1 >>>>> >>>>> Btw, this is built on top of JDK-8199850 (Move parsing of VerifyGCType >>>>> to G1) and JDK-8199918 (Shorten names of CollectedHeap::Name members), >>>>> and the GCConfig::is_gc_* functions will eventually be used by >>>>> JDK-8199927 (Make WhiteBox more GC agnostic). >>>>> >>>>> /Per >>>>> >>>>>> On 03/21/2018 11:33 AM, Per Liden wrote: >>>>>> Hi Roman, >>>>>> >>>>>>> On 03/21/2018 10:15 AM, Roman Kennke wrote: >>>>>>>> Am 21.03.2018 um 09:12 schrieb Per Liden: >>>>>>>> In an attempt to make WhiteBox more GC agnostic, and in turn make >>>>> it >>>>>>>> easier to plugin new GC without touching a lot of non-GC code, this >>>>>>>> patch breaks out the GC selection logic from GCArguments to >>>>> GCSelector. >>>>>>>> There are two reasons why I think this makes sense: >>>>>>>> 1) The GC selection logic is self-contained and fairly unrelated to >>>>> the >>>>>>>> rest of the flags processing done by GCArguments and its >>>>> GC-specific >>>>>>>> children. >>>>>>>> 2) Parts of the GC selection logic is needed by WhiteBox to answer >>>>>>>> questions about which GC are supported, selected, etc. >>>>>>>> >>>>>>>> A follow up patch (JDK-8199927) will change WhiteBox (WB_CurrentGC, >>>>>>>> WB_AllSupportedGC, WB_GCSelectedByErgo), to use the GCSelector. >>>>>>>> >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8199925 >>>>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8199925/webrev.0 >>>>>>>> >>>>>>>> /Per >>>>>>> >>>>>>> My original proposal for the GCArguments did have a separate >>>>>>> GCFactory. :-) >>>>>>> >>>>>>> In generally I like your change. >>>>>>> >>>>>>> I'd probably go even further and let GCSelector initialize and >>>>> return an >>>>>>> instance of GCArguments. This way we wouldn't end up with two places >>>>>>> that have knowledge of all possible GCs, but would only have one >>>>> place >>>>>>> (GCSelector) where one had to hook up for adding a new GC. >>>>>>> >>>>>>> What do you think? >>>>>> >>>>>> My intention here was that GCSelector would be a pure translator of >>>>>> Use*GC flags to CollectedHeap::Name, i.e. a small and well defined >>>>>> component, which knows nothing about GCArguments. The purpose of this >>>>> >>>>>> was to clean out the WhiteBox stuff. >>>>>> >>>>>> On the other hand, I see what you mean, and too would like to reach a >>>>> >>>>>> state where we have one place in the code that "knows about all the >>>>> GCs". >>>>>> >>>>>> Let me think about it some more and maybe prototype an alternative, >>>>> and >>>>>> we'll see if that gets us to a better place. >>>>>> >>>>>> /Per >>>>>> >>>>>>> >>>>>>> Roman >>>>>>> >>>> > From thomas.schatzl at oracle.com Mon Mar 26 08:58:27 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 26 Mar 2018 10:58:27 +0200 Subject: RFR/RFC: 8196071: Change G1 Full GC heap and thread sizing ergonomics In-Reply-To: <7497d3fb-044d-35f2-f02f-70ea79f0e2aa@oracle.com> References: <7497d3fb-044d-35f2-f02f-70ea79f0e2aa@oracle.com> Message-ID: <1522054707.2471.7.camel@oracle.com> Hi, On Thu, 2018-03-22 at 16:42 +0100, Stefan Johansson wrote: > Hi, > > Please review or comment on this change to let the G1 Full GC > calculate the number of worker threads. > > Links > JBS: https://bugs.openjdk.java.net/browse/JDK-8196071 > Webrev: http://cr.openjdk.java.net/~sjohanss/8196071/00/ > > Summary > After making UseDynamicNumberOfGCThreads true by default, having a > sane thread sizing model for the G1 Full GC is important. Before this > change the Full GC would use the same number of threads as the > previous collection, and if being the first just use 1 thread. The > proposed change now have the Full GC consider two metrics to get a > good number of worker threads. > > The first metric is to try to avoid to much wasted space. On average > each worker used by the parallel full GC will cause half a region of > unused memory. To make sure we don't use too many workers we use > G1HeapWastePercent to limit the number of workers. > > The other metric is the one used in AdaptiveSizePolicy, it looks at > the flag HeapSizePerGCWorker and limit the number by this. In most > cases this is more restrictive than using G1HeapWastePercent, but for > small heaps setting a large G1HeapRegionSize, G1HeapWastePercent will > come into play. > > This change also changes the default value of HeapSizePerGCWorker to > 32M instead of 64M, this can be reverted if anyone has a good > argument to leave it as is. My testing shows that for DaCapo, > SpecJVM98 and GCBasher (gc intensive stress test) with small heaps we > get better pause times with the flag set to 32M. There might be other > arguments, such as not using to much resources on small systems that > might also be important so please let me know if you have reasons why > the flag should be left unchanged. > > Testing > Functional testing through Mach5 and manual performance testing with > small heaps and different values for HeapSizePerGCThread. > looks good, although I would prefer to separate the HeapSizePerGCWorker change into a separate CR. Please update the copyrights on push. Thanks, Thomas From stefan.johansson at oracle.com Mon Mar 26 09:02:56 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 26 Mar 2018 11:02:56 +0200 Subject: RFR: 8199850: Move parsing of VerifyGCType to G1 In-Reply-To: <1522054521.2471.5.camel@oracle.com> References: <5d3f62d2-f4c7-affa-a21c-be3330262af9@oracle.com> <1521735659.2264.5.camel@oracle.com> <3da59cea-4040-1446-a591-85240404b25f@oracle.com> <1521801065.14740.3.camel@oracle.com> <20217c01-6f58-1eb4-2847-740fe4fb932b@oracle.com> <9ec6fc3c-1d89-4079-e198-ff7edeeda681@oracle.com> <1522054521.2471.5.camel@oracle.com> Message-ID: On 2018-03-26 10:55, Thomas Schatzl wrote: > Hi, > > On Fri, 2018-03-23 at 13:58 +0100, Per Liden wrote: >> Sorry, I noticed that I had missed the gtest, which also needed an >> adjustments: >> >> Diff: http://cr.openjdk.java.net/~pliden/8199850/webrev.1vs2 >> Full: http://cr.openjdk.java.net/~pliden/8199850/webrev.2 >> >> Testing: Passed hg-tier1 > still good. +1 Stefan > > Thomas From aph at redhat.com Mon Mar 26 09:38:24 2018 From: aph at redhat.com (Andrew Haley) Date: Mon, 26 Mar 2018 10:38:24 +0100 Subject: RFR: JDK-8199781: Don't use naked == for comparing oops In-Reply-To: <096620eb-e252-9689-c157-098314ac9119@redhat.com> References: <74ca6651-49aa-2b77-d7e9-a588fb2b8417@oracle.com> <096620eb-e252-9689-c157-098314ac9119@redhat.com> Message-ID: <650c2e5c-88d7-3896-1526-386557f5d2df@redhat.com> On 03/21/2018 04:58 PM, Roman Kennke wrote: > I believe that inline is over-used in OpenJDK too. It might be > better to start out with regular methods and *if* they show up in > profiles, consider to inline them. Would that be acceptable? I don't believe so. Efficient systems are made from thousands of tiny incremental improvements, each one of which is too small to be measured on its own. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From thomas.schatzl at oracle.com Mon Mar 26 09:41:04 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 26 Mar 2018 11:41:04 +0200 Subject: RFR (XS): 8193067: gc/g1/TestVerifyGCType.java still unstable Message-ID: <1522057264.2471.13.camel@oracle.com> Hi all, can I have reviews for this small change to the TestVerifyGCType test to make it stable? The problem has been that due to different number of threads used, different amount of heap waste has been "produced", only randomly triggering the expected mixed gcs. By fixing the number of threads used and decreasing G1HeapWastePercent the mixed gcs can be forced reproducably. CR: https://bugs.openjdk.java.net/browse/JDK-8193067 Webrev: http://cr.openjdk.java.net/~tschatzl/8193067/webrev/ Testing: failing test passing multiple times in CI Thanks, Thomas From stefan.johansson at oracle.com Mon Mar 26 09:47:20 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 26 Mar 2018 11:47:20 +0200 Subject: RFR/RFC: 8196071: Change G1 Full GC heap and thread sizing ergonomics In-Reply-To: <1522054707.2471.7.camel@oracle.com> References: <7497d3fb-044d-35f2-f02f-70ea79f0e2aa@oracle.com> <1522054707.2471.7.camel@oracle.com> Message-ID: <217a9dd1-5e01-d46d-7e0e-7ab79e3381fd@oracle.com> On 2018-03-26 10:58, Thomas Schatzl wrote: > Hi, > > On Thu, 2018-03-22 at 16:42 +0100, Stefan Johansson wrote: >> Hi, >> >> Please review or comment on this change to let the G1 Full GC >> calculate the number of worker threads. >> >> Links >> JBS: https://bugs.openjdk.java.net/browse/JDK-8196071 >> Webrev: http://cr.openjdk.java.net/~sjohanss/8196071/00/ >> >> Summary >> After making UseDynamicNumberOfGCThreads true by default, having a >> sane thread sizing model for the G1 Full GC is important. Before this >> change the Full GC would use the same number of threads as the >> previous collection, and if being the first just use 1 thread. The >> proposed change now have the Full GC consider two metrics to get a >> good number of worker threads. >> >> The first metric is to try to avoid to much wasted space. On average >> each worker used by the parallel full GC will cause half a region of >> unused memory. To make sure we don't use too many workers we use >> G1HeapWastePercent to limit the number of workers. >> >> The other metric is the one used in AdaptiveSizePolicy, it looks at >> the flag HeapSizePerGCWorker and limit the number by this. In most >> cases this is more restrictive than using G1HeapWastePercent, but for >> small heaps setting a large G1HeapRegionSize, G1HeapWastePercent will >> come into play. >> >> This change also changes the default value of HeapSizePerGCWorker to >> 32M instead of 64M, this can be reverted if anyone has a good >> argument to leave it as is. My testing shows that for DaCapo, >> SpecJVM98 and GCBasher (gc intensive stress test) with small heaps we >> get better pause times with the flag set to 32M. There might be other >> arguments, such as not using to much resources on small systems that >> might also be important so please let me know if you have reasons why >> the flag should be left unchanged. >> >> Testing >> Functional testing through Mach5 and manual performance testing with >> small heaps and different values for HeapSizePerGCThread. >> > looks good, although I would prefer to separate the > HeapSizePerGCWorker change into a separate CR. Good point. Created JDK-8200228 for this and just realized I probably need a CSR for that as well. Here's a new webrev without the flag-change: http://cr.openjdk.java.net/~sjohanss/8196071/01/ Thanks, Stefan > Please update the copyrights on push. > > Thanks, > Thomas > From stefan.johansson at oracle.com Mon Mar 26 09:56:00 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 26 Mar 2018 11:56:00 +0200 Subject: RFR (XS): 8193067: gc/g1/TestVerifyGCType.java still unstable In-Reply-To: <1522057264.2471.13.camel@oracle.com> References: <1522057264.2471.13.camel@oracle.com> Message-ID: <3c8c48eb-a35d-7bd1-1ef0-2d65e53ebfb3@oracle.com> Hi Thomas, On 2018-03-26 11:41, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this small change to the TestVerifyGCType test > to make it stable? The problem has been that due to different number of > threads used, different amount of heap waste has been "produced", only > randomly triggering the expected mixed gcs. > > By fixing the number of threads used and decreasing G1HeapWastePercent > the mixed gcs can be forced reproducably. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8193067 > > Webrev: > http://cr.openjdk.java.net/~tschatzl/8193067/webrev/ Looks good, thanks for fixing this Thomas! Thanks, Stefan > > Testing: > failing test passing multiple times in CI > > Thanks, > Thomas > From thomas.schatzl at oracle.com Mon Mar 26 10:11:06 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 26 Mar 2018 12:11:06 +0200 Subject: RFR (XS): 8193067: gc/g1/TestVerifyGCType.java still unstable In-Reply-To: <3c8c48eb-a35d-7bd1-1ef0-2d65e53ebfb3@oracle.com> References: <1522057264.2471.13.camel@oracle.com> <3c8c48eb-a35d-7bd1-1ef0-2d65e53ebfb3@oracle.com> Message-ID: <1522059066.8723.0.camel@oracle.com> Hi, On Mon, 2018-03-26 at 11:56 +0200, Stefan Johansson wrote: > Hi Thomas, > > On 2018-03-26 11:41, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this small change to the TestVerifyGCType > > test to make it stable? [...] > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8193067 > > > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8193067/webrev/ > > Looks good, thanks for fixing this Thomas! > thanks Stefan! Thomas From thomas.schatzl at oracle.com Mon Mar 26 10:12:42 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 26 Mar 2018 12:12:42 +0200 Subject: RFR: 8196071: Change G1 Full GC heap and thread sizing ergonomics In-Reply-To: <217a9dd1-5e01-d46d-7e0e-7ab79e3381fd@oracle.com> References: <7497d3fb-044d-35f2-f02f-70ea79f0e2aa@oracle.com> <1522054707.2471.7.camel@oracle.com> <217a9dd1-5e01-d46d-7e0e-7ab79e3381fd@oracle.com> Message-ID: <1522059162.8723.1.camel@oracle.com> Hi, On Mon, 2018-03-26 at 11:47 +0200, Stefan Johansson wrote: > > On 2018-03-26 10:58, Thomas Schatzl wrote: > > Hi, > > > > On Thu, 2018-03-22 at 16:42 +0100, Stefan Johansson wrote: > > > Hi, > > > > > > Please review or comment on this change to let the G1 Full GC > > > calculate the number of worker threads. > > > [...] > > > > looks good, although I would prefer to separate the > > HeapSizePerGCWorker change into a separate CR. > > Good point. Created JDK-8200228 for this and just realized I > probably > need a CSR for that as well. Here's a new webrev without the flag- > change: > http://cr.openjdk.java.net/~sjohanss/8196071/01/ still good. Thomas From thomas.schatzl at oracle.com Mon Mar 26 10:04:41 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 26 Mar 2018 12:04:41 +0200 Subject: RFR (S): 8196485: FromCardCache default card index can cause crashes In-Reply-To: <7187c820-0e40-6187-67a8-9f3bb56500ca@redhat.com> References: <1520499556.3121.8.camel@oracle.com> <7187c820-0e40-6187-67a8-9f3bb56500ca@redhat.com> Message-ID: <1522058681.2471.14.camel@oracle.com> Hi Aleksey, On Fri, 2018-03-23 at 09:45 +0100, Aleksey Shipilev wrote: > On 03/08/2018 09:59 AM, Thomas Schatzl wrote: > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8196485 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8196485/webrev/ > > Ha! Ouch. Looks good, except: > Yeah :/ > *) I feel uneasy about assigning negative values to unsigned vars. > Intended to be UINTPTR_MAX? > > static const uintptr_t InvalidCard = -1; > > *) "heap region" here, probably: > > 134 // of the given heapregion. I updated the webrev in place with these fixes. Thanks for your review. Thomas From shade at redhat.com Mon Mar 26 10:22:49 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 26 Mar 2018 12:22:49 +0200 Subject: RFR (S): 8196485: FromCardCache default card index can cause crashes In-Reply-To: <1522058681.2471.14.camel@oracle.com> References: <1520499556.3121.8.camel@oracle.com> <7187c820-0e40-6187-67a8-9f3bb56500ca@redhat.com> <1522058681.2471.14.camel@oracle.com> Message-ID: <6db0dfe1-ff83-8b7f-2fbd-a42bb12bf6d7@redhat.com> On 03/26/2018 12:04 PM, Thomas Schatzl wrote: >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8196485/webrev/ >> > I updated the webrev in place with these fixes. Looks good! -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Mon Mar 26 10:34:15 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 26 Mar 2018 12:34:15 +0200 Subject: RFR: 8199850: Move parsing of VerifyGCType to G1 In-Reply-To: References: <5d3f62d2-f4c7-affa-a21c-be3330262af9@oracle.com> <1521735659.2264.5.camel@oracle.com> <3da59cea-4040-1446-a591-85240404b25f@oracle.com> <1521801065.14740.3.camel@oracle.com> <20217c01-6f58-1eb4-2847-740fe4fb932b@oracle.com> <9ec6fc3c-1d89-4079-e198-ff7edeeda681@oracle.com> <1522054521.2471.5.camel@oracle.com> Message-ID: Thanks Stefan and Thomas! /Per On 03/26/2018 11:02 AM, Stefan Johansson wrote: > > > On 2018-03-26 10:55, Thomas Schatzl wrote: >> Hi, >> >> On Fri, 2018-03-23 at 13:58 +0100, Per Liden wrote: >>> Sorry, I noticed that I had missed the gtest, which also needed an >>> adjustments: >>> >>> Diff: http://cr.openjdk.java.net/~pliden/8199850/webrev.1vs2 >>> Full: http://cr.openjdk.java.net/~pliden/8199850/webrev.2 >>> >>> Testing: Passed hg-tier1 >> ?? still good. > +1 > Stefan >> >> Thomas > From stefan.johansson at oracle.com Mon Mar 26 10:37:20 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 26 Mar 2018 12:37:20 +0200 Subject: RFR (S): 8196485: FromCardCache default card index can cause crashes In-Reply-To: <1522058681.2471.14.camel@oracle.com> References: <1520499556.3121.8.camel@oracle.com> <7187c820-0e40-6187-67a8-9f3bb56500ca@redhat.com> <1522058681.2471.14.camel@oracle.com> Message-ID: <8a043b0e-c99c-64ce-58f7-a0531a3fbab5@oracle.com> Looks good, StefanJ On 2018-03-26 12:04, Thomas Schatzl wrote: > Hi Aleksey, > > On Fri, 2018-03-23 at 09:45 +0100, Aleksey Shipilev wrote: >> On 03/08/2018 09:59 AM, Thomas Schatzl wrote: >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8196485 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8196485/webrev/ >> Ha! Ouch. Looks good, except: >> > Yeah :/ > >> *) I feel uneasy about assigning negative values to unsigned vars. >> Intended to be UINTPTR_MAX? >> >> static const uintptr_t InvalidCard = -1; >> >> *) "heap region" here, probably: >> >> 134 // of the given heapregion. > I updated the webrev in place with these fixes. > > Thanks for your review. > > Thomas From thomas.schatzl at oracle.com Mon Mar 26 10:51:41 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 26 Mar 2018 12:51:41 +0200 Subject: RFR (S): 8196485: FromCardCache default card index can cause crashes In-Reply-To: <8a043b0e-c99c-64ce-58f7-a0531a3fbab5@oracle.com> References: <1520499556.3121.8.camel@oracle.com> <7187c820-0e40-6187-67a8-9f3bb56500ca@redhat.com> <1522058681.2471.14.camel@oracle.com> <8a043b0e-c99c-64ce-58f7-a0531a3fbab5@oracle.com> Message-ID: <1522061501.8723.5.camel@oracle.com> Hi Stefan, Aleksey, On Mon, 2018-03-26 at 12:37 +0200, Stefan Johansson wrote: > Looks good, > StefanJ > thanks for your reviews. Thomas From thomas.schatzl at oracle.com Mon Mar 26 11:48:55 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 26 Mar 2018 13:48:55 +0200 Subject: RFR (S): 8200233: Simple G1 evacuation path performance enhancements Message-ID: <1522064935.8723.7.camel@oracle.com> Hi all, can I have reviews for these very tiny (and ultimately note really measureable) improvements to the G1 evacuation path? They are: - some stores can be made OOP_NOT_NULL - the path for compressed/uncompressed oops when popping elements from the task queue during evacuation can be specialized: compressed oops are never array oops so saving the useless test - some load of the "from" region in the evacuation path can be delayed and in many cases actually avoided CR: https://bugs.openjdk.java.net/browse/JDK-8200233 Webrev: http://cr.openjdk.java.net/~tschatzl/8200233/webrev/ Testing: hs-tier-1 Thanks, Thomas From rkennke at redhat.com Mon Mar 26 13:16:56 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 26 Mar 2018 15:16:56 +0200 Subject: RFR: JDK-8199781: Don't use naked == for comparing oops In-Reply-To: <650c2e5c-88d7-3896-1526-386557f5d2df@redhat.com> References: <74ca6651-49aa-2b77-d7e9-a588fb2b8417@oracle.com> <096620eb-e252-9689-c157-098314ac9119@redhat.com> <650c2e5c-88d7-3896-1526-386557f5d2df@redhat.com> Message-ID: <2683d96d-4170-f8b7-a48d-12fd0465ec4c@redhat.com> Am 26.03.2018 um 11:38 schrieb Andrew Haley: > On 03/21/2018 04:58 PM, Roman Kennke wrote: > >> I believe that inline is over-used in OpenJDK too. It might be >> better to start out with regular methods and *if* they show up in >> profiles, consider to inline them. Would that be acceptable? > > I don't believe so. Efficient systems are made from thousands of tiny > incremental improvements, each one of which is too small to be > measured on its own. > I don't generally agree. Efficient systems are made of efficient algorithms. Tiny little improvements (aka microoptimizations) like inlining all sorts of stuff just because.. can also backfire. I have checked the uses of the new equals() function, and there are not many that look like they would be performance sensitive. However, in order to not cause regressions in any code I might have overlooked, I did change the new stuff to inlining, and hunted down the places where it breaks compilation. Differential: http://cr.openjdk.java.net/~rkennke/JDK-8199781/webrev.02.diff/ Full: http://cr.openjdk.java.net/~rkennke/JDK-8199781/webrev.02/ One thing I was not sure of if it's worth refactoring into .inline.hpp pattern is the change around stackValue.hpp. There is still one slightly problematic use of oopDesc::equals() in growableArray.hpp, but I don't see how to fix that (i.e. refactor into growableArray.inline.hpp or such) without causing a mess (this stuff is used throughout C2 headers, e.g. compile.hpp). This means that code that includes growableArray.hpp also currently also needs to include oop.inline.hpp. Is the patch acceptable now? Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From stefan.karlsson at oracle.com Mon Mar 26 13:41:13 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 26 Mar 2018 15:41:13 +0200 Subject: RFR (S) 8199007: ClassLoaderDataGraph::unload_list_contains() is wrong In-Reply-To: <714f00e1-6938-e78e-d485-b3efd7297db4@oracle.com> References: <714f00e1-6938-e78e-d485-b3efd7297db4@oracle.com> Message-ID: <084beb5b-6acc-f96b-c866-2e603b54417b@oracle.com> Looks good. StefanK On 2018-03-23 15:29, coleen.phillimore at oracle.com wrote: > Summary: The function isn't needed any longer. > > I believe the function existed from an earlier day when > Metaspace::contains walked the CLDG, so you needed a function to walk > the unloaded list as well. > > Tested with Per's GC test suite and mach5 tier1-5 (until it failed in > infra for some platforms), and my own local testing like: > > jtreg -exclude:open/test/hotspot/jtreg/ProblemList.txt > open/test/hotspot/jtreg:hotspot_all_no_apps > > (and the closed jtreg tests too). > > open webrev at http://cr.openjdk.java.net/~coleenp/8199007.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8199007 > > Thanks, > Coleen From coleen.phillimore at oracle.com Mon Mar 26 14:06:43 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 26 Mar 2018 10:06:43 -0400 Subject: RFR (S) 8199007: ClassLoaderDataGraph::unload_list_contains() is wrong In-Reply-To: <084beb5b-6acc-f96b-c866-2e603b54417b@oracle.com> References: <714f00e1-6938-e78e-d485-b3efd7297db4@oracle.com> <084beb5b-6acc-f96b-c866-2e603b54417b@oracle.com> Message-ID: <27f12173-ba73-b5f1-a101-6eaef9e9db0b@oracle.com> Thank you! Coleen On 3/26/18 9:41 AM, Stefan Karlsson wrote: > Looks good. > > StefanK > > On 2018-03-23 15:29, coleen.phillimore at oracle.com wrote: >> Summary: The function isn't needed any longer. >> >> I believe the function existed from an earlier day when >> Metaspace::contains walked the CLDG, so you needed a function to walk >> the unloaded list as well. >> >> Tested with Per's GC test suite and mach5 tier1-5 (until it failed in >> infra for some platforms), and my own local testing like: >> >> jtreg -exclude:open/test/hotspot/jtreg/ProblemList.txt >> open/test/hotspot/jtreg:hotspot_all_no_apps >> >> (and the closed jtreg tests too). >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8199007.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8199007 >> >> Thanks, >> Coleen From erik.osterlund at oracle.com Mon Mar 26 15:04:39 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 26 Mar 2018 17:04:39 +0200 Subject: RFR: JDK-8199735: Mark word updates need to use Access API In-Reply-To: <63199799-0194-a1d4-9c89-d95251c7aa7d@redhat.com> References: <7bd73e32-a26b-2504-f069-b74807ae5e53@redhat.com> <859ac657-78c9-c7ab-0ec0-eddda331001b@redhat.com> <5AB3B052.4000304@oracle.com> <63199799-0194-a1d4-9c89-d95251c7aa7d@redhat.com> Message-ID: <5AB90C07.3020507@oracle.com> Hi Roman, This looks good to me. Thank you for doing this. The only thing I am a bit puzzled by is why the oopDesc::set_mark member function takes a volatile markOop as parameter. That seems rather weird. But that is unrelated to your changes. Thanks, /Erik On 2018-03-25 23:03, Roman Kennke wrote: > Hi Erik, > > Thanks for having a look. > >> A few comments: >> >> 1) Please make sure all files referencing these accessors that have been >> moved to the oop.inline.hpp file now #include the oop.inline.hpp file. I >> found that there are quite a few missing by looking for references to >> oopDesc::mark() with rtags. >> 2) I think the mark_addr() function should now be called mark_addr_raw() >> for consistency. >> 3) I wonder if there should be _raw() accessors that the GCs use >> instead. It feels like the GC should not go through the Access API to >> acquire mark words. > You are of course right in all 3 points. Unfortunately, that causes > massive ripples through the code base ;-) > > So here is what I did: I added _raw() variants for stuff that is used by > GC. I temporarily renamed all relevant methods, e.g. mark() -> > mark_fluff() to make the compiler help me. Then for each occurance of > any such call decided whether it should call the usual method, or the > raw variant (as a general rule, everything under src/hotspot/share/gc > uses the _raw() variants). Also, I added #include "oop.inline.hpp" for > every such occurance. Then I renamed all _fluff() methods and calls back > to usual. Did this exercise with and without PCH, with and without zero. > Checked with grep if I might have missed some. > > Notes: bytecodeInterpreter.cpp needed some additional fixes to use > cas_set_mark() instead of the Atomic::cmpxchg(). > > There are a bunch of methods in oop.hpp that are only used by GC, like > is_locked() etc. I haven't changed them to _raw(). > > This passes tier1 tests release+fastdebug, and as mentioned above builds > with and without PCH and zero. > > Differential: > http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.03.diff/ > Full: > http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.03/ > > Better now? > > Best regards, > Roman > > > >> Thanks, >> /Erik >> >> >> On 2018-03-21 20:21, Roman Kennke wrote: >>> Am 19.03.2018 um 12:07 schrieb Roman Kennke: >>>> Am 19.03.2018 um 11:40 schrieb Roman Kennke: >>>>> Currently, the mark word is accessed directly in oopDesc::mark() >>>>> set_mark() and a bunch of other accessors. Those need to use the Access >>>>> API instead because GC might want to employ barriers on those accesses, >>>>> pretty much like every other field in oopDesc. >>>>> >>>>> Notice that this is *not* about accessing the bits and fields inside >>>>> the >>>>> markOop, but about accessing the header itself. >>>>> >>>>> http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.01/ >>>>> >>>>> Testing: build fastdebug/release with and without PCH, passed tier1 >>>>> fastdebug/release. >>>>> >>>>> Can I please get reviews? >>>>> >>>>> Thanks, Roman >>>>> >>>> Just when I sent it, I realized that this is dropping the volatile from >>>> the mark word access. Fixed here: >>>> >>>> >>>> http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.02/ >>>> >>> Ping? >>> > From thomas.schatzl at oracle.com Mon Mar 26 15:06:52 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 26 Mar 2018 17:06:52 +0200 Subject: RFR (M): 8199742: Clean up state flags in G1CollectorState Message-ID: <1522076812.2567.21.camel@oracle.com> Hi all, I would like to request reviews for this change that cleans up the flags in G1CollectorState, applying uniform naming, removing members that were basically temporary variables for a single method, and remove redundant flags (the four flags in the "XXX" block were basically a single one). While this is just a step towards my goal of removing the flags as much as possible and merging them into enums, I think this is a very worthwhile step on that way. Further it blocks a few more changes of mine :P Naming guidelines: - flags indicating a particular GC were named "in_XXX_gc" - durations spanning multiple garbage collections, i.e. phases have the "phase" postfix. Other than that there were general updates for the names to hopefully make them reflect reality a bit closer. Main changes apart from naming were - in g1CollectionSet.cpp where I made add_young_region_common() to not update statistics. This avoids the need to make the full gc also a young gc (at least as far as flags are concerned); to fix this properly we would need to know the previous phase we were in when coming into the full gc. I just do not think it is worth adding a new state here (and the code has been wrong previously too, unconditionally adding these statistics to the young-only phase). I made a note in the follow-up change JDK-8080226. - the changes in G1Policy::record_collection_pause_end() were mostly due to removing the last_gc_was_young() member which was basically a reminder just for this method that the current gc was a young-only gc. There is still _initiate_conc_mark_if_possible flag that indicates that a state change to initial mark should occur asap, but I think that should be moved to e.g. g1policy in a next change. CR: https://bugs.openjdk.java.net/browse/JDK-8199742 Webrev: http://cr.openjdk.java.net/~tschatzl/8199742/webrev Testing: hs-tier 1-5 These changes are based on the rebuild remembered sets concurrently changes. As it may be a bit of work to get them working (I already pushed to first four), here is a webrev that updates latest jdk/hs to that level at http://cr.openjdk.java.net/~tschatzl/8199742/baseline/web rev/ Thanks, Thomas From thomas.schatzl at oracle.com Mon Mar 26 15:14:35 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 26 Mar 2018 17:14:35 +0200 Subject: RFR (M): 8151171: Bring g1ConcurrentMark files up to current coding conventions Message-ID: <1522077275.2567.27.camel@oracle.com> Hi all, can I have reviews for this mostly renaming patch that cleans up g1ConcurrentMark files (once again *sigh*). Changes: - camelCase variable and method naming - missing brackets around one-line if-statement bodies. - method naming (getters/setters) - use _g1h member variable instead of _g1 or another method local variable as discussed earlier - use assert_at_safepoint* macros - remove incomprehensible comments - changed "CheckPointRoots" to "Remark" throughout ... CR: https://bugs.openjdk.java.net/browse/JDK-8151171 Webrev: http://cr.openjdk.java.net/~tschatzl/8151171/webrev Testing: hs-tier 1-5 in conjunction with JDK-8200234, hs-tier 1 standalone The webrev is based on JDK-8199742 also out for review, but has no particular semantic connection. Thanks, Thomas From stefan.johansson at oracle.com Mon Mar 26 15:19:58 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 26 Mar 2018 17:19:58 +0200 Subject: RFR (S): 8200233: Simple G1 evacuation path performance enhancements In-Reply-To: <1522064935.8723.7.camel@oracle.com> References: <1522064935.8723.7.camel@oracle.com> Message-ID: <5eb8de4d-6f61-0e88-5147-7db69f4cec3f@oracle.com> Looks good, Stefan On 2018-03-26 13:48, Thomas Schatzl wrote: > Hi all, > > can I have reviews for these very tiny (and ultimately note really > measureable) improvements to the G1 evacuation path? > > They are: > > - some stores can be made OOP_NOT_NULL > - the path for compressed/uncompressed oops when popping elements from > the task queue during evacuation can be specialized: compressed oops > are never array oops so saving the useless test > - some load of the "from" region in the evacuation path can be delayed > and in many cases actually avoided > > CR: > https://bugs.openjdk.java.net/browse/JDK-8200233 > > Webrev: > http://cr.openjdk.java.net/~tschatzl/8200233/webrev/ > > Testing: > hs-tier-1 > > Thanks, > Thomas From thomas.schatzl at oracle.com Mon Mar 26 15:22:29 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 26 Mar 2018 17:22:29 +0200 Subject: RFR (S): 8200233: Simple G1 evacuation path performance enhancements In-Reply-To: <5eb8de4d-6f61-0e88-5147-7db69f4cec3f@oracle.com> References: <1522064935.8723.7.camel@oracle.com> <5eb8de4d-6f61-0e88-5147-7db69f4cec3f@oracle.com> Message-ID: <1522077749.2567.30.camel@oracle.com> Hi, On Mon, 2018-03-26 at 17:19 +0200, Stefan Johansson wrote: > Looks good, > Stefan > thanks for your review. Thomas From thomas.schatzl at oracle.com Mon Mar 26 15:59:49 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 26 Mar 2018 17:59:49 +0200 Subject: RFR (S/M): 8200234: Cleanup Remark and Cleanup pause code Message-ID: <1522079989.2567.43.camel@oracle.com> Hi all, can I have reviews for this change that cleans up Remark and Cleanup pause code? It mainly moves the record_concurrent_mark_xxx_start/end() methods to the top/bottom of the remark() and cleanup() methods, and refactors the verification code into a single method. There is one more somewhat tricky related change here that fixes the check_bitmaps() method of that verification: after we swapped bitmaps (in Cleanup right now) until the previous prev-bitmap has been cleared, its bits are invalid, and G1HeapVerifier::check_bitmaps() should not check the next bitmap. Previously this has been done using the somewhat tricky condition 674 if (_g1h->collector_state()->mark_or_rebuild_in_progress() || !_g1h->_cmThread->in_progress()) { 675 res_n = verify_no_bits_over_tams("next", next_bitmap, ntams, end); 676 } which you might immediately understand as the best way to prevent checking the next bitmap from the Cleanup until the (new) next bitmap has been cleared, but I did not. Further, there is no similarly easy condition if the bitmap swapping is moved to the Remark pause, so I created a new flag in G1CollectorState that directly records that we are currently clearing the next bitmap. CR: https://bugs.openjdk.java.net/browse/JDK-8200234 Webrev: http://cr.openjdk.java.net/~tschatzl/8200234/webrev/ Testing: hs-tier 1-5 It is based on JDK-8151171 which is also out for review currently. Thanks, Thomas From rkennke at redhat.com Mon Mar 26 16:39:40 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 26 Mar 2018 18:39:40 +0200 Subject: RFR: JDK-8199735: Mark word updates need to use Access API In-Reply-To: <5AB90C07.3020507@oracle.com> References: <7bd73e32-a26b-2504-f069-b74807ae5e53@redhat.com> <859ac657-78c9-c7ab-0ec0-eddda331001b@redhat.com> <5AB3B052.4000304@oracle.com> <63199799-0194-a1d4-9c89-d95251c7aa7d@redhat.com> <5AB90C07.3020507@oracle.com> Message-ID: <89bc1081-71f8-2906-e890-035c5922c1af@redhat.com> > This looks good to me. Thank you for doing this. Thanks Erik for reviewing! > The only thing I am a bit puzzled by is why the oopDesc::set_mark member > function takes a volatile markOop as parameter. That seems rather weird. > But that is unrelated to your changes. Yeah. What does that even mean in C++? Roman > > Thanks, > /Erik > > On 2018-03-25 23:03, Roman Kennke wrote: >> Hi Erik, >> >> Thanks for having a look. >> >>> A few comments: >>> >>> 1) Please make sure all files referencing these accessors that have been >>> moved to the oop.inline.hpp file now #include the oop.inline.hpp file. I >>> found that there are quite a few missing by looking for references to >>> oopDesc::mark() with rtags. >>> 2) I think the mark_addr() function should now be called mark_addr_raw() >>> for consistency. >>> 3) I wonder if there should be _raw() accessors that the GCs use >>> instead. It feels like the GC should not go through the Access API to >>> acquire mark words. >> You are of course right in all 3 points. Unfortunately, that causes >> massive ripples through the code base ;-) >> >> So here is what I did: I added _raw() variants for stuff that is used by >> GC. I temporarily renamed all relevant methods, e.g. mark() -> >> mark_fluff() to make the compiler help me. Then for each occurance of >> any such call decided whether it should call the usual method, or the >> raw variant (as a general rule, everything under src/hotspot/share/gc >> uses the _raw() variants). Also, I added #include "oop.inline.hpp" for >> every such occurance. Then I renamed all _fluff() methods and calls back >> to usual. Did this exercise with and without PCH, with and without zero. >> Checked with grep if I might have missed some. >> >> Notes: bytecodeInterpreter.cpp needed some additional fixes to use >> cas_set_mark() instead of the Atomic::cmpxchg(). >> >> There are a bunch of methods in oop.hpp that are only used by GC, like >> is_locked() etc. I haven't changed them to _raw(). >> >> This passes tier1 tests release+fastdebug, and as mentioned above builds >> with and without PCH and zero. >> >> Differential: >> http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.03.diff/ >> Full: >> http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.03/ >> >> Better now? >> >> Best regards, >> Roman >> >> >> >>> Thanks, >>> /Erik >>> >>> >>> On 2018-03-21 20:21, Roman Kennke wrote: >>>> Am 19.03.2018 um 12:07 schrieb Roman Kennke: >>>>> Am 19.03.2018 um 11:40 schrieb Roman Kennke: >>>>>> Currently, the mark word is accessed directly in oopDesc::mark() >>>>>> set_mark() and a bunch of other accessors. Those need to use the >>>>>> Access >>>>>> API instead because GC might want to employ barriers on those >>>>>> accesses, >>>>>> pretty much like every other field in oopDesc. >>>>>> >>>>>> Notice that this is *not* about accessing the bits and fields inside >>>>>> the >>>>>> markOop, but about accessing the header itself. >>>>>> >>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.01/ >>>>>> >>>>>> Testing: build fastdebug/release with and without PCH, passed tier1 >>>>>> fastdebug/release. >>>>>> >>>>>> Can I please get reviews? >>>>>> >>>>>> Thanks, Roman >>>>>> >>>>> Just when I sent it, I realized that this is dropping the volatile >>>>> from >>>>> the mark word access. Fixed here: >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.02/ >>>>> >>>> Ping? >>>> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Mon Mar 26 16:40:15 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 26 Mar 2018 18:40:15 +0200 Subject: RFR: JDK-8199735: Mark word updates need to use Access API In-Reply-To: <5AB90C07.3020507@oracle.com> References: <7bd73e32-a26b-2504-f069-b74807ae5e53@redhat.com> <859ac657-78c9-c7ab-0ec0-eddda331001b@redhat.com> <5AB3B052.4000304@oracle.com> <63199799-0194-a1d4-9c89-d95251c7aa7d@redhat.com> <5AB90C07.3020507@oracle.com> Message-ID: <4cdc8f4d-3f27-5c20-a606-251077ed9a82@redhat.com> Oh and I need one more review :-) Roman > This looks good to me. Thank you for doing this. > The only thing I am a bit puzzled by is why the oopDesc::set_mark member > function takes a volatile markOop as parameter. That seems rather weird. > But that is unrelated to your changes. > > Thanks, > /Erik > > On 2018-03-25 23:03, Roman Kennke wrote: >> Hi Erik, >> >> Thanks for having a look. >> >>> A few comments: >>> >>> 1) Please make sure all files referencing these accessors that have been >>> moved to the oop.inline.hpp file now #include the oop.inline.hpp file. I >>> found that there are quite a few missing by looking for references to >>> oopDesc::mark() with rtags. >>> 2) I think the mark_addr() function should now be called mark_addr_raw() >>> for consistency. >>> 3) I wonder if there should be _raw() accessors that the GCs use >>> instead. It feels like the GC should not go through the Access API to >>> acquire mark words. >> You are of course right in all 3 points. Unfortunately, that causes >> massive ripples through the code base ;-) >> >> So here is what I did: I added _raw() variants for stuff that is used by >> GC. I temporarily renamed all relevant methods, e.g. mark() -> >> mark_fluff() to make the compiler help me. Then for each occurance of >> any such call decided whether it should call the usual method, or the >> raw variant (as a general rule, everything under src/hotspot/share/gc >> uses the _raw() variants). Also, I added #include "oop.inline.hpp" for >> every such occurance. Then I renamed all _fluff() methods and calls back >> to usual. Did this exercise with and without PCH, with and without zero. >> Checked with grep if I might have missed some. >> >> Notes: bytecodeInterpreter.cpp needed some additional fixes to use >> cas_set_mark() instead of the Atomic::cmpxchg(). >> >> There are a bunch of methods in oop.hpp that are only used by GC, like >> is_locked() etc. I haven't changed them to _raw(). >> >> This passes tier1 tests release+fastdebug, and as mentioned above builds >> with and without PCH and zero. >> >> Differential: >> http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.03.diff/ >> Full: >> http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.03/ >> >> Better now? >> >> Best regards, >> Roman >> >> >> >>> Thanks, >>> /Erik >>> >>> >>> On 2018-03-21 20:21, Roman Kennke wrote: >>>> Am 19.03.2018 um 12:07 schrieb Roman Kennke: >>>>> Am 19.03.2018 um 11:40 schrieb Roman Kennke: >>>>>> Currently, the mark word is accessed directly in oopDesc::mark() >>>>>> set_mark() and a bunch of other accessors. Those need to use the >>>>>> Access >>>>>> API instead because GC might want to employ barriers on those >>>>>> accesses, >>>>>> pretty much like every other field in oopDesc. >>>>>> >>>>>> Notice that this is *not* about accessing the bits and fields inside >>>>>> the >>>>>> markOop, but about accessing the header itself. >>>>>> >>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.01/ >>>>>> >>>>>> Testing: build fastdebug/release with and without PCH, passed tier1 >>>>>> fastdebug/release. >>>>>> >>>>>> Can I please get reviews? >>>>>> >>>>>> Thanks, Roman >>>>>> >>>>> Just when I sent it, I realized that this is dropping the volatile >>>>> from >>>>> the mark word access. Fixed here: >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.02/ >>>>> >>>> Ping? >>>> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Mon Mar 26 17:55:16 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 26 Mar 2018 19:55:16 +0200 Subject: RFR (XS): 8193067: gc/g1/TestVerifyGCType.java still unstable In-Reply-To: <1522057264.2471.13.camel@oracle.com> References: <1522057264.2471.13.camel@oracle.com> Message-ID: On 03/26/2018 11:41 AM, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this small change to the TestVerifyGCType test > to make it stable? The problem has been that due to different number of > threads used, different amount of heap waste has been "produced", only > randomly triggering the expected mixed gcs. > > By fixing the number of threads used and decreasing G1HeapWastePercent > the mixed gcs can be forced reproducably. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8193067 > > Webrev: > http://cr.openjdk.java.net/~tschatzl/8193067/webrev/ Looks good! /Per > > Testing: > failing test passing multiple times in CI > > Thanks, > Thomas > From thomas.schatzl at oracle.com Mon Mar 26 18:06:13 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 26 Mar 2018 20:06:13 +0200 Subject: RFR (XS): 8193067: gc/g1/TestVerifyGCType.java still unstable In-Reply-To: References: <1522057264.2471.13.camel@oracle.com> Message-ID: <1522087573.2567.49.camel@oracle.com> Hi Per, On Mon, 2018-03-26 at 19:55 +0200, Per Liden wrote: > On 03/26/2018 11:41 AM, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this small change to the TestVerifyGCType > > test to make it stable? The problem has been that due to different > > number of threads used, different amount of heap waste has been > > "produced", only randomly triggering the expected mixed gcs. > > > > By fixing the number of threads used and decreasing > > G1HeapWastePercent the mixed gcs can be forced reproducably. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8193067 > > > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8193067/webrev/ > > Looks good! > thanks for your review. Thomas From thomas.schatzl at oracle.com Mon Mar 26 18:17:30 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 26 Mar 2018 20:17:30 +0200 Subject: RFR (S): 8200074: Remove G1ConcurrentMark::_concurrent_marking_in_progress Message-ID: <1522088250.2567.55.camel@oracle.com> Hi all, can I have reviews for the following change that removes G1ConcurrentMark::_concurrent_marking_in_progress since it is only used in some asserts, and at the same time mostly the same as G1ConcurrentMark::_concurrent (ie. it is used to verify _concurrent). CR: https://bugs.openjdk.java.net/browse/JDK-8200074 Webrev: http://cr.openjdk.java.net/~tschatzl/8200074/webrev Testing: hs-tier 1-5 with other changes Change needs JDK-8200255 to apply cleanly. Thanks, Thomas From thomas.schatzl at oracle.com Mon Mar 26 18:25:54 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 26 Mar 2018 20:25:54 +0200 Subject: RFR (S): 8200255: Remove G1CMTask::_concurrent Message-ID: <1522088754.2567.56.camel@oracle.com> Hi all, can I have reviews for this small change that removes G1CMTask::_concurrent. That member is a duplicate of G1ConcurrentMark::_concurrent, but there does not seem to be any advantage of keeping this duplicate, as it's only used in asserts or non-performance critical code. This removes some imho unnecessary code, so I suggest to do that. CR: https://bugs.openjdk.java.net/browse/JDK-8200255 Webrev: http://cr.openjdk.java.net/~tschatzl/8200255/webrev/ Testing: local compilation, gcbasher, testing with other changes Thanks, Thomas From john.r.rose at oracle.com Mon Mar 26 19:11:49 2018 From: john.r.rose at oracle.com (John Rose) Date: Mon, 26 Mar 2018 12:11:49 -0700 Subject: RFR: JDK-8199781: Don't use naked == for comparing oops In-Reply-To: <2683d96d-4170-f8b7-a48d-12fd0465ec4c@redhat.com> References: <74ca6651-49aa-2b77-d7e9-a588fb2b8417@oracle.com> <096620eb-e252-9689-c157-098314ac9119@redhat.com> <650c2e5c-88d7-3896-1526-386557f5d2df@redhat.com> <2683d96d-4170-f8b7-a48d-12fd0465ec4c@redhat.com> Message-ID: On Mar 26, 2018, at 6:16 AM, Roman Kennke wrote: > > However, in order to not cause regressions in any code I might have > overlooked, I did change the new stuff to inlining, and hunted down the > places where it breaks compilation. Thank you; I think this is the right step even if there is evidence that an out-of-line oopDesc::equals might be harmless, in today's code base with today's benchmarks. HotSpot is built (today) using the C++ performance model where inlining is the safe way to boil down abstract operations to concrete one. In the case of operator==, the concrete operation is a single instruction (or sometimes no instructions at all, with a clever optimizer). We know it's often harmless to replace such an optimizable operation with a much heavier one (a function call that won't be inlined, in this case). But, in the case of *very simple* concrete operations like op==, there is always a risk of a future well-meant code change causing a sudden and surprising performance regression, as well as a rarely taken code path suddenly failing to scale in the field. So our customary treatment of very simple concrete operations is to abstract them however we like, but express their abstractions in terms of inlines, wherever there might be risk of performance surprises. This is clearly one of those cases. Let's code against surprises (both present and future) by abstracting simpler operations with inlines. ? John -------------- next part -------------- An HTML attachment was scrubbed... URL: From rkennke at redhat.com Mon Mar 26 21:21:40 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 26 Mar 2018 23:21:40 +0200 Subject: RFR: JDK-8199781: Don't use naked == for comparing oops In-Reply-To: References: <74ca6651-49aa-2b77-d7e9-a588fb2b8417@oracle.com> <096620eb-e252-9689-c157-098314ac9119@redhat.com> <650c2e5c-88d7-3896-1526-386557f5d2df@redhat.com> <2683d96d-4170-f8b7-a48d-12fd0465ec4c@redhat.com> Message-ID: <1fa7bccc-d827-0a6e-ecbb-8ac1352a6449@redhat.com> Am 26.03.2018 um 21:11 schrieb John Rose: > On Mar 26, 2018, at 6:16 AM, Roman Kennke > wrote: >> >> However, in order to not cause regressions in any code I might have >> overlooked, I did change the new stuff to inlining, and hunted down the >> places where it breaks compilation. > > Thank you; I think this is the right step even if there is evidence > that an out-of-line oopDesc::equals might be harmless, in today's > code base with today's benchmarks. > > HotSpot is built (today) using the C++ performance model where > inlining is the safe way to boil down abstract operations to concrete > one. ?In the case of operator==, the concrete operation is a single > instruction (or sometimes no instructions at all, with a clever optimizer). > > We know it's often harmless to replace such an optimizable operation > with a much heavier one (a function call that won't be inlined, in this > case). ?But, in the case of *very simple* concrete operations like op==, > there is always a risk of a future well-meant code change causing a > sudden and surprising performance regression, as well as a rarely > taken code path suddenly failing to scale in the field. > > So our customary treatment of very simple concrete operations > is to abstract them however we like, but express their abstractions > in terms of inlines, wherever there might be risk of performance > surprises. > > This is clearly one of those cases. ?Let's code against surprises > (both present and future) by abstracting simpler operations with > inlines. Thanks John for clarifying. I generally do agree with you that it makes sense to inline something like an operator that will compile to a single instruction. Unfortunately, this new abstraction will generate a non-inlineable call. Thanks Erik's work with the Access API, it should be a straight call to the right implementation, not even a virtual call, but it's a call nonetheless. The reason why I'm proposing it is that the GC might want to have a say about object equality. In the case of Shenandoah, there may be two copies of any one object floating around, and comparing the two copies might otherwise result in false negatives. So... with inlining oopDesc::equals() we will get one straight call (from the caller of oopDesc::equals() to the impl), instead of 2 (caller -> oopDesc::equals() -> impl). It's the best I could come up with that does what (Shenandoah) GC needs, and it hasn't shown up in any performance testing that we did, yet. Still good? Thanks and best regards, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From sangheon.kim at oracle.com Mon Mar 26 22:38:16 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Mon, 26 Mar 2018 15:38:16 -0700 Subject: RFR [S][5/7]: 8197573: Remove concurrent cleanup and secondary free list handling In-Reply-To: <1520515912.3121.23.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520262995.2532.99.camel@oracle.com> <1520510384.3121.17.camel@oracle.com> <1520515912.3121.23.camel@oracle.com> Message-ID: Hi Thomas, On 03/08/2018 05:31 AM, Thomas Schatzl wrote: > Hi, > > On Thu, 2018-03-08 at 13:28 +0100, Stefan Johansson wrote: >> On 2018-03-08 12:59, Thomas Schatzl wrote: >>> Hi, >>> >>> On Mon, 2018-03-05 at 16:16 +0100, Thomas Schatzl wrote: >>>> Hi all, >>>> >>>> can I have reviews for this change that removes the concurrent >>>> cleanup phase and the secondary free list handling entirely. It >>>> moves leftover work into the Cleanup pause. >>>> >>>> In the previous change the remembered set scrubbing, which took >>>> in cases where it mattered like 95%+ of time, has been removed >>>> because it is not required any more. >>>> >>>> The reasons for removal are: >>>> - due to JDK-8180415 *freeing* remembered sets (as opposed to >>>> calculating occupancy) is really fast, so this phase has never >>>> been very long for a long time. >>>> - it adds a lock in the LAB allocation path >>>> >>>> It also allows us to remove the "mixed_gc_pending" flag in the >>>> CollectorState, as there is no further time to wait until G1 can >>>> schedule the "last young gc". >>>> >>>> CR: >>>> https://bugs.openjdk.java.net/browse/JDK-8197573 >>>> Webrev: >>>> http://cr.openjdk.java.net/~tschatzl/8197573/webrev/index.html >>>> Testing: >>>> hs-tier 1-3, .... >>>> >>>> (I consider this an "S" changeset because it is mostly removal of >>>> code). >>>> >>> Some ripple-through after latest changes to JDK-8180415 requires an >>> update of the webrevs: >>> http://cr.openjdk.java.net/~tschatzl/8197573/webrev.0_to_1 (diff) >>> http://cr.openjdk.java.net/~tschatzl/8197573/webrev.1 (full) >> Looks lovely, just some minor things: >> src/hotspot/share/gc/g1/concurrentMarkThread.cpp >> 436 log_info(gc, marking)("Concurrent Mark Abort"); >> >> This line was removed, and it looks like will never log a concurrent >> mark abort now. Looks like this could be added to >> G1ConcurrentMark::concurrent_cycle_end() where we trace a potential >> cm failure. > Nice catch! > >> --- >> src/hotspot/share/gc/g1/g1FullGCPrepareTask.cpp >> 106 _g1h->free_humongous_region(hr, &dummy_free_list); >> >> Should be safe to call free_humongous_region with locked = true >> here, since this will be the only thread working on that region. > Fixed. > >> 115 _g1h->card_table()->clear(MemRegion(hr->bottom(), hr->end())); >> >> Pre-existing, but since you have added the clear_cardtable() function >> we could use it here as well, seems to be the only place where the >> old "scheme" is used. > Done. > >> --- >> >>> I also found some more opportunity to do minor cleanup after >>> Stefan's >>> last remark about log messages... >> Nice, and you also restructured the code exactly like I planned to >> comment :) > :) > > New webrevs: > http://cr.openjdk.java.net/~tschatzl/8197573/webrev.1_to_2/ (diff) > http://cr.openjdk.java.net/~tschatzl/8197573/webrev.2/ (full) webrev.2 looks good. I only have minor nits. =============================== src/hotspot/share/gc/g1/concurrentMarkThread.cpp ?362?????? if (!cm()->has_aborted()) { ... ?364?????? } ?365 ?366?????? if (!cm()->has_aborted()) { - Any reason for having 2 if clause at line:362 and 366? =============================== src/hotspot/share/gc/g1/g1CollectedHeap.cpp (from original file) 1030?? // following two methods. - Do you know which method is referring originally? I guess wait_while_free_regions_coming() and _cm->root_regions()->abort()? And this is why you didn't remove the comment above wait_while_free_regions_coming(), right? =============================== src/hotspot/share/gc/g1/heapRegionSet.cpp - Copyright year update Thanks, Sangheon > > Thanks, > Thomas > From sangheon.kim at oracle.com Mon Mar 26 22:43:38 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Mon, 26 Mar 2018 15:43:38 -0700 Subject: RFR [XS][6/7]: 8197928: Only enqueue deferred cards with references into regions that have a tracked remembered set during GC In-Reply-To: <1520263515.2532.106.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520263515.2532.106.camel@oracle.com> Message-ID: <131b5761-df61-de09-fe89-dc20c1b8d7b0@oracle.com> Hi Thomas, On 03/05/2018 07:25 AM, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this tiny change that optimizes evacuation a > little: in particular, when updating the remembered sets (enqueuing > cards), we do not need to do this for regions where we do not track the > remembered sets. > > The suggested implementation is rather primitive, I have not noticed > pause time impact on doing so (of course it can lessen the concurrent > work later). > > An alternative implementation I considered has been to create a per- > region map with a single flag that contains the "needs-update/does-not- > need-update" information, like the _fast_in_cset_state table. > > However creating such a table requires G1 to iterate over all regions > before evacuation, and that would be something that needs to be added > as G1 does not do that yet. > > Another would be to move remembered set state/remembered sets outside > of the region, so it can be accessed quickly during GC. > > In any case I thought this implementation is better than not doing it, > hoping for some input here. :) > > CR: > https://bugs.openjdk.java.net/browse/JDK-8197928 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8197928/webrev/ Looks good to me too. Thanks, Sangheon > Testing: > hs-tier1-5, ... > > Thanks, > Thomas > From john.r.rose at oracle.com Mon Mar 26 22:46:20 2018 From: john.r.rose at oracle.com (John Rose) Date: Mon, 26 Mar 2018 15:46:20 -0700 Subject: RFR: JDK-8199781: Don't use naked == for comparing oops In-Reply-To: <1fa7bccc-d827-0a6e-ecbb-8ac1352a6449@redhat.com> References: <74ca6651-49aa-2b77-d7e9-a588fb2b8417@oracle.com> <096620eb-e252-9689-c157-098314ac9119@redhat.com> <650c2e5c-88d7-3896-1526-386557f5d2df@redhat.com> <2683d96d-4170-f8b7-a48d-12fd0465ec4c@redhat.com> <1fa7bccc-d827-0a6e-ecbb-8ac1352a6449@redhat.com> Message-ID: <25E3C8FE-089E-4AB8-A9B4-1DA48B2FF17D@oracle.com> On Mar 26, 2018, at 2:21 PM, Roman Kennke wrote: > > Am 26.03.2018 um 21:11 schrieb John Rose: >> ... >> This is clearly one of those cases. Let's code against surprises >> (both present and future) by abstracting simpler operations with >> inlines. > > Thanks John for clarifying. I generally do agree with you that it makes > sense to inline something like an operator that will compile to a single > instruction. > > Unfortunately, this new abstraction will generate a non-inlineable call. > Thanks Erik's work with the Access API, it should be a straight call to > the right implementation, not even a virtual call, but it's a call > nonetheless. What would it take to inline the call, so there's no new performance hazard introduced? This might be a place to add #ifdefs, if the templates don't let us control the decision statically. (I confess I find the templates complex and hard to reason about. But that's a potential maintenance issue with the access API, not your work.) > The reason why I'm proposing it is that the GC might want to have a say > about object equality. In the case of Shenandoah, there may be two > copies of any one object floating around, and comparing the two copies > might otherwise result in false negatives. 1. Of course we need a way for some GC algorithms to control object identity. 2. It's good coding discipline to find all the places where native op== is used and upgrade them to the needed control. (2a. We also need a way to avoid having op== creep back in in an uncontrolled way.) 3. We still need an unsurprising performance model for GC's which don't require the extra identity hook. So we're not there yet, I think. If there's no easy way to adjust the access API to get the last inlining, I suggest adding an #ifdef into oopDesc::equals. I suspect the happy final solution will be to templatize hot loops using making the inlinable adjusted op== be part of the "decorations" of those loops. Then we will have two copies of hot loops in the JVM, one with the out-of-line hook and one without. For now, I think it's enough to have separate builds of the JVM, one with the #ifdef set "the old way" and one that allows more flexibility. (Is this the first time we've run into an occasion to make a separate Shenandaoah build? Surely not.) > So... with inlining oopDesc::equals() we will get one straight call > (from the caller of oopDesc::equals() to the impl), instead of 2 (caller > -> oopDesc::equals() -> impl). > > It's the best I could come up with that does what (Shenandoah) GC needs, > and it hasn't shown up in any performance testing that we did, yet. > > Still good? I think it's too risky to out-of-line the "cmpq" instruction. Let's find an agreeable way to handle that via a trade-off that keeps the old code shape in VM builds which don't need the new code shape. I'm speaking up here because I don't believe this is the last time we'll have to consider adding something gross like an #ifdef to support new features, and I want to aim at trade-offs that keep the existing hot paths efficient, while still allowing new code shapes to become part of the mix. (I'd be happy to see something cleaner than an #ifdef, of course. Seems to me that a constant non-product flag -XX:+ShortCircuitOopEquals would work too, with a binding of falseInShenandoah.) ? John From leonid.mesnik at oracle.com Mon Mar 26 23:32:01 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Mon, 26 Mar 2018 16:32:01 -0700 Subject: RFR(XS): 8200187: Exclude 3 long-running tests from tier1 In-Reply-To: References: Message-ID: Hi There is no strict rules and time budgets for hotspot tiers. It is assumed that faster tests should be executed earlier tiers. The order and total impact of of these test depends on how they are executed. Here is time of execution of tier1 groups with and without these excluded tests: The whole :tier1 time reduced from 40 to 30 minutes (on dedicated HW with 32 core) Time for :tier1_serviceability reduced from 15 to 6 min (in HS CI) Time for :tier1_gc_1 reduced from 12 to 8-10 min (in HS CI) The benefits of exclusion of gc/g1/ihop/TestIHOPErgo.java are not so significant. However it is the only one GC stress test executed in tier1 while intention is to don?t run stress testing in tier1. Tests serviceability/sa/ClhsdbScanOops.java serviceability/sa/TestHeapDumpForLargeArray.java are now hotspot_tier3_runtime which includes all hotspot_serviceability tests which are not a part of tier1. They are executed as a part of tier3 now as well as most of serviceability tests. Test gc/g1/ihop/TestIHOPErgo.java in now in group :hotspot_gc only and also it is marked as ?stress? test. So it is executed with all other GC stress tests Leonid > On Mar 25, 2018, at 3:48 PM, David Holmes wrote: > > Hi Leonid, > > On 24/03/2018 9:31 AM, Leonid Mesnik wrote: >> Hi >> Could you please review following fix which exclude following tests from tier1 testing: >> serviceability/sa/ClhsdbScanOops.java >> serviceability/sa/TestHeapDumpForLargeArray.java >> gc/g1/ihop/TestIHOPErgo.java >> Each of them takes more then 5 minutes to complete and significantly increase overall time to complete tier1. > > I'd need to see a much more detailed analysis of all the tests run, the order they run and the execution times to ascertain what impact this actually has on overall test execution time. > > But assuming 5 minutes is too long for tier1, this seems okay. But the tests must still be run in some other tier, and I'm not clear where that would be now? I would expect them to move to tier 3 perhaps, depending on what the time criteria for tier 3 is. > > Thanks, > David > >> Please let me know if there are any reasons to run these tests in tier1 despite on their execution time. >> webrev:http://cr.openjdk.java.net/~lmesnik/8200187/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8200187 >> Leonid From david.holmes at oracle.com Tue Mar 27 00:53:32 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 27 Mar 2018 10:53:32 +1000 Subject: RFR(XS): 8200187: Exclude 3 long-running tests from tier1 In-Reply-To: References: Message-ID: Thanks for that Leonid. As long as they are still included somewhere that is fine. David On 27/03/2018 9:32 AM, Leonid Mesnik wrote: > Hi > > There is no strict rules and time budgets for hotspot tiers. It is assumed that faster tests should be executed earlier tiers. The order and total impact of of these test depends on how they are executed. > > Here is time of execution of tier1 groups with and without these excluded tests: > The whole :tier1 time reduced from 40 to 30 minutes (on dedicated HW with 32 core) > Time for :tier1_serviceability reduced from 15 to 6 min (in HS CI) > Time for :tier1_gc_1 reduced from 12 to 8-10 min (in HS CI) > > The benefits of exclusion of gc/g1/ihop/TestIHOPErgo.java are not so significant. However it is the only one GC stress test executed in tier1 while intention is to don?t run stress testing in tier1. > > Tests > serviceability/sa/ClhsdbScanOops.java > serviceability/sa/TestHeapDumpForLargeArray.java > are now hotspot_tier3_runtime which includes all hotspot_serviceability tests which are not a part of tier1. They are executed as a part of tier3 now as well as most of serviceability tests. > > Test gc/g1/ihop/TestIHOPErgo.java in now in group :hotspot_gc only and also it is marked as ?stress? test. So it is executed with all other GC stress tests > > Leonid > >> On Mar 25, 2018, at 3:48 PM, David Holmes wrote: >> >> Hi Leonid, >> >> On 24/03/2018 9:31 AM, Leonid Mesnik wrote: >>> Hi >>> Could you please review following fix which exclude following tests from tier1 testing: >>> serviceability/sa/ClhsdbScanOops.java >>> serviceability/sa/TestHeapDumpForLargeArray.java >>> gc/g1/ihop/TestIHOPErgo.java >>> Each of them takes more then 5 minutes to complete and significantly increase overall time to complete tier1. >> >> I'd need to see a much more detailed analysis of all the tests run, the order they run and the execution times to ascertain what impact this actually has on overall test execution time. >> >> But assuming 5 minutes is too long for tier1, this seems okay. But the tests must still be run in some other tier, and I'm not clear where that would be now? I would expect them to move to tier 3 perhaps, depending on what the time criteria for tier 3 is. >> >> Thanks, >> David >> >>> Please let me know if there are any reasons to run these tests in tier1 despite on their execution time. >>> webrev:http://cr.openjdk.java.net/~lmesnik/8200187/webrev.00/ >>> bug: https://bugs.openjdk.java.net/browse/JDK-8200187 >>> Leonid > From rkennke at redhat.com Tue Mar 27 07:14:28 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 27 Mar 2018 09:14:28 +0200 Subject: RFR: JDK-8199781: Don't use naked == for comparing oops In-Reply-To: <25E3C8FE-089E-4AB8-A9B4-1DA48B2FF17D@oracle.com> References: <74ca6651-49aa-2b77-d7e9-a588fb2b8417@oracle.com> <096620eb-e252-9689-c157-098314ac9119@redhat.com> <650c2e5c-88d7-3896-1526-386557f5d2df@redhat.com> <2683d96d-4170-f8b7-a48d-12fd0465ec4c@redhat.com> <1fa7bccc-d827-0a6e-ecbb-8ac1352a6449@redhat.com> <25E3C8FE-089E-4AB8-A9B4-1DA48B2FF17D@oracle.com> Message-ID: <5e3f8ffc-0eb9-e4da-d0c0-72becb677b7b@redhat.com> Am 27.03.2018 um 00:46 schrieb John Rose: > On Mar 26, 2018, at 2:21 PM, Roman Kennke wrote: >> >> Am 26.03.2018 um 21:11 schrieb John Rose: >>> ... >>> This is clearly one of those cases. Let's code against surprises >>> (both present and future) by abstracting simpler operations with >>> inlines. >> >> Thanks John for clarifying. I generally do agree with you that it makes >> sense to inline something like an operator that will compile to a single >> instruction. >> >> Unfortunately, this new abstraction will generate a non-inlineable call. >> Thanks Erik's work with the Access API, it should be a straight call to >> the right implementation, not even a virtual call, but it's a call >> nonetheless. > > What would it take to inline the call, so there's no new performance > hazard introduced? > > This might be a place to add #ifdefs, if the templates don't let us > control the decision statically. > > (I confess I find the templates complex and hard to reason about. > But that's a potential maintenance issue with the access API, not > your work.) The Access API supports static inclusion/exclusion of stuff, and we've used it before to ensable barriers on primitive field accesses for Shenandoah-enabled builds. It works by passing -DSUPPORT_BARRIER_ON_PRIMITIVES to gcc. This is much better than #ifdefs and does the same thing. I can add something similar for object equality. >> The reason why I'm proposing it is that the GC might want to have a say >> about object equality. In the case of Shenandoah, there may be two >> copies of any one object floating around, and comparing the two copies >> might otherwise result in false negatives. > > 1. Of course we need a way for some GC algorithms to control > object identity. > > 2. It's good coding discipline to find all the places where native > op== is used and upgrade them to the needed control. I hope I did that with the proposed patch (it's based on years of finding all those places in Shenandoah land ;-) ). > (2a. We also need a way to avoid having op== creep back in > in an uncontrolled way.) We have something for this in Shenandoah too. It works by overriding == in oopsHierarch.hpp and running into ShouldNotReachHere() if both operands are != NULL. This catches all uses of naked == at runtime, when running with +CheckUnhandledOops. It's not perfect, because it doesn't catch uses in places that are not executed in tests, but it's still incredibly useful. The caveat here is that we also need another abstraction for code paths that actually want naked ==, e.g. in the GC. In Shenandoah we have an oopDesc::unsafe_equals() for that. The other alternative would be to drop operators from oop to oopDesc* and compare that (which is exactly what oopDesc::unsafe_equals() does). > 3. We still need an unsurprising performance model for GC's > which don't require the extra identity hook. Yeah. Unfortunately this is C++ and not Java, and the decision which GC to use is made at runtime. The 'best' that I can see is to rip out the indirection in Shenandoah-disabled builds, as proposed above. I don't like it much, but if this is the only way we can get Shenandoah in, then let's go for it? > So we're not there yet, I think. If there's no easy way to adjust > the access API to get the last inlining, I suggest adding an #ifdef > into oopDesc::equals. > > I suspect the happy final solution will be to templatize hot loops > using making the inlinable adjusted op== be part of the "decorations" > of those loops. Then we will have two copies of hot loops in the > JVM, one with the out-of-line hook and one without. > > For now, I think it's enough to have separate builds of the JVM, > one with the #ifdef set "the old way" and one that allows more > flexibility. > > (Is this the first time we've run into an occasion to make a separate > Shenandaoah build? Surely not.) Primitive field accesses required similar jumping through hoops ;-) >> So... with inlining oopDesc::equals() we will get one straight call >> (from the caller of oopDesc::equals() to the impl), instead of 2 (caller >> -> oopDesc::equals() -> impl). >> >> It's the best I could come up with that does what (Shenandoah) GC needs, >> and it hasn't shown up in any performance testing that we did, yet. >> >> Still good? > > I think it's too risky to out-of-line the "cmpq" instruction. Let's > find an agreeable way to handle that via a trade-off that keeps > the old code shape in VM builds which don't need the new code > shape. I'm speaking up here because I don't believe this is the > last time we'll have to consider adding something gross like an > #ifdef to support new features, and I want to aim at trade-offs > that keep the existing hot paths efficient, while still allowing > new code shapes to become part of the mix. > > (I'd be happy to see something cleaner than an #ifdef, of > course. Seems to me that a constant non-product flag > -XX:+ShortCircuitOopEquals would work too, with a > binding of falseInShenandoah.) That would require coding 2 versions of hot loops that involve oop==oop and switch to one or the other based on that flag. I have checked all the uses of oopDesc::equals() in my patch, and none seemed particularily critical. I think this would be overkill. And notice also that none interpreter/c1/c2 compiled code will suffer this problem because we'll just generate the right code at runtime. Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From erik.osterlund at oracle.com Tue Mar 27 08:09:27 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 27 Mar 2018 10:09:27 +0200 Subject: RFR: JDK-8199781: Don't use naked == for comparing oops In-Reply-To: <5e3f8ffc-0eb9-e4da-d0c0-72becb677b7b@redhat.com> References: <74ca6651-49aa-2b77-d7e9-a588fb2b8417@oracle.com> <096620eb-e252-9689-c157-098314ac9119@redhat.com> <650c2e5c-88d7-3896-1526-386557f5d2df@redhat.com> <2683d96d-4170-f8b7-a48d-12fd0465ec4c@redhat.com> <1fa7bccc-d827-0a6e-ecbb-8ac1352a6449@redhat.com> <25E3C8FE-089E-4AB8-A9B4-1DA48B2FF17D@oracle.com> <5e3f8ffc-0eb9-e4da-d0c0-72becb677b7b@redhat.com> Message-ID: <5AB9FC37.8020508@oracle.com> Hi, This is indeed exactly what the build-time decorators were designed for. Specifically, the INTERNAL_BT_TO_SPACE_INVARIANT decorator fits the bill here. Access::equals should use the same hardwiring techniques that resolve did, where it inlines equals for builds that only have to-space invariant GCs. When Shenandoah is included in the build, there must be a static variation point. Then a function pointer call is performed. The overhead of that is small. As for the include .inline.hpp problems, it is worth mentioning that previous incarnations of the Access API (before it went out for review) had a mechanism for generating the required backend functions that the function pointers (that indeed require a variation point in a given build) resolve to in .cpp files, and hence allow a chosen subset of common Access operations to be used directly from access.hpp instead. The main motivation then was reducing build times. I later optimized my templates so the build times were no longer a problem, and therefore removed the support for doing that as it seemed no longer necessary. But I did not consider using it to reduce .inline.hpp dependencies. It sounds like this is worth a shot. If you would like this, I can revive the support for doing that. Then you can include only access.hpp for equals, and solve the include problems. Thanks, /Erik On 2018-03-27 09:14, Roman Kennke wrote: > Am 27.03.2018 um 00:46 schrieb John Rose: >> On Mar 26, 2018, at 2:21 PM, Roman Kennke wrote: >>> Am 26.03.2018 um 21:11 schrieb John Rose: >>>> ... >>>> This is clearly one of those cases. Let's code against surprises >>>> (both present and future) by abstracting simpler operations with >>>> inlines. >>> Thanks John for clarifying. I generally do agree with you that it makes >>> sense to inline something like an operator that will compile to a single >>> instruction. >>> >>> Unfortunately, this new abstraction will generate a non-inlineable call. >>> Thanks Erik's work with the Access API, it should be a straight call to >>> the right implementation, not even a virtual call, but it's a call >>> nonetheless. >> What would it take to inline the call, so there's no new performance >> hazard introduced? >> >> This might be a place to add #ifdefs, if the templates don't let us >> control the decision statically. >> >> (I confess I find the templates complex and hard to reason about. >> But that's a potential maintenance issue with the access API, not >> your work.) > The Access API supports static inclusion/exclusion of stuff, and we've > used it before to ensable barriers on primitive field accesses for > Shenandoah-enabled builds. It works by passing > -DSUPPORT_BARRIER_ON_PRIMITIVES to gcc. This is much better than #ifdefs > and does the same thing. I can add something similar for object equality. > >>> The reason why I'm proposing it is that the GC might want to have a say >>> about object equality. In the case of Shenandoah, there may be two >>> copies of any one object floating around, and comparing the two copies >>> might otherwise result in false negatives. >> 1. Of course we need a way for some GC algorithms to control >> object identity. >> >> 2. It's good coding discipline to find all the places where native >> op== is used and upgrade them to the needed control. > I hope I did that with the proposed patch (it's based on years of > finding all those places in Shenandoah land ;-) ). > >> (2a. We also need a way to avoid having op== creep back in >> in an uncontrolled way.) > We have something for this in Shenandoah too. It works by overriding == > in oopsHierarch.hpp and running into ShouldNotReachHere() if both > operands are != NULL. This catches all uses of naked == at runtime, when > running with +CheckUnhandledOops. It's not perfect, because it doesn't > catch uses in places that are not executed in tests, but it's still > incredibly useful. The caveat here is that we also need another > abstraction for code paths that actually want naked ==, e.g. in the GC. > In Shenandoah we have an oopDesc::unsafe_equals() for that. The other > alternative would be to drop operators from oop to oopDesc* and compare > that (which is exactly what oopDesc::unsafe_equals() does). > >> 3. We still need an unsurprising performance model for GC's >> which don't require the extra identity hook. > Yeah. Unfortunately this is C++ and not Java, and the decision which GC > to use is made at runtime. The 'best' that I can see is to rip out the > indirection in Shenandoah-disabled builds, as proposed above. I don't > like it much, but if this is the only way we can get Shenandoah in, then > let's go for it? > >> So we're not there yet, I think. If there's no easy way to adjust >> the access API to get the last inlining, I suggest adding an #ifdef >> into oopDesc::equals. >> >> I suspect the happy final solution will be to templatize hot loops >> using making the inlinable adjusted op== be part of the "decorations" >> of those loops. Then we will have two copies of hot loops in the >> JVM, one with the out-of-line hook and one without. >> >> For now, I think it's enough to have separate builds of the JVM, >> one with the #ifdef set "the old way" and one that allows more >> flexibility. >> >> (Is this the first time we've run into an occasion to make a separate >> Shenandaoah build? Surely not.) > Primitive field accesses required similar jumping through hoops ;-) > >>> So... with inlining oopDesc::equals() we will get one straight call >>> (from the caller of oopDesc::equals() to the impl), instead of 2 (caller >>> -> oopDesc::equals() -> impl). >>> >>> It's the best I could come up with that does what (Shenandoah) GC needs, >>> and it hasn't shown up in any performance testing that we did, yet. >>> >>> Still good? >> I think it's too risky to out-of-line the "cmpq" instruction. Let's >> find an agreeable way to handle that via a trade-off that keeps >> the old code shape in VM builds which don't need the new code >> shape. I'm speaking up here because I don't believe this is the >> last time we'll have to consider adding something gross like an >> #ifdef to support new features, and I want to aim at trade-offs >> that keep the existing hot paths efficient, while still allowing >> new code shapes to become part of the mix. >> >> (I'd be happy to see something cleaner than an #ifdef, of >> course. Seems to me that a constant non-product flag >> -XX:+ShortCircuitOopEquals would work too, with a >> binding of falseInShenandoah.) > That would require coding 2 versions of hot loops that involve oop==oop > and switch to one or the other based on that flag. I have checked all > the uses of oopDesc::equals() in my patch, and none seemed particularily > critical. I think this would be overkill. > > And notice also that none interpreter/c1/c2 compiled code will suffer > this problem because we'll just generate the right code at runtime. > > Roman > From thomas.schatzl at oracle.com Tue Mar 27 08:58:19 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 27 Mar 2018 10:58:19 +0200 Subject: RFR [S][5/7]: 8197573: Remove concurrent cleanup and secondary free list handling In-Reply-To: References: <1520258985.2532.40.camel@oracle.com> <1520262995.2532.99.camel@oracle.com> <1520510384.3121.17.camel@oracle.com> <1520515912.3121.23.camel@oracle.com> Message-ID: <1522141099.3162.6.camel@oracle.com> Hi Sangheon, thanks for your review. On Mon, 2018-03-26 at 15:38 -0700, sangheon.kim wrote: > Hi Thomas, [..] > > > > New webrevs: > > http://cr.openjdk.java.net/~tschatzl/8197573/webrev.1_to_2/ (diff) > > http://cr.openjdk.java.net/~tschatzl/8197573/webrev.2/ (full) > > webrev.2 looks good. I only have minor nits. > > =============================== > src/hotspot/share/gc/g1/concurrentMarkThread.cpp > 362 if (!cm()->has_aborted()) { > ... > 364 } > 365 > 366 if (!cm()->has_aborted()) { > - Any reason for having 2 if clause at line:362 and 366? > The delay might take a long while, so the abort state might have changed already. So it does not hurt checking to avoid going into the VM operation. > =============================== > src/hotspot/share/gc/g1/g1CollectedHeap.cpp (from original file) > 1030 // following two methods. > - Do you know which method is referring originally? I guess > wait_while_free_regions_coming() and _cm->root_regions()->abort()? > And this is why you didn't remove the comment above > wait_while_free_regions_coming(), right? As far as I understand the two methods were wait_while_free_regions_coming() and wait_until_scan_finished(). The latter is still there so I only adapted the comment. I removed the comment in the new webrev and filed JDK-8200291. We should not have todo-lists in the code. > > =============================== > src/hotspot/share/gc/g1/heapRegionSet.cpp > - Copyright year update > Fixed. New webrevs (just the comment removals and the copyright update): http://cr.openjdk.java.net/~tschatzl/8197573/webrev.2_to_3/ (diff) http://cr.openjdk.java.net/~tschatzl/8197573/webrev.3/ (full) Thanks, Thomas From thomas.schatzl at oracle.com Tue Mar 27 08:59:40 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 27 Mar 2018 10:59:40 +0200 Subject: RFR [XS][6/7]: 8197928: Only enqueue deferred cards with references into regions that have a tracked remembered set during GC In-Reply-To: <131b5761-df61-de09-fe89-dc20c1b8d7b0@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520263515.2532.106.camel@oracle.com> <131b5761-df61-de09-fe89-dc20c1b8d7b0@oracle.com> Message-ID: <1522141180.3162.7.camel@oracle.com> Hi Sangheon, Stefan, On Mon, 2018-03-26 at 15:43 -0700, sangheon.kim wrote: > Hi Thomas, > > On 03/05/2018 07:25 AM, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this tiny change that optimizes > > evacuation a little: in particular, when updating the remembered > > sets (enqueuing cards), we do not need to do this for regions where > > we do not track the remembered sets. [...] > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8197928 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8197928/webrev/ > > Looks good to me too. thanks for your reviews, Thomas From stefan.karlsson at oracle.com Tue Mar 27 09:05:04 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 27 Mar 2018 11:05:04 +0200 Subject: RFR: 8200292: Move PushAndMarkVerifyClosure::do_oop_work to concurrentMarkSweepGeneration.cpp Message-ID: Hi all, Please review this trivial patch to move PushAndMarkVerifyClosure::do_oop_work to concurrentMarkSweepGeneration.cpp. This allows us to get rid of the access.inline.hp include in concurrentMarkSweepGeneration.hpp. http://cr.openjdk.java.net/~stefank/8200292/webrev.01/ Thanks, StefanK From thomas.schatzl at oracle.com Tue Mar 27 09:11:23 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 27 Mar 2018 11:11:23 +0200 Subject: RFR: 8200292: Move PushAndMarkVerifyClosure::do_oop_work to concurrentMarkSweepGeneration.cpp In-Reply-To: References: Message-ID: <1522141883.3162.9.camel@oracle.com> Hi, On Tue, 2018-03-27 at 11:05 +0200, Stefan Karlsson wrote: > Hi all, > > Please review this trivial patch to move > PushAndMarkVerifyClosure::do_oop_work to > concurrentMarkSweepGeneration.cpp. This allows us to get rid of the > access.inline.hp include in concurrentMarkSweepGeneration.hpp. > > http://cr.openjdk.java.net/~stefank/8200292/webrev.01/ looks good. Thomas From stefan.karlsson at oracle.com Tue Mar 27 09:14:04 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 27 Mar 2018 11:14:04 +0200 Subject: RFR: 8200292: Move PushAndMarkVerifyClosure::do_oop_work to concurrentMarkSweepGeneration.cpp In-Reply-To: <1522141883.3162.9.camel@oracle.com> References: <1522141883.3162.9.camel@oracle.com> Message-ID: <78390fba-1d55-2f10-da4e-5e805f85fb3f@oracle.com> Thanks Thomas. StefanK On 2018-03-27 11:11, Thomas Schatzl wrote: > Hi, > > On Tue, 2018-03-27 at 11:05 +0200, Stefan Karlsson wrote: >> Hi all, >> >> Please review this trivial patch to move >> PushAndMarkVerifyClosure::do_oop_work to >> concurrentMarkSweepGeneration.cpp. This allows us to get rid of the >> access.inline.hp include in concurrentMarkSweepGeneration.hpp. >> >> http://cr.openjdk.java.net/~stefank/8200292/webrev.01/ > > looks good. > > Thomas > From stefan.johansson at oracle.com Tue Mar 27 09:16:29 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 27 Mar 2018 11:16:29 +0200 Subject: RFR: 8200292: Move PushAndMarkVerifyClosure::do_oop_work to concurrentMarkSweepGeneration.cpp In-Reply-To: References: Message-ID: <78d9a224-29a7-df3e-3864-d9b6eea955e0@oracle.com> Looks good! StefanJ On 2018-03-27 11:05, Stefan Karlsson wrote: > Hi all, > > Please review this trivial patch to move > PushAndMarkVerifyClosure::do_oop_work to > concurrentMarkSweepGeneration.cpp. This allows us to get rid of the > access.inline.hp include in concurrentMarkSweepGeneration.hpp. > > http://cr.openjdk.java.net/~stefank/8200292/webrev.01/ > > Thanks, > StefanK From stefan.johansson at oracle.com Tue Mar 27 09:20:19 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 27 Mar 2018 11:20:19 +0200 Subject: RFR [S][5/7]: 8197573: Remove concurrent cleanup and secondary free list handling In-Reply-To: <1522141099.3162.6.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520262995.2532.99.camel@oracle.com> <1520510384.3121.17.camel@oracle.com> <1520515912.3121.23.camel@oracle.com> <1522141099.3162.6.camel@oracle.com> Message-ID: On 2018-03-27 10:58, Thomas Schatzl wrote: > Hi Sangheon, > > thanks for your review. > > On Mon, 2018-03-26 at 15:38 -0700, sangheon.kim wrote: >> Hi Thomas, > [..] >>> New webrevs: >>> http://cr.openjdk.java.net/~tschatzl/8197573/webrev.1_to_2/ (diff) >>> http://cr.openjdk.java.net/~tschatzl/8197573/webrev.2/ (full) >> webrev.2 looks good. I only have minor nits. >> >> =============================== >> src/hotspot/share/gc/g1/concurrentMarkThread.cpp >> 362 if (!cm()->has_aborted()) { >> ... >> 364 } >> 365 >> 366 if (!cm()->has_aborted()) { >> - Any reason for having 2 if clause at line:362 and 366? >> > The delay might take a long while, so the abort state might have > changed already. So it does not hurt checking to avoid going into the > VM operation. > >> =============================== >> src/hotspot/share/gc/g1/g1CollectedHeap.cpp (from original file) >> 1030 // following two methods. >> - Do you know which method is referring originally? I guess >> wait_while_free_regions_coming() and _cm->root_regions()->abort()? >> And this is why you didn't remove the comment above >> wait_while_free_regions_coming(), right? > As far as I understand the two methods were > wait_while_free_regions_coming() and wait_until_scan_finished(). The > latter is still there so I only adapted the comment. > > I removed the comment in the new webrev and filed JDK-8200291. We > should not have todo-lists in the code. > >> =============================== >> src/hotspot/share/gc/g1/heapRegionSet.cpp >> - Copyright year update >> > Fixed. New webrevs (just the comment removals and the copyright > update): > > http://cr.openjdk.java.net/~tschatzl/8197573/webrev.2_to_3/ (diff) > http://cr.openjdk.java.net/~tschatzl/8197573/webrev.3/ (full) Still good! StefanJ > Thanks, > Thomas > From stefan.karlsson at oracle.com Tue Mar 27 09:21:53 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 27 Mar 2018 11:21:53 +0200 Subject: RFR: 8200292: Move PushAndMarkVerifyClosure::do_oop_work to concurrentMarkSweepGeneration.cpp In-Reply-To: <78d9a224-29a7-df3e-3864-d9b6eea955e0@oracle.com> References: <78d9a224-29a7-df3e-3864-d9b6eea955e0@oracle.com> Message-ID: Thanks StefanJ. StefanK On 2018-03-27 11:16, Stefan Johansson wrote: > Looks good! > StefanJ > > On 2018-03-27 11:05, Stefan Karlsson wrote: >> Hi all, >> >> Please review this trivial patch to move >> PushAndMarkVerifyClosure::do_oop_work to >> concurrentMarkSweepGeneration.cpp. This allows us to get rid of the >> access.inline.hp include in concurrentMarkSweepGeneration.hpp. >> >> http://cr.openjdk.java.net/~stefank/8200292/webrev.01/ >> >> Thanks, >> StefanK > From stefan.johansson at oracle.com Tue Mar 27 12:58:48 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 27 Mar 2018 14:58:48 +0200 Subject: RFR (M): 8199742: Clean up state flags in G1CollectorState In-Reply-To: <1522076812.2567.21.camel@oracle.com> References: <1522076812.2567.21.camel@oracle.com> Message-ID: <9f028672-bc86-525e-3c19-a3f347b1c118@oracle.com> Hi Thomas, Thanks for this much needed cleanup :) On 2018-03-26 17:06, Thomas Schatzl wrote: > Hi all, > > I would like to request reviews for this change that cleans up the > flags in G1CollectorState, applying uniform naming, removing members > that were basically temporary variables for a single method, and remove > redundant flags (the four flags in the "XXX" block were basically a > single one). > > While this is just a step towards my goal of removing the flags as much > as possible and merging them into enums, I think this is a very > worthwhile step on that way. Further it blocks a few more changes of > mine :P > > Naming guidelines: > - flags indicating a particular GC were named "in_XXX_gc" > - durations spanning multiple garbage collections, i.e. phases have > the "phase" postfix. > > Other than that there were general updates for the names to hopefully > make them reflect reality a bit closer. > > Main changes apart from naming were > > - in g1CollectionSet.cpp where I made add_young_region_common() to not > update statistics. This avoids the need to make the full gc also a > young gc (at least as far as flags are concerned); to fix this properly > we would need to know the previous phase we were in when coming into > the full gc. I just do not think it is worth adding a new state here > (and the code has been wrong previously too, unconditionally adding > these statistics to the young-only phase). > > I made a note in the follow-up change JDK-8080226. > > - the changes in G1Policy::record_collection_pause_end() were mostly > due to removing the last_gc_was_young() member which was basically a > reminder just for this method that the current gc was a young-only gc. > > There is still _initiate_conc_mark_if_possible flag that indicates that > a state change to initial mark should occur asap, but I think that > should be moved to e.g. g1policy in a next change. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8199742 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8199742/webrev The change looks good. Just some minor things, gc_are_young and during_im are used as local variables at some places. I think we could change those to reflect the new better names as well. Thanks, Stefan > Testing: > hs-tier 1-5 > > These changes are based on the rebuild remembered sets concurrently > changes. As it may be a bit of work to get them working (I already > pushed to first four), here is a webrev that updates latest jdk/hs to > that level at http://cr.openjdk.java.net/~tschatzl/8199742/baseline/web > rev/ > > Thanks, > Thomas > From thomas.schatzl at oracle.com Tue Mar 27 13:25:20 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 27 Mar 2018 15:25:20 +0200 Subject: RFR (M): 8199742: Clean up state flags in G1CollectorState In-Reply-To: <9f028672-bc86-525e-3c19-a3f347b1c118@oracle.com> References: <1522076812.2567.21.camel@oracle.com> <9f028672-bc86-525e-3c19-a3f347b1c118@oracle.com> Message-ID: <1522157120.28255.2.camel@oracle.com> Hi Stefan, On Tue, 2018-03-27 at 14:58 +0200, Stefan Johansson wrote: > Hi Thomas, > > Thanks for this much needed cleanup :) > > On 2018-03-26 17:06, Thomas Schatzl wrote: > > Hi all, > > > > I would like to request reviews for this change that cleans up > > the flags in G1CollectorState, applying uniform naming, removing > > members that were basically temporary variables for a single > > method, and [...] > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8199742 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8199742/webrev > > The change looks good. Just some minor things, gc_are_young and > during_im are used as local variables at some places. I think we > could change those to reflect the new better names as well. I grepped through the sources for the old names and changed the uses to something better (I hope). http://cr.openjdk.java.net/~tschatzl/8199742/webrev.0_to_1 (diff) http://cr.openjdk.java.net/~tschatzl/8199742/webrev.1 (full) Thanks, Thomas From stefan.johansson at oracle.com Tue Mar 27 13:48:28 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 27 Mar 2018 15:48:28 +0200 Subject: RFR (M): 8199742: Clean up state flags in G1CollectorState In-Reply-To: <1522157120.28255.2.camel@oracle.com> References: <1522076812.2567.21.camel@oracle.com> <9f028672-bc86-525e-3c19-a3f347b1c118@oracle.com> <1522157120.28255.2.camel@oracle.com> Message-ID: On 2018-03-27 15:25, Thomas Schatzl wrote: > Hi Stefan, > > On Tue, 2018-03-27 at 14:58 +0200, Stefan Johansson wrote: >> Hi Thomas, >> >> Thanks for this much needed cleanup :) >> >> On 2018-03-26 17:06, Thomas Schatzl wrote: >>> Hi all, >>> >>> I would like to request reviews for this change that cleans up >>> the flags in G1CollectorState, applying uniform naming, removing >>> members that were basically temporary variables for a single >>> method, and [...] >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8199742 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8199742/webrev >> The change looks good. Just some minor things, gc_are_young and >> during_im are used as local variables at some places. I think we >> could change those to reflect the new better names as well. > I grepped through the sources for the old names and changed the uses to > something better (I hope). > > http://cr.openjdk.java.net/~tschatzl/8199742/webrev.0_to_1 (diff) > http://cr.openjdk.java.net/~tschatzl/8199742/webrev.1 (full) Much better :) Reviewed. StefanJ > > Thanks, > Thomas > From chris.plummer at oracle.com Tue Mar 27 00:48:51 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 26 Mar 2018 17:48:51 -0700 Subject: RFR(XS): 8200187: Exclude 3 long-running tests from tier1 In-Reply-To: References: Message-ID: <95567282-45e1-7c8c-9404-43da3eb6f9d4@oracle.com> Hi Leonid, The exclusion of these 3 tests from tier1 looks ok to me. It did lead me to some other questions however. The first was that if you exclude them from tier1, how to they get included in a later tier. The answer, for the SA tests, is that all of serviceability is included in tier2. So the next question was whether or not we want to repeat testing in higher tiers. I suppose since higher tiers include new platforms and build options, the answer is yes, but I hope the upper tiers are constructed well enough that we aren't wasting too much time repeating the same test(s) on the same platforms with the same options. thanks, Chris On 3/26/18 4:32 PM, Leonid Mesnik wrote: > Hi > > There is no strict rules and time budgets for hotspot tiers. It is assumed that faster tests should be executed earlier tiers. The order and total impact of of these test depends on how they are executed. > > Here is time of execution of tier1 groups with and without these excluded tests: > The whole :tier1 time reduced from 40 to 30 minutes (on dedicated HW with 32 core) > Time for :tier1_serviceability reduced from 15 to 6 min (in HS CI) > Time for :tier1_gc_1 reduced from 12 to 8-10 min (in HS CI) > > The benefits of exclusion of gc/g1/ihop/TestIHOPErgo.java are not so significant. However it is the only one GC stress test executed in tier1 while intention is to don?t run stress testing in tier1. > > Tests > serviceability/sa/ClhsdbScanOops.java > serviceability/sa/TestHeapDumpForLargeArray.java > are now hotspot_tier3_runtime which includes all hotspot_serviceability tests which are not a part of tier1. They are executed as a part of tier3 now as well as most of serviceability tests. > > Test gc/g1/ihop/TestIHOPErgo.java in now in group :hotspot_gc only and also it is marked as ?stress? test. So it is executed with all other GC stress tests > > Leonid > >> On Mar 25, 2018, at 3:48 PM, David Holmes wrote: >> >> Hi Leonid, >> >> On 24/03/2018 9:31 AM, Leonid Mesnik wrote: >>> Hi >>> Could you please review following fix which exclude following tests from tier1 testing: >>> serviceability/sa/ClhsdbScanOops.java >>> serviceability/sa/TestHeapDumpForLargeArray.java >>> gc/g1/ihop/TestIHOPErgo.java >>> Each of them takes more then 5 minutes to complete and significantly increase overall time to complete tier1. >> I'd need to see a much more detailed analysis of all the tests run, the order they run and the execution times to ascertain what impact this actually has on overall test execution time. >> >> But assuming 5 minutes is too long for tier1, this seems okay. But the tests must still be run in some other tier, and I'm not clear where that would be now? I would expect them to move to tier 3 perhaps, depending on what the time criteria for tier 3 is. >> >> Thanks, >> David >> >>> Please let me know if there are any reasons to run these tests in tier1 despite on their execution time. >>> webrev:http://cr.openjdk.java.net/~lmesnik/8200187/webrev.00/ >>> bug: https://bugs.openjdk.java.net/browse/JDK-8200187 >>> Leonid From sangheon.kim at oracle.com Tue Mar 27 17:12:31 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Tue, 27 Mar 2018 10:12:31 -0700 Subject: RFR [S][5/7]: 8197573: Remove concurrent cleanup and secondary free list handling In-Reply-To: <1522141099.3162.6.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520262995.2532.99.camel@oracle.com> <1520510384.3121.17.camel@oracle.com> <1520515912.3121.23.camel@oracle.com> <1522141099.3162.6.camel@oracle.com> Message-ID: Hi Thomas, On 03/27/2018 01:58 AM, Thomas Schatzl wrote: > Hi Sangheon, > > thanks for your review. > > On Mon, 2018-03-26 at 15:38 -0700, sangheon.kim wrote: >> Hi Thomas, > [..] >>> New webrevs: >>> http://cr.openjdk.java.net/~tschatzl/8197573/webrev.1_to_2/ (diff) >>> http://cr.openjdk.java.net/~tschatzl/8197573/webrev.2/ (full) >> webrev.2 looks good. I only have minor nits. >> >> =============================== >> src/hotspot/share/gc/g1/concurrentMarkThread.cpp >> 362 if (!cm()->has_aborted()) { >> ... >> 364 } >> 365 >> 366 if (!cm()->has_aborted()) { >> - Any reason for having 2 if clause at line:362 and 366? >> > The delay might take a long while, so the abort state might have > changed already. So it does not hurt checking to avoid going into the > VM operation. Right. > >> =============================== >> src/hotspot/share/gc/g1/g1CollectedHeap.cpp (from original file) >> 1030 // following two methods. >> - Do you know which method is referring originally? I guess >> wait_while_free_regions_coming() and _cm->root_regions()->abort()? >> And this is why you didn't remove the comment above >> wait_while_free_regions_coming(), right? > As far as I understand the two methods were > wait_while_free_regions_coming() and wait_until_scan_finished(). The > latter is still there so I only adapted the comment. > > I removed the comment in the new webrev and filed JDK-8200291. We > should not have todo-lists in the code. Okay, thanks for filing the CR. > >> =============================== >> src/hotspot/share/gc/g1/heapRegionSet.cpp >> - Copyright year update >> > Fixed. New webrevs (just the comment removals and the copyright > update): > > http://cr.openjdk.java.net/~tschatzl/8197573/webrev.2_to_3/ (diff) > http://cr.openjdk.java.net/~tschatzl/8197573/webrev.3/ (full) webrev.3 looks good to me. Thanks, Sangheon > > Thanks, > Thomas > From sangheon.kim at oracle.com Tue Mar 27 17:50:45 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Tue, 27 Mar 2018 10:50:45 -0700 Subject: RFR [M][7/7]: 8197932: Better split work in rebuild remembered sets phase In-Reply-To: <1521715562.2448.15.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520265115.2532.126.camel@oracle.com> <1520953401.3100.41.camel@oracle.com> <1521461029.2323.11.camel@oracle.com> <4110ec65-c66f-eca0-3b7a-6580f0edf00d@oracle.com> <1521715562.2448.15.camel@oracle.com> Message-ID: Hi Thomas, Finally the last piece! :) On 03/22/2018 03:46 AM, Thomas Schatzl wrote: > Hi Stefan, > > thanks for your review, and sorry for the late reply... > > On Mon, 2018-03-19 at 16:39 +0100, Stefan Johansson wrote: >> Thanks Thomas, >> >> On 2018-03-19 13:03, Thomas Schatzl wrote: >>> Hi all, >>> >>> Stefan proposed to use some iterator over the work area that >>> internally handles the boundary at the TAMS wrt to liveness when >>> scanning. >>> >>> This makes the change a bit more compact, performance does not seem >>> to differ, so why not... :) >> I really like this approach :) A few additional small things: >> src/hotspot/share/gc/g1/g1RemSet.cpp >> 787 if (!is_live(_current)) { >> >> I would like to invert the if-statement in the LiveObjIterator >> constructor and move the call to move_if_below_ntams() into the >> if-statement updating _current, something like this: >> if (is_live(_current)) { >> // Non-objArrays were scanned by the previous part of that region. >> if (_current < mr.start() && !oop(_current)->is_objArray()) { >> _current += oop(_current)->size(); >> // We might have positioned _current on a non-live object. >> Reposition to the next >> // live one if needed. >> move_if_below_ntams(); >> } >> } >> } else { >> ... >> } >> >> 926 #ifdef ASSERT >> 927 // In the final iteration of the loop the region might >> have been eagerly reclaimed. >> 928 // Simply filter out those regions. We can not just use >> region >> type because there >> 929 // might have already been new allocations into these >> regions. >> 930 HeapWord* const top_at_rebuild_start = >> _cm->top_at_rebuild_start(region_idx); >> 931 assert(!hr->is_old() || >> >> What do you think about making this assert into a guarantee or use >> DEBUG_ONLY() to avoid the #ifdef ASSERT? Also maybe adding something >> to the comment explaining that it is the check on >> top_at_rebuild_start that filters out eagerly reclaimed regions. > Done. > >> --- >> src/hotspot/share/gc/g1/g1_globals.hpp >> 259 experimental(size_t, G1RebuildRemSetChunkSize, 256 * K, >> >> Did you test to see if changing the G1RebuildRemSetChunkSize flag >> had any impact on performance, otherwise maybe we could make it a >> constant instead. > The problem is not so much increasing the value, but decreasing it for > very slow systems. Decreasing it further adds some overhead. > > The other point is that since we track the time a chunk takes, it seems > strange to not have any knobs to deal with problems in that area. > > New webrev: > http://cr.openjdk.java.net/~tschatzl/8197932/webrev.2_to_3 (diff) > http://cr.openjdk.java.net/~tschatzl/8197932/webrev.3 (full) webrev.3 looks good! Minor nits below. ============================= src/hotspot/share/gc/g1/g1RemSet.cpp ?805?????? void move_to_next() { ?810?????? oop next() const { (Ignore if you don't agree) - I like the idea of using iterator here. But for me, current naming seems misleading. Current move_to_next() seems next(). And something else for current next(). to_oop()? ============================= src/hotspot/share/gc/g1/g1RemSetTrackingPolicy.cpp ? 37?? // All non-young and non-closed archive regions need to be scanned for references; ? 38?? // At every gc we gather references to other regions in young, and closed archive ? 39?? // regions by definition do not have references going outside the closed archive. - Add some explanation about r->is_free() case as well? ============================= src/hotspot/share/gc/g1/g1_globals.hpp ?261?????????? range(4 * K, 32 * M)????????????????????????????????????????????? \ - Just question. Why the min is 4K? Thanks, Sangheon > > In this webrev I also did some somewhat messy looking renaming of nTAMS > to TAMS (removing the n) and passing the (n)TAMS around in conjunction > with the bitmap. > This will make a follow-up change a lot easier to read. > > Thanks, > Thomas > From leonid.mesnik at oracle.com Wed Mar 28 00:22:43 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Tue, 27 Mar 2018 17:22:43 -0700 Subject: RFR(XS): 8200187: Exclude 3 long-running tests from tier1 In-Reply-To: <95567282-45e1-7c8c-9404-43da3eb6f9d4@oracle.com> References: <95567282-45e1-7c8c-9404-43da3eb6f9d4@oracle.com> Message-ID: <5B7068DF-7FFC-41A8-AFC5-4D8DFDFE2B3D@oracle.com> Chris, David Thank you for review. > On Mar 26, 2018, at 5:48 PM, Chris Plummer wrote: > > Hi Leonid, > > The exclusion of these 3 tests from tier1 looks ok to me. It did lead me to some other questions however. The first was that if you exclude them from tier1, how to they get included in a later tier. The answer, for the SA tests, is that all of serviceability is included in tier2. So the next question was whether or not we want to repeat testing in higher tiers. I suppose since higher tiers include new platforms and build options, the answer is yes, but I hope the upper tiers are constructed well enough that we aren't wasting too much time repeating the same test(s) on the same platforms with the same options. > It depends from component . Runtime the groups organized by excluding previous tiers like: hotspot_tier2_runtime = \ runtime/ \ serviceability/ \ -runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java \ -runtime/Thread/TestThreadDumpMonitorContention.java \ -runtime/containers/ \ -:tier1_runtime \ -:tier1_serviceability \ -:hotspot_tier2_runtime_platform_agnostic ? hotspot_tier3_runtime = \ runtime/ \ serviceability/ \ -runtime/containers/ \ -:tier1_runtime \ -:tier1_serviceability \ -:hotspot_tier2_runtime_platform_agnostic \ -:hotspot_tier2_runtime ? So once test is excluded from tier1 it automatically is executed as a part of tier 2/3. While these tiers doesn?t execute tests from tier1. The other teams organize their testing slightly different. However the whole idea is that test excluded from tier1 should be executed in latest tiers and test should not re-executed in fast tiers once it was done already. However during testing like PIT we might just run whole hotspot_[component] team even some subset was run always in CI. Leonid > thanks, > > Chris > > On 3/26/18 4:32 PM, Leonid Mesnik wrote: >> Hi >> >> There is no strict rules and time budgets for hotspot tiers. It is assumed that faster tests should be executed earlier tiers. The order and total impact of of these test depends on how they are executed. >> >> Here is time of execution of tier1 groups with and without these excluded tests: >> The whole :tier1 time reduced from 40 to 30 minutes (on dedicated HW with 32 core) >> Time for :tier1_serviceability reduced from 15 to 6 min (in HS CI) >> Time for :tier1_gc_1 reduced from 12 to 8-10 min (in HS CI) >> >> The benefits of exclusion of gc/g1/ihop/TestIHOPErgo.java are not so significant. However it is the only one GC stress test executed in tier1 while intention is to don?t run stress testing in tier1. >> >> Tests >> serviceability/sa/ClhsdbScanOops.java >> serviceability/sa/TestHeapDumpForLargeArray.java >> are now hotspot_tier3_runtime which includes all hotspot_serviceability tests which are not a part of tier1. They are executed as a part of tier3 now as well as most of serviceability tests. >> >> Test gc/g1/ihop/TestIHOPErgo.java in now in group :hotspot_gc only and also it is marked as ?stress? test. So it is executed with all other GC stress tests >> >> Leonid >> >>> On Mar 25, 2018, at 3:48 PM, David Holmes wrote: >>> >>> Hi Leonid, >>> >>> On 24/03/2018 9:31 AM, Leonid Mesnik wrote: >>>> Hi >>>> Could you please review following fix which exclude following tests from tier1 testing: >>>> serviceability/sa/ClhsdbScanOops.java >>>> serviceability/sa/TestHeapDumpForLargeArray.java >>>> gc/g1/ihop/TestIHOPErgo.java >>>> Each of them takes more then 5 minutes to complete and significantly increase overall time to complete tier1. >>> I'd need to see a much more detailed analysis of all the tests run, the order they run and the execution times to ascertain what impact this actually has on overall test execution time. >>> >>> But assuming 5 minutes is too long for tier1, this seems okay. But the tests must still be run in some other tier, and I'm not clear where that would be now? I would expect them to move to tier 3 perhaps, depending on what the time criteria for tier 3 is. >>> >>> Thanks, >>> David >>> >>>> Please let me know if there are any reasons to run these tests in tier1 despite on their execution time. >>>> webrev:http://cr.openjdk.java.net/~lmesnik/8200187/webrev.00/ >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8200187 >>>> Leonid > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From per.liden at oracle.com Wed Mar 28 08:12:51 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 28 Mar 2018 10:12:51 +0200 Subject: RFR: 8199927: Make WhiteBox more GC agnostic In-Reply-To: <5ABB4E25.8000807@oracle.com> References: <8e0b6cdb-c1d4-37f9-1170-b5596051865d@oracle.com> <95ce0e01-69ba-b7fa-bace-2e56a7a8d3ea@oracle.com> <5ABB4E25.8000807@oracle.com> Message-ID: Thanks for reviewing Erik! /Per On 03/28/2018 10:11 AM, Erik ?sterlund wrote: > Hi Per, > > Looks good. > > Thanks, > /Erik > > On 2018-03-21 15:50, Per Liden wrote: >> Updated webrev to reflect that GCSelector changed name to GCConfig in >> the patch for JDK-8199925. >> >> http://cr.openjdk.java.net/~pliden/8199927/webrev.1 >> >> /Per >> >> On 03/21/2018 09:14 AM, Per Liden wrote: >>> Make WhiteBox more GC agnostic by having it use the GCSelector to >>> answer questions about which GCs are supported, selected, etc. >>> >>> Going forward, this will make it easier to plug in new GC without >>> having to touch and adjust the WhiteBox code. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8199927 >>> Webrev: http://cr.openjdk.java.net/~pliden/8199927/webrev.0 >>> >>> /Per > From erik.osterlund at oracle.com Wed Mar 28 08:11:17 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 28 Mar 2018 10:11:17 +0200 Subject: RFR: 8199927: Make WhiteBox more GC agnostic In-Reply-To: <95ce0e01-69ba-b7fa-bace-2e56a7a8d3ea@oracle.com> References: <8e0b6cdb-c1d4-37f9-1170-b5596051865d@oracle.com> <95ce0e01-69ba-b7fa-bace-2e56a7a8d3ea@oracle.com> Message-ID: <5ABB4E25.8000807@oracle.com> Hi Per, Looks good. Thanks, /Erik On 2018-03-21 15:50, Per Liden wrote: > Updated webrev to reflect that GCSelector changed name to GCConfig in > the patch for JDK-8199925. > > http://cr.openjdk.java.net/~pliden/8199927/webrev.1 > > /Per > > On 03/21/2018 09:14 AM, Per Liden wrote: >> Make WhiteBox more GC agnostic by having it use the GCSelector to >> answer questions about which GCs are supported, selected, etc. >> >> Going forward, this will make it easier to plug in new GC without >> having to touch and adjust the WhiteBox code. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8199927 >> Webrev: http://cr.openjdk.java.net/~pliden/8199927/webrev.0 >> >> /Per From thomas.schatzl at oracle.com Wed Mar 28 11:23:30 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 28 Mar 2018 13:23:30 +0200 Subject: RFR [M][7/7]: 8197932: Better split work in rebuild remembered sets phase In-Reply-To: References: <1520258985.2532.40.camel@oracle.com> <1520265115.2532.126.camel@oracle.com> <1520953401.3100.41.camel@oracle.com> <1521461029.2323.11.camel@oracle.com> <4110ec65-c66f-eca0-3b7a-6580f0edf00d@oracle.com> <1521715562.2448.15.camel@oracle.com> Message-ID: <1522236210.2391.7.camel@oracle.com> Hi, On Tue, 2018-03-27 at 10:50 -0700, sangheon.kim wrote: > Hi Thomas, > > Finally the last piece! :) > yes :) Unfortunately there are already a few more changes out for review, although much smaller in scope and complexity. [...] > > > > New webrev: > > http://cr.openjdk.java.net/~tschatzl/8197932/webrev.2_to_3 (diff) > > http://cr.openjdk.java.net/~tschatzl/8197932/webrev.3 (full) > > webrev.3 looks good! > Minor nits below. > > ============================= > src/hotspot/share/gc/g1/g1RemSet.cpp > 805 void move_to_next() { > 810 oop next() const { > (Ignore if you don't agree) > - I like the idea of using iterator here. But for me, current naming > seems misleading. Current move_to_next() seems next(). And something > else for current next(). to_oop()? I would like to keep it as is until we in the team figured out how to do iterators in hotspot, and then change as needed. Apart from the naming this style is 1:1 compatible to STL iterators (i.e. move_to_next() -> "++" operator, next() -> * operator) (which although I expect us to use them in the future, I have no particular preference for any style), so I chose that. > ============================= > src/hotspot/share/gc/g1/g1RemSetTrackingPolicy.cpp > 37 // All non-young and non-closed archive regions need to be > scanned for references; > 38 // At every gc we gather references to other regions in > young, > and closed archive > 39 // regions by definition do not have references going > outside > the closed archive. > - Add some explanation about r->is_free() case as well? I will fix this before pushing. > ============================= > src/hotspot/share/gc/g1/g1_globals.hpp > 261 range(4 * K, 32 * > M) \ > - Just question. Why the min is 4K? 4k is like 1024 references max. This seemed to be a small enough increment to not cause too much safepointing delay for the smallest machine we are going to run with G1. I am going to push this change as is - your review did not seem to pose any significant issue. Otherwise there is enough opportunity to fix this immediately. Thanks, Thomas From stefan.johansson at oracle.com Wed Mar 28 11:26:24 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 28 Mar 2018 13:26:24 +0200 Subject: RFR (M): 8151171: Bring g1ConcurrentMark files up to current coding conventions In-Reply-To: <1522077275.2567.27.camel@oracle.com> References: <1522077275.2567.27.camel@oracle.com> Message-ID: Hi Thomas, Very nice cleanup. On 2018-03-26 17:14, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this mostly renaming patch that cleans up > g1ConcurrentMark files (once again *sigh*). > > Changes: > - camelCase variable and method naming > - missing brackets around one-line if-statement bodies. > - method naming (getters/setters) > - use _g1h member variable instead of _g1 or another method local > variable as discussed earlier > - use assert_at_safepoint* macros > - remove incomprehensible comments > - changed "CheckPointRoots" to "Remark" throughout > ... > > CR: > https://bugs.openjdk.java.net/browse/JDK-8151171 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8151171/webrev Looks good, just to have one comment ;) src/hotspot/share/gc/g1/concurrentMarkThread.cpp 331?????????? CMRemark final_cl(_cm); 362???????? CMCleanup cl_cl(_cm); Rename the instances to remark and cleanup. --- Thanks, Stefan > Testing: > hs-tier 1-5 in conjunction with JDK-8200234, hs-tier 1 standalone > > The webrev is based on JDK-8199742 also out for review, but has no > particular semantic connection. > > Thanks, > Thomas > From stefan.johansson at oracle.com Wed Mar 28 12:18:15 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 28 Mar 2018 14:18:15 +0200 Subject: RFR (S/M): 8200234: Cleanup Remark and Cleanup pause code In-Reply-To: <1522079989.2567.43.camel@oracle.com> References: <1522079989.2567.43.camel@oracle.com> Message-ID: <283b9e7b-7729-718b-7bc4-22282fa3d7e1@oracle.com> Hi Thomas, On 2018-03-26 17:59, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that cleans up Remark and Cleanup > pause code? > > It mainly moves the record_concurrent_mark_xxx_start/end() methods to > the top/bottom of the remark() and cleanup() methods, and refactors the > verification code into a single method. > > There is one more somewhat tricky related change here that fixes the > check_bitmaps() method of that verification: after we swapped bitmaps > (in Cleanup right now) until the previous prev-bitmap has been cleared, > its bits are invalid, and G1HeapVerifier::check_bitmaps() should not > check the next bitmap. > > Previously this has been done using the somewhat tricky condition > > 674 if (_g1h->collector_state()->mark_or_rebuild_in_progress() || > !_g1h->_cmThread->in_progress()) { > 675 res_n = verify_no_bits_over_tams("next", next_bitmap, ntams, > end); > 676 } > > which you might immediately understand as the best way to prevent > checking the next bitmap from the Cleanup until the (new) next bitmap > has been cleared, but I did not. Further, there is no similarly easy > condition if the bitmap swapping is moved to the Remark pause, so I > created a new flag in G1CollectorState that directly records that we > are currently clearing the next bitmap. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8200234 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8200234/webrev/ Looks good in general, but a few questions and suggestions: src/hotspot/share/gc/g1/g1ConcurrentMark.cpp ?539?? G1CMBitMap* const bitmap = _g1h->collector_state()->mark_or_rebuild_in_progress() ? _next_mark_bitmap : _prev_mark_bitmap; The old code always used the _next_mark_bitmap, was this a bug or has any timing changed? 1010???? os::snprintf(buffer, BufLen, "During GC (%s)", caller); Seems like jio_snprintf is more used in the HotSpot code, so maybe change to use that one instead. --- src/hotspot/share/memory/iterator.hpp ?136 #ifdef ASSERT ?137?? bool should_verify_oops() { return false; } ?138 #endif Why is this needed? I don't see any use of NoHeaderExtendedOopClosure in the new code. If needed I would prefer: debug_only(bool should_verify_oops() { return false; }) --- Thanks, Stefan > Testing: > hs-tier 1-5 > > It is based on JDK-8151171 which is also out for review currently. > > Thanks, > Thomas > From thomas.schatzl at oracle.com Wed Mar 28 12:21:39 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 28 Mar 2018 14:21:39 +0200 Subject: RFR (M): 8151171: Bring g1ConcurrentMark files up to current coding conventions In-Reply-To: References: <1522077275.2567.27.camel@oracle.com> Message-ID: <1522239699.2391.10.camel@oracle.com> Hi Stefan, On Wed, 2018-03-28 at 13:26 +0200, Stefan Johansson wrote: > Hi Thomas, > > Very nice cleanup. thanks for your review. > > On 2018-03-26 17:14, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this mostly renaming patch that cleans up > > g1ConcurrentMark files (once again *sigh*). > > > > Changes: > > - camelCase variable and method naming > > - missing brackets around one-line if-statement bodies. > > - method naming (getters/setters) > > - use _g1h member variable instead of _g1 or another method local > > variable as discussed earlier > > - use assert_at_safepoint* macros > > - remove incomprehensible comments > > - changed "CheckPointRoots" to "Remark" throughout > > ... > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8151171 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8151171/webrev > > Looks good, just to have one comment ;) > src/hotspot/share/gc/g1/concurrentMarkThread.cpp > 331 CMRemark final_cl(_cm); > 362 CMCleanup cl_cl(_cm); > > Rename the instances to remark and cleanup. :) Done. http://cr.openjdk.java.net/~tschatzl/8151171/webrev.0_to_1 (diff) http://cr.openjdk.java.net/~tschatzl/8151171/webrev.1 (full) I found some more (very minor) renaming work to be done in concurrentMarkThread.[hc]pp. Nothing really fancy. Thanks, Thomas From stefan.johansson at oracle.com Wed Mar 28 12:23:07 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 28 Mar 2018 14:23:07 +0200 Subject: RFR (S): 8200074: Remove G1ConcurrentMark::_concurrent_marking_in_progress In-Reply-To: <1522088250.2567.55.camel@oracle.com> References: <1522088250.2567.55.camel@oracle.com> Message-ID: Looks good, StefanJ On 2018-03-26 20:17, Thomas Schatzl wrote: > Hi all, > > can I have reviews for the following change that removes > G1ConcurrentMark::_concurrent_marking_in_progress since it is only used > in some asserts, and at the same time mostly the same > as G1ConcurrentMark::_concurrent (ie. it is used to verify > _concurrent). > > CR: > https://bugs.openjdk.java.net/browse/JDK-8200074 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8200074/webrev > Testing: > hs-tier 1-5 with other changes > > Change needs JDK-8200255 to apply cleanly. > > Thanks, > Thomas From stefan.johansson at oracle.com Wed Mar 28 12:24:21 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 28 Mar 2018 14:24:21 +0200 Subject: RFR (S): 8200255: Remove G1CMTask::_concurrent In-Reply-To: <1522088754.2567.56.camel@oracle.com> References: <1522088754.2567.56.camel@oracle.com> Message-ID: <512feca3-2dba-a6a4-87a3-d1f1e830787e@oracle.com> Looks good, StefanJ On 2018-03-26 20:25, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this small change that removes > G1CMTask::_concurrent. That member is a duplicate of > G1ConcurrentMark::_concurrent, but there does not seem to be any > advantage of keeping this duplicate, as it's only used in asserts or > non-performance critical code. > > This removes some imho unnecessary code, so I suggest to do that. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8200255 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8200255/webrev/ > Testing: > local compilation, gcbasher, testing with other changes > > Thanks, > Thomas From erik.osterlund at oracle.com Wed Mar 28 13:00:49 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 28 Mar 2018 15:00:49 +0200 Subject: RFR: JDK-8199781: Don't use naked == for comparing oops In-Reply-To: <5e3f8ffc-0eb9-e4da-d0c0-72becb677b7b@redhat.com> References: <74ca6651-49aa-2b77-d7e9-a588fb2b8417@oracle.com> <096620eb-e252-9689-c157-098314ac9119@redhat.com> <650c2e5c-88d7-3896-1526-386557f5d2df@redhat.com> <2683d96d-4170-f8b7-a48d-12fd0465ec4c@redhat.com> <1fa7bccc-d827-0a6e-ecbb-8ac1352a6449@redhat.com> <25E3C8FE-089E-4AB8-A9B4-1DA48B2FF17D@oracle.com> <5e3f8ffc-0eb9-e4da-d0c0-72becb677b7b@redhat.com> Message-ID: <5ABB9201.7010806@oracle.com> Hi, The access.hpp support has reincarnated. Here is an incremental patch on Roman's work (rebased on latest jdk-hs) that makes it possible to include access.hpp to call Access<>::equals specifically. The infrastructure allows picking more accesses that can be used from the hpp file, but I have kept it down to the minimum requirements now, so for now it only works for equals. Perhaps in the future, we might want to move more operations to be reachable from hpp, but not today. I changed the involved header files to include access.hpp, and made it possible to inline this again for non-Shenandoah builds, by checking for the INTERNAL_BT_TO_SPACE_INVARIANT build-time decorator. Builds that only have to-space invariant GCs inline equals to a normal native comparison of the pointers. Builds that have at least one to-space invariant GC instead generate a single function pointer call. This function pointer may be resolved in the access.cpp file, instead of the access.inline.hpp file. Here is an explanation how this works: The template pipeline for each operation consists of 5 steps, and they were previously all performed in the access.inline.hpp file. The steps are: 1) Set default decorators and decay types 2) Reduce types 3) Pre-runtime dispatch 4) Runtime-dispatch 5.a) Barrier resolution (check which GC backend accessor to be used upon the first invocation, then patch function pointer to 5.b) 5.b) Post-runtime dispatch I have moved steps 1-4 to the accessBackend.hpp file, which is included by access.hpp. This allows performing all steps until the backend barrier resolution in only hpp files. This makes it possible to use, e.g. raw accesses (although without compressed oops, atomic or order access) from only the access.hpp file. The resolution of barriers in the GC backends (comprising steps 4-5) remains in the access.inline.hpp file. But they can be instantiated from the access.cpp file for well chosen accesses and decorator combinations that can be pre-generated in the libjvm.so file. The function pointers used in builds that have not to-space invariant GCs will be resolved to point right into these functions in the access.cpp file at runtime. This change required moving the decorators to a new file called accessDecorators.hpp. I kind of like having them there in a separate file. Incremental webrev to Roman's patch: http://cr.openjdk.java.net/~eosterlund/8199781/webrev.00_01/ Full webrev: http://cr.openjdk.java.net/~eosterlund/8199781/webrev.01/ Thanks, /Erik On 2018-03-27 09:14, Roman Kennke wrote: > Am 27.03.2018 um 00:46 schrieb John Rose: >> On Mar 26, 2018, at 2:21 PM, Roman Kennke wrote: >>> Am 26.03.2018 um 21:11 schrieb John Rose: >>>> ... >>>> This is clearly one of those cases. Let's code against surprises >>>> (both present and future) by abstracting simpler operations with >>>> inlines. >>> Thanks John for clarifying. I generally do agree with you that it makes >>> sense to inline something like an operator that will compile to a single >>> instruction. >>> >>> Unfortunately, this new abstraction will generate a non-inlineable call. >>> Thanks Erik's work with the Access API, it should be a straight call to >>> the right implementation, not even a virtual call, but it's a call >>> nonetheless. >> What would it take to inline the call, so there's no new performance >> hazard introduced? >> >> This might be a place to add #ifdefs, if the templates don't let us >> control the decision statically. >> >> (I confess I find the templates complex and hard to reason about. >> But that's a potential maintenance issue with the access API, not >> your work.) > The Access API supports static inclusion/exclusion of stuff, and we've > used it before to ensable barriers on primitive field accesses for > Shenandoah-enabled builds. It works by passing > -DSUPPORT_BARRIER_ON_PRIMITIVES to gcc. This is much better than #ifdefs > and does the same thing. I can add something similar for object equality. > >>> The reason why I'm proposing it is that the GC might want to have a say >>> about object equality. In the case of Shenandoah, there may be two >>> copies of any one object floating around, and comparing the two copies >>> might otherwise result in false negatives. >> 1. Of course we need a way for some GC algorithms to control >> object identity. >> >> 2. It's good coding discipline to find all the places where native >> op== is used and upgrade them to the needed control. > I hope I did that with the proposed patch (it's based on years of > finding all those places in Shenandoah land ;-) ). > >> (2a. We also need a way to avoid having op== creep back in >> in an uncontrolled way.) > We have something for this in Shenandoah too. It works by overriding == > in oopsHierarch.hpp and running into ShouldNotReachHere() if both > operands are != NULL. This catches all uses of naked == at runtime, when > running with +CheckUnhandledOops. It's not perfect, because it doesn't > catch uses in places that are not executed in tests, but it's still > incredibly useful. The caveat here is that we also need another > abstraction for code paths that actually want naked ==, e.g. in the GC. > In Shenandoah we have an oopDesc::unsafe_equals() for that. The other > alternative would be to drop operators from oop to oopDesc* and compare > that (which is exactly what oopDesc::unsafe_equals() does). > >> 3. We still need an unsurprising performance model for GC's >> which don't require the extra identity hook. > Yeah. Unfortunately this is C++ and not Java, and the decision which GC > to use is made at runtime. The 'best' that I can see is to rip out the > indirection in Shenandoah-disabled builds, as proposed above. I don't > like it much, but if this is the only way we can get Shenandoah in, then > let's go for it? > >> So we're not there yet, I think. If there's no easy way to adjust >> the access API to get the last inlining, I suggest adding an #ifdef >> into oopDesc::equals. >> >> I suspect the happy final solution will be to templatize hot loops >> using making the inlinable adjusted op== be part of the "decorations" >> of those loops. Then we will have two copies of hot loops in the >> JVM, one with the out-of-line hook and one without. >> >> For now, I think it's enough to have separate builds of the JVM, >> one with the #ifdef set "the old way" and one that allows more >> flexibility. >> >> (Is this the first time we've run into an occasion to make a separate >> Shenandaoah build? Surely not.) > Primitive field accesses required similar jumping through hoops ;-) > >>> So... with inlining oopDesc::equals() we will get one straight call >>> (from the caller of oopDesc::equals() to the impl), instead of 2 (caller >>> -> oopDesc::equals() -> impl). >>> >>> It's the best I could come up with that does what (Shenandoah) GC needs, >>> and it hasn't shown up in any performance testing that we did, yet. >>> >>> Still good? >> I think it's too risky to out-of-line the "cmpq" instruction. Let's >> find an agreeable way to handle that via a trade-off that keeps >> the old code shape in VM builds which don't need the new code >> shape. I'm speaking up here because I don't believe this is the >> last time we'll have to consider adding something gross like an >> #ifdef to support new features, and I want to aim at trade-offs >> that keep the existing hot paths efficient, while still allowing >> new code shapes to become part of the mix. >> >> (I'd be happy to see something cleaner than an #ifdef, of >> course. Seems to me that a constant non-product flag >> -XX:+ShortCircuitOopEquals would work too, with a >> binding of falseInShenandoah.) > That would require coding 2 versions of hot loops that involve oop==oop > and switch to one or the other based on that flag. I have checked all > the uses of oopDesc::equals() in my patch, and none seemed particularily > critical. I think this would be overkill. > > And notice also that none interpreter/c1/c2 compiled code will suffer > this problem because we'll just generate the right code at runtime. > > Roman > From stefan.johansson at oracle.com Wed Mar 28 13:52:42 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 28 Mar 2018 15:52:42 +0200 Subject: RFR: 8200228: Change default value of HeapSizePerGCThread Message-ID: <2690ce00-921f-9310-648f-69fd41afca19@oracle.com> Hi, Please review this change of the default value for HeapSizePerGCThread. Links JBS: https://bugs.openjdk.java.net/browse/JDK-8200228 Webrev: http://cr.openjdk.java.net/~sjohanss/8200228/00/ Summary My testing shows that for DaCapo, SpecJVM98 and GCBasher (gc intensive stress test) with small heaps we get better pause times with the flag set to 32M. There might be other arguments, such as not using to much resources on small systems that might also be important so please let me know if you have reasons why the flag should be left unchanged. Thanks, Stefan From rkennke at redhat.com Wed Mar 28 14:03:27 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 28 Mar 2018 16:03:27 +0200 Subject: RFR: JDK-8199781: Don't use naked == for comparing oops In-Reply-To: <5ABB9201.7010806@oracle.com> References: <74ca6651-49aa-2b77-d7e9-a588fb2b8417@oracle.com> <096620eb-e252-9689-c157-098314ac9119@redhat.com> <650c2e5c-88d7-3896-1526-386557f5d2df@redhat.com> <2683d96d-4170-f8b7-a48d-12fd0465ec4c@redhat.com> <1fa7bccc-d827-0a6e-ecbb-8ac1352a6449@redhat.com> <25E3C8FE-089E-4AB8-A9B4-1DA48B2FF17D@oracle.com> <5e3f8ffc-0eb9-e4da-d0c0-72becb677b7b@redhat.com> <5ABB9201.7010806@oracle.com> Message-ID: <73626aa7-9584-5cf2-3efa-25dd66d25a28@redhat.com> Hi Erik, Thanks for this work! without having looked too deeply at the code (it might be in vain anyway, what with all the template voodoo stuff going on :-P), I have a few conceptual questions: - For the issue at hand (replacing naked == with oopDesc::equals() or such), wouldn't it have been enough to mark it with the INTERNAL_BT_TO_SPACE_INVARIANT decorator, to get inlining for non-Shenandoah (or more generally, non-to-space-invariant) builds. ? - What is the advantage of this refactoring? It is a refactoring, right? or do we get new features here? - So now we can use the Access API by only including access.hpp. Fine. But this comes at the cost of having actual code in .hpp files? I am not sure about Hotspot style, but I try to avoid actual code in .hpp files whenever I can, except *maybe* for the most trivial of stuff. Thanks, Roman > Hi, > > The access.hpp support has reincarnated. > > Here is an incremental patch on Roman's work (rebased on latest jdk-hs) > that makes it possible to include access.hpp to call Access<>::equals > specifically. The infrastructure allows picking more accesses that can > be used from the hpp file, but I have kept it down to the minimum > requirements now, so for now it only works for equals. Perhaps in the > future, we might want to move more operations to be reachable from hpp, > but not today. > > I changed the involved header files to include access.hpp, and made it > possible to inline this again for non-Shenandoah builds, by checking for > the INTERNAL_BT_TO_SPACE_INVARIANT build-time decorator. > > Builds that only have to-space invariant GCs inline equals to a normal > native comparison of the pointers. Builds that have at least one > to-space invariant GC instead generate a single function pointer call. > This function pointer may be resolved in the access.cpp file, instead of > the access.inline.hpp file. > > Here is an explanation how this works: > > The template pipeline for each operation consists of 5 steps, and they > were previously all performed in the access.inline.hpp file. The steps are: > > 1) Set default decorators and decay types > 2) Reduce types > 3) Pre-runtime dispatch > 4) Runtime-dispatch > 5.a) Barrier resolution (check which GC backend accessor to be used upon > the first invocation, then patch function pointer to 5.b) > 5.b) Post-runtime dispatch > > I have moved steps 1-4 to the accessBackend.hpp file, which is included > by access.hpp. This allows performing all steps until the backend > barrier resolution in only hpp files. This makes it possible to use, > e.g. raw accesses (although without compressed oops, atomic or order > access) from only the access.hpp file. The resolution of barriers in the > GC backends (comprising steps 4-5) remains in the access.inline.hpp > file. But they can be instantiated from the access.cpp file for well > chosen accesses and decorator combinations that can be pre-generated in > the libjvm.so file. The function pointers used in builds that have not > to-space invariant GCs will be resolved to point right into these > functions in the access.cpp file at runtime. > > This change required moving the decorators to a new file called > accessDecorators.hpp. I kind of like having them there in a separate file. > > Incremental webrev to Roman's patch: > http://cr.openjdk.java.net/~eosterlund/8199781/webrev.00_01/ > > Full webrev: > http://cr.openjdk.java.net/~eosterlund/8199781/webrev.01/ > > Thanks, > /Erik > > On 2018-03-27 09:14, Roman Kennke wrote: >> Am 27.03.2018 um 00:46 schrieb John Rose: >>> On Mar 26, 2018, at 2:21 PM, Roman Kennke wrote: >>>> Am 26.03.2018 um 21:11 schrieb John Rose: >>>>> ... >>>>> This is clearly one of those cases.? Let's code against surprises >>>>> (both present and future) by abstracting simpler operations with >>>>> inlines. >>>> Thanks John for clarifying. I generally do agree with you that it makes >>>> sense to inline something like an operator that will compile to a >>>> single >>>> instruction. >>>> >>>> Unfortunately, this new abstraction will generate a non-inlineable >>>> call. >>>> Thanks Erik's work with the Access API, it should be a straight call to >>>> the right implementation, not even a virtual call, but it's a call >>>> nonetheless. >>> What would it take to inline the call, so there's no new performance >>> hazard introduced? >>> >>> This might be a place to add #ifdefs, if the templates don't let us >>> control the decision statically. >>> >>> (I confess I find the templates complex and hard to reason about. >>> But that's a potential maintenance issue with the access API, not >>> your work.) >> The Access API supports static inclusion/exclusion of stuff, and we've >> used it before to ensable barriers on primitive field accesses for >> Shenandoah-enabled builds. It works by passing >> -DSUPPORT_BARRIER_ON_PRIMITIVES to gcc. This is much better than #ifdefs >> and does the same thing. I can add something similar for object equality. >> >>>> The reason why I'm proposing it is that the GC might want to have a say >>>> about object equality. In the case of Shenandoah, there may be two >>>> copies of any one object floating around, and comparing the two copies >>>> might otherwise result in false negatives. >>> 1. Of course we need a way for some GC algorithms to control >>> object identity. >>> >>> 2. It's good coding discipline to find all the places where native >>> op== is used and upgrade them to the needed control. >> I hope I did that with the proposed patch (it's based on years of >> finding all those places in Shenandoah land ;-) ). >> >>> (2a. We also need a way to avoid having op== creep back in >>> in an uncontrolled way.) >> We have something for this in Shenandoah too. It works by overriding == >> in oopsHierarch.hpp and running into ShouldNotReachHere() if both >> operands are != NULL. This catches all uses of naked == at runtime, when >> running with +CheckUnhandledOops. It's not perfect, because it doesn't >> catch uses in places that are not executed in tests, but it's still >> incredibly useful. The caveat here is that we also need another >> abstraction for code paths that actually want naked ==, e.g. in the GC. >> In Shenandoah we have an oopDesc::unsafe_equals() for that. The other >> alternative would be to drop operators from oop to oopDesc* and compare >> that (which is exactly what oopDesc::unsafe_equals() does). >> >>> 3. We still need an unsurprising performance model for GC's >>> which don't require the extra identity hook. >> Yeah. Unfortunately this is C++ and not Java, and the decision which GC >> to use is made at runtime. The 'best' that I can see is to rip out the >> indirection in Shenandoah-disabled builds, as proposed above. I don't >> like it much, but if this is the only way we can get Shenandoah in, then >> let's go for it? >> >>> So we're not there yet, I think.? If there's no easy way to adjust >>> the access API to get the last inlining, I suggest adding an #ifdef >>> into oopDesc::equals. >>> >>> I suspect the happy final solution will be to templatize hot loops >>> using making the inlinable adjusted op== be part of the "decorations" >>> of those loops.? Then we will have two copies of hot loops in the >>> JVM, one with the out-of-line hook and one without. >>> >>> For now, I think it's enough to have separate builds of the JVM, >>> one with the #ifdef set "the old way" and one that allows more >>> flexibility. >>> >>> (Is this the first time we've run into an occasion to make a separate >>> Shenandaoah build?? Surely not.) >> Primitive field accesses required similar jumping through hoops ;-) >> >>>> So... with inlining oopDesc::equals() we will get one straight call >>>> (from the caller of oopDesc::equals() to the impl), instead of 2 >>>> (caller >>>> -> oopDesc::equals() -> impl). >>>> >>>> It's the best I could come up with that does what (Shenandoah) GC >>>> needs, >>>> and it hasn't shown up in any performance testing that we did, yet. >>>> >>>> Still good? >>> I think it's too risky to out-of-line the "cmpq" instruction.? Let's >>> find an agreeable way to handle that via a trade-off that keeps >>> the old code shape in VM builds which don't need the new code >>> shape.? I'm speaking up here because I don't believe this is the >>> last time we'll have to consider adding something gross like an >>> #ifdef to support new features, and I want to aim at trade-offs >>> that keep the existing hot paths efficient, while still allowing >>> new code shapes to become part of the mix. >>> >>> (I'd be happy to see something cleaner than an #ifdef, of >>> course.? Seems to me that a constant non-product flag >>> -XX:+ShortCircuitOopEquals would work too, with a >>> binding of falseInShenandoah.) >> That would require coding 2 versions of hot loops that involve oop==oop >> and switch to one or the other based on that flag. I have checked all >> the uses of oopDesc::equals() in my patch, and none seemed particularily >> critical. I think this would be overkill. >> >> And notice also that none interpreter/c1/c2 compiled code will suffer >> this problem because we'll just generate the right code at runtime. >> >> Roman >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Wed Mar 28 14:07:58 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 28 Mar 2018 16:07:58 +0200 Subject: RFR: 8200228: Change default value of HeapSizePerGCThread In-Reply-To: <2690ce00-921f-9310-648f-69fd41afca19@oracle.com> References: <2690ce00-921f-9310-648f-69fd41afca19@oracle.com> Message-ID: <1522246078.2391.43.camel@oracle.com> Hi, On Wed, 2018-03-28 at 15:52 +0200, Stefan Johansson wrote: > Hi, > > Please review this change of the default value for > HeapSizePerGCThread. > > Links > JBS: https://bugs.openjdk.java.net/browse/JDK-8200228 > Webrev: http://cr.openjdk.java.net/~sjohanss/8200228/00/ > > Summary > My testing shows that for DaCapo, SpecJVM98 and GCBasher (gc > intensive stress test) with small heaps we get better pause times > with the flag set to 32M. There might be other arguments, such as not > using to much resources on small systems that might also be important > so please let me know if you have reasons why the flag should be left > unchanged. looks good. Thomas From thomas.schatzl at oracle.com Wed Mar 28 14:06:06 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 28 Mar 2018 16:06:06 +0200 Subject: RFR (S/M): 8200234: Cleanup Remark and Cleanup pause code In-Reply-To: <283b9e7b-7729-718b-7bc4-22282fa3d7e1@oracle.com> References: <1522079989.2567.43.camel@oracle.com> <283b9e7b-7729-718b-7bc4-22282fa3d7e1@oracle.com> Message-ID: <1522245966.2391.42.camel@oracle.com> Hi, On Wed, 2018-03-28 at 14:18 +0200, Stefan Johansson wrote: > Hi Thomas, > > On 2018-03-26 17:59, Thomas Schatzl wrote: > > Hi all, > > [...] > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8200234 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8200234/webrev/ > > Looks good in general, but a few questions and suggestions: > src/hotspot/share/gc/g1/g1ConcurrentMark.cpp > 539 G1CMBitMap* const bitmap = > _g1h->collector_state()->mark_or_rebuild_in_progress() ? > _next_mark_bitmap : _prev_mark_bitmap; > The old code always used the _next_mark_bitmap, was this a bug or > has any timing changed? This is a (benign?) bug, and actually thinking about it again the current implementation is not completely correct either. :/ Thanks for making me think about this again. The prev ("current") bitmap may contain the mark because of the humongous object surviving some previous marking. Now if you eagerly reclaim that object, the mark will simply remain until the next time we swap the bitmaps and clear the now "next" bitmap, so it's kind of self-healing after all. As for the actual impact, I do not think it has any except for stray verification errors I saw with some of our tests: we never use the bitmap for walking free regions (they are not walked at all), and if we reallocated into that region again, the objects would all be above tams, and so we would not use the bitmap either and live anyway. But I think this issue is unrelated to this cleanup change and will create an extra CR. > 1010 os::snprintf(buffer, BufLen, "During GC (%s)", caller); > Seems like jio_snprintf is more used in the HotSpot code, so maybe > change to use that one instead. Okay, I just did not know which to use. Both seemed equal to me. Fixed. > --- > src/hotspot/share/memory/iterator.hpp > 136 #ifdef ASSERT > 137 bool should_verify_oops() { return false; } > 138 #endif > Why is this needed? I don't see any use of NoHeaderExtendedOopClosure > in the new code. If needed I would prefer: > debug_only(bool should_verify_oops() { return false; }) The reason is that without that change this prevents an OopClosure that does verification to actually trigger its typically more useful error message. But looking at it again at the other collector's verification closure, the consensus is that these verification closures should themselves override this method and not disable it globally. The particular issue is about G1Mux2Closure that if there is an error, it errors out too early, not showing G1 specific information as this assert triggers before the other; I will make a seperate change for that one too though. http://cr.openjdk.java.net/~tschatzl/8200234/webrev.0_to_1 (diff) http://cr.openjdk.java.net/~tschatzl/8200234/webrev.1 (full) Thanks, Thomas From stefan.johansson at oracle.com Wed Mar 28 14:12:57 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 28 Mar 2018 16:12:57 +0200 Subject: RFR (S/M): 8200234: Cleanup Remark and Cleanup pause code In-Reply-To: <1522245966.2391.42.camel@oracle.com> References: <1522079989.2567.43.camel@oracle.com> <283b9e7b-7729-718b-7bc4-22282fa3d7e1@oracle.com> <1522245966.2391.42.camel@oracle.com> Message-ID: <74c9806c-9824-2168-9e34-fd46b869ef37@oracle.com> On 2018-03-28 16:06, Thomas Schatzl wrote: > Hi, > > On Wed, 2018-03-28 at 14:18 +0200, Stefan Johansson wrote: >> Hi Thomas, >> >> On 2018-03-26 17:59, Thomas Schatzl wrote: >>> Hi all, >>> > [...] >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8200234 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8200234/webrev/ >> Looks good in general, but a few questions and suggestions: >> src/hotspot/share/gc/g1/g1ConcurrentMark.cpp >> 539 G1CMBitMap* const bitmap = >> _g1h->collector_state()->mark_or_rebuild_in_progress() ? >> _next_mark_bitmap : _prev_mark_bitmap; >> The old code always used the _next_mark_bitmap, was this a bug or >> has any timing changed? > This is a (benign?) bug, and actually thinking about it again the > current implementation is not completely correct either. :/ > > Thanks for making me think about this again. > > The prev ("current") bitmap may contain the mark because of the > humongous object surviving some previous marking. > > Now if you eagerly reclaim that object, the mark will simply remain > until the next time we swap the bitmaps and clear the now "next" > bitmap, so it's kind of self-healing after all. > > As for the actual impact, I do not think it has any except for stray > verification errors I saw with some of our tests: we never use the > bitmap for walking free regions (they are not walked at all), and if we > reallocated into that region again, the objects would all be above > tams, and so we would not use the bitmap either and live anyway. > > But I think this issue is unrelated to this cleanup change and will > create an extra CR. > >> 1010 os::snprintf(buffer, BufLen, "During GC (%s)", caller); >> Seems like jio_snprintf is more used in the HotSpot code, so maybe >> change to use that one instead. > Okay, I just did not know which to use. Both seemed equal to me. Fixed. > >> --- >> src/hotspot/share/memory/iterator.hpp >> 136 #ifdef ASSERT >> 137 bool should_verify_oops() { return false; } >> 138 #endif >> Why is this needed? I don't see any use of NoHeaderExtendedOopClosure >> in the new code. If needed I would prefer: >> debug_only(bool should_verify_oops() { return false; }) > The reason is that without that change this prevents an OopClosure that > does verification to actually trigger its typically more useful error > message. > > But looking at it again at the other collector's verification closure, > the consensus is that these verification closures should themselves > override this method and not disable it globally. > > The particular issue is about G1Mux2Closure that if there is an error, > it errors out too early, not showing G1 specific information as this > assert triggers before the other; I will make a seperate change for > that one too though. > > http://cr.openjdk.java.net/~tschatzl/8200234/webrev.0_to_1 (diff) > http://cr.openjdk.java.net/~tschatzl/8200234/webrev.1 (full) Thanks for the detailed answers Thomas, This looks good, StefanJ > Thanks, > Thomas > From thomas.schatzl at oracle.com Wed Mar 28 14:28:50 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 28 Mar 2018 16:28:50 +0200 Subject: RFR (S): 8200074: Remove G1ConcurrentMark::_concurrent_marking_in_progress In-Reply-To: References: <1522088250.2567.55.camel@oracle.com> Message-ID: <1522247330.2391.50.camel@oracle.com> Hi Stefan, On Wed, 2018-03-28 at 14:23 +0200, Stefan Johansson wrote: > Looks good, > StefanJ > thanks for your review. Thomas From thomas.schatzl at oracle.com Wed Mar 28 14:29:11 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 28 Mar 2018 16:29:11 +0200 Subject: RFR (S): 8200255: Remove G1CMTask::_concurrent In-Reply-To: <512feca3-2dba-a6a4-87a3-d1f1e830787e@oracle.com> References: <1522088754.2567.56.camel@oracle.com> <512feca3-2dba-a6a4-87a3-d1f1e830787e@oracle.com> Message-ID: <1522247351.2391.51.camel@oracle.com> Hi Stefan, On Wed, 2018-03-28 at 14:24 +0200, Stefan Johansson wrote: > Looks good, > StefanJ > thanks for your review. Thomas From erik.osterlund at oracle.com Wed Mar 28 15:04:21 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 28 Mar 2018 17:04:21 +0200 Subject: RFR: JDK-8199781: Don't use naked == for comparing oops In-Reply-To: <73626aa7-9584-5cf2-3efa-25dd66d25a28@redhat.com> References: <74ca6651-49aa-2b77-d7e9-a588fb2b8417@oracle.com> <096620eb-e252-9689-c157-098314ac9119@redhat.com> <650c2e5c-88d7-3896-1526-386557f5d2df@redhat.com> <2683d96d-4170-f8b7-a48d-12fd0465ec4c@redhat.com> <1fa7bccc-d827-0a6e-ecbb-8ac1352a6449@redhat.com> <25E3C8FE-089E-4AB8-A9B4-1DA48B2FF17D@oracle.com> <5e3f8ffc-0eb9-e4da-d0c0-72becb677b7b@redhat.com> <5ABB9201.7010806@oracle.com> <73626aa7-9584-5cf2-3efa-25dd66d25a28@redhat.com> Message-ID: <5ABBAEF5.4080109@oracle.com> Hi Roman, On 2018-03-28 16:03, Roman Kennke wrote: > Hi Erik, > > Thanks for this work! > > without having looked too deeply at the code (it might be in vain > anyway, what with all the template voodoo stuff going on :-P), I have a > few conceptual questions: > > - For the issue at hand (replacing naked == with oopDesc::equals() or > such), wouldn't it have been enough to mark it with the > INTERNAL_BT_TO_SPACE_INVARIANT decorator, to get inlining for > non-Shenandoah (or more generally, non-to-space-invariant) builds. ? I have done that as well to get inlining for non-to-space-invariant builds. But obviously, by moving e.g. Handles::operator == to the cpp file, and oopDesc::equals to the cpp file, you broke inlining anyway. To get the inlining back, choices were considering comparing two naked oops as not trivial and hence forcing all transitive dependencies to code that compares two naked oops to move to .inline.hpp files, or to make it allowed from .hpp files. This is my solution for allowing existing callsites that compare naked oops in .hpp files (because it was IMO rightfully considered trivial), to remain in .hpp files, and preserve all inlining. Therefore, only looking for that build-time decorator would not have been enough to make this legally allowed from .hpp files, because the very logic for folding away the non-to-space-invariant path was in access.inline.hpp.There must also exist a path for the non-to-space-invariant build from hpp files only to get to get to the GC backend implementation, which relies on .inline.hpp file support. That is why I have shuffled things around to be able to reach all the way up until but not including barrier resolution through the hpp files, and be able to finish the rest of the path in access.cpp for selected operations. > - What is the advantage of this refactoring? It is a refactoring, right? > or do we get new features here? This is a refactoring that allows using Access<>::equals() from old header files that have been included all over the place, that have used raw oop comparisons in .hpp files in the past. One alternative is moving the caller of Access<>::equals to cpp files (and I heard objections to that from both Andrew Haley and John Rose, arguing that this should be a fast operation, and losing the inlining purely because of file dependency technicalities would be a shame). Another alternative is moving these callsites out to .inline.hpp files, and then expanding the transitive closure of required .inline.hpp dependency changes that creates. This is my attempt at making everyone happy - maximized inlining with or without to-space-invariant GCs in the build, without violating transitive .inline.hpp include dependency rules, and without having to move everything that ever transitively depends on comparing two oops to .inline.hpp files. > - So now we can use the Access API by only including access.hpp. Fine. > But this comes at the cost of having actual code in .hpp files? I am not > sure about Hotspot style, but I try to avoid actual code in .hpp files > whenever I can, except *maybe* for the most trivial of stuff. For now, this is only intended for Access<>::equals to solve your problem at hand in this review: not violating transitive .inline.hpp dependencies, while preserving inlining all the way for non-to-space-invariant builds, and at the same time injecting variation points for Shenandoah without any GC backend .inline.hpp dependencies leaking into .hpp files. While this mechanism can be extended for more operations, there is currently no need for it. About code style of calling "actual code" from .hpp files, I agree that it is a kind of gray area, where it is preferable to move non-trivial things to .inline.hpp files. Comparing naked oops has been considered trivial in the past by engineers, (despite calling oop::operator ==() when CHECK_UNHANDLED_OOPS is enabled), or now by calling oopDesc::equals, and I can see why. This patch allows existing code that considered oop1 == oop2 to be trivial enough to be in a header file to continue doing so, without violating transitive include dependency rules. Thanks, /Erik > > Thanks, > Roman > > >> Hi, >> >> The access.hpp support has reincarnated. >> >> Here is an incremental patch on Roman's work (rebased on latest jdk-hs) >> that makes it possible to include access.hpp to call Access<>::equals >> specifically. The infrastructure allows picking more accesses that can >> be used from the hpp file, but I have kept it down to the minimum >> requirements now, so for now it only works for equals. Perhaps in the >> future, we might want to move more operations to be reachable from hpp, >> but not today. >> >> I changed the involved header files to include access.hpp, and made it >> possible to inline this again for non-Shenandoah builds, by checking for >> the INTERNAL_BT_TO_SPACE_INVARIANT build-time decorator. >> >> Builds that only have to-space invariant GCs inline equals to a normal >> native comparison of the pointers. Builds that have at least one >> to-space invariant GC instead generate a single function pointer call. >> This function pointer may be resolved in the access.cpp file, instead of >> the access.inline.hpp file. >> >> Here is an explanation how this works: >> >> The template pipeline for each operation consists of 5 steps, and they >> were previously all performed in the access.inline.hpp file. The steps are: >> >> 1) Set default decorators and decay types >> 2) Reduce types >> 3) Pre-runtime dispatch >> 4) Runtime-dispatch >> 5.a) Barrier resolution (check which GC backend accessor to be used upon >> the first invocation, then patch function pointer to 5.b) >> 5.b) Post-runtime dispatch >> >> I have moved steps 1-4 to the accessBackend.hpp file, which is included >> by access.hpp. This allows performing all steps until the backend >> barrier resolution in only hpp files. This makes it possible to use, >> e.g. raw accesses (although without compressed oops, atomic or order >> access) from only the access.hpp file. The resolution of barriers in the >> GC backends (comprising steps 4-5) remains in the access.inline.hpp >> file. But they can be instantiated from the access.cpp file for well >> chosen accesses and decorator combinations that can be pre-generated in >> the libjvm.so file. The function pointers used in builds that have not >> to-space invariant GCs will be resolved to point right into these >> functions in the access.cpp file at runtime. >> >> This change required moving the decorators to a new file called >> accessDecorators.hpp. I kind of like having them there in a separate file. >> >> Incremental webrev to Roman's patch: >> http://cr.openjdk.java.net/~eosterlund/8199781/webrev.00_01/ >> >> Full webrev: >> http://cr.openjdk.java.net/~eosterlund/8199781/webrev.01/ >> >> Thanks, >> /Erik >> >> On 2018-03-27 09:14, Roman Kennke wrote: >>> Am 27.03.2018 um 00:46 schrieb John Rose: >>>> On Mar 26, 2018, at 2:21 PM, Roman Kennke wrote: >>>>> Am 26.03.2018 um 21:11 schrieb John Rose: >>>>>> ... >>>>>> This is clearly one of those cases. Let's code against surprises >>>>>> (both present and future) by abstracting simpler operations with >>>>>> inlines. >>>>> Thanks John for clarifying. I generally do agree with you that it makes >>>>> sense to inline something like an operator that will compile to a >>>>> single >>>>> instruction. >>>>> >>>>> Unfortunately, this new abstraction will generate a non-inlineable >>>>> call. >>>>> Thanks Erik's work with the Access API, it should be a straight call to >>>>> the right implementation, not even a virtual call, but it's a call >>>>> nonetheless. >>>> What would it take to inline the call, so there's no new performance >>>> hazard introduced? >>>> >>>> This might be a place to add #ifdefs, if the templates don't let us >>>> control the decision statically. >>>> >>>> (I confess I find the templates complex and hard to reason about. >>>> But that's a potential maintenance issue with the access API, not >>>> your work.) >>> The Access API supports static inclusion/exclusion of stuff, and we've >>> used it before to ensable barriers on primitive field accesses for >>> Shenandoah-enabled builds. It works by passing >>> -DSUPPORT_BARRIER_ON_PRIMITIVES to gcc. This is much better than #ifdefs >>> and does the same thing. I can add something similar for object equality. >>> >>>>> The reason why I'm proposing it is that the GC might want to have a say >>>>> about object equality. In the case of Shenandoah, there may be two >>>>> copies of any one object floating around, and comparing the two copies >>>>> might otherwise result in false negatives. >>>> 1. Of course we need a way for some GC algorithms to control >>>> object identity. >>>> >>>> 2. It's good coding discipline to find all the places where native >>>> op== is used and upgrade them to the needed control. >>> I hope I did that with the proposed patch (it's based on years of >>> finding all those places in Shenandoah land ;-) ). >>> >>>> (2a. We also need a way to avoid having op== creep back in >>>> in an uncontrolled way.) >>> We have something for this in Shenandoah too. It works by overriding == >>> in oopsHierarch.hpp and running into ShouldNotReachHere() if both >>> operands are != NULL. This catches all uses of naked == at runtime, when >>> running with +CheckUnhandledOops. It's not perfect, because it doesn't >>> catch uses in places that are not executed in tests, but it's still >>> incredibly useful. The caveat here is that we also need another >>> abstraction for code paths that actually want naked ==, e.g. in the GC. >>> In Shenandoah we have an oopDesc::unsafe_equals() for that. The other >>> alternative would be to drop operators from oop to oopDesc* and compare >>> that (which is exactly what oopDesc::unsafe_equals() does). >>> >>>> 3. We still need an unsurprising performance model for GC's >>>> which don't require the extra identity hook. >>> Yeah. Unfortunately this is C++ and not Java, and the decision which GC >>> to use is made at runtime. The 'best' that I can see is to rip out the >>> indirection in Shenandoah-disabled builds, as proposed above. I don't >>> like it much, but if this is the only way we can get Shenandoah in, then >>> let's go for it? >>> >>>> So we're not there yet, I think. If there's no easy way to adjust >>>> the access API to get the last inlining, I suggest adding an #ifdef >>>> into oopDesc::equals. >>>> >>>> I suspect the happy final solution will be to templatize hot loops >>>> using making the inlinable adjusted op== be part of the "decorations" >>>> of those loops. Then we will have two copies of hot loops in the >>>> JVM, one with the out-of-line hook and one without. >>>> >>>> For now, I think it's enough to have separate builds of the JVM, >>>> one with the #ifdef set "the old way" and one that allows more >>>> flexibility. >>>> >>>> (Is this the first time we've run into an occasion to make a separate >>>> Shenandaoah build? Surely not.) >>> Primitive field accesses required similar jumping through hoops ;-) >>> >>>>> So... with inlining oopDesc::equals() we will get one straight call >>>>> (from the caller of oopDesc::equals() to the impl), instead of 2 >>>>> (caller >>>>> -> oopDesc::equals() -> impl). >>>>> >>>>> It's the best I could come up with that does what (Shenandoah) GC >>>>> needs, >>>>> and it hasn't shown up in any performance testing that we did, yet. >>>>> >>>>> Still good? >>>> I think it's too risky to out-of-line the "cmpq" instruction. Let's >>>> find an agreeable way to handle that via a trade-off that keeps >>>> the old code shape in VM builds which don't need the new code >>>> shape. I'm speaking up here because I don't believe this is the >>>> last time we'll have to consider adding something gross like an >>>> #ifdef to support new features, and I want to aim at trade-offs >>>> that keep the existing hot paths efficient, while still allowing >>>> new code shapes to become part of the mix. >>>> >>>> (I'd be happy to see something cleaner than an #ifdef, of >>>> course. Seems to me that a constant non-product flag >>>> -XX:+ShortCircuitOopEquals would work too, with a >>>> binding of falseInShenandoah.) >>> That would require coding 2 versions of hot loops that involve oop==oop >>> and switch to one or the other based on that flag. I have checked all >>> the uses of oopDesc::equals() in my patch, and none seemed particularily >>> critical. I think this would be overkill. >>> >>> And notice also that none interpreter/c1/c2 compiled code will suffer >>> this problem because we'll just generate the right code at runtime. >>> >>> Roman >>> > From rkennke at redhat.com Wed Mar 28 15:23:07 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 28 Mar 2018 17:23:07 +0200 Subject: RFR: JDK-8199781: Don't use naked == for comparing oops In-Reply-To: <5ABBAEF5.4080109@oracle.com> References: <74ca6651-49aa-2b77-d7e9-a588fb2b8417@oracle.com> <096620eb-e252-9689-c157-098314ac9119@redhat.com> <650c2e5c-88d7-3896-1526-386557f5d2df@redhat.com> <2683d96d-4170-f8b7-a48d-12fd0465ec4c@redhat.com> <1fa7bccc-d827-0a6e-ecbb-8ac1352a6449@redhat.com> <25E3C8FE-089E-4AB8-A9B4-1DA48B2FF17D@oracle.com> <5e3f8ffc-0eb9-e4da-d0c0-72becb677b7b@redhat.com> <5ABB9201.7010806@oracle.com> <73626aa7-9584-5cf2-3efa-25dd66d25a28@redhat.com> <5ABBAEF5.4080109@oracle.com> Message-ID: <6427fa81-62a9-f394-f9e0-c7fc4177c8d6@redhat.com> Am 28.03.2018 um 17:04 schrieb Erik ?sterlund: > Hi Roman, > > On 2018-03-28 16:03, Roman Kennke wrote: >> Hi Erik, >> >> Thanks for this work! >> >> without having looked too deeply at the code (it might be in vain >> anyway, what with all the template voodoo stuff going on :-P), I have a >> few conceptual questions: >> >> - For the issue at hand (replacing naked == with oopDesc::equals() or >> such), wouldn't it have been enough to mark it with the >> INTERNAL_BT_TO_SPACE_INVARIANT decorator, to get inlining for >> non-Shenandoah (or more generally, non-to-space-invariant) builds. ? > > I have done that as well to get inlining for non-to-space-invariant > builds. But obviously, by moving e.g. Handles::operator == to the cpp > file, and oopDesc::equals to the cpp file, you broke inlining anyway. To > get the inlining back, choices were considering comparing two naked oops > as not trivial and hence forcing all transitive dependencies to code > that compares two naked oops to move to .inline.hpp files, or to make it > allowed from .hpp files. This is my solution for allowing existing > callsites that compare naked oops in .hpp files (because it was IMO > rightfully considered trivial), to remain in .hpp files, and preserve > all inlining. Therefore, only looking for that build-time decorator > would not have been enough to make this legally allowed from .hpp files, > because the very logic for folding away the non-to-space-invariant path > was in access.inline.hpp.There must also exist a path for the > non-to-space-invariant build from hpp files only to get to get to the GC > backend implementation, which relies on .inline.hpp file support. That > is why I have shuffled things around to be able to reach all the way up > until but not including barrier resolution through the hpp files, and be > able to finish the rest of the path in access.cpp for selected operations. Ok. But my last patch moved all uses of == into .inline.hpp or .cpp files, out of .hpp to be sure. So that should have solved that, and not break inlining. >> - What is the advantage of this refactoring? It is a refactoring, right? >> or do we get new features here? > > This is a refactoring that allows using Access<>::equals() from old > header files that have been included all over the place, that have used > raw oop comparisons in .hpp files in the past. One alternative is moving > the caller of Access<>::equals to cpp files (and I heard objections to > that from both Andrew Haley and John Rose, arguing that this should be a > fast operation, and losing the inlining purely because of file > dependency technicalities would be a shame). Another alternative is > moving these callsites out to .inline.hpp files, and then expanding the > transitive closure of required .inline.hpp dependency changes that > creates. This is my attempt at making everyone happy - maximized > inlining with or without to-space-invariant GCs in the build, without > violating transitive .inline.hpp include dependency rules, and without > having to move everything that ever transitively depends on comparing > two oops to .inline.hpp files. Sounds ok. I'll give the change a thorough look later today. >> - So now we can use the Access API by only including access.hpp. Fine. >> But this comes at the cost of having actual code in .hpp files? I am not >> sure about Hotspot style, but I try to avoid actual code in .hpp files >> whenever I can, except *maybe* for the most trivial of stuff. > > For now, this is only intended for Access<>::equals to solve your > problem at hand in this review: not violating transitive .inline.hpp > dependencies, while preserving inlining all the way for > non-to-space-invariant builds, and at the same time injecting variation > points for Shenandoah without any GC backend .inline.hpp dependencies > leaking into .hpp files. While this mechanism can be extended for more > operations, there is currently no need for it. > > About code style of calling "actual code" from .hpp files, I agree that > it is a kind of gray area, where it is preferable to move non-trivial > things to .inline.hpp files. Comparing naked oops has been considered > trivial in the past by engineers, (despite calling oop::operator ==() > when CHECK_UNHANDLED_OOPS is enabled), or now by calling > oopDesc::equals, and I can see why. This patch allows existing code that > considered oop1 == oop2 to be trivial enough to be in a header file to > continue doing so, without violating transitive include dependency rules. Ok. Thanks for this support! I'll review it later today, need to rush out for a bit... Cheers, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Wed Mar 28 15:26:52 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 28 Mar 2018 17:26:52 +0200 Subject: RFR (S): 8200305: Update gc,liveness output with remset state after rebuild remset concurrently changes Message-ID: <1522250812.15321.7.camel@oracle.com> Hi all, can I have reviews for this change that updates the gc,liveness output to add the remembered set state? This change adds a column next to "remset" called "state" that can be either UNTRA (Untracked), UPDAT (Updating) and CMPLT (Complete). There is a log snippet attached that shows the new output. If somebody asks, I kind of agree that we should probably rethink this output (remove some of the addresses, add region number), but this is imho out of scope for this change. ;] CR: https://bugs.openjdk.java.net/browse/JDK-8200305 Webrev: http://cr.openjdk.java.net/~tschatzl/8200305/webrev/ Testing: local verification Thanks, Thomas -------------- next part -------------- [1.458s][trace][gc,liveness ] GC(6) ### type address-range used prev-live next-live gc-eff remset state code-roots [1.458s][trace][gc,liveness ] GC(6) ### (bytes) (bytes) (bytes) (bytes/ms) (bytes) (bytes) [1.458s][trace][gc,liveness ] GC(6) ### OLD 0x00000000f8000000-0x00000000f8100000 1048576 1048216 1048216 0.0 12440 UNTRA 9680 From thomas.schatzl at oracle.com Wed Mar 28 15:27:11 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 28 Mar 2018 17:27:11 +0200 Subject: RFR (XS): 8200362: G1Mux2Closure should disable implicit oop verification Message-ID: <1522250831.15321.8.camel@oracle.com> Hi all, can I have reviews for this small change that fixes some annoyance when debugging with G1? In particular, G1Mux2Closure is used by verification to verify and in case of error print out g1 specific error messages. However, since G1Mux2Closure is an OopClosure, it is actually wrapped by NoHeaderExtendedOopClosure in oop_iterate* calls. ExtendedOopClosure has its own verification that is very generic, and may trigger in the same situations as G1Mux2Closure. Disable this implicit verification for G1Mux2Closure so that we always get the g1 specific error messages. CR: https://bugs.openjdk.java.net/browse/JDK-8200362 Webrev: http://cr.openjdk.java.net/~tschatzl/8200362/webrev/ Testing: (mostly finished) hs-tier 1+2 Thanks, Thomas From erik.osterlund at oracle.com Wed Mar 28 15:47:28 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 28 Mar 2018 17:47:28 +0200 Subject: RFR: JDK-8199781: Don't use naked == for comparing oops In-Reply-To: <6427fa81-62a9-f394-f9e0-c7fc4177c8d6@redhat.com> References: <74ca6651-49aa-2b77-d7e9-a588fb2b8417@oracle.com> <096620eb-e252-9689-c157-098314ac9119@redhat.com> <650c2e5c-88d7-3896-1526-386557f5d2df@redhat.com> <2683d96d-4170-f8b7-a48d-12fd0465ec4c@redhat.com> <1fa7bccc-d827-0a6e-ecbb-8ac1352a6449@redhat.com> <25E3C8FE-089E-4AB8-A9B4-1DA48B2FF17D@oracle.com> <5e3f8ffc-0eb9-e4da-d0c0-72becb677b7b@redhat.com> <5ABB9201.7010806@oracle.com> <73626aa7-9584-5cf2-3efa-25dd66d25a28@redhat.com> <5ABBAEF5.4080109@oracle.com> <6427fa81-62a9-f394-f9e0-c7fc4177c8d6@redhat.com> Message-ID: <5ABBB910.2060307@oracle.com> Hi Roman, On 2018-03-28 17:23, Roman Kennke wrote: > Am 28.03.2018 um 17:04 schrieb Erik ?sterlund: >> Hi Roman, >> >> On 2018-03-28 16:03, Roman Kennke wrote: >>> Hi Erik, >>> >>> Thanks for this work! >>> >>> without having looked too deeply at the code (it might be in vain >>> anyway, what with all the template voodoo stuff going on :-P), I have a >>> few conceptual questions: >>> >>> - For the issue at hand (replacing naked == with oopDesc::equals() or >>> such), wouldn't it have been enough to mark it with the >>> INTERNAL_BT_TO_SPACE_INVARIANT decorator, to get inlining for >>> non-Shenandoah (or more generally, non-to-space-invariant) builds. ? >> I have done that as well to get inlining for non-to-space-invariant >> builds. But obviously, by moving e.g. Handles::operator == to the cpp >> file, and oopDesc::equals to the cpp file, you broke inlining anyway. To >> get the inlining back, choices were considering comparing two naked oops >> as not trivial and hence forcing all transitive dependencies to code >> that compares two naked oops to move to .inline.hpp files, or to make it >> allowed from .hpp files. This is my solution for allowing existing >> callsites that compare naked oops in .hpp files (because it was IMO >> rightfully considered trivial), to remain in .hpp files, and preserve >> all inlining. Therefore, only looking for that build-time decorator >> would not have been enough to make this legally allowed from .hpp files, >> because the very logic for folding away the non-to-space-invariant path >> was in access.inline.hpp.There must also exist a path for the >> non-to-space-invariant build from hpp files only to get to get to the GC >> backend implementation, which relies on .inline.hpp file support. That >> is why I have shuffled things around to be able to reach all the way up >> until but not including barrier resolution through the hpp files, and be >> able to finish the rest of the path in access.cpp for selected operations. > Ok. But my last patch moved all uses of == into .inline.hpp or .cpp > files, out of .hpp to be sure. So that should have solved that, and not > break inlining. I'm afraid not. You called oopDesc::equals (that had its definition in oop.inline.hpp) from GrowableArray::safe_equals, defined in growableArray.hpp, hence violating our include dependency rules. You mentioned in the review you did not know how to fix that, as respecting the include dependencies properly would cause a mess when expanding the transitive include dependencies. My patch allows it to stay there without violating our include dependency rules. And obviously, moving things to cpp files is prone to breaking inlining in the general case. >>> - What is the advantage of this refactoring? It is a refactoring, right? >>> or do we get new features here? >> This is a refactoring that allows using Access<>::equals() from old >> header files that have been included all over the place, that have used >> raw oop comparisons in .hpp files in the past. One alternative is moving >> the caller of Access<>::equals to cpp files (and I heard objections to >> that from both Andrew Haley and John Rose, arguing that this should be a >> fast operation, and losing the inlining purely because of file >> dependency technicalities would be a shame). Another alternative is >> moving these callsites out to .inline.hpp files, and then expanding the >> transitive closure of required .inline.hpp dependency changes that >> creates. This is my attempt at making everyone happy - maximized >> inlining with or without to-space-invariant GCs in the build, without >> violating transitive .inline.hpp include dependency rules, and without >> having to move everything that ever transitively depends on comparing >> two oops to .inline.hpp files. > Sounds ok. I'll give the change a thorough look later today. Thank you for having a look at this. >>> - So now we can use the Access API by only including access.hpp. Fine. >>> But this comes at the cost of having actual code in .hpp files? I am not >>> sure about Hotspot style, but I try to avoid actual code in .hpp files >>> whenever I can, except *maybe* for the most trivial of stuff. >> For now, this is only intended for Access<>::equals to solve your >> problem at hand in this review: not violating transitive .inline.hpp >> dependencies, while preserving inlining all the way for >> non-to-space-invariant builds, and at the same time injecting variation >> points for Shenandoah without any GC backend .inline.hpp dependencies >> leaking into .hpp files. While this mechanism can be extended for more >> operations, there is currently no need for it. >> >> About code style of calling "actual code" from .hpp files, I agree that >> it is a kind of gray area, where it is preferable to move non-trivial >> things to .inline.hpp files. Comparing naked oops has been considered >> trivial in the past by engineers, (despite calling oop::operator ==() >> when CHECK_UNHANDLED_OOPS is enabled), or now by calling >> oopDesc::equals, and I can see why. This patch allows existing code that >> considered oop1 == oop2 to be trivial enough to be in a header file to >> continue doing so, without violating transitive include dependency rules. > Ok. Thanks for this support! I'll review it later today, need to rush > out for a bit... No problem Roman, I am happy to help. Thank you for reviewing. /Erik > Cheers, > Roman > From thomas.schatzl at oracle.com Wed Mar 28 15:55:50 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 28 Mar 2018 17:55:50 +0200 Subject: RFR (M): 8199742: Clean up state flags in G1CollectorState In-Reply-To: <1522076812.2567.21.camel@oracle.com> References: <1522076812.2567.21.camel@oracle.com> Message-ID: <1522252550.2315.0.camel@oracle.com> Hi all, On Mon, 2018-03-26 at 17:06 +0200, Thomas Schatzl wrote: > Hi all, > > [...] > These changes are based on the rebuild remembered sets concurrently > changes. As it may be a bit of work to get them working (I already > pushed to first four), here is a webrev that updates latest jdk/hs to > that level at http://cr.openjdk.java.net/~tschatzl/8199742/baseline/w > eb > rev/ since the baseline changes have been pushed now, this manual update of the repo should not be required any more. Thanks, Thomas From sangheon.kim at oracle.com Wed Mar 28 17:03:21 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Wed, 28 Mar 2018 10:03:21 -0700 Subject: RFR [M][7/7]: 8197932: Better split work in rebuild remembered sets phase In-Reply-To: <1522236210.2391.7.camel@oracle.com> References: <1520258985.2532.40.camel@oracle.com> <1520265115.2532.126.camel@oracle.com> <1520953401.3100.41.camel@oracle.com> <1521461029.2323.11.camel@oracle.com> <4110ec65-c66f-eca0-3b7a-6580f0edf00d@oracle.com> <1521715562.2448.15.camel@oracle.com> <1522236210.2391.7.camel@oracle.com> Message-ID: <33b599f4-e317-1044-0564-501db097805f@oracle.com> Hi Thomas, On 03/28/2018 04:23 AM, Thomas Schatzl wrote: > Hi, > > On Tue, 2018-03-27 at 10:50 -0700, sangheon.kim wrote: >> Hi Thomas, >> >> Finally the last piece! :) >> > yes :) > > Unfortunately there are already a few more changes out for review, > although much smaller in scope and complexity. :) > > [...] >>> New webrev: >>> http://cr.openjdk.java.net/~tschatzl/8197932/webrev.2_to_3 (diff) >>> http://cr.openjdk.java.net/~tschatzl/8197932/webrev.3 (full) >> webrev.3 looks good! >> Minor nits below. >> >> ============================= >> src/hotspot/share/gc/g1/g1RemSet.cpp >> 805 void move_to_next() { >> 810 oop next() const { >> (Ignore if you don't agree) >> - I like the idea of using iterator here. But for me, current naming >> seems misleading. Current move_to_next() seems next(). And something >> else for current next(). to_oop()? > I would like to keep it as is until we in the team figured out how to > do iterators in hotspot, and then change as needed. > > Apart from the naming this style is 1:1 compatible to STL iterators > (i.e. move_to_next() -> "++" operator, next() -> * operator) (which > although I expect us to use them in the future, I have no particular > preference for any style), so I chose that. Okay! > >> ============================= >> src/hotspot/share/gc/g1/g1RemSetTrackingPolicy.cpp >> 37 // All non-young and non-closed archive regions need to be >> scanned for references; >> 38 // At every gc we gather references to other regions in >> young, >> and closed archive >> 39 // regions by definition do not have references going >> outside >> the closed archive. >> - Add some explanation about r->is_free() case as well? > I will fix this before pushing. OK > >> ============================= >> src/hotspot/share/gc/g1/g1_globals.hpp >> 261 range(4 * K, 32 * >> M) \ >> - Just question. Why the min is 4K? > 4k is like 1024 references max. This seemed to be a small enough > increment to not cause too much safepointing delay for the smallest > machine we are going to run with G1. Thanks for the answer, agree. > > I am going to push this change as is - your review did not seem to pose > any significant issue. Otherwise there is enough opportunity to fix > this immediately. Thank you for the nice patches!! Sangheon > > Thanks, > Thomas From sangheon.kim at oracle.com Wed Mar 28 17:43:19 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Wed, 28 Mar 2018 10:43:19 -0700 Subject: RFR (M): 8151171: Bring g1ConcurrentMark files up to current coding conventions In-Reply-To: <1522239699.2391.10.camel@oracle.com> References: <1522077275.2567.27.camel@oracle.com> <1522239699.2391.10.camel@oracle.com> Message-ID: <7261ae53-9945-ff37-73ed-eaa3b41ca799@oracle.com> Hi Thomas, On 03/28/2018 05:21 AM, Thomas Schatzl wrote: > Hi Stefan, > > On Wed, 2018-03-28 at 13:26 +0200, Stefan Johansson wrote: >> Hi Thomas, >> >> Very nice cleanup. > thanks for your review. > >> On 2018-03-26 17:14, Thomas Schatzl wrote: >>> Hi all, >>> >>> can I have reviews for this mostly renaming patch that cleans up >>> g1ConcurrentMark files (once again *sigh*). >>> >>> Changes: >>> - camelCase variable and method naming >>> - missing brackets around one-line if-statement bodies. >>> - method naming (getters/setters) >>> - use _g1h member variable instead of _g1 or another method local >>> variable as discussed earlier >>> - use assert_at_safepoint* macros >>> - remove incomprehensible comments >>> - changed "CheckPointRoots" to "Remark" throughout >>> ... >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8151171 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8151171/webrev >> Looks good, just to have one comment ;) >> src/hotspot/share/gc/g1/concurrentMarkThread.cpp >> 331 CMRemark final_cl(_cm); >> 362 CMCleanup cl_cl(_cm); >> >> Rename the instances to remark and cleanup. > :) Done. > > http://cr.openjdk.java.net/~tschatzl/8151171/webrev.0_to_1 (diff) > http://cr.openjdk.java.net/~tschatzl/8151171/webrev.1 (full) Webrev.1 looks good to me. It would be better to update copyright year for concurrentMarkThread.hpp before pushing it. Thanks, Sangheon > > I found some more (very minor) renaming work to be done in > concurrentMarkThread.[hc]pp. Nothing really fancy. > > Thanks, > Thomas > From sangheon.kim at oracle.com Wed Mar 28 17:45:38 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Wed, 28 Mar 2018 10:45:38 -0700 Subject: RFR (S): 8200255: Remove G1CMTask::_concurrent In-Reply-To: <1522088754.2567.56.camel@oracle.com> References: <1522088754.2567.56.camel@oracle.com> Message-ID: Hi Thomas, On 03/26/2018 11:25 AM, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this small change that removes > G1CMTask::_concurrent. That member is a duplicate of > G1ConcurrentMark::_concurrent, but there does not seem to be any > advantage of keeping this duplicate, as it's only used in asserts or > non-performance critical code. > > This removes some imho unnecessary code, so I suggest to do that. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8200255 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8200255/webrev/ > Testing: > local compilation, gcbasher, testing with other changes Looks good to me too. Thanks, Sangheon > > Thanks, > Thomas From sangheon.kim at oracle.com Wed Mar 28 17:54:53 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Wed, 28 Mar 2018 10:54:53 -0700 Subject: RFR (S): 8200074: Remove G1ConcurrentMark::_concurrent_marking_in_progress In-Reply-To: <1522088250.2567.55.camel@oracle.com> References: <1522088250.2567.55.camel@oracle.com> Message-ID: <847d16a9-0e0b-098e-7bf8-b0e7b0575568@oracle.com> Hi Thomas, On 03/26/2018 11:17 AM, Thomas Schatzl wrote: > Hi all, > > can I have reviews for the following change that removes > G1ConcurrentMark::_concurrent_marking_in_progress since it is only used > in some asserts, and at the same time mostly the same > as G1ConcurrentMark::_concurrent (ie. it is used to verify > _concurrent). > > CR: > https://bugs.openjdk.java.net/browse/JDK-8200074 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8200074/webrev > Testing: > hs-tier 1-5 with other changes Looks good to me. Thanks, Sangheon > > Change needs JDK-8200255 to apply cleanly. > > Thanks, > Thomas From thomas.schatzl at oracle.com Wed Mar 28 18:06:30 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 28 Mar 2018 20:06:30 +0200 Subject: RFR (S): 8200074: Remove G1ConcurrentMark::_concurrent_marking_in_progress In-Reply-To: <847d16a9-0e0b-098e-7bf8-b0e7b0575568@oracle.com> References: <1522088250.2567.55.camel@oracle.com> <847d16a9-0e0b-098e-7bf8-b0e7b0575568@oracle.com> Message-ID: <1522260390.5658.1.camel@oracle.com> Hi, On Wed, 2018-03-28 at 10:54 -0700, sangheon.kim wrote: > Hi Thomas, > > On 03/26/2018 11:17 AM, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for the following change that removes > > G1ConcurrentMark::_concurrent_marking_in_progress since it is only > > used > > in some asserts, and at the same time mostly the same > > as G1ConcurrentMark::_concurrent (ie. it is used to verify > > _concurrent). > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8200074 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8200074/webrev > > Testing: > > hs-tier 1-5 with other changes > > Looks good to me. thanks for your review. Thomas From thomas.schatzl at oracle.com Wed Mar 28 18:07:05 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 28 Mar 2018 20:07:05 +0200 Subject: RFR (S): 8200255: Remove G1CMTask::_concurrent In-Reply-To: References: <1522088754.2567.56.camel@oracle.com> Message-ID: <1522260425.5658.2.camel@oracle.com> Hi Sangheon, On Wed, 2018-03-28 at 10:45 -0700, sangheon.kim wrote: > Hi Thomas, > > On 03/26/2018 11:25 AM, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this small change [...] > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8200255 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8200255/webrev/ > > Testing: > > local compilation, gcbasher, testing with other changes > > Looks good to me too. thanks for your review. Thomas From sangheon.kim at oracle.com Wed Mar 28 18:18:50 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Wed, 28 Mar 2018 11:18:50 -0700 Subject: RFR (S): 8200305: Update gc,liveness output with remset state after rebuild remset concurrently changes In-Reply-To: <1522250812.15321.7.camel@oracle.com> References: <1522250812.15321.7.camel@oracle.com> Message-ID: Hi Thomas, On 03/28/2018 08:26 AM, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that updates the gc,liveness > output to add the remembered set state? > > This change adds a column next to "remset" called "state" that can be > either UNTRA (Untracked), UPDAT (Updating) and CMPLT (Complete). > > There is a log snippet attached that shows the new output. > > If somebody asks, I kind of agree that we should probably rethink this > output (remove some of the addresses, add region number), but this is > imho out of scope for this change. ;] > > CR: > https://bugs.openjdk.java.net/browse/JDK-8200305 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8200305/webrev/ > Testing: > local verification Looks good as is. Adding the state is a good idea. But these short version strings are not easy to catch. Just printing original strings(9 characters) are too long? :) Thanks, Sangheon > > Thanks, > Thomas > From thomas.schatzl at oracle.com Wed Mar 28 18:32:08 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 28 Mar 2018 20:32:08 +0200 Subject: RFR (M): 8151171: Bring g1ConcurrentMark files up to current coding conventions In-Reply-To: <7261ae53-9945-ff37-73ed-eaa3b41ca799@oracle.com> References: <1522077275.2567.27.camel@oracle.com> <1522239699.2391.10.camel@oracle.com> <7261ae53-9945-ff37-73ed-eaa3b41ca799@oracle.com> Message-ID: <1522261928.5658.3.camel@oracle.com> Hi Sangheon, On Wed, 2018-03-28 at 10:43 -0700, sangheon.kim wrote: > Hi Thomas, > > On 03/28/2018 05:21 AM, Thomas Schatzl wrote: > > Hi Stefan, > > [...] > > http://cr.openjdk.java.net/~tschatzl/8151171/webrev.0_to_1 (diff) > > http://cr.openjdk.java.net/~tschatzl/8151171/webrev.1 (full) > > Webrev.1 looks good to me. > > It would be better to update copyright year for > concurrentMarkThread.hpp before pushing it. Done in the local copy. Thanks for your review. Thomas From thomas.schatzl at oracle.com Wed Mar 28 18:35:38 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 28 Mar 2018 20:35:38 +0200 Subject: RFR (S): 8200305: Update gc,liveness output with remset state after rebuild remset concurrently changes In-Reply-To: References: <1522250812.15321.7.camel@oracle.com> Message-ID: <1522262138.5658.6.camel@oracle.com> Hi, On Wed, 2018-03-28 at 11:18 -0700, sangheon.kim wrote: > Hi Thomas, > > On 03/28/2018 08:26 AM, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this change that updates the gc,liveness > > output to add the remembered set state? > > > > This change adds a column next to "remset" called "state" that can > > be > > either UNTRA (Untracked), UPDAT (Updating) and CMPLT (Complete). > > > > There is a log snippet attached that shows the new output. > > > > If somebody asks, I kind of agree that we should probably rethink > > this > > output (remove some of the addresses, add region number), but this > > is > > imho out of scope for this change. ;] > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8200305 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8200305/webrev/ > > Testing: > > local verification > > Looks good as is. > > Adding the state is a good idea. > But these short version strings are not easy to catch. Just printing > original strings(9 characters) are too long? :) > yes, I am aware of this issue, I actually thought about this for an unusual amount of time too. After all I decided to keep the abbreviations. And I understand that particularly UNTRA vs. UPDAT might be a candidate for confusion, but then again it seemed better than the long strings. However if somebody else also prefers the long strings, I will change that without further delay. Thanks, Thomas From rkennke at redhat.com Wed Mar 28 22:31:15 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 29 Mar 2018 00:31:15 +0200 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: <30509316-0d9e-40c0-deec-8767990c070d@oracle.com> References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> <5AAFDF8A.1080309@oracle.com> <30509316-0d9e-40c0-deec-8767990c070d@oracle.com> Message-ID: <7f3dc351-4184-466d-8aaf-c777acb8fe3f@redhat.com> BTW, we do have a working prototype of 'JNI lazy critical functions' or whatever it is called in Shenandoah now, using pinning: http://hg.openjdk.java.net/shenandoah/jdk/rev/232d819bc452 It looks like it could use the same API that I proposed (pin_object() / unpin_object()), plus an extra method to indicate if the GC would like to use it, and generate the corresponding stub code. I came to think that it was an unfortunate decision to put the GCLocker stuff in pin_object() / unpin_object(). I think I outlined the reasons elsewhere in this thread, it basically boils down to what I consider unnecessary coupling of the two possible ways to implement JNI critical fluff. Roman > This looks much better, thanks for fixing. Reviewed. > > Btw, as Erik mentioned, there are JavaCritical_ libraries out there on > many other platforms (I've seen performance critical stuff like OpenGL > library bindings using this... anyway, Shenandoah can opt-out from that) > > cheers, > /Per > > On 03/19/2018 06:07 PM, Roman Kennke wrote: >> Hi Erik, >> >> thanks for reviewing! >> >> I can do that, but a GC would get the opportunity to do both, or one or >> the other in any case. As it stands, the JNI side of the GCLocker >> protocol is always executed. If a GC decides to participate in GCLocker >> protocol, by calling GCLocker::check_active_before_gc() and related API, >> then will get JNI Critical function support, otherwise GCLocker is >> basically counting up and down counters. >> >> If it instead wishes to use pinning and ignore JNI critical functions, >> then it only needs to override the pinning methods and not participate >> in GCLocker, and everything would be fine. >> >> However, I don't mind either way, so here's the alternative: >> Diff: >> http://cr.openjdk.java.net/~rkennke/8199620/webrev.02.diff/ >> Full: >> http://cr.openjdk.java.net/~rkennke/8199620/webrev.02/ >> >> Better? >> >> Thanks, >> Roman >> >>> Would there be a problem for you to move the GCLocker stuff into the >>> default implementation of object pinning? >>> I have seen code in the wild (not just Solaris) outside of hotspot that >>> exploits critical native. I would feel less worried if a given GC is not >>> forced to choose between completely ignoring the GC locking protocol >>> (and hence breaking such applications), or only locking. >>> >>> Thanks, >>> /Erik >>> >>> On 2018-03-19 15:23, Roman Kennke wrote: >>>> Am 14.03.2018 um 19:13 schrieb Roman Kennke: >>>>> Currently, the Get/Release*Critical() family of functions use the >>>>> GCLocker protocol to ensure that no JNI critical arrays are in use >>>>> when >>>>> a GC pause is entered. >>>>> >>>>> Some GCs may instead want to use object pinning and guarantee that the >>>>> object does not move. For example, this is easy to do with >>>>> region-based >>>>> GCs (G1, Shenandoah, ZGC) by simply not including regions with pinned >>>>> objects in the collection set. >>>>> >>>>> The implementation/API that I'm proposing is fairly simple: add two >>>>> methods oop pin_object(oop) and void unpin_object(oop) to >>>>> CollectedHeap, >>>>> and call them from JNI's Get/Release*Critical methods. >>>>> >>>>> This approach has been working perfectly fine since a long time in >>>>> Shenandoah. >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8199620 >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~rkennke/8199620/webrev.00/ >>>>> >>>>> >>>> Ping? Any takers for this? >>>> >>>> Roman >>>> >>> >> >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From sangheon.kim at oracle.com Thu Mar 29 06:27:17 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Wed, 28 Mar 2018 23:27:17 -0700 Subject: RFR (M): 8199742: Clean up state flags in G1CollectorState In-Reply-To: <1522157120.28255.2.camel@oracle.com> References: <1522076812.2567.21.camel@oracle.com> <9f028672-bc86-525e-3c19-a3f347b1c118@oracle.com> <1522157120.28255.2.camel@oracle.com> Message-ID: Hi Thomas, On 03/27/2018 06:25 AM, Thomas Schatzl wrote: > Hi Stefan, > > On Tue, 2018-03-27 at 14:58 +0200, Stefan Johansson wrote: >> Hi Thomas, >> >> Thanks for this much needed cleanup :) >> >> On 2018-03-26 17:06, Thomas Schatzl wrote: >>> Hi all, >>> >>> I would like to request reviews for this change that cleans up >>> the flags in G1CollectorState, applying uniform naming, removing >>> members that were basically temporary variables for a single >>> method, and [...] >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8199742 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8199742/webrev >> The change looks good. Just some minor things, gc_are_young and >> during_im are used as local variables at some places. I think we >> could change those to reflect the new better names as well. > I grepped through the sources for the old names and changed the uses to > something better (I hope). > > http://cr.openjdk.java.net/~tschatzl/8199742/webrev.0_to_1 (diff) > http://cr.openjdk.java.net/~tschatzl/8199742/webrev.1 (full) webrev.1 looks good to me. I have some minor comments. ======================= src/hotspot/share/gc/g1/g1CollectorState.hpp 41?? // If initiate_conc_mark_if_possible() is set at the beginning of a - _initiate_conc_mark_if_possible instead of initiate_conc_mark_if_possible() as other cases are using a variable name. ======================= src/hotspot/share/gc/g1/g1ConcurrentMark.cpp 1899?? if (!G1CollectedHeap::heap()->collector_state()->mark_or_rebuild_in_progress()) { - Pre-existing issue: _g1h instead of G1CollectedHeap::heap(). This should be commented on JDK-8151171 but I missed. :) Probably need an extra CR for G1CollectedHeap::heap() cleanup as other classes also would use it even though the class has a member of G1CollectedHeap. e.g. g1CollectionSet.cpp:260 I don't need an extra webrev for above comment(s) if you will fix. Thanks, Sangheon > > Thanks, > Thomas > From stefan.johansson at oracle.com Thu Mar 29 07:52:03 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 29 Mar 2018 09:52:03 +0200 Subject: RFR (XS): 8200362: G1Mux2Closure should disable implicit oop verification In-Reply-To: <1522250831.15321.8.camel@oracle.com> References: <1522250831.15321.8.camel@oracle.com> Message-ID: <95b30fe5-f2c8-7ff0-efa6-3dac25a220dc@oracle.com> Hi Thomas, On 2018-03-28 17:27, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this small change that fixes some annoyance > when debugging with G1? > > In particular, G1Mux2Closure is used by verification to verify and in > case of error print out g1 specific error messages. > > However, since G1Mux2Closure is an OopClosure, it is actually wrapped > by NoHeaderExtendedOopClosure in oop_iterate* calls. ExtendedOopClosure > has its own verification that is very generic, and may trigger in the > same situations as G1Mux2Closure. > > Disable this implicit verification for G1Mux2Closure so that we always > get the g1 specific error messages. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8200362 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8200362/webrev/ Nice change, always good to get detailed information. I think you need to change the code using the G1Mux2Closure as well. On line 666 in heapRegion.cpp: ? G1Mux2Closure mux(&vl_cl, &vr_cl); ? obj->oop_iterate_no_header(&mux); This should now be a normal oop_iterate(&mux) to make sure the right verification is done. Thanks, Stefan > Testing: > (mostly finished) hs-tier 1+2 > > Thanks, > Thomas > From stefan.johansson at oracle.com Thu Mar 29 07:58:26 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 29 Mar 2018 09:58:26 +0200 Subject: RFR (M): 8151171: Bring g1ConcurrentMark files up to current coding conventions In-Reply-To: <1522261928.5658.3.camel@oracle.com> References: <1522077275.2567.27.camel@oracle.com> <1522239699.2391.10.camel@oracle.com> <7261ae53-9945-ff37-73ed-eaa3b41ca799@oracle.com> <1522261928.5658.3.camel@oracle.com> Message-ID: On 2018-03-28 20:32, Thomas Schatzl wrote: > Hi Sangheon, > > On Wed, 2018-03-28 at 10:43 -0700, sangheon.kim wrote: >> Hi Thomas, >> >> On 03/28/2018 05:21 AM, Thomas Schatzl wrote: >>> Hi Stefan, >>> > [...] >>> http://cr.openjdk.java.net/~tschatzl/8151171/webrev.0_to_1 (diff) >>> http://cr.openjdk.java.net/~tschatzl/8151171/webrev.1 (full) >> Webrev.1 looks good to me. >> >> It would be better to update copyright year for >> concurrentMarkThread.hpp before pushing it. > Done in the local copy. Thanks for your review. Ship it! > Thomas > From thomas.schatzl at oracle.com Thu Mar 29 08:33:21 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 29 Mar 2018 10:33:21 +0200 Subject: RFR (M): 8199742: Clean up state flags in G1CollectorState In-Reply-To: References: <1522076812.2567.21.camel@oracle.com> <9f028672-bc86-525e-3c19-a3f347b1c118@oracle.com> <1522157120.28255.2.camel@oracle.com> Message-ID: <1522312401.2400.1.camel@oracle.com> Hi, thanks for your review :) On Wed, 2018-03-28 at 23:27 -0700, sangheon.kim wrote: > Hi Thomas, > > On 03/27/2018 06:25 AM, Thomas Schatzl wrote: > > Hi Stefan, > > > > On Tue, 2018-03-27 at 14:58 +0200, Stefan Johansson wrote: > > > Hi Thomas, > > > > > > Thanks for this much needed cleanup :) > > > > > > On 2018-03-26 17:06, Thomas Schatzl wrote: > > > > Hi all, > > > > > > > > I would like to request reviews for this change that cleans > > > > up > > > > the flags in G1CollectorState, applying uniform naming, > > > > removing > > > > members that were basically temporary variables for a single > > > > method, and [...] > > > > CR: > > > > https://bugs.openjdk.java.net/browse/JDK-8199742 > > > > Webrev: > > > > http://cr.openjdk.java.net/~tschatzl/8199742/webrev > > > > > > The change looks good. Just some minor things, gc_are_young and > > > during_im are used as local variables at some places. I think we > > > could change those to reflect the new better names as well. > > > > I grepped through the sources for the old names and changed the > > uses to > > something better (I hope). > > > > http://cr.openjdk.java.net/~tschatzl/8199742/webrev.0_to_1 (diff) > > http://cr.openjdk.java.net/~tschatzl/8199742/webrev.1 (full) > > webrev.1 looks good to me. > I have some minor comments. > > ======================= > src/hotspot/share/gc/g1/g1CollectorState.hpp > 41 // If initiate_conc_mark_if_possible() is set at the beginning > of a > - _initiate_conc_mark_if_possible instead of > initiate_conc_mark_if_possible() as other cases are using a variable > name. Fixed. > > ======================= > src/hotspot/share/gc/g1/g1ConcurrentMark.cpp > 1899 if > (!G1CollectedHeap::heap()->collector_state()- > >mark_or_rebuild_in_progress()) > { > - Pre-existing issue: _g1h instead of G1CollectedHeap::heap(). This > should be commented on JDK-8151171 but I missed. :) Probably need an I added this tiny change to JDK-8151171 before pushing. Nice catch. > extra CR for G1CollectedHeap::heap() cleanup as other classes also > would use it even though the class has a member of G1CollectedHeap. > e.g. g1CollectionSet.cpp:260 I know that other classes are guilty of the same issue. Thanks, Thomas From stefan.johansson at oracle.com Thu Mar 29 08:52:43 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 29 Mar 2018 10:52:43 +0200 Subject: RFR: 8200228: Change default value of HeapSizePerGCThread In-Reply-To: <2690ce00-921f-9310-648f-69fd41afca19@oracle.com> References: <2690ce00-921f-9310-648f-69fd41afca19@oracle.com> Message-ID: <05493def-e244-8710-04e0-cbb70a40f3f2@oracle.com> I've now created a CSR as well: https://bugs.openjdk.java.net/browse/JDK-8200417 Please have a look at it and add yourself as a reviewer if everything looks good. Thanks, Stefan On 2018-03-28 15:52, Stefan Johansson wrote: > Hi, > > Please review this change of the default value for HeapSizePerGCThread. > > Links > JBS: https://bugs.openjdk.java.net/browse/JDK-8200228 > Webrev: http://cr.openjdk.java.net/~sjohanss/8200228/00/ > > Summary > My testing shows that for DaCapo, SpecJVM98 and GCBasher (gc intensive > stress test) with small heaps we get better pause times with the flag > set to 32M. There might be other arguments, such as not using to much > resources on small systems that might also be important so please let > me know if you have reasons why the flag should be left unchanged. > > Thanks, > Stefan > From stefan.johansson at oracle.com Thu Mar 29 08:53:04 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 29 Mar 2018 10:53:04 +0200 Subject: RFR: 8200228: Change default value of HeapSizePerGCThread In-Reply-To: <1522246078.2391.43.camel@oracle.com> References: <2690ce00-921f-9310-648f-69fd41afca19@oracle.com> <1522246078.2391.43.camel@oracle.com> Message-ID: <507e34ce-646b-c864-ca54-6f432e59c630@oracle.com> Thanks for the review Thomas, Stefan On 2018-03-28 16:07, Thomas Schatzl wrote: > Hi, > > On Wed, 2018-03-28 at 15:52 +0200, Stefan Johansson wrote: >> Hi, >> >> Please review this change of the default value for >> HeapSizePerGCThread. >> >> Links >> JBS: https://bugs.openjdk.java.net/browse/JDK-8200228 >> Webrev: http://cr.openjdk.java.net/~sjohanss/8200228/00/ >> >> Summary >> My testing shows that for DaCapo, SpecJVM98 and GCBasher (gc >> intensive stress test) with small heaps we get better pause times >> with the flag set to 32M. There might be other arguments, such as not >> using to much resources on small systems that might also be important >> so please let me know if you have reasons why the flag should be left >> unchanged. > looks good. > > Thomas > From thomas.schatzl at oracle.com Thu Mar 29 09:02:48 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 29 Mar 2018 11:02:48 +0200 Subject: RFR (XS): 8200362: G1Mux2Closure should disable implicit oop verification In-Reply-To: <95b30fe5-f2c8-7ff0-efa6-3dac25a220dc@oracle.com> References: <1522250831.15321.8.camel@oracle.com> <95b30fe5-f2c8-7ff0-efa6-3dac25a220dc@oracle.com> Message-ID: <1522314168.2400.3.camel@oracle.com> Hi Stefan, On Thu, 2018-03-29 at 09:52 +0200, Stefan Johansson wrote: > Hi Thomas, > > On 2018-03-28 17:27, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this small change that fixes some > > annoyance > > when debugging with G1? > > > > In particular, G1Mux2Closure is used by verification to verify and > > in > > case of error print out g1 specific error messages. > > > > However, since G1Mux2Closure is an OopClosure, it is actually > > wrapped > > by NoHeaderExtendedOopClosure in oop_iterate* calls. > > ExtendedOopClosure > > has its own verification that is very generic, and may trigger in > > the > > same situations as G1Mux2Closure. > > > > Disable this implicit verification for G1Mux2Closure so that we > > always > > get the g1 specific error messages. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8200362 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8200362/webrev/ > > Nice change, always good to get detailed information. I think you > need > to change the code using the G1Mux2Closure as well. On line 666 in > heapRegion.cpp: > G1Mux2Closure mux(&vl_cl, &vr_cl); > obj->oop_iterate_no_header(&mux); > This should now be a normal oop_iterate(&mux) to make sure the right > verification is done. Fixed. Also looked through other closures used for verification. http://cr.openjdk.java.net/~tschatzl/8200362/webrev.0_to_1 (diff) http://cr.openjdk.java.net/~tschatzl/8200362/webrev.1 (full) Thomas From stefan.johansson at oracle.com Thu Mar 29 09:07:36 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 29 Mar 2018 11:07:36 +0200 Subject: RFR (XS): 8200362: G1Mux2Closure should disable implicit oop verification In-Reply-To: <1522314168.2400.3.camel@oracle.com> References: <1522250831.15321.8.camel@oracle.com> <95b30fe5-f2c8-7ff0-efa6-3dac25a220dc@oracle.com> <1522314168.2400.3.camel@oracle.com> Message-ID: <92c86e9f-407d-9403-a735-42d521a5bd07@oracle.com> Thanks Thomas, On 2018-03-29 11:02, Thomas Schatzl wrote: > Hi Stefan, > > On Thu, 2018-03-29 at 09:52 +0200, Stefan Johansson wrote: >> Hi Thomas, >> >> On 2018-03-28 17:27, Thomas Schatzl wrote: >>> Hi all, >>> >>> can I have reviews for this small change that fixes some >>> annoyance >>> when debugging with G1? >>> >>> In particular, G1Mux2Closure is used by verification to verify and >>> in >>> case of error print out g1 specific error messages. >>> >>> However, since G1Mux2Closure is an OopClosure, it is actually >>> wrapped >>> by NoHeaderExtendedOopClosure in oop_iterate* calls. >>> ExtendedOopClosure >>> has its own verification that is very generic, and may trigger in >>> the >>> same situations as G1Mux2Closure. >>> >>> Disable this implicit verification for G1Mux2Closure so that we >>> always >>> get the g1 specific error messages. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8200362 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8200362/webrev/ >> Nice change, always good to get detailed information. I think you >> need >> to change the code using the G1Mux2Closure as well. On line 666 in >> heapRegion.cpp: >> G1Mux2Closure mux(&vl_cl, &vr_cl); >> obj->oop_iterate_no_header(&mux); >> This should now be a normal oop_iterate(&mux) to make sure the right >> verification is done. > Fixed. Also looked through other closures used for verification. > > http://cr.openjdk.java.net/~tschatzl/8200362/webrev.0_to_1 (diff) > http://cr.openjdk.java.net/~tschatzl/8200362/webrev.1 (full) Looks good, Stefan > Thomas From thomas.schatzl at oracle.com Thu Mar 29 09:12:07 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 29 Mar 2018 11:12:07 +0200 Subject: RFR (S): 8200385: Eagerly reclaimed humongous objects leave mark in prev bitmap Message-ID: <1522314727.2400.10.camel@oracle.com> Hi all, can I have reviews for this small fix to a benign bug, that is I haven't seen any actual product failure from it but some very rare test failures, where when we eagerly reclaim humongous objects we leave a mark on the prev bitmap in some cases? The suggested fix is to always look at the prev bitmap and clear it, and if needed also clear potential marks in the next bitmap. To make the failure appear basically 100% in that test, I added a simple assert after reclaiming the humongous object. With the fix, this failure goes away completely. Note that this is more a "data structure hygiene" fix - the stray mark on the prev bitmap will be automatically cleared after switching the bitmaps at cleanup and preparing that bitmap for the next mark. CR: https://bugs.openjdk.java.net/browse/JDK-8200385 Webrev: http://cr.openjdk.java.net/~tschatzl/8200385/webrev/ Testing: local testing with existing tests (gc/g1/TestEagerReclaimHumongousRegionsClearMarkBits.java) Thanks, Thomas From stefan.karlsson at oracle.com Thu Mar 29 09:20:59 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 29 Mar 2018 11:20:59 +0200 Subject: RFR (XS): 8200362: G1Mux2Closure should disable implicit oop verification In-Reply-To: <1522314168.2400.3.camel@oracle.com> References: <1522250831.15321.8.camel@oracle.com> <95b30fe5-f2c8-7ff0-efa6-3dac25a220dc@oracle.com> <1522314168.2400.3.camel@oracle.com> Message-ID: <5225d203-2207-85aa-c7f2-fa624d4585e9@oracle.com> On 2018-03-29 11:02, Thomas Schatzl wrote: > Hi Stefan, > > On Thu, 2018-03-29 at 09:52 +0200, Stefan Johansson wrote: >> Hi Thomas, >> >> On 2018-03-28 17:27, Thomas Schatzl wrote: >>> Hi all, >>> >>> can I have reviews for this small change that fixes some >>> annoyance >>> when debugging with G1? >>> >>> In particular, G1Mux2Closure is used by verification to verify and >>> in >>> case of error print out g1 specific error messages. >>> >>> However, since G1Mux2Closure is an OopClosure, it is actually >>> wrapped >>> by NoHeaderExtendedOopClosure in oop_iterate* calls. >>> ExtendedOopClosure >>> has its own verification that is very generic, and may trigger in >>> the >>> same situations as G1Mux2Closure. >>> >>> Disable this implicit verification for G1Mux2Closure so that we >>> always >>> get the g1 specific error messages. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8200362 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8200362/webrev/ >> >> Nice change, always good to get detailed information. I think you >> need >> to change the code using the G1Mux2Closure as well. On line 666 in >> heapRegion.cpp: >> G1Mux2Closure mux(&vl_cl, &vr_cl); >> obj->oop_iterate_no_header(&mux); >> This should now be a normal oop_iterate(&mux) to make sure the right >> verification is done. > > Fixed. Also looked through other closures used for verification. > > http://cr.openjdk.java.net/~tschatzl/8200362/webrev.0_to_1 (diff) > http://cr.openjdk.java.net/~tschatzl/8200362/webrev.1 (full) Looks good. I like that we get rid of usages of oop_iterate_no_header. Maybe we should remove the other usages in G1, as a separate RFE. StefanK > > Thomas > From stefan.johansson at oracle.com Thu Mar 29 09:25:01 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 29 Mar 2018 11:25:01 +0200 Subject: RFR (XS): 8200362: G1Mux2Closure should disable implicit oop verification In-Reply-To: <5225d203-2207-85aa-c7f2-fa624d4585e9@oracle.com> References: <1522250831.15321.8.camel@oracle.com> <95b30fe5-f2c8-7ff0-efa6-3dac25a220dc@oracle.com> <1522314168.2400.3.camel@oracle.com> <5225d203-2207-85aa-c7f2-fa624d4585e9@oracle.com> Message-ID: <4ba778ab-31c1-0bdd-a929-7b6bec4941d4@oracle.com> On 2018-03-29 11:20, Stefan Karlsson wrote: > > > On 2018-03-29 11:02, Thomas Schatzl wrote: >> Hi Stefan, >> >> On Thu, 2018-03-29 at 09:52 +0200, Stefan Johansson wrote: >>> Hi Thomas, >>> >>> On 2018-03-28 17:27, Thomas Schatzl wrote: >>>> Hi all, >>>> >>>> ??? can I have reviews for this small change that fixes some >>>> annoyance >>>> when debugging with G1? >>>> >>>> In particular, G1Mux2Closure is used by verification to verify and >>>> in >>>> case of error print out g1 specific error messages. >>>> >>>> However, since G1Mux2Closure is an OopClosure, it is actually >>>> wrapped >>>> by NoHeaderExtendedOopClosure in oop_iterate* calls. >>>> ExtendedOopClosure >>>> has its own verification that is very generic, and may trigger in >>>> the >>>> same situations as G1Mux2Closure. >>>> >>>> Disable this implicit verification for G1Mux2Closure so that we >>>> always >>>> get the g1 specific error messages. >>>> >>>> CR: >>>> https://bugs.openjdk.java.net/browse/JDK-8200362 >>>> Webrev: >>>> http://cr.openjdk.java.net/~tschatzl/8200362/webrev/ >>> >>> Nice change, always good to get detailed information. I think you >>> need >>> to change the code using the G1Mux2Closure as well. On line 666 in >>> heapRegion.cpp: >>> ??? G1Mux2Closure mux(&vl_cl, &vr_cl); >>> ??? obj->oop_iterate_no_header(&mux); >>> This should now be a normal oop_iterate(&mux) to make sure the right >>> verification is done. >> >> Fixed. Also looked through other closures used for verification. >> >> http://cr.openjdk.java.net/~tschatzl/8200362/webrev.0_to_1 (diff) >> http://cr.openjdk.java.net/~tschatzl/8200362/webrev.1 (full) > > Looks good. > > I like that we get rid of usages of oop_iterate_no_header. Maybe we > should remove the other usages in G1, as a separate RFE. > I like the sound of that, maybe something for Leo to look at. Any history or reason why we have these? StefanJ > StefanK > >> >> Thomas >> From per.liden at oracle.com Thu Mar 29 09:29:04 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 29 Mar 2018 11:29:04 +0200 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: <7f3dc351-4184-466d-8aaf-c777acb8fe3f@redhat.com> References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> <5AAFDF8A.1080309@oracle.com> <30509316-0d9e-40c0-deec-8767990c070d@oracle.com> <7f3dc351-4184-466d-8aaf-c777acb8fe3f@redhat.com> Message-ID: <4f40f1c7-36b4-a1aa-b8df-defee13643de@oracle.com> Hi Roman, On 03/29/2018 12:31 AM, Roman Kennke wrote: > BTW, we do have a working prototype of 'JNI lazy critical functions' or > whatever it is called in Shenandoah now, using pinning: > > http://hg.openjdk.java.net/shenandoah/jdk/rev/232d819bc452 > > It looks like it could use the same API that I proposed (pin_object() / > unpin_object()), plus an extra method to indicate if the GC would like > to use it, and generate the corresponding stub code. > > I came to think that it was an unfortunate decision to put the GCLocker > stuff in pin_object() / unpin_object(). I think I outlined the reasons > elsewhere in this thread, it basically boils down to what I consider > unnecessary coupling of the two possible ways to implement JNI critical > fluff. The main objection people had with your original patch was that it both grabbed the GCLocker _and_ potentially pinned the object. I think that was perceived as both unnecessary and confusing. However, the design we have now isn't set in stone by any means. How about changing it to this: http://cr.openjdk.java.net/~pliden/supports_object_pinning/webrev.0 This gives you the ability to ask the collector if pinning is supported, and then choose what to do. I think this should cover the need you have, right? I'm currently running it though testing. /Per > > Roman > > >> This looks much better, thanks for fixing. Reviewed. >> >> Btw, as Erik mentioned, there are JavaCritical_ libraries out there on >> many other platforms (I've seen performance critical stuff like OpenGL >> library bindings using this... anyway, Shenandoah can opt-out from that) >> >> cheers, >> /Per >> >> On 03/19/2018 06:07 PM, Roman Kennke wrote: >>> Hi Erik, >>> >>> thanks for reviewing! >>> >>> I can do that, but a GC would get the opportunity to do both, or one or >>> the other in any case. As it stands, the JNI side of the GCLocker >>> protocol is always executed. If a GC decides to participate in GCLocker >>> protocol, by calling GCLocker::check_active_before_gc() and related API, >>> then will get JNI Critical function support, otherwise GCLocker is >>> basically counting up and down counters. >>> >>> If it instead wishes to use pinning and ignore JNI critical functions, >>> then it only needs to override the pinning methods and not participate >>> in GCLocker, and everything would be fine. >>> >>> However, I don't mind either way, so here's the alternative: >>> Diff: >>> http://cr.openjdk.java.net/~rkennke/8199620/webrev.02.diff/ >>> Full: >>> http://cr.openjdk.java.net/~rkennke/8199620/webrev.02/ >>> >>> Better? >>> >>> Thanks, >>> Roman >>> >>>> Would there be a problem for you to move the GCLocker stuff into the >>>> default implementation of object pinning? >>>> I have seen code in the wild (not just Solaris) outside of hotspot that >>>> exploits critical native. I would feel less worried if a given GC is not >>>> forced to choose between completely ignoring the GC locking protocol >>>> (and hence breaking such applications), or only locking. >>>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2018-03-19 15:23, Roman Kennke wrote: >>>>> Am 14.03.2018 um 19:13 schrieb Roman Kennke: >>>>>> Currently, the Get/Release*Critical() family of functions use the >>>>>> GCLocker protocol to ensure that no JNI critical arrays are in use >>>>>> when >>>>>> a GC pause is entered. >>>>>> >>>>>> Some GCs may instead want to use object pinning and guarantee that the >>>>>> object does not move. For example, this is easy to do with >>>>>> region-based >>>>>> GCs (G1, Shenandoah, ZGC) by simply not including regions with pinned >>>>>> objects in the collection set. >>>>>> >>>>>> The implementation/API that I'm proposing is fairly simple: add two >>>>>> methods oop pin_object(oop) and void unpin_object(oop) to >>>>>> CollectedHeap, >>>>>> and call them from JNI's Get/Release*Critical methods. >>>>>> >>>>>> This approach has been working perfectly fine since a long time in >>>>>> Shenandoah. >>>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8199620 >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~rkennke/8199620/webrev.00/ >>>>>> >>>>>> >>>>> Ping? Any takers for this? >>>>> >>>>> Roman >>>>> >>>> >>> >>> > > From stefan.johansson at oracle.com Thu Mar 29 09:53:25 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 29 Mar 2018 11:53:25 +0200 Subject: RFR (S): 8200385: Eagerly reclaimed humongous objects leave mark in prev bitmap In-Reply-To: <1522314727.2400.10.camel@oracle.com> References: <1522314727.2400.10.camel@oracle.com> Message-ID: <52e92ebe-e962-2af7-59dd-df8b35fe320e@oracle.com> On 2018-03-29 11:12, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this small fix to a benign bug, that is I > haven't seen any actual product failure from it but some very rare test > failures, where when we eagerly reclaim humongous objects we leave a > mark on the prev bitmap in some cases? > > The suggested fix is to always look at the prev bitmap and clear it, > and if needed also clear potential marks in the next bitmap. > > To make the failure appear basically 100% in that test, I added a > simple assert after reclaiming the humongous object. > > With the fix, this failure goes away completely. > > Note that this is more a "data structure hygiene" fix - the stray mark > on the prev bitmap will be automatically cleared after switching the > bitmaps at cleanup and preparing that bitmap for the next mark. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8200385 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8200385/webrev/ Good fix, even though the problem is benign it is always nice to have a consistent state. A few comments: src/hotspot/share/gc/g1/g1ConcurrentMark.cpp ?532 static void maybe_clear_bitmap_if_set(G1CMBitMap* bitmap, HeapWord* addr) It will always clear if set, so I think we should rename it. What do you think about clear_bitmap_if_marked() or clear_mark_in_bitmap()? ?544?? G1CollectorState* collector_state = _g1h->collector_state(); ?545?? if (collector_state->mark_or_rebuild_in_progress() || ?546?????? collector_state->clearing_next_bitmap()) { ?547???? maybe_clear_bitmap_if_set(_next_mark_bitmap, r->bottom()); ?548?? } I get that this might be a bit more efficient, but I would prefer always clearing both bitmaps, to not have to depend on the state. Or would there be any problem with that? There is also a difference here that will now clear_statistics even if mark_or_rebuild_in_progress() returns false. Is this on purpose? I don't see any problem with this just wanted to check. --- Thanks, Stefan > Testing: > local testing with existing tests > (gc/g1/TestEagerReclaimHumongousRegionsClearMarkBits.java) > > Thanks, > Thomas From stefan.karlsson at oracle.com Thu Mar 29 10:19:57 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 29 Mar 2018 12:19:57 +0200 Subject: RFR (XS): 8200362: G1Mux2Closure should disable implicit oop verification In-Reply-To: <4ba778ab-31c1-0bdd-a929-7b6bec4941d4@oracle.com> References: <1522250831.15321.8.camel@oracle.com> <95b30fe5-f2c8-7ff0-efa6-3dac25a220dc@oracle.com> <1522314168.2400.3.camel@oracle.com> <5225d203-2207-85aa-c7f2-fa624d4585e9@oracle.com> <4ba778ab-31c1-0bdd-a929-7b6bec4941d4@oracle.com> Message-ID: On 2018-03-29 11:25, Stefan Johansson wrote: > > > On 2018-03-29 11:20, Stefan Karlsson wrote: >> >> >> On 2018-03-29 11:02, Thomas Schatzl wrote: >>> Hi Stefan, >>> >>> On Thu, 2018-03-29 at 09:52 +0200, Stefan Johansson wrote: >>>> Hi Thomas, >>>> >>>> On 2018-03-28 17:27, Thomas Schatzl wrote: >>>>> Hi all, >>>>> >>>>> ??? can I have reviews for this small change that fixes some >>>>> annoyance >>>>> when debugging with G1? >>>>> >>>>> In particular, G1Mux2Closure is used by verification to verify and >>>>> in >>>>> case of error print out g1 specific error messages. >>>>> >>>>> However, since G1Mux2Closure is an OopClosure, it is actually >>>>> wrapped >>>>> by NoHeaderExtendedOopClosure in oop_iterate* calls. >>>>> ExtendedOopClosure >>>>> has its own verification that is very generic, and may trigger in >>>>> the >>>>> same situations as G1Mux2Closure. >>>>> >>>>> Disable this implicit verification for G1Mux2Closure so that we >>>>> always >>>>> get the g1 specific error messages. >>>>> >>>>> CR: >>>>> https://bugs.openjdk.java.net/browse/JDK-8200362 >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~tschatzl/8200362/webrev/ >>>> >>>> Nice change, always good to get detailed information. I think you >>>> need >>>> to change the code using the G1Mux2Closure as well. On line 666 in >>>> heapRegion.cpp: >>>> ??? G1Mux2Closure mux(&vl_cl, &vr_cl); >>>> ??? obj->oop_iterate_no_header(&mux); >>>> This should now be a normal oop_iterate(&mux) to make sure the right >>>> verification is done. >>> >>> Fixed. Also looked through other closures used for verification. >>> >>> http://cr.openjdk.java.net/~tschatzl/8200362/webrev.0_to_1 (diff) >>> http://cr.openjdk.java.net/~tschatzl/8200362/webrev.1 (full) >> >> Looks good. >> >> I like that we get rid of usages of oop_iterate_no_header. Maybe we >> should remove the other usages in G1, as a separate RFE. >> > I like the sound of that, maybe something for Leo to look at. Any > history or reason why we have these? Some years ago ExtendedOopClosure and OopClosure were only one class, the OopClosure. The ExtendedOopClosure was created and all GC specific code needed for the oop_iterate functions were pulled out from OopClosure and put into ExtendedOopClosure. ExtendedOopClosure is very GC centric, while OopClosure is a much leaner API to apply a closure to an oop* (or an narrowOop*). The iterate_oop_no_header (and NoHeaderExtendedOopClosure) was intended to simplify non-GC JVM code to apply an OopClosure to all oop* of an object, without having to understand the ExtendedOopClosure class. The no_header name is legacy from the Permgen when we sometimes walked the "header", the klass pointer. Today no_header means "don't care about metadata pointers, or other GC stuff". We've recently realized that instanceRefKlass::oop_oop_iterate is broken when we run GCs with load barriers and concurrent reference processing, and non-GC code applies a normal, strong, load barrier to the referent field. That strong load barrier will try to use the stale pointer, and most likely cause crashes. Therefore, oop_iterate_no_header is problematic, and should probably be completely removed. The GCs can deal with this by simply implementing ExtendedOopClosure and use oop_iterate, just like we do in this patch. StefanK > > StefanJ >> StefanK >> >>> >>> Thomas >>> > From thomas.schatzl at oracle.com Thu Mar 29 10:55:22 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 29 Mar 2018 12:55:22 +0200 Subject: RFR (S): 8200385: Eagerly reclaimed humongous objects leave mark in prev bitmap In-Reply-To: <52e92ebe-e962-2af7-59dd-df8b35fe320e@oracle.com> References: <1522314727.2400.10.camel@oracle.com> <52e92ebe-e962-2af7-59dd-df8b35fe320e@oracle.com> Message-ID: <1522320922.2400.18.camel@oracle.com> On Thu, 2018-03-29 at 11:53 +0200, Stefan Johansson wrote: > > On 2018-03-29 11:12, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this small fix to a benign bug, that is I > > haven't seen any actual product failure from it but some very rare > > test failures, where when we eagerly reclaim humongous objects we > > leave a mark on the prev bitmap in some cases? > > > > The suggested fix is to always look at the prev bitmap and clear > > it, and if needed also clear potential marks in the next bitmap. > > > > To make the failure appear basically 100% in that test, I added a > > simple assert after reclaiming the humongous object. > > > > With the fix, this failure goes away completely. > > > > Note that this is more a "data structure hygiene" fix - the stray > > mark on the prev bitmap will be automatically cleared after > > switching the bitmaps at cleanup and preparing that bitmap for the > > next mark. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8200385 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8200385/webrev/ > > Good fix, even though the problem is benign it is always nice to have > a consistent state. A few comments: > src/hotspot/share/gc/g1/g1ConcurrentMark.cpp > 532 static void maybe_clear_bitmap_if_set(G1CMBitMap* bitmap, > HeapWord* addr) > > It will always clear if set, so I think we should rename it. What do > you think about clear_bitmap_if_marked() or clear_mark_in_bitmap()? I changed it to clear_mark_if_set. > > 544 G1CollectorState* collector_state = _g1h->collector_state(); > 545 if (collector_state->mark_or_rebuild_in_progress() || > 546 collector_state->clearing_next_bitmap()) { > 547 maybe_clear_bitmap_if_set(_next_mark_bitmap, r->bottom()); > 548 } > > I get that this might be a bit more efficient, but I would prefer > always clearing both bitmaps, to not have to depend on the state. Or > would there be any problem with that? I do not think so. I will change that as suggested. > There is also a difference here that will now clear_statistics even > if mark_or_rebuild_in_progress() returns false. Is this on purpose? I > don't see any problem with this just wanted to check. Not on purpose, it does not matter. The statistics are not used beyond the mark_or_rebuild_in_progress() part. I will revert to the old code. http://cr.openjdk.java.net/~tschatzl/8200385/webrev.1 (full only; sorry I messed up the incremental diff) Thanks, Thomas From stefan.johansson at oracle.com Thu Mar 29 12:25:23 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 29 Mar 2018 14:25:23 +0200 Subject: RFR (S): 8200385: Eagerly reclaimed humongous objects leave mark in prev bitmap In-Reply-To: <1522320922.2400.18.camel@oracle.com> References: <1522314727.2400.10.camel@oracle.com> <52e92ebe-e962-2af7-59dd-df8b35fe320e@oracle.com> <1522320922.2400.18.camel@oracle.com> Message-ID: Thanks Thomas, On 2018-03-29 12:55, Thomas Schatzl wrote: > On Thu, 2018-03-29 at 11:53 +0200, Stefan Johansson wrote: >> On 2018-03-29 11:12, Thomas Schatzl wrote: >>> Hi all, >>> >>> can I have reviews for this small fix to a benign bug, that is I >>> haven't seen any actual product failure from it but some very rare >>> test failures, where when we eagerly reclaim humongous objects we >>> leave a mark on the prev bitmap in some cases? >>> >>> The suggested fix is to always look at the prev bitmap and clear >>> it, and if needed also clear potential marks in the next bitmap. >>> >>> To make the failure appear basically 100% in that test, I added a >>> simple assert after reclaiming the humongous object. >>> >>> With the fix, this failure goes away completely. >>> >>> Note that this is more a "data structure hygiene" fix - the stray >>> mark on the prev bitmap will be automatically cleared after >>> switching the bitmaps at cleanup and preparing that bitmap for the >>> next mark. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8200385 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8200385/webrev/ >> Good fix, even though the problem is benign it is always nice to have >> a consistent state. A few comments: >> src/hotspot/share/gc/g1/g1ConcurrentMark.cpp >> 532 static void maybe_clear_bitmap_if_set(G1CMBitMap* bitmap, >> HeapWord* addr) >> >> It will always clear if set, so I think we should rename it. What do >> you think about clear_bitmap_if_marked() or clear_mark_in_bitmap()? > I changed it to clear_mark_if_set. > >> 544 G1CollectorState* collector_state = _g1h->collector_state(); >> 545 if (collector_state->mark_or_rebuild_in_progress() || >> 546 collector_state->clearing_next_bitmap()) { >> 547 maybe_clear_bitmap_if_set(_next_mark_bitmap, r->bottom()); >> 548 } >> >> I get that this might be a bit more efficient, but I would prefer >> always clearing both bitmaps, to not have to depend on the state. Or >> would there be any problem with that? > I do not think so. I will change that as suggested. > >> There is also a difference here that will now clear_statistics even >> if mark_or_rebuild_in_progress() returns false. Is this on purpose? I >> don't see any problem with this just wanted to check. > Not on purpose, it does not matter. The statistics are not used beyond > the mark_or_rebuild_in_progress() part. I will revert to the old code. > > http://cr.openjdk.java.net/~tschatzl/8200385/webrev.1 (full only; sorry > I messed up the incremental diff) No problem, this looks good! Cheers, Stefan > Thanks, > Thomas > From stefan.johansson at oracle.com Thu Mar 29 13:50:33 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 29 Mar 2018 15:50:33 +0200 Subject: RFR (S): 8200305: Update gc,liveness output with remset state after rebuild remset concurrently changes In-Reply-To: <1522262138.5658.6.camel@oracle.com> References: <1522250812.15321.7.camel@oracle.com> <1522262138.5658.6.camel@oracle.com> Message-ID: <4c392558-fa1f-9703-6880-d996bc90083c@oracle.com> Hi, On 2018-03-28 20:35, Thomas Schatzl wrote: > Hi, > > On Wed, 2018-03-28 at 11:18 -0700, sangheon.kim wrote: >> Hi Thomas, >> >> On 03/28/2018 08:26 AM, Thomas Schatzl wrote: >>> Hi all, >>> >>> can I have reviews for this change that updates the gc,liveness >>> output to add the remembered set state? >>> >>> This change adds a column next to "remset" called "state" that can >>> be >>> either UNTRA (Untracked), UPDAT (Updating) and CMPLT (Complete). >>> >>> There is a log snippet attached that shows the new output. >>> >>> If somebody asks, I kind of agree that we should probably rethink >>> this >>> output (remove some of the addresses, add region number), but this >>> is >>> imho out of scope for this change. ;] >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8200305 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8200305/webrev/ >>> Testing: >>> local verification >> Looks good as is. >> >> Adding the state is a good idea. >> But these short version strings are not easy to catch. Just printing >> original strings(9 characters) are too long? :) >> > yes, I am aware of this issue, I actually thought about this for an > unusual amount of time too. After all I decided to keep the > abbreviations. And I understand that particularly UNTRA vs. UPDAT might > be a candidate for confusion, but then again it seemed better than the > long strings. > > However if somebody else also prefers the long strings, I will change > that without further delay. I'm fine either way, but an alternative, not sure it's better, could be to add a footer explaining the short names. In that case the short name could be even shorter: - = Untracked + = Updating # = Complete An other alternative is would be to revisit the state-names and maybe come up with something only needing one state_string, but I don't have any ideas here. So I suggest going with either what you have or the footer-approach. Thanks, Stefan > Thanks, > Thomas > From rkennke at redhat.com Thu Mar 29 13:54:13 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 29 Mar 2018 15:54:13 +0200 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: <4f40f1c7-36b4-a1aa-b8df-defee13643de@oracle.com> References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> <5AAFDF8A.1080309@oracle.com> <30509316-0d9e-40c0-deec-8767990c070d@oracle.com> <7f3dc351-4184-466d-8aaf-c777acb8fe3f@redhat.com> <4f40f1c7-36b4-a1aa-b8df-defee13643de@oracle.com> Message-ID: <11080d79-4c55-0b01-1e45-deb25540d458@redhat.com> Am 29.03.2018 um 11:29 schrieb Per Liden: > Hi Roman, > > On 03/29/2018 12:31 AM, Roman Kennke wrote: >> BTW, we do have a working prototype of 'JNI lazy critical functions' or >> whatever it is called in Shenandoah now, using pinning: >> >> http://hg.openjdk.java.net/shenandoah/jdk/rev/232d819bc452 >> >> It looks like it could use the same API that I proposed (pin_object() / >> unpin_object()), plus an extra method to indicate if the GC would like >> to use it, and generate the corresponding stub code. >> >> I came to think that it was an unfortunate decision to put the GCLocker >> stuff in pin_object() / unpin_object(). I think I outlined the reasons >> elsewhere in this thread, it basically boils down to what I consider >> unnecessary coupling of the two possible ways to implement JNI critical >> fluff. > > The main objection people had with your original patch was that it both > grabbed the GCLocker _and_ potentially pinned the object. I think that > was perceived as both unnecessary and confusing. However, the design we > have now isn't set in stone by any means. How about changing it to this: > > http://cr.openjdk.java.net/~pliden/supports_object_pinning/webrev.0 > > This gives you the ability to ask the collector if pinning is supported, > and then choose what to do. I think this should cover the need you have, > right? > yes, that looks good. Thank you! Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Thu Mar 29 14:19:06 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 29 Mar 2018 16:19:06 +0200 Subject: RFR (S): 8200426: Make G1 code use _g1h members Message-ID: <1522333146.2451.5.camel@oracle.com> Hi all, can I have reviews for this change that tries to make the use of locally cached members of G1CollectedHeap* uniform by actually using it and renaming the members uniformly to "g1h". This change has been suggested in a recent review. There has not actually been a lot to change. CR: https://bugs.openjdk.java.net/browse/JDK-8200426 Webrev: http://cr.openjdk.java.net/~tschatzl/8200426/webrev Testing: local compilation Thanks, Thomas From stefan.johansson at oracle.com Thu Mar 29 14:31:08 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 29 Mar 2018 16:31:08 +0200 Subject: RFR (S): 8200426: Make G1 code use _g1h members In-Reply-To: <1522333146.2451.5.camel@oracle.com> References: <1522333146.2451.5.camel@oracle.com> Message-ID: Hi Thomas, On 2018-03-29 16:19, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that tries to make the use of > locally cached members of G1CollectedHeap* uniform by actually using it > and renaming the members uniformly to "g1h". > > This change has been suggested in a recent review. > > There has not actually been a lot to change. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8200426 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8200426/webrev Looks good, Stefan > Testing: > local compilation > > Thanks, > Thomas From thomas.schatzl at oracle.com Thu Mar 29 14:35:36 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 29 Mar 2018 16:35:36 +0200 Subject: RFR (S/M): 8178105: Switch mark bitmaps during Remark Message-ID: <1522334136.2451.15.camel@oracle.com> Hi all, can I have reviews for this small change that makes G1 switch the mark bitmaps already during the Remark pause as opposed waiting until Cleanup? This is the last step before G1 can reclaim regions at Remark, which means that empty regions can be cleaned up more more quickly than before (JDK-8154528, coming soon :)). The main changes apart from actually switching the bitmaps consist of updating the liveness information, i.e. how many bytes are live in a region, now already available at Remark since JDK-8197850, also at the same time. Previously G1 used the Rebuild Remsets phase to calculate that at the same time. I could not find significant differences in timing. This change depends on the recent g1ConcurrentMark refactorings, particularly JDK-8200234. CR: https://bugs.openjdk.java.net/browse/JDK-8178105 Webrev: http://cr.openjdk.java.net/~tschatzl/8178105/ Testing: hs-tier 1-5 Thanks, Thomas From per.liden at oracle.com Thu Mar 29 14:42:52 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 29 Mar 2018 16:42:52 +0200 Subject: RFR: JDK-8199620: Support for JNI object pinning In-Reply-To: <11080d79-4c55-0b01-1e45-deb25540d458@redhat.com> References: <7061678b-92df-5169-4b86-5e1165bfc59e@redhat.com> <5AAFDF8A.1080309@oracle.com> <30509316-0d9e-40c0-deec-8767990c070d@oracle.com> <7f3dc351-4184-466d-8aaf-c777acb8fe3f@redhat.com> <4f40f1c7-36b4-a1aa-b8df-defee13643de@oracle.com> <11080d79-4c55-0b01-1e45-deb25540d458@redhat.com> Message-ID: On 03/29/2018 03:54 PM, Roman Kennke wrote: > Am 29.03.2018 um 11:29 schrieb Per Liden: >> Hi Roman, >> >> On 03/29/2018 12:31 AM, Roman Kennke wrote: >>> BTW, we do have a working prototype of 'JNI lazy critical functions' or >>> whatever it is called in Shenandoah now, using pinning: >>> >>> http://hg.openjdk.java.net/shenandoah/jdk/rev/232d819bc452 >>> >>> It looks like it could use the same API that I proposed (pin_object() / >>> unpin_object()), plus an extra method to indicate if the GC would like >>> to use it, and generate the corresponding stub code. >>> >>> I came to think that it was an unfortunate decision to put the GCLocker >>> stuff in pin_object() / unpin_object(). I think I outlined the reasons >>> elsewhere in this thread, it basically boils down to what I consider >>> unnecessary coupling of the two possible ways to implement JNI critical >>> fluff. >> >> The main objection people had with your original patch was that it both >> grabbed the GCLocker _and_ potentially pinned the object. I think that >> was perceived as both unnecessary and confusing. However, the design we >> have now isn't set in stone by any means. How about changing it to this: >> >> http://cr.openjdk.java.net/~pliden/supports_object_pinning/webrev.0 >> >> This gives you the ability to ask the collector if pinning is supported, >> and then choose what to do. I think this should cover the need you have, >> right? >> > > yes, that looks good. Thank you! Good! I'll file an RFE and send it out for review on hotspot-dev. /Per > > Roman > > From thomas.schatzl at oracle.com Thu Mar 29 14:49:20 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 29 Mar 2018 16:49:20 +0200 Subject: RFR (S): 8154528: Reclaim regions emptied by marking in Remark pause Message-ID: <1522334960.2451.24.camel@oracle.com> Hi all, can I have reviews for this change that moves the space reclamation of empty regions into the Remark pause, which makes these regions available much sooner than before, with the obvious benefits of doing so. The Cleanup pause now only contains updating the remembered set states after rebuilding remembered sets and determining the collection set candidates. In the future we might be able to make those concurrent and drop the Cleanup pause completely. There is not much too say here, after all the recent refactorings this patch is almost trivial :) >From a timing POV this adds a few ms to the Remark pause in my measurements, mostly from the code root purging moved here. I think there is enough opportunity in the remark pause to decrease its duration in the future that outweighs the mentioned benefits. Thanks go to everyone reviewing so far, particularly StefanJ and Sangheon! Depends on JDK-8178105. CR: https://bugs.openjdk.java.net/browse/JDK-8154528 Webrev: http://cr.openjdk.java.net/~tschatzl/8154528/webrev/ Testing: hs-tier 1-5 Thanks, Thomas From leo.korinth at oracle.com Thu Mar 29 15:09:59 2018 From: leo.korinth at oracle.com (Leo Korinth) Date: Thu, 29 Mar 2018 17:09:59 +0200 Subject: RFR: 8200371: In g1, rename ConcurrentMarkThread to G1ConcurrentMarkThread Message-ID: <427cf9a1-118c-6b2d-f035-a7794f0c65e9@oracle.com> Hi, I am renaming ConcurrentMarkThread to G1ConcurrentMarkThread. Bug: https://bugs.openjdk.java.net/browse/JDK-8200371 Webrev: http://cr.openjdk.java.net/~lkorinth/8200371/00/index.html Testing: - building with and without precompiled headers - hs-tier1, hs-tier2 Thanks, Leo From sangheon.kim at oracle.com Thu Mar 29 17:10:41 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Thu, 29 Mar 2018 10:10:41 -0700 Subject: RFR (S): 8200426: Make G1 code use _g1h members In-Reply-To: <1522333146.2451.5.camel@oracle.com> References: <1522333146.2451.5.camel@oracle.com> Message-ID: <7ea7c816-66ca-c6f5-84c3-2d7772674fce@oracle.com> Hi Thomas, Thank you for addressing this cleanup! On 03/29/2018 07:19 AM, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that tries to make the use of > locally cached members of G1CollectedHeap* uniform by actually using it > and renaming the members uniformly to "g1h". > > This change has been suggested in a recent review. > > There has not actually been a lot to change. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8200426 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8200426/webrev Looks good. And I found more. ================================================ src/hotspot/share/gc/g1/g1CollectedHeap.cpp 3644 G1CollectedHeap* _g1; 3663 G1CollectedHeap* _g1; ================================================ src/hotspot/share/gc/g1/g1CollectedHeap.hpp 111??? G1CollectedHeap* _g1; ================================================ src/hotspot/share/gc/g1/g1EvacFailure.cpp 42?? G1CollectedHeap* _g1; 73?? G1CollectedHeap* _g1; ================================================ src/hotspot/share/gc/g1/g1HeapSizingPolicy.hpp 39?? const G1CollectedHeap* _g1; ================================================ src/hotspot/share/gc/g1/g1OopClosures.hpp 44? G1CollectedHeap* _g1; 107??? G1CollectedHeap* _g1; 191??? G1CollectedHeap* _g1; 209??? G1CollectedHeap* _g1; ================================================ src/hotspot/share/gc/g1/g1Policy.hpp ? 184??? G1CollectedHeap* _g1 ================================================ src/hotspot/share/gc/g1/g1RemSet.hpp 72? G1CollectedHeap* _g1; ================================================ src/hotspot/share/gc/g1/g1MonitoringSupport.cpp 243?? G1CollectedHeap* g1 = g1h(); (This is a bonus :) ) - We don't need this line as line 248 also gets g1h. Thanks, Sangheon > Testing: > local compilation > > Thanks, > Thomas From thomas.schatzl at oracle.com Thu Mar 29 18:29:55 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 29 Mar 2018 20:29:55 +0200 Subject: RFR (S): 8200426: Make G1 code use _g1h members In-Reply-To: <7ea7c816-66ca-c6f5-84c3-2d7772674fce@oracle.com> References: <1522333146.2451.5.camel@oracle.com> <7ea7c816-66ca-c6f5-84c3-2d7772674fce@oracle.com> Message-ID: <1522348195.2273.1.camel@oracle.com> Hi, On Thu, 2018-03-29 at 10:10 -0700, sangheon.kim wrote: > Hi Thomas, > > Thank you for addressing this cleanup! > > On 03/29/2018 07:19 AM, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this change that tries to make the use of > > locally cached members of G1CollectedHeap* uniform by actually > > using it > > and renaming the members uniformly to "g1h". > > > > This change has been suggested in a recent review. > > > > There has not actually been a lot to change. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8200426 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8200426/webrev > > Looks good. > And I found more. > I should improve my grep-fu... Anyway, I think I fixed all of this and some more locations at http://cr.openjdk.java.net/~tschatzl/8200426/webrev.0_to_1 (diff) http://cr.openjdk.java.net/~tschatzl/8200426/webrev.1 (full) Thanks, Thomas > ================================================ > src/hotspot/share/gc/g1/g1CollectedHeap.cpp > 3644 G1CollectedHeap* _g1; > 3663 G1CollectedHeap* _g1; > > ================================================ > src/hotspot/share/gc/g1/g1CollectedHeap.hpp > 111 G1CollectedHeap* _g1; > > ================================================ > src/hotspot/share/gc/g1/g1EvacFailure.cpp > 42 G1CollectedHeap* _g1; > 73 G1CollectedHeap* _g1; > > ================================================ > src/hotspot/share/gc/g1/g1HeapSizingPolicy.hpp > 39 const G1CollectedHeap* _g1; > > ================================================ > src/hotspot/share/gc/g1/g1OopClosures.hpp > 44 G1CollectedHeap* _g1; > 107 G1CollectedHeap* _g1; > 191 G1CollectedHeap* _g1; > 209 G1CollectedHeap* _g1; > > ================================================ > src/hotspot/share/gc/g1/g1Policy.hpp > 184 G1CollectedHeap* _g1 > > ================================================ > src/hotspot/share/gc/g1/g1RemSet.hpp > 72 G1CollectedHeap* _g1; > > ================================================ > src/hotspot/share/gc/g1/g1MonitoringSupport.cpp > 243 G1CollectedHeap* g1 = g1h(); > (This is a bonus :) ) > - We don't need this line as line 248 also gets g1h. > > Thanks, > Sangheon > > > Testing: > > local compilation > > > > Thanks, > > Thomas > > From sangheon.kim at oracle.com Fri Mar 30 04:40:23 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Thu, 29 Mar 2018 21:40:23 -0700 Subject: RFR (S): 8200426: Make G1 code use _g1h members In-Reply-To: <1522348195.2273.1.camel@oracle.com> References: <1522333146.2451.5.camel@oracle.com> <7ea7c816-66ca-c6f5-84c3-2d7772674fce@oracle.com> <1522348195.2273.1.camel@oracle.com> Message-ID: <53ee6863-3988-2c73-e649-bd39b533c8f2@oracle.com> Hi Thomas, On 03/29/2018 11:29 AM, Thomas Schatzl wrote: > Hi, > > On Thu, 2018-03-29 at 10:10 -0700, sangheon.kim wrote: >> Hi Thomas, >> >> Thank you for addressing this cleanup! >> >> On 03/29/2018 07:19 AM, Thomas Schatzl wrote: >>> Hi all, >>> >>> can I have reviews for this change that tries to make the use of >>> locally cached members of G1CollectedHeap* uniform by actually >>> using it >>> and renaming the members uniformly to "g1h". >>> >>> This change has been suggested in a recent review. >>> >>> There has not actually been a lot to change. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8200426 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8200426/webrev >> Looks good. >> And I found more. >> > I should improve my grep-fu... Sometimes good IDE works better. :-) > > Anyway, I think I fixed all of this and some more locations at > > http://cr.openjdk.java.net/~tschatzl/8200426/webrev.0_to_1 (diff) > http://cr.openjdk.java.net/~tschatzl/8200426/webrev.1 (full) Looks good, but below is not fixed. I don't need a new webrev for this. src/hotspot/share/gc/g1/g1OopClosures.hpp: ? 191:?? G1CollectedHeap* _g1; ? 196:???? _g1(g1h), src/hotspot/share/gc/g1/g1OopClosures.inline.hpp: ? 146:?? HeapRegionRemSet* to_rem_set = _g1->heap_region_containing(obj)->rem_set(); Thanks, Sangheon > > Thanks, > Thomas > >> ================================================ >> src/hotspot/share/gc/g1/g1CollectedHeap.cpp >> 3644 G1CollectedHeap* _g1; >> 3663 G1CollectedHeap* _g1; >> >> ================================================ >> src/hotspot/share/gc/g1/g1CollectedHeap.hpp >> 111 G1CollectedHeap* _g1; >> >> ================================================ >> src/hotspot/share/gc/g1/g1EvacFailure.cpp >> 42 G1CollectedHeap* _g1; >> 73 G1CollectedHeap* _g1; >> >> ================================================ >> src/hotspot/share/gc/g1/g1HeapSizingPolicy.hpp >> 39 const G1CollectedHeap* _g1; >> >> ================================================ >> src/hotspot/share/gc/g1/g1OopClosures.hpp >> 44 G1CollectedHeap* _g1; >> 107 G1CollectedHeap* _g1; >> 191 G1CollectedHeap* _g1; >> 209 G1CollectedHeap* _g1; >> >> ================================================ >> src/hotspot/share/gc/g1/g1Policy.hpp >> 184 G1CollectedHeap* _g1 >> >> ================================================ >> src/hotspot/share/gc/g1/g1RemSet.hpp >> 72 G1CollectedHeap* _g1; >> >> ================================================ >> src/hotspot/share/gc/g1/g1MonitoringSupport.cpp >> 243 G1CollectedHeap* g1 = g1h(); >> (This is a bonus :) ) >> - We don't need this line as line 248 also gets g1h. >> >> Thanks, >> Sangheon >> >>> Testing: >>> local compilation >>> >>> Thanks, >>> Thomas >> From thomas.schatzl at oracle.com Fri Mar 30 10:49:17 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 30 Mar 2018 12:49:17 +0200 Subject: RFR (S): 8200426: Make G1 code use _g1h members In-Reply-To: <53ee6863-3988-2c73-e649-bd39b533c8f2@oracle.com> References: <1522333146.2451.5.camel@oracle.com> <7ea7c816-66ca-c6f5-84c3-2d7772674fce@oracle.com> <1522348195.2273.1.camel@oracle.com> <53ee6863-3988-2c73-e649-bd39b533c8f2@oracle.com> Message-ID: <1522406957.2390.1.camel@oracle.com> Hi, On Thu, 2018-03-29 at 21:40 -0700, sangheon.kim wrote: > Hi Thomas, > > On 03/29/2018 11:29 AM, Thomas Schatzl wrote: > > Hi, > > > > On Thu, 2018-03-29 at 10:10 -0700, sangheon.kim wrote: > > > Hi Thomas, > > > > > > Thank you for addressing this cleanup! > > > > > > On 03/29/2018 07:19 AM, Thomas Schatzl wrote: > > > > Hi all, > > > > > > > > can I have reviews for this change that tries to make the > > > > use of locally cached members of G1CollectedHeap* uniform by > > > > actually using it and renaming the members uniformly to "g1h". > > > > > > > > This change has been suggested in a recent review. > > > > > > > > There has not actually been a lot to change. > > > > > > > > CR: > > > > https://bugs.openjdk.java.net/browse/JDK-8200426 > > > > Webrev: > > > > http://cr.openjdk.java.net/~tschatzl/8200426/webrev > > > > > > Looks good. > > > And I found more. > > > > > > > I should improve my grep-fu... > > Sometimes good IDE works better. :-) > > > > > Anyway, I think I fixed all of this and some more locations at > > > > http://cr.openjdk.java.net/~tschatzl/8200426/webrev.0_to_1 (diff) > > http://cr.openjdk.java.net/~tschatzl/8200426/webrev.1 (full) > > Looks good, but below is not fixed. > I don't need a new webrev for this. > > src/hotspot/share/gc/g1/g1OopClosures.hpp: > 191: G1CollectedHeap* _g1; > 196: _g1(g1h), > > src/hotspot/share/gc/g1/g1OopClosures.inline.hpp: > 146: HeapRegionRemSet* to_rem_set = > _g1->heap_region_containing(obj)->rem_set(); > I updated the webrev in place. Thanks for your review. Thanks, Thomas From thomas.schatzl at oracle.com Fri Mar 30 10:56:58 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 30 Mar 2018 12:56:58 +0200 Subject: RFR: 8200371: In g1, rename ConcurrentMarkThread to G1ConcurrentMarkThread In-Reply-To: <427cf9a1-118c-6b2d-f035-a7794f0c65e9@oracle.com> References: <427cf9a1-118c-6b2d-f035-a7794f0c65e9@oracle.com> Message-ID: <1522407418.2390.3.camel@oracle.com> Hi Leo, On Thu, 2018-03-29 at 17:09 +0200, Leo Korinth wrote: > Hi, > > I am renaming ConcurrentMarkThread to G1ConcurrentMarkThread. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8200371 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/8200371/00/index.html > > Testing: > - building with and without precompiled headers > - hs-tier1, hs-tier2 Looks good. Some potential followup change could rename the _cmThread variable in G1CollectedHeap to _cm_thread; not sure if it worth an extra CR for that though, depending on how many other similar problems the G1CollectedHeap files have. Thanks, Thomas From sangheon.kim at oracle.com Fri Mar 30 16:44:56 2018 From: sangheon.kim at oracle.com (sangheon.kim) Date: Fri, 30 Mar 2018 09:44:56 -0700 Subject: RFR: 8200371: In g1, rename ConcurrentMarkThread to G1ConcurrentMarkThread In-Reply-To: <427cf9a1-118c-6b2d-f035-a7794f0c65e9@oracle.com> References: <427cf9a1-118c-6b2d-f035-a7794f0c65e9@oracle.com> Message-ID: <6c4c74d7-a9ec-3b5e-58e6-6b97a0d18e72@oracle.com> Hi Leo, On 03/29/2018 08:09 AM, Leo Korinth wrote: > Hi, > > I am renaming ConcurrentMarkThread to G1ConcurrentMarkThread. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8200371 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/8200371/00/index.html > > Testing: > - building with and without precompiled headers > - hs-tier1, hs-tier2 Looks good. And I agree to Thomas about changing _cmThread in G1CollectedHeap. Thanks, Sangheon > > Thanks, > Leo From rkennke at redhat.com Fri Mar 30 17:53:33 2018 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 30 Mar 2018 19:53:33 +0200 Subject: RFR: JDK-8199735: Mark word updates need to use Access API In-Reply-To: <5AB90C07.3020507@oracle.com> References: <7bd73e32-a26b-2504-f069-b74807ae5e53@redhat.com> <859ac657-78c9-c7ab-0ec0-eddda331001b@redhat.com> <5AB3B052.4000304@oracle.com> <63199799-0194-a1d4-9c89-d95251c7aa7d@redhat.com> <5AB90C07.3020507@oracle.com> Message-ID: Hi Erik, thanks for reviewing! All: can I get one more reviewer for this? I shall probably mention that this falls under the build-time switch for primitive heap accesses, and therefore doesn't create any hazards for builds without Shenandoah (or more generally GCs that require barriers on primitive accesses). Thanks, Roman > Hi Roman, > > This looks good to me. Thank you for doing this. > The only thing I am a bit puzzled by is why the oopDesc::set_mark member > function takes a volatile markOop as parameter. That seems rather weird. > But that is unrelated to your changes. > > Thanks, > /Erik > > On 2018-03-25 23:03, Roman Kennke wrote: >> Hi Erik, >> >> Thanks for having a look. >> >>> A few comments: >>> >>> 1) Please make sure all files referencing these accessors that have been >>> moved to the oop.inline.hpp file now #include the oop.inline.hpp file. I >>> found that there are quite a few missing by looking for references to >>> oopDesc::mark() with rtags. >>> 2) I think the mark_addr() function should now be called mark_addr_raw() >>> for consistency. >>> 3) I wonder if there should be _raw() accessors that the GCs use >>> instead. It feels like the GC should not go through the Access API to >>> acquire mark words. >> You are of course right in all 3 points. Unfortunately, that causes >> massive ripples through the code base ;-) >> >> So here is what I did: I added _raw() variants for stuff that is used by >> GC. I temporarily renamed all relevant methods, e.g. mark() -> >> mark_fluff() to make the compiler help me. Then for each occurance of >> any such call decided whether it should call the usual method, or the >> raw variant (as a general rule, everything under src/hotspot/share/gc >> uses the _raw() variants). Also, I added #include "oop.inline.hpp" for >> every such occurance. Then I renamed all _fluff() methods and calls back >> to usual. Did this exercise with and without PCH, with and without zero. >> Checked with grep if I might have missed some. >> >> Notes: bytecodeInterpreter.cpp needed some additional fixes to use >> cas_set_mark() instead of the Atomic::cmpxchg(). >> >> There are a bunch of methods in oop.hpp that are only used by GC, like >> is_locked() etc. I haven't changed them to _raw(). >> >> This passes tier1 tests release+fastdebug, and as mentioned above builds >> with and without PCH and zero. >> >> Differential: >> http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.03.diff/ >> Full: >> http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.03/ >> >> Better now? >> >> Best regards, >> Roman >> >> >> >>> Thanks, >>> /Erik >>> >>> >>> On 2018-03-21 20:21, Roman Kennke wrote: >>>> Am 19.03.2018 um 12:07 schrieb Roman Kennke: >>>>> Am 19.03.2018 um 11:40 schrieb Roman Kennke: >>>>>> Currently, the mark word is accessed directly in oopDesc::mark() >>>>>> set_mark() and a bunch of other accessors. Those need to use the >>>>>> Access >>>>>> API instead because GC might want to employ barriers on those >>>>>> accesses, >>>>>> pretty much like every other field in oopDesc. >>>>>> >>>>>> Notice that this is *not* about accessing the bits and fields inside >>>>>> the >>>>>> markOop, but about accessing the header itself. >>>>>> >>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.01/ >>>>>> >>>>>> Testing: build fastdebug/release with and without PCH, passed tier1 >>>>>> fastdebug/release. >>>>>> >>>>>> Can I please get reviews? >>>>>> >>>>>> Thanks, Roman >>>>>> >>>>> Just when I sent it, I realized that this is dropping the volatile >>>>> from >>>>> the mark word access. Fixed here: >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~rkennke/JDK-8199735/webrev.02/ >>>>> >>>> Ping? >>>> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From david.holmes at oracle.com Sat Mar 31 02:53:02 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 31 Mar 2018 12:53:02 +1000 Subject: space.inline.hpp build failure In-Reply-To: References: Message-ID: <8d453d26-4f9b-d872-4175-fa6b3f2e1dbe@oracle.com> Moving discussion to hotspot-gc-dev David On 30/03/2018 4:36 PM, Peter Johnson wrote: > When cross compiling either 9 or 10 to ARM with gcc 5.5.0, I get the > following linker error: > /tmp/cc7TKVtK.ltrans2.ltrans.o: In function > `OffsetTableContigSpace::block_start_const(void const*) const': > :(.text+0x26e4): undefined reference to > `BlockOffsetTable::block_start(void const*) const' > /tmp/cc7TKVtK.ltrans13.ltrans.o: In function > `OffsetTableContigSpace::verify() const': > :(.text+0x2ca6): undefined reference to > `BlockOffsetTable::block_start(void const*) const' > collect2: error: ld returned 1 exit status > > This appears to be due to a missing include in space.inline.hpp. Applying > the following patch fixes the build for me. > > Thanks, > Peter > > diff -r 5ab7a67bc155 src/share/vm/gc/shared/space.inline.hpp > --- a/src/share/vm/gc/shared/space.inline.hpp Fri Sep 08 18:24:18 2017 > +0000 > +++ b/src/share/vm/gc/shared/space.inline.hpp Thu Mar 29 23:25:25 2018 > -0700 > @@ -26,6 +26,7 @@ > #define SHARE_VM_GC_SHARED_SPACE_INLINE_HPP > > #include "gc/serial/markSweep.inline.hpp" > +#include "gc/shared/blockOffsetTable.inline.hpp" > #include "gc/shared/collectedHeap.hpp" > #include "gc/shared/generation.hpp" > #include "gc/shared/space.hpp" >