From per.liden at oracle.com Mon Oct 1 07:31:45 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 1 Oct 2018 09:31:45 +0200 Subject: RFR: JDK-8211279: Verify missing object equals barriers In-Reply-To: <935559b9-6df2-e337-97b5-5807cf57a154@redhat.com> References: <935559b9-6df2-e337-97b5-5807cf57a154@redhat.com> Message-ID: Hi Roman, On 09/28/2018 04:59 PM, Roman Kennke wrote: > GCs like Shenandoah require an extra barrier for comparing objects > (oop==oop). It is easy to forget or overlook this. It would be very > useful to have a flag to turn on extra verification that catches missing > object equality barriers. > > This change inserts an assert into == and != operators for the oop class > in oopsHierarchy.hpp. This only gets compiled in fastdebug builds (when > CHECK_UNHANDLED_OOPS is on). > > It also adds a develop flag VerifyObjectEquals that is used to turn on > this verification. > > It also adds a method oopDesc::unsafe_equals(..) to be used in cases > where you know what what you are doing, and really want to use direct == > comparison without using barriers. This is used in e.g. > ReferenceProcessor or all over the place in ShenandoahGC. > > The change also fixes a couple of places where oops are compared to > non-oops like Universe::non_oop_word() to use the oop==void* operator > instead, so those don't falsely trip the verification. > > It doesn't make sense to turn this check on if you're not running a GC > that needs it, unless you want to go fix all the oop==oop in the GC itself. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8211279 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.00/ > > What do you think? So this means we would have a verification option that, when enabled, always crashes the VM unless you run Shenandoah? That doesn't sound quite right to me. This should just be a noop when not using Shenandoah, don't you think? cheers, Per From rkennke at redhat.com Mon Oct 1 12:48:40 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 1 Oct 2018 14:48:40 +0200 Subject: RFR: JDK-8211279: Verify missing object equals barriers In-Reply-To: References: <935559b9-6df2-e337-97b5-5807cf57a154@redhat.com> Message-ID: Hi Per, >> GCs like Shenandoah require an extra barrier for comparing objects >> (oop==oop). It is easy to forget or overlook this. It would be very >> useful to have a flag to turn on extra verification that catches missing >> object equality barriers. >> >> This change inserts an assert into == and != operators for the oop class >> in oopsHierarchy.hpp. This only gets compiled in fastdebug builds (when >> CHECK_UNHANDLED_OOPS is on). >> >> It also adds a develop flag VerifyObjectEquals that is used to turn on >> this verification. >> >> It also adds a method oopDesc::unsafe_equals(..) to be used in cases >> where you know what what you are doing, and really want to use direct == >> comparison without using barriers. This is used in e.g. >> ReferenceProcessor or all over the place in ShenandoahGC. >> >> The change also fixes a couple of places where oops are compared to >> non-oops like Universe::non_oop_word() to use the oop==void* operator >> instead, so those don't falsely trip the verification. >> >> It doesn't make sense to turn this check on if you're not running a GC >> that needs it, unless you want to go fix all the oop==oop in the GC >> itself. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8211279 >> Webrev: >> http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.00/ >> >> What do you think? > > So this means we would have a verification option that, when enabled, > always crashes the VM unless you run Shenandoah? That doesn't sound > quite right to me. This should just be a noop when not using Shenandoah, > don't you think? Hmm, right. Let's add some BarrierSet-infrastructure to handle this, and remove the option (it would be a GC-'private' option). It would probably have looked slightly better to do this in BarrierSet::Access, next to the Access::equals() API, but I don't feel like adding tons of boilerplate just to add this. (Infact, this is a big red warning signal regarding the Access API...) http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.01/ How does this look now? It still passes hotspot/jtreg:tier1 here. Thanks for looking at this! Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From poonam.bajaj at oracle.com Mon Oct 1 16:39:45 2018 From: poonam.bajaj at oracle.com (Poonam Parhar) Date: Mon, 1 Oct 2018 09:39:45 -0700 Subject: RFR: JDK-8211150: G1 Full GC not purging code root memory and hence causing memory leak Message-ID: Hello, Please review this simple code change that fixes a memory leak during the G1 Full GC cycle. JDK-8211150 : G1 Full GC not purging code root memory and hence causing memory leak Webrev: http://cr.openjdk.java.net/~poonam/8211150/webrev/ Problem and fix: During G1 Full collections, the code root memory is not being deallocated, and that causes a leak for processes invoking only Full GCs (e.g. using System.gc()). The above change fixes that leak. Please see the attached images showing the memory allocations of a process creating lots of Thread objects and invoking System.gc(), before and after this fix. The allocations profile is collected using jemalloc. Before-fix image clearly shows that the new entries created in G1CodeRootSetTableEntry are not being deleted in Full collections. Thanks, Poonam -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 12_afterfix.png Type: image/png Size: 227982 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 12_beforefix.png Type: image/png Size: 224734 bytes Desc: not available URL: From thomas.schatzl at oracle.com Mon Oct 1 20:33:00 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 01 Oct 2018 22:33:00 +0200 Subject: RFR: JDK-8211150: G1 Full GC not purging code root memory and hence causing memory leak In-Reply-To: References: Message-ID: <99e16f41cf0e50e2d9e2f8348169621ddc79f041.camel@oracle.com> Hi Poonam, On Mon, 2018-10-01 at 09:39 -0700, Poonam Parhar wrote: > Hello, > > Please review this simple code change that fixes a memory leak during > the G1 Full GC cycle. > > JDK-8211150: G1 Full GC not purging code root memory and hence > causing memory leak > > Webrev: http://cr.openjdk.java.net/~poonam/8211150/webrev/ > > Problem and fix: During G1 Full collections, the code root memory is > not being deallocated, and that causes a leak for processes invoking > only Full GCs (e.g. using System.gc()). The above change fixes that > leak. Please see the attached images showing the memory allocations > of a process creating lots of Thread objects and invoking > System.gc(), before and after this fix. The allocations profile is > collected using jemalloc. Before-fix image clearly shows that the new > entries created in G1CodeRootSetTableEntry are not being deleted in > Full collections. while the change is correct, I would expect that placing the purge_code_root_memory() call after the rebuild_strong_code_roots() call, as otherwise to-be-purged objects created during that time would carry over to the next GC. I.e. needlessly keep them alive. Thanks, Thomas From kim.barrett at oracle.com Tue Oct 2 00:25:58 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 1 Oct 2018 20:25:58 -0400 Subject: RFR(XS): 8210716: Detailed GC logging request misses some In-Reply-To: <0515E1CE-8665-48D4-826A-DFB4AC9A8118@oracle.com> References: <0515E1CE-8665-48D4-826A-DFB4AC9A8118@oracle.com> Message-ID: <8409FBE2-836C-49B9-B1DE-78556C9A02E3@oracle.com> > On Sep 29, 2018, at 3:19 PM, Kim Barrett wrote: > >> On Sep 28, 2018, at 9:46 PM, Man Cao wrote: >> >> Hi, >> >> Could someone review this minor logging tag change for ParallelGC? It avoids missing log messages with -Xlog:gc*=debug and -Xlog:gc*=trace. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8210716 >> Webrev: http://cr.openjdk.java.net/~manc/8210716/webrev.00/ >> >> PS: I'd need someone help me push it as I just became an Author. >> >> Thanks, >> Man > > All other uses of the "ergo" tag in gc logging place it as the second > tag, e.g. "(gc, ergo, ...)" or "(gc, ergo)". Not that the order > matters for logging control, but consistency can help with searching > and pattern matching while reading. FYI, it turns out the tag ordering does need to be consistent. There is a gtest that complains if they aren?t. [ RUN ] LogTagSet.duplicates_test test/hotspot/gtest/logging/test_logTagSet.cpp:169: Failure Failed duplicate LogTagSets found: 'gc,ergo,heap' vs 'gc,heap,ergo' (tags must always be specified in the same order for each tagset) [ FAILED ] LogTagSet.duplicates_test (1 ms) Was the original patch run through dev-submit? I think that should have failed because of the failing gtest. From manc at google.com Tue Oct 2 03:31:50 2018 From: manc at google.com (Man Cao) Date: Mon, 1 Oct 2018 20:31:50 -0700 Subject: RFR(XS): 8210716: Detailed GC logging request misses some In-Reply-To: <8409FBE2-836C-49B9-B1DE-78556C9A02E3@oracle.com> References: <0515E1CE-8665-48D4-826A-DFB4AC9A8118@oracle.com> <8409FBE2-836C-49B9-B1DE-78556C9A02E3@oracle.com> Message-ID: Ah, sorry about that. I only tested locally with "test/hotspot/jtreg:tier1". And my configure script has --disable-hotspot-gtest, so it didn't execute the gtest (I'll remove that configure flag). Usually JC would help me run tests through the submit repo , but he's on vacation. Will do better testing next time. I saw the patch has been pushed. Thanks for the feedback and help! -Man On Mon, Oct 1, 2018 at 5:26 PM Kim Barrett wrote: > > On Sep 29, 2018, at 3:19 PM, Kim Barrett wrote: > > > >> On Sep 28, 2018, at 9:46 PM, Man Cao wrote: > >> > >> Hi, > >> > >> Could someone review this minor logging tag change for ParallelGC? It > avoids missing log messages with -Xlog:gc*=debug and -Xlog:gc*=trace. > >> > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8210716 > >> Webrev: http://cr.openjdk.java.net/~manc/8210716/webrev.00/ > >> > >> PS: I'd need someone help me push it as I just became an Author. > >> > >> Thanks, > >> Man > > > > All other uses of the "ergo" tag in gc logging place it as the second > > tag, e.g. "(gc, ergo, ...)" or "(gc, ergo)". Not that the order > > matters for logging control, but consistency can help with searching > > and pattern matching while reading. > > FYI, it turns out the tag ordering does need to be consistent. There is a > gtest that complains if they aren?t. > > [ RUN ] LogTagSet.duplicates_test > test/hotspot/gtest/logging/test_logTagSet.cpp:169: Failure > Failed > duplicate LogTagSets found: 'gc,ergo,heap' vs 'gc,heap,ergo' (tags must > always be specified in the same order for each tagset) > [ FAILED ] LogTagSet.duplicates_test (1 ms) > > Was the original patch run through dev-submit? I think that should have > failed because of the failing gtest. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Tue Oct 2 11:28:57 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 02 Oct 2018 13:28:57 +0200 Subject: RFR (M): 8071913: Filter out entries to free/uncommitted regions during iteration Message-ID: <809796367d72a91d192bfdb6fdb05617a7506914.camel@oracle.com> Hi all, can I have reviews for this change that makes sure that remembered set entries to free or uncommitted regions are not touched during GC? In order to implement uncommitting regions at remark we need to make sure that these regions and their corresponding side data structures are not touched by GC any more. This change adds a simple filter in the remembered set iterator plus some careful changes in various locations of the code to avoid just that. There are alternatives to this changes that have been considered but ultimately rejected: - drop remembered sets when freeing regions: this would require us to scan all remembered sets for such regions which is slow. - do the filtering directly in the iterator: while this has been prototyped, and even gives a significant improvement in pause times (~5%), the change would be much larger (1500 LOC). Since I am currently reworking the remembered sets completely, I found it a waste of reviewer's time to let you do this review now, and shortly afterwards throw away all that code. I will incorporate the improvements into the new code though. :) If the remembered set changes won't make it e.g. for 12, we might consider taking these changes in at a later time. CR: https://bugs.openjdk.java.net/browse/JDK-8071913 Webrev: http://cr.openjdk.java.net/~tschatzl/8071913/webrev/ Testing: hs-tier1-5,jdk-tier1-3, perf tested, no change in pause times or (throughput) scores Quite a few gc tests do extensive commit/uncommit of regions with the corresponding exercise of the code. Thanks, Thomas From poonam.bajaj at oracle.com Tue Oct 2 13:41:26 2018 From: poonam.bajaj at oracle.com (Poonam Parhar) Date: Tue, 2 Oct 2018 06:41:26 -0700 Subject: RFR: JDK-8211150: G1 Full GC not purging code root memory and hence causing memory leak In-Reply-To: <99e16f41cf0e50e2d9e2f8348169621ddc79f041.camel@oracle.com> References: <99e16f41cf0e50e2d9e2f8348169621ddc79f041.camel@oracle.com> Message-ID: Hello Thomas, Thanks for the review! I have moved purge_code_root_memory() after the rebuild_strong_code_roots() call. http://cr.openjdk.java.net/~poonam/8211150/webrev.01/ Thanks, Poonam On 10/1/18 1:33 PM, Thomas Schatzl wrote: > Hi Poonam, > > On Mon, 2018-10-01 at 09:39 -0700, Poonam Parhar wrote: >> Hello, >> >> Please review this simple code change that fixes a memory leak during >> the G1 Full GC cycle. >> >> JDK-8211150: G1 Full GC not purging code root memory and hence >> causing memory leak >> >> Webrev: http://cr.openjdk.java.net/~poonam/8211150/webrev/ >> >> Problem and fix: During G1 Full collections, the code root memory is >> not being deallocated, and that causes a leak for processes invoking >> only Full GCs (e.g. using System.gc()). The above change fixes that >> leak. Please see the attached images showing the memory allocations >> of a process creating lots of Thread objects and invoking >> System.gc(), before and after this fix. The allocations profile is >> collected using jemalloc. Before-fix image clearly shows that the new >> entries created in G1CodeRootSetTableEntry are not being deleted in >> Full collections. > while the change is correct, I would expect that placing the > purge_code_root_memory() call after the rebuild_strong_code_roots() > call, as otherwise to-be-purged objects created during that time would > carry over to the next GC. I.e. needlessly keep them alive. > > Thanks, > Thomas > > From thomas.schatzl at oracle.com Tue Oct 2 13:45:01 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 02 Oct 2018 15:45:01 +0200 Subject: RFR: JDK-8211150: G1 Full GC not purging code root memory and hence causing memory leak In-Reply-To: References: <99e16f41cf0e50e2d9e2f8348169621ddc79f041.camel@oracle.com> Message-ID: Hi Poonam, On Tue, 2018-10-02 at 06:41 -0700, Poonam Parhar wrote: > Hello Thomas, > > Thanks for the review! I have moved purge_code_root_memory() after > the rebuild_strong_code_roots() call. > > http://cr.openjdk.java.net/~poonam/8211150/webrev.01/ looks good to me :) Thomas From thomas.schatzl at oracle.com Tue Oct 2 15:57:43 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 02 Oct 2018 17:57:43 +0200 Subject: RFR (S): 6490394: G1: Allow heap shrinking / memory unmapping after reclaiming regions during Remark Message-ID: <10549c6a3761fe3a2ec76a1dc3b001fa708520f6.camel@oracle.com> Hi all, allow me to provide a fix for the oldest g1-specific open issue on the bug tracker (from 2006 :)): resize the heap (eg. uncommit) according to existing sizing policies at remark time. It does exactly what the subject of the CR says. The feature is enabled by default - I did not see any reason to disable it as the functionality seems to actually be what would be expected of the concurrent cycle (as it mirrors the functionality of a full gc). Needs the change for JDK-8071913 currently out for review applied to work (i.e. not crash after uncommit). The current work for "JDK-8204089: Timely Reducing Unused Committed Memory" also depends on this change. CR: https://bugs.openjdk.java.net/browse/JDK-6490394 Webrev: http://cr.openjdk.java.net/~tschatzl/6490394/webrev/ Testing: hs-tier1-5, jdk-tier1-3, perf testing ongoing but not expecting a difference Thanks, Thomas From thomas.schatzl at oracle.com Tue Oct 2 15:58:02 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 02 Oct 2018 17:58:02 +0200 Subject: RFR (S): 8211388: Make OtherRegionsTable independent of the region it is for Message-ID: <13515ce5470f232d34dfb1d232624770fe4dff28.camel@oracle.com> Hi, the OtherRegionsTable is a set of a set of cards that comprises the remembered set. It stores the HeapRegion it collects cards for to do filtering in the FromCardCache. The set of cards should not have knowledge about what region it is for, from an abstraction POV it should just be a dumb container of cards. Move out the HeapRegion id from OtherRegionsTable (the "set of cards") into the enclosing HeapRegionRememberedSet (managing set of cards), separating the FCC functionality from the raw data storage. This will make code that assigns a HeapRegionRememberedSet to multiple regions in the future a bit simpler too. CR: https://bugs.openjdk.java.net/browse/JDK-8211388 Webrev: http://cr.openjdk.java.net/~tschatzl/8211388/webrev/ Testing: hs-tier1-3 Thanks, Thomas From serguei.spitsyn at oracle.com Wed Oct 3 01:14:18 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 2 Oct 2018 18:14:18 -0700 Subject: RFR: 8208686: [AOT] JVMTI ResourceExhausted event repeated for same allocation In-Reply-To: <5F46B759-4AF0-4EA2-B2B8-F076DD350F5A@oracle.com> References: <910A4A3C-3EEF-4167-84D5-9819C83D6FC1@oracle.com> <64cbe730-5c9a-a04d-9eee-a56abfbb8e07@oracle.com> <015e8416-a948-fdc5-46db-8b5d80ba52e8@oracle.com> <5F46B759-4AF0-4EA2-B2B8-F076DD350F5A@oracle.com> Message-ID: Hi Doug, The JVMTI related part looks good to me. Thank you for fixing it! Thanks, Serguei On 10/2/18 1:11 AM, Doug Simon wrote: > It would be great to get some input from the non-compilers teams on this RFR. > > -Doug > >> On 28 Sep 2018, at 19:51, Vladimir Kozlov wrote: >> >> To let you know, me and Tom R. did review these changes and agreed that it is the least intrusive changes for Hotspot shared code. >> >> Thanks, >> Vladimir >> >> On 9/25/18 8:11 AM, Daniel D. Daugherty wrote: >>> Adding serviceability-dev at ... since this is JVM/TI... >>> Dan >>> On 9/25/18 10:48 AM, Doug Simon wrote: >>>> A major design point of Graal is to treat allocations as non-side effecting to give more freedom to the optimizer by reducing the number of distinct FrameStates that need to be managed. When failing an allocation, Graal will deoptimize to the last side effecting instruction before the allocation. This mean the VM code for heap allocation will potentially be executed twice, once from Graal compiled code and then again in the interpreter. While this is perfectly fine according to the JVM specification, it can cause confusing behavior for JVMTI based tools. They will receive 2 ResourceExhausted events for a single allocation. Furthermore, the first ResourceExhausted event (on the Graal allocation slow path) might denote a bytecode instruction that performs no allocation, making it hard to debug the memory failure. >>>> >>>> The proposed solution is to add an extra set of JVMCI VM runtime calls for allocation. These entry points will attempt the allocation and upon failure, >>>> skip side-effects such as posting JVMTI events or handling -XX:OnOutOfMemoryError. The compiled code using these entry points is expected deoptmize on null. >>>> >>>> The path from these new entry points to where allocation can fail goes through quite a bit of VM code. One could modify all these paths by: >>>> * Returning null instead of throwing an exception on failure. >>>> * Adding a `bool null_on_fail` argument to all relevant methods. >>>> * Adding extra null checking where necessary after each call to these methods when `null_on_fail == true`. >>>> This represents a significant number of changes. >>>> >>>> Instead, the proposed solution introduces a new _in_retryable_allocation thread-local. This way, only the entry points and allocation routines that raise an exception need to be modified. Failure is communicated back to the new entry points by throwing a special pre-allocated OOME object (i.e., Universe::out_of_memory_error_retry()) which must not propagate back to Java code. Use of this object is not strictly necessary; it is introduced to highlight/document the special allocation mode. >>>> >>>> The proposed solution is at http://cr.openjdk.java.net/~dnsimon/8208686. >>>> THE JBS bug is: https://bugs.openjdk.java.net/browse/JDK-8208686 >>>> >>>> -Doug From stefan.johansson at oracle.com Wed Oct 3 07:43:56 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 3 Oct 2018 09:43:56 +0200 Subject: RFR: JDK-8211150: G1 Full GC not purging code root memory and hence causing memory leak In-Reply-To: References: <99e16f41cf0e50e2d9e2f8348169621ddc79f041.camel@oracle.com> Message-ID: <8c50618c-e41d-af60-eb1a-4e172c7a1f2b@oracle.com> Looks good to me too, thanks for finding and fixing this. Stefan On 2018-10-02 15:45, Thomas Schatzl wrote: > Hi Poonam, > > On Tue, 2018-10-02 at 06:41 -0700, Poonam Parhar wrote: >> Hello Thomas, >> >> Thanks for the review! I have moved purge_code_root_memory() after >> the rebuild_strong_code_roots() call. >> >> http://cr.openjdk.java.net/~poonam/8211150/webrev.01/ > > looks good to me :) > > Thomas > From per.liden at oracle.com Wed Oct 3 08:42:51 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 3 Oct 2018 10:42:51 +0200 Subject: RFR: JDK-8211279: Verify missing object equals barriers In-Reply-To: References: <935559b9-6df2-e337-97b5-5807cf57a154@redhat.com> Message-ID: Hi Roman, On 10/01/2018 02:48 PM, Roman Kennke wrote: > Hi Per, > > >>> GCs like Shenandoah require an extra barrier for comparing objects >>> (oop==oop). It is easy to forget or overlook this. It would be very >>> useful to have a flag to turn on extra verification that catches missing >>> object equality barriers. >>> >>> This change inserts an assert into == and != operators for the oop class >>> in oopsHierarchy.hpp. This only gets compiled in fastdebug builds (when >>> CHECK_UNHANDLED_OOPS is on). >>> >>> It also adds a develop flag VerifyObjectEquals that is used to turn on >>> this verification. >>> >>> It also adds a method oopDesc::unsafe_equals(..) to be used in cases >>> where you know what what you are doing, and really want to use direct == >>> comparison without using barriers. This is used in e.g. >>> ReferenceProcessor or all over the place in ShenandoahGC. >>> >>> The change also fixes a couple of places where oops are compared to >>> non-oops like Universe::non_oop_word() to use the oop==void* operator >>> instead, so those don't falsely trip the verification. >>> >>> It doesn't make sense to turn this check on if you're not running a GC >>> that needs it, unless you want to go fix all the oop==oop in the GC >>> itself. >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8211279 >>> Webrev: >>> http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.00/ >>> >>> What do you think? >> >> So this means we would have a verification option that, when enabled, >> always crashes the VM unless you run Shenandoah? That doesn't sound >> quite right to me. This should just be a noop when not using Shenandoah, >> don't you think? > > > Hmm, right. Let's add some BarrierSet-infrastructure to handle this, and > remove the option (it would be a GC-'private' option). It would probably > have looked slightly better to do this in BarrierSet::Access, next to > the Access::equals() API, but I don't feel like adding tons of > boilerplate just to add this. (Infact, this is a big red warning signal > regarding the Access API...) > > http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.01/ > > How does this look now? src/hotspot/share/oops/oop.hpp ------------------------------ 157 inline static bool unsafe_equals(oop o1, oop o2) { 158 return (void*) o1 == (void*) o2; 159 } I think this should be called oopDesc::equals_raw() to follow the naming convention we have for these types of functions. Also, it should do: return RawAccess<>::equals(o1, o2); Also, please make it a one-liner to better match the look of oopDesc::equals(). src/hotspot/share/gc/shared/referenceProcessor.cpp -------------------------------------------------- 477 while (! oopDesc::unsafe_equals(next, obj)) { Stray white-space, please remove. src/hotspot/share/gc/shared/referenceProcessor.hpp -------------------------------------------------- 152 assert(! oopDesc::unsafe_equals(_current_discovered, _first_seen), "cyclic ref_list found"); Stray white-space, please remove. src/hotspot/share/oops/accessBackend.hpp ---------------------------------------- 413 static bool equals(oop o1, oop o2) { return (void*) o1 == (void*) o2; } Stray white-spaces, please make that "(void*)o1 == (void*)o2". src/hotspot/share/gc/shared/barrierSet.hpp ------------------------------------------ 134 virtual void verify_equals(oop o1, oop o2) { } I'm thinking this should be: virtual bool oop_equals_operator_allowed() { return true; } And let oop::operator==(...) do: assert(BarrierSet::barrier_set()->oop_equals_operator_allowed(), "Not allowed"); Erik, can you live with this, or do you have any better ideas here? I'm not ecstatic about having a new function on BarrierSet just for this. Should we just make oop::operator==() private and fix all the places where it's used? One could also argue the oop::operator==() _is_ the raw equals and that we should be allowed to use it. Any other ideas? cheers, Per > > It still passes hotspot/jtreg:tier1 here. > > Thanks for looking at this! > Roman > From stefan.johansson at oracle.com Wed Oct 3 09:12:44 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 3 Oct 2018 11:12:44 +0200 Subject: RFR (M): 8071913: Filter out entries to free/uncommitted regions during iteration In-Reply-To: <809796367d72a91d192bfdb6fdb05617a7506914.camel@oracle.com> References: <809796367d72a91d192bfdb6fdb05617a7506914.camel@oracle.com> Message-ID: On 2018-10-02 13:28, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that makes sure that remembered > set entries to free or uncommitted regions are not touched during GC? > > In order to implement uncommitting regions at remark we need to make > sure that these regions and their corresponding side data structures > are not touched by GC any more. > > This change adds a simple filter in the remembered set iterator plus > some careful changes in various locations of the code to avoid just > that. > > There are alternatives to this changes that have been considered but > ultimately rejected: > > - drop remembered sets when freeing regions: this would require us to > scan all remembered sets for such regions which is slow. > > - do the filtering directly in the iterator: while this has been > prototyped, and even gives a significant improvement in pause times > (~5%), the change would be much larger (1500 LOC). Since I am currently > reworking the remembered sets completely, I found it a waste of > reviewer's time to let you do this review now, and shortly afterwards > throw away all that code. I will incorporate the improvements into the > new code though. :) > > If the remembered set changes won't make it e.g. for 12, we might > consider taking these changes in at a later time. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8071913 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8071913/webrev/ Looks good, I agree that going with this simpler approach is a good first step. Thanks Stefan > Testing: > hs-tier1-5,jdk-tier1-3, perf tested, no change in pause times or > (throughput) scores > > Quite a few gc tests do extensive commit/uncommit of regions with the > corresponding exercise of the code. > > Thanks, > Thomas > > From erik.osterlund at oracle.com Wed Oct 3 09:23:03 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 3 Oct 2018 11:23:03 +0200 Subject: RFR: JDK-8211279: Verify missing object equals barriers In-Reply-To: References: <935559b9-6df2-e337-97b5-5807cf57a154@redhat.com> Message-ID: <5BB48A77.7020905@oracle.com> Hi Per and Roman, My favourite solution to this problem would be to in the oop::operator() definition assert that RawAccess<>::equals(o1, o2) == Access<>::equals(o1, o2). This verification could have false positives where it considers things safe that are in fact unsafe at runtime. On the other hand, the already proposed verification could have false negatives, where a completely valid == is considered not safe, despite being safe. For example if the operands have already been Access<>::resolved, then it is safe to use == to compare them. In practice, I would be surprised if my proposed solution did not immediately hit the assert during testing when the usage is wrong. And every time it hits the assert, it would be due to provably wrong usage of ==. As a benefit, we do not have to clutter the shared barrier set with a function that essentially says if (UseShenandoah) ShouldNotReachHere(); What do you think? Thanks, /Erik On 2018-10-03 10:42, Per Liden wrote: > Hi Roman, > > On 10/01/2018 02:48 PM, Roman Kennke wrote: >> Hi Per, >> >> >>>> GCs like Shenandoah require an extra barrier for comparing objects >>>> (oop==oop). It is easy to forget or overlook this. It would be very >>>> useful to have a flag to turn on extra verification that catches >>>> missing >>>> object equality barriers. >>>> >>>> This change inserts an assert into == and != operators for the oop >>>> class >>>> in oopsHierarchy.hpp. This only gets compiled in fastdebug builds >>>> (when >>>> CHECK_UNHANDLED_OOPS is on). >>>> >>>> It also adds a develop flag VerifyObjectEquals that is used to turn on >>>> this verification. >>>> >>>> It also adds a method oopDesc::unsafe_equals(..) to be used in cases >>>> where you know what what you are doing, and really want to use >>>> direct == >>>> comparison without using barriers. This is used in e.g. >>>> ReferenceProcessor or all over the place in ShenandoahGC. >>>> >>>> The change also fixes a couple of places where oops are compared to >>>> non-oops like Universe::non_oop_word() to use the oop==void* operator >>>> instead, so those don't falsely trip the verification. >>>> >>>> It doesn't make sense to turn this check on if you're not running a GC >>>> that needs it, unless you want to go fix all the oop==oop in the GC >>>> itself. >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8211279 >>>> Webrev: >>>> http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.00/ >>>> >>>> What do you think? >>> >>> So this means we would have a verification option that, when enabled, >>> always crashes the VM unless you run Shenandoah? That doesn't sound >>> quite right to me. This should just be a noop when not using >>> Shenandoah, >>> don't you think? >> >> >> Hmm, right. Let's add some BarrierSet-infrastructure to handle this, and >> remove the option (it would be a GC-'private' option). It would probably >> have looked slightly better to do this in BarrierSet::Access, next to >> the Access::equals() API, but I don't feel like adding tons of >> boilerplate just to add this. (Infact, this is a big red warning signal >> regarding the Access API...) >> >> http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.01/ >> >> How does this look now? > > > src/hotspot/share/oops/oop.hpp > ------------------------------ > > 157 inline static bool unsafe_equals(oop o1, oop o2) { > 158 return (void*) o1 == (void*) o2; > 159 } > > I think this should be called oopDesc::equals_raw() to follow the > naming convention we have for these types of functions. Also, it > should do: > > return RawAccess<>::equals(o1, o2); > > Also, please make it a one-liner to better match the look of > oopDesc::equals(). > > > src/hotspot/share/gc/shared/referenceProcessor.cpp > -------------------------------------------------- > > 477 while (! oopDesc::unsafe_equals(next, obj)) { > > Stray white-space, please remove. > > > src/hotspot/share/gc/shared/referenceProcessor.hpp > -------------------------------------------------- > > 152 assert(! oopDesc::unsafe_equals(_current_discovered, > _first_seen), "cyclic ref_list found"); > > Stray white-space, please remove. > > > src/hotspot/share/oops/accessBackend.hpp > ---------------------------------------- > > 413 static bool equals(oop o1, oop o2) { return (void*) o1 == > (void*) o2; } > > Stray white-spaces, please make that "(void*)o1 == (void*)o2". > > > src/hotspot/share/gc/shared/barrierSet.hpp > ------------------------------------------ > > 134 virtual void verify_equals(oop o1, oop o2) { } > > I'm thinking this should be: > > virtual bool oop_equals_operator_allowed() { return true; } > > And let oop::operator==(...) do: > > assert(BarrierSet::barrier_set()->oop_equals_operator_allowed(), "Not > allowed"); > > > Erik, can you live with this, or do you have any better ideas here? > I'm not ecstatic about having a new function on BarrierSet just for > this. Should we just make oop::operator==() private and fix all the > places where it's used? One could also argue the oop::operator==() > _is_ the raw equals and that we should be allowed to use it. Any other > ideas? > > > cheers, > Per > >> >> It still passes hotspot/jtreg:tier1 here. >> >> Thanks for looking at this! >> Roman >> From per.liden at oracle.com Wed Oct 3 10:21:01 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 3 Oct 2018 12:21:01 +0200 Subject: RFR: JDK-8211279: Verify missing object equals barriers In-Reply-To: <5BB48A77.7020905@oracle.com> References: <935559b9-6df2-e337-97b5-5807cf57a154@redhat.com> <5BB48A77.7020905@oracle.com> Message-ID: Hi, On 2018-10-03 11:23, Erik ?sterlund wrote: > Hi Per and Roman, > > My favourite solution to this problem would be to in the oop::operator() > definition assert that RawAccess<>::equals(o1, o2) == > Access<>::equals(o1, o2). > This verification could have false positives where it considers things > safe that are in fact unsafe at runtime. On the other hand, the already > proposed verification could have false negatives, where a completely > valid == is considered not safe, despite being safe. For example if the > operands have already been Access<>::resolved, then it is safe to use == > to compare them. > > In practice, I would be surprised if my proposed solution did not > immediately hit the assert during testing when the usage is wrong. And > every time it hits the assert, it would be due to provably wrong usage > of ==. > > As a benefit, we do not have to clutter the shared barrier set with a > function that essentially says if (UseShenandoah) ShouldNotReachHere(); > > What do you think? That would be a nicely GC agnostic and much less intrusive approach, which turns all this into a one-liner fix. I like it. cheers, Per > > Thanks, > /Erik > > On 2018-10-03 10:42, Per Liden wrote: >> Hi Roman, >> >> On 10/01/2018 02:48 PM, Roman Kennke wrote: >>> Hi Per, >>> >>> >>>>> GCs like Shenandoah require an extra barrier for comparing objects >>>>> (oop==oop). It is easy to forget or overlook this. It would be very >>>>> useful to have a flag to turn on extra verification that catches >>>>> missing >>>>> object equality barriers. >>>>> >>>>> This change inserts an assert into == and != operators for the oop >>>>> class >>>>> in oopsHierarchy.hpp. This only gets compiled in fastdebug builds >>>>> (when >>>>> CHECK_UNHANDLED_OOPS is on). >>>>> >>>>> It also adds a develop flag VerifyObjectEquals that is used to turn on >>>>> this verification. >>>>> >>>>> It also adds a method oopDesc::unsafe_equals(..) to be used in cases >>>>> where you know what what you are doing, and really want to use >>>>> direct == >>>>> comparison without using barriers. This is used in e.g. >>>>> ReferenceProcessor or all over the place in ShenandoahGC. >>>>> >>>>> The change also fixes a couple of places where oops are compared to >>>>> non-oops like Universe::non_oop_word() to use the oop==void* operator >>>>> instead, so those don't falsely trip the verification. >>>>> >>>>> It doesn't make sense to turn this check on if you're not running a GC >>>>> that needs it, unless you want to go fix all the oop==oop in the GC >>>>> itself. >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8211279 >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.00/ >>>>> >>>>> What do you think? >>>> >>>> So this means we would have a verification option that, when enabled, >>>> always crashes the VM unless you run Shenandoah? That doesn't sound >>>> quite right to me. This should just be a noop when not using >>>> Shenandoah, >>>> don't you think? >>> >>> >>> Hmm, right. Let's add some BarrierSet-infrastructure to handle this, and >>> remove the option (it would be a GC-'private' option). It would probably >>> have looked slightly better to do this in BarrierSet::Access, next to >>> the Access::equals() API, but I don't feel like adding tons of >>> boilerplate just to add this. (Infact, this is a big red warning signal >>> regarding the Access API...) >>> >>> http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.01/ >>> >>> How does this look now? >> >> >> src/hotspot/share/oops/oop.hpp >> ------------------------------ >> >> ?157?? inline static bool unsafe_equals(oop o1, oop o2) { >> ?158???? return (void*) o1 == (void*) o2; >> ?159?? } >> >> I think this should be called oopDesc::equals_raw() to follow the >> naming convention we have for these types of functions. Also, it >> should do: >> >> ? return RawAccess<>::equals(o1, o2); >> >> Also, please make it a one-liner to better match the look of >> oopDesc::equals(). >> >> >> src/hotspot/share/gc/shared/referenceProcessor.cpp >> -------------------------------------------------- >> >> ?477?? while (! oopDesc::unsafe_equals(next, obj)) { >> >> Stray white-space, please remove. >> >> >> src/hotspot/share/gc/shared/referenceProcessor.hpp >> -------------------------------------------------- >> >> ?152???? assert(! oopDesc::unsafe_equals(_current_discovered, >> _first_seen), "cyclic ref_list found"); >> >> Stray white-space, please remove. >> >> >> src/hotspot/share/oops/accessBackend.hpp >> ---------------------------------------- >> >> ?413?? static bool equals(oop o1, oop o2) { return (void*) o1 == >> (void*) o2; } >> >> Stray white-spaces, please make that "(void*)o1 == (void*)o2". >> >> >> src/hotspot/share/gc/shared/barrierSet.hpp >> ------------------------------------------ >> >> ?134?? virtual void verify_equals(oop o1, oop o2) { } >> >> I'm thinking this should be: >> >> ? virtual bool oop_equals_operator_allowed() { return true; } >> >> And let oop::operator==(...) do: >> >> assert(BarrierSet::barrier_set()->oop_equals_operator_allowed(), "Not >> allowed"); >> >> >> Erik, can you live with this, or do you have any better ideas here? >> I'm not ecstatic about having a new function on BarrierSet just for >> this. Should we just make oop::operator==() private and fix all the >> places where it's used? One could also argue the oop::operator==() >> _is_ the raw equals and that we should be allowed to use it. Any other >> ideas? >> >> >> cheers, >> Per >> >>> >>> It still passes hotspot/jtreg:tier1 here. >>> >>> Thanks for looking at this! >>> Roman >>> > From stefan.johansson at oracle.com Wed Oct 3 10:37:40 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 3 Oct 2018 12:37:40 +0200 Subject: RFR (S): 8211388: Make OtherRegionsTable independent of the region it is for In-Reply-To: <13515ce5470f232d34dfb1d232624770fe4dff28.camel@oracle.com> References: <13515ce5470f232d34dfb1d232624770fe4dff28.camel@oracle.com> Message-ID: On 2018-10-02 17:58, Thomas Schatzl wrote: > Hi, > > the OtherRegionsTable is a set of a set of cards that comprises the > remembered set. > > It stores the HeapRegion it collects cards for to do filtering in the > FromCardCache. The set of cards should not have knowledge about what > region it is for, from an abstraction POV it should just be a dumb > container of cards. > > Move out the HeapRegion id from OtherRegionsTable (the "set of cards") > into the enclosing HeapRegionRememberedSet (managing set of cards), > separating the FCC functionality from the raw data storage. > > This will make code that assigns a HeapRegionRememberedSet to multiple > regions in the future a bit simpler too. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8211388 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8211388/webrev/ Nice cleanup Thomas, I agree that the old abstraction was a bit confusing and I've been looking a doing a similar cleanup myself. Thanks for fixing, Stefan > Testing: > hs-tier1-3 > > Thanks, > Thomas > > From rkennke at redhat.com Wed Oct 3 12:04:30 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 3 Oct 2018 14:04:30 +0200 Subject: RFR: JDK-8211279: Verify missing object equals barriers In-Reply-To: <5BB48A77.7020905@oracle.com> References: <935559b9-6df2-e337-97b5-5807cf57a154@redhat.com> <5BB48A77.7020905@oracle.com> Message-ID: <782fc8a1-cd2e-5966-1873-e717c9a9443e@redhat.com> The problem is that it would get false positive *in most cases*. With the solution that I proposed, one would do a run on purpose to catch missing eq barriers, and find all the relevant places *before it actually blows up*. The solution you propose would lead to rare headscratching failures instead. I'd rather prefer the explicit flag-controlled verification. Roman Am 03.10.18 um 11:23 schrieb Erik ?sterlund: > Hi Per and Roman, > > My favourite solution to this problem would be to in the oop::operator() > definition assert that RawAccess<>::equals(o1, o2) == > Access<>::equals(o1, o2). > This verification could have false positives where it considers things > safe that are in fact unsafe at runtime. On the other hand, the already > proposed verification could have false negatives, where a completely > valid == is considered not safe, despite being safe. For example if the > operands have already been Access<>::resolved, then it is safe to use == > to compare them. > > In practice, I would be surprised if my proposed solution did not > immediately hit the assert during testing when the usage is wrong. And > every time it hits the assert, it would be due to provably wrong usage > of ==. > > As a benefit, we do not have to clutter the shared barrier set with a > function that essentially says if (UseShenandoah) ShouldNotReachHere(); > > What do you think? > > Thanks, > /Erik > > On 2018-10-03 10:42, Per Liden wrote: >> Hi Roman, >> >> On 10/01/2018 02:48 PM, Roman Kennke wrote: >>> Hi Per, >>> >>> >>>>> GCs like Shenandoah require an extra barrier for comparing objects >>>>> (oop==oop). It is easy to forget or overlook this. It would be very >>>>> useful to have a flag to turn on extra verification that catches >>>>> missing >>>>> object equality barriers. >>>>> >>>>> This change inserts an assert into == and != operators for the oop >>>>> class >>>>> in oopsHierarchy.hpp. This only gets compiled in fastdebug builds >>>>> (when >>>>> CHECK_UNHANDLED_OOPS is on). >>>>> >>>>> It also adds a develop flag VerifyObjectEquals that is used to turn on >>>>> this verification. >>>>> >>>>> It also adds a method oopDesc::unsafe_equals(..) to be used in cases >>>>> where you know what what you are doing, and really want to use >>>>> direct == >>>>> comparison without using barriers. This is used in e.g. >>>>> ReferenceProcessor or all over the place in ShenandoahGC. >>>>> >>>>> The change also fixes a couple of places where oops are compared to >>>>> non-oops like Universe::non_oop_word() to use the oop==void* operator >>>>> instead, so those don't falsely trip the verification. >>>>> >>>>> It doesn't make sense to turn this check on if you're not running a GC >>>>> that needs it, unless you want to go fix all the oop==oop in the GC >>>>> itself. >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8211279 >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.00/ >>>>> >>>>> What do you think? >>>> >>>> So this means we would have a verification option that, when enabled, >>>> always crashes the VM unless you run Shenandoah? That doesn't sound >>>> quite right to me. This should just be a noop when not using >>>> Shenandoah, >>>> don't you think? >>> >>> >>> Hmm, right. Let's add some BarrierSet-infrastructure to handle this, and >>> remove the option (it would be a GC-'private' option). It would probably >>> have looked slightly better to do this in BarrierSet::Access, next to >>> the Access::equals() API, but I don't feel like adding tons of >>> boilerplate just to add this. (Infact, this is a big red warning signal >>> regarding the Access API...) >>> >>> http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.01/ >>> >>> How does this look now? >> >> >> src/hotspot/share/oops/oop.hpp >> ------------------------------ >> >> ?157?? inline static bool unsafe_equals(oop o1, oop o2) { >> ?158???? return (void*) o1 == (void*) o2; >> ?159?? } >> >> I think this should be called oopDesc::equals_raw() to follow the >> naming convention we have for these types of functions. Also, it >> should do: >> >> ? return RawAccess<>::equals(o1, o2); >> >> Also, please make it a one-liner to better match the look of >> oopDesc::equals(). >> >> >> src/hotspot/share/gc/shared/referenceProcessor.cpp >> -------------------------------------------------- >> >> ?477?? while (! oopDesc::unsafe_equals(next, obj)) { >> >> Stray white-space, please remove. >> >> >> src/hotspot/share/gc/shared/referenceProcessor.hpp >> -------------------------------------------------- >> >> ?152???? assert(! oopDesc::unsafe_equals(_current_discovered, >> _first_seen), "cyclic ref_list found"); >> >> Stray white-space, please remove. >> >> >> src/hotspot/share/oops/accessBackend.hpp >> ---------------------------------------- >> >> ?413?? static bool equals(oop o1, oop o2) { return (void*) o1 == >> (void*) o2; } >> >> Stray white-spaces, please make that "(void*)o1 == (void*)o2". >> >> >> src/hotspot/share/gc/shared/barrierSet.hpp >> ------------------------------------------ >> >> ?134?? virtual void verify_equals(oop o1, oop o2) { } >> >> I'm thinking this should be: >> >> ? virtual bool oop_equals_operator_allowed() { return true; } >> >> And let oop::operator==(...) do: >> >> assert(BarrierSet::barrier_set()->oop_equals_operator_allowed(), "Not >> allowed"); >> >> >> Erik, can you live with this, or do you have any better ideas here? >> I'm not ecstatic about having a new function on BarrierSet just for >> this. Should we just make oop::operator==() private and fix all the >> places where it's used? One could also argue the oop::operator==() >> _is_ the raw equals and that we should be allowed to use it. Any other >> ideas? >> >> >> cheers, >> Per >> >>> >>> It still passes hotspot/jtreg:tier1 here. >>> >>> Thanks for looking at this! >>> Roman >>> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Wed Oct 3 12:12:31 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 3 Oct 2018 14:12:31 +0200 Subject: RFR: JDK-8211279: Verify missing object equals barriers In-Reply-To: <782fc8a1-cd2e-5966-1873-e717c9a9443e@redhat.com> References: <935559b9-6df2-e337-97b5-5807cf57a154@redhat.com> <5BB48A77.7020905@oracle.com> <782fc8a1-cd2e-5966-1873-e717c9a9443e@redhat.com> Message-ID: On 10/03/2018 02:04 PM, Roman Kennke wrote: > The problem is that it would get false positive *in most cases*. With > the solution that I proposed, one would do a run on purpose to catch > missing eq barriers, and find all the relevant places *before it > actually blows up*. The solution you propose would lead to rare > headscratching failures instead. I'd rather prefer the explicit > flag-controlled verification. That is what we actually do in Shenandoah: we have the test runs with this verification enabled. This captures all the missing acmp barriers at once, which is very handy for maintenance, for example when doing massive upstream merges. It is the question whether you want to assert the missing barriers statically, e.g. reporting all risky comparisons as failures, or you want to do this dynamically by checking the actual failures. IIRC, in Shenandoah, the frequency of actually hitting the acmp slowpath for false negative handling (i.e. both sides point to the different copies of the same object) is way too low for dynamic way to be reliable, even in "aggressive" / "move everything all the time" mode. I can get by Erik's idea that in some cases the comparisons are provably safe from the code inspection. In these places, we have unsafe_equals that clearly captures that intent, and can be used on high-performance paths. Think about unsafe_equals as the brother of RawAccess; in fact, maybe it should be called raw_equals :) -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed Oct 3 13:25:06 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 3 Oct 2018 15:25:06 +0200 Subject: RFR: JDK-8211279: Verify missing object equals barriers In-Reply-To: References: <935559b9-6df2-e337-97b5-5807cf57a154@redhat.com> Message-ID: Hi Per, > On 10/01/2018 02:48 PM, Roman Kennke wrote: >> Hi Per, >> >> >>>> GCs like Shenandoah require an extra barrier for comparing objects >>>> (oop==oop). It is easy to forget or overlook this. It would be very >>>> useful to have a flag to turn on extra verification that catches >>>> missing >>>> object equality barriers. >>>> >>>> This change inserts an assert into == and != operators for the oop >>>> class >>>> in oopsHierarchy.hpp. This only gets compiled in fastdebug builds (when >>>> CHECK_UNHANDLED_OOPS is on). >>>> >>>> It also adds a develop flag VerifyObjectEquals that is used to turn on >>>> this verification. >>>> >>>> It also adds a method oopDesc::unsafe_equals(..) to be used in cases >>>> where you know what what you are doing, and really want to use >>>> direct == >>>> comparison without using barriers. This is used in e.g. >>>> ReferenceProcessor or all over the place in ShenandoahGC. >>>> >>>> The change also fixes a couple of places where oops are compared to >>>> non-oops like Universe::non_oop_word() to use the oop==void* operator >>>> instead, so those don't falsely trip the verification. >>>> >>>> It doesn't make sense to turn this check on if you're not running a GC >>>> that needs it, unless you want to go fix all the oop==oop in the GC >>>> itself. >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8211279 >>>> Webrev: >>>> http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.00/ >>>> >>>> What do you think? >>> >>> So this means we would have a verification option that, when enabled, >>> always crashes the VM unless you run Shenandoah? That doesn't sound >>> quite right to me. This should just be a noop when not using Shenandoah, >>> don't you think? >> >> >> Hmm, right. Let's add some BarrierSet-infrastructure to handle this, and >> remove the option (it would be a GC-'private' option). It would probably >> have looked slightly better to do this in BarrierSet::Access, next to >> the Access::equals() API, but I don't feel like adding tons of >> boilerplate just to add this. (Infact, this is a big red warning signal >> regarding the Access API...) >> >> http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.01/ >> >> How does this look now? > > > src/hotspot/share/oops/oop.hpp > ------------------------------ > > ?157?? inline static bool unsafe_equals(oop o1, oop o2) { > ?158???? return (void*) o1 == (void*) o2; > ?159?? } > > I think this should be called oopDesc::equals_raw() to follow the naming > convention we have for these types of functions. Also, it should do: > > ? return RawAccess<>::equals(o1, o2); > > Also, please make it a one-liner to better match the look of > oopDesc::equals(). > > > src/hotspot/share/gc/shared/referenceProcessor.cpp > -------------------------------------------------- > > ?477?? while (! oopDesc::unsafe_equals(next, obj)) { > > Stray white-space, please remove. > > > src/hotspot/share/gc/shared/referenceProcessor.hpp > -------------------------------------------------- > > ?152???? assert(! oopDesc::unsafe_equals(_current_discovered, > _first_seen), "cyclic ref_list found"); > > Stray white-space, please remove. > > > src/hotspot/share/oops/accessBackend.hpp > ---------------------------------------- > > ?413?? static bool equals(oop o1, oop o2) { return (void*) o1 == (void*) > o2; } > > Stray white-spaces, please make that "(void*)o1 == (void*)o2". > > > src/hotspot/share/gc/shared/barrierSet.hpp > ------------------------------------------ > > ?134?? virtual void verify_equals(oop o1, oop o2) { } > > I'm thinking this should be: > > ? virtual bool oop_equals_operator_allowed() { return true; } > > And let oop::operator==(...) do: > > ? assert(BarrierSet::barrier_set()->oop_equals_operator_allowed(), "Not > allowed"); > > > Erik, can you live with this, or do you have any better ideas here? > I'm not ecstatic about having a new function on BarrierSet just for > this. Should we just make oop::operator==() private and fix all the > places where it's used? One could also argue the oop::operator==() _is_ > the raw equals and that we should be allowed to use it. Any other ideas? This addresses all the issues mentioned above: http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.02/ It does not implement Erik's suggestion, because I think it's inferior. I'm open to discussions though. Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From stefan.johansson at oracle.com Wed Oct 3 14:59:51 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 3 Oct 2018 16:59:51 +0200 Subject: RFR: bug: Timely Reducing Unused Committed Memory In-Reply-To: References: <8af5c2468f5133bc1ba2517bf52c3078af516977.camel@oracle.com> <65063e5f31702649e12b485b570ecab9ebd32d01.camel@oracle.com> <66c536399b2f218af0d85bd3bced44f204e3dbf4.camel@oracle.com> <2821cf70-c4d5-016f-b976-5ecc850f377c@oracle.com> <57b10708-b870-cb98-0df0-fab650f431e4@oracle.com> <3927596f-8847-ba95-d4c6-3c5f3656330c@oracle.com> Message-ID: <9bbf4a13-8490-d918-3cda-5c34acd16fa6@oracle.com> On 2018-09-29 09:26, Ruslan Synytsky wrote: > > Another question, when > running in the cloud, what load is the user expecting us to compare > against, the overall system or the local container. I'm actually not > entirely sure what the getloadavg() call return in case of running in a > container. > > Good question! It depends on the used container technology. In short, if > it?s a system container then it shows the load of the container, if it?s > an application container then the load of the host machine. There is an > article on a related topic > https://jelastic.com/blog/java-and-memory-limits-in-containers-lxc-docker-and-openvz/ > > Can we measure CPU usage of JVM process itself and use it for decisions? > There is code currently only used by JFR that reports the CPU usage for the JVM process. So I guess this is another metric that also could be used to define idleness. > > > > > >> > Another thing that I don't fully > >> > understand is why the flags are manageable if there isn't > supposed > >> to be > >> > some external logic that sets them? > > Some advanced users, for example cloud platform or software vendors, > > will be able to apply an additional logic based on their custom > needs / > > specifics. Such flexibility enables more use cases and it helps to > > collect more feedback for the further default improvements. > > That's how I would expect it to be used as well, thanks for clarifying > your viewpoint. > > >> > >> >> We can also change the semantics of "idleness".? Currently it > >> checks the load. > >> >> I think that checking the allocation rate might be another good > >> option (instead of > >> >> load). The only corner case is? an application that does not > >> allocate but consumes > >> >> a lot of CPU. For this case, we might only trigger compaction at > >> most once because, > >> >> as it does not allocate memory, we will not get over committed > >> memory (i.e., the other > >> >> checks will prevent it). The opposite is also possible > (almost idle > >> application that allocates > >> >> a lot of memory) but in this scenario I don't think we want to > >> trigger an idle compaction. > >> >> > >> > >> > This is my main problem when it comes to determine "idleness", > for some > >> > applications allocation rate will be the correct metric, for > others it > >> > will be the load and for a third something different. It feels > like it > >> > is always possible to come up with a case that needs something > >> different. > > I would prefer to start with the most obvious one - based on CPU, > give > > it to try to more people by promoting the fact that JVM is > elastic now, > > and we will get more feedback that can be converted into an > additional > > logic later. > > > So basically, the first version would have two flags, one to turn on > periodic GCs (currently named GCFrequency) and one to control at which > average load (MaxLoadGC) these GCs will kick in? > > I think it?s a good starting point. > > Nice to hear. We discussed this a bit further today and I'll try to summarize what we currently think is the way forward and what a first version should include: * Periodic concurrent cycles (which after JDK-6490394 will uncommit memory) are on by default, but will only trigger if no other GC has occurred during the interval. The length of the interval should be controlled by a flag and if set to -1, the periodic GCs will be turned off. The name should probably be something other than GCFrequency, like PeriodicGCInterval or so. This will be similar to what currently is called CMSTriggerInterval for CMS. * You can also control the periodic GCs by specifying a system average load threshold, which the current avg load must be under for a periodic GC to occur. This value will not be consider by default, and I think we should try to come up with at more describing name than MaxLoadGC. * The code checking if a periodic GC should trigger will be added to the already existing G1YoungRemSetSamplingThread. This thread currently is pretty idle and there are ideas around removing it, but if that is done we can move the idle-checking code then. * If you want the periodic GC to be a full collection, there will be a flag to control this. Those were the key parts in our discussion, and I think Thomas plans to update his webrev in a few days or so. Does this still sound like a good starting point? And also, other users having opinions are very welcome :) Thanks, Stefan > > -- > Ruslan > CEO @ Jelastic From doug.simon at oracle.com Tue Oct 2 08:11:24 2018 From: doug.simon at oracle.com (Doug Simon) Date: Tue, 2 Oct 2018 10:11:24 +0200 Subject: RFR: 8208686: [AOT] JVMTI ResourceExhausted event repeated for same allocation In-Reply-To: <015e8416-a948-fdc5-46db-8b5d80ba52e8@oracle.com> References: <910A4A3C-3EEF-4167-84D5-9819C83D6FC1@oracle.com> <64cbe730-5c9a-a04d-9eee-a56abfbb8e07@oracle.com> <015e8416-a948-fdc5-46db-8b5d80ba52e8@oracle.com> Message-ID: <5F46B759-4AF0-4EA2-B2B8-F076DD350F5A@oracle.com> It would be great to get some input from the non-compilers teams on this RFR. -Doug > On 28 Sep 2018, at 19:51, Vladimir Kozlov wrote: > > To let you know, me and Tom R. did review these changes and agreed that it is the least intrusive changes for Hotspot shared code. > > Thanks, > Vladimir > > On 9/25/18 8:11 AM, Daniel D. Daugherty wrote: >> Adding serviceability-dev at ... since this is JVM/TI... >> Dan >> On 9/25/18 10:48 AM, Doug Simon wrote: >>> A major design point of Graal is to treat allocations as non-side effecting to give more freedom to the optimizer by reducing the number of distinct FrameStates that need to be managed. When failing an allocation, Graal will deoptimize to the last side effecting instruction before the allocation. This mean the VM code for heap allocation will potentially be executed twice, once from Graal compiled code and then again in the interpreter. While this is perfectly fine according to the JVM specification, it can cause confusing behavior for JVMTI based tools. They will receive 2 ResourceExhausted events for a single allocation. Furthermore, the first ResourceExhausted event (on the Graal allocation slow path) might denote a bytecode instruction that performs no allocation, making it hard to debug the memory failure. >>> >>> The proposed solution is to add an extra set of JVMCI VM runtime calls for allocation. These entry points will attempt the allocation and upon failure, >>> skip side-effects such as posting JVMTI events or handling -XX:OnOutOfMemoryError. The compiled code using these entry points is expected deoptmize on null. >>> >>> The path from these new entry points to where allocation can fail goes through quite a bit of VM code. One could modify all these paths by: >>> * Returning null instead of throwing an exception on failure. >>> * Adding a `bool null_on_fail` argument to all relevant methods. >>> * Adding extra null checking where necessary after each call to these methods when `null_on_fail == true`. >>> This represents a significant number of changes. >>> >>> Instead, the proposed solution introduces a new _in_retryable_allocation thread-local. This way, only the entry points and allocation routines that raise an exception need to be modified. Failure is communicated back to the new entry points by throwing a special pre-allocated OOME object (i.e., Universe::out_of_memory_error_retry()) which must not propagate back to Java code. Use of this object is not strictly necessary; it is introduced to highlight/document the special allocation mode. >>> >>> The proposed solution is at http://cr.openjdk.java.net/~dnsimon/8208686. >>> THE JBS bug is: https://bugs.openjdk.java.net/browse/JDK-8208686 >>> >>> -Doug From doug.simon at oracle.com Wed Oct 3 22:20:48 2018 From: doug.simon at oracle.com (Doug Simon) Date: Thu, 4 Oct 2018 00:20:48 +0200 Subject: RFR: 8208686: [AOT] JVMTI ResourceExhausted event repeated for same allocation In-Reply-To: References: <910A4A3C-3EEF-4167-84D5-9819C83D6FC1@oracle.com> <64cbe730-5c9a-a04d-9eee-a56abfbb8e07@oracle.com> <015e8416-a948-fdc5-46db-8b5d80ba52e8@oracle.com> <5F46B759-4AF0-4EA2-B2B8-F076DD350F5A@oracle.com> Message-ID: <7FD94ACE-4CF4-488C-9A2B-C440429B915A@oracle.com> Thanks for the reviews Serguei and Vladimir. Unless I hear objections in the next 24 hours, I'll push this webrev. -Doug > On 3 Oct 2018, at 03:14, serguei.spitsyn at oracle.com wrote: > > Hi Doug, > > The JVMTI related part looks good to me. > Thank you for fixing it! > > Thanks, > Serguei > > On 10/2/18 1:11 AM, Doug Simon wrote: >> It would be great to get some input from the non-compilers teams on this RFR. >> >> -Doug >> >>> On 28 Sep 2018, at 19:51, Vladimir Kozlov wrote: >>> >>> To let you know, me and Tom R. did review these changes and agreed that it is the least intrusive changes for Hotspot shared code. >>> >>> Thanks, >>> Vladimir >>> >>> On 9/25/18 8:11 AM, Daniel D. Daugherty wrote: >>>> Adding serviceability-dev at ... since this is JVM/TI... >>>> Dan >>>> On 9/25/18 10:48 AM, Doug Simon wrote: >>>>> A major design point of Graal is to treat allocations as non-side effecting to give more freedom to the optimizer by reducing the number of distinct FrameStates that need to be managed. When failing an allocation, Graal will deoptimize to the last side effecting instruction before the allocation. This mean the VM code for heap allocation will potentially be executed twice, once from Graal compiled code and then again in the interpreter. While this is perfectly fine according to the JVM specification, it can cause confusing behavior for JVMTI based tools. They will receive 2 ResourceExhausted events for a single allocation. Furthermore, the first ResourceExhausted event (on the Graal allocation slow path) might denote a bytecode instruction that performs no allocation, making it hard to debug the memory failure. >>>>> >>>>> The proposed solution is to add an extra set of JVMCI VM runtime calls for allocation. These entry points will attempt the allocation and upon failure, >>>>> skip side-effects such as posting JVMTI events or handling -XX:OnOutOfMemoryError. The compiled code using these entry points is expected deoptmize on null. >>>>> >>>>> The path from these new entry points to where allocation can fail goes through quite a bit of VM code. One could modify all these paths by: >>>>> * Returning null instead of throwing an exception on failure. >>>>> * Adding a `bool null_on_fail` argument to all relevant methods. >>>>> * Adding extra null checking where necessary after each call to these methods when `null_on_fail == true`. >>>>> This represents a significant number of changes. >>>>> >>>>> Instead, the proposed solution introduces a new _in_retryable_allocation thread-local. This way, only the entry points and allocation routines that raise an exception need to be modified. Failure is communicated back to the new entry points by throwing a special pre-allocated OOME object (i.e., Universe::out_of_memory_error_retry()) which must not propagate back to Java code. Use of this object is not strictly necessary; it is introduced to highlight/document the special allocation mode. >>>>> >>>>> The proposed solution is at http://cr.openjdk.java.net/~dnsimon/8208686. >>>>> THE JBS bug is: https://bugs.openjdk.java.net/browse/JDK-8208686 >>>>> >>>>> -Doug > From kishor.kharbas at intel.com Thu Oct 4 01:24:39 2018 From: kishor.kharbas at intel.com (Kharbas, Kishor) Date: Thu, 4 Oct 2018 01:24:39 +0000 Subject: RFR(M): 8204908: NVDIMM for POGC and G1GC - ReserveSpace.cpp changes are mostly eliminated/no collector specific code. In-Reply-To: References: <5df6d080894cfad5e6486a00f28b6ccfc5ca633f.camel@oracle.com> <87e5373f-a2e0-083e-6421-0209f519dca5@oracle.com> <177b91b2-c0ed-d7ad-84ab-ce479f07474a@oracle.com> <0b6a7684-3120-ce80-2069-1a0d163b14ba@oracle.com> Message-ID: Hi, I have an update to the webrev which addresses comments from Sangheon during an offline discussion. The JEP has been moved to an enhancement, consequently bug id '8204908' has been closed. I have created a new issue for this implementation - https://bugs.openjdk.java.net/browse/JDK-8211424. Check the comments in main issue - https://bugs.openjdk.java.net/browse/JDK-8202286. Any suggestion on whether I continue on this thread or start new one. Not only the bug id has changed, but also the design, implementation. Full: http://cr.openjdk.java.net/~kkharbas/8211424/webrev-8204908.03 Incremental : http://cr.openjdk.java.net/~kkharbas/8211424/webrev-8204908.03_to_02 Testing : Passed tier1_gc and tier1_runtime jtret tests. I added extra options "-XX:+UnlockExperimentalVMOptions -XX:AllocateOldGenAt=/home/Kishor" to each test. There are some tests which I had to exclude since they won't work with this flag. Example: tests in ConcMarkSweepGC which does not support this flag, tests requiring CompressedOops to be enabled, etc. Thanks, Kishor > -----Original Message----- > From: Kharbas, Kishor > Sent: Wednesday, September 19, 2018 3:57 PM > To: sangheon.kim at oracle.com; Thomas St?fe > ; Thomas Schatzl > > Cc: hotspot-gc-dev at openjdk.java.net; Hotspot dev runtime runtime-dev at openjdk.java.net>; Viswanathan, Sandhya > ; Aundhe, Shirish > ; Kharbas, Kishor > Subject: RE: RFR(M): 8204908: NVDIMM for POGC and G1GC - > ReserveSpace.cpp changes are mostly eliminated/no collector specific code. > > Hi, > I have an small update to the patch to disable UseCompressedOops. > http://cr.openjdk.java.net/~kkharbas/8204908/webrev-8204908.02/ > http://cr.openjdk.java.net/~kkharbas/8204908/webrev-8204908.02_to_01/ > > Regards, > Kishor > > > -----Original Message----- > > From: Kharbas, Kishor > > Sent: Wednesday, September 19, 2018 9:35 AM > > To: 'sangheon.kim at oracle.com' ; Thomas > St?fe > > ; Thomas Schatzl > > > Cc: hotspot-gc-dev at openjdk.java.net; Hotspot dev runtime > runtime-dev at openjdk.java.net>; Viswanathan, Sandhya > > ; Aundhe, Shirish > > ; Kharbas, Kishor > > Subject: RE: RFR(M): 8204908: NVDIMM for POGC and G1GC - > > ReserveSpace.cpp changes are mostly eliminated/no collector specific > code. > > > > Thanks Sangheon, > > > > I have uploaded the updated patch at, > > http://cr.openjdk.java.net/~kkharbas/8204908/webrev-8204908.01/ > > http://cr.openjdk.java.net/~kkharbas/8204908/webrev- > 8204908.01_to_00/ > > > > I have fixed all the indentation and format related comments, others I > > have pasted here below with my comments inline. > > > > ============================= > > > src/hotspot/share/gc/parallel/adjoiningGenerations.hpp > > > - Copyright update > > > > > > 62 AdjoiningGenerations(); > > > - Why we need this ctor? > > > > > >> I need this default ctor for constructing > > adjoiningGenerationsForHeteroHeap, since I don't want to call the non- > > default constructor of adjoiningGenerations. > > > > > ========================= > > > /src/hotspot/share/gc/parallel/adjoiningVirtualSpaces.hpp > > > - Copyright update > > > > > > 88 virtual PSVirtualSpace* high() { return _high; } > > > 89 virtual PSVirtualSpace* low() { return _low; } > > > - Those methods don't need 'virtual' specifier as high()/low() > > > methods are only used at ctor of AdjoiningGenerations. The other > > > calling site is ctor of AdjoiningGenerationsForHeteroHeap but it is > > > used another type of > > > high()/low() which returns _young_vs or _old_vs. > > > > > >> I feel overriding these methods is a good idea from design > > >> standpoint; > > code changes in future would take benefit of this. > > > > ======================== > > > src/hotspot/share/gc/parallel/adjoiningGenerationsForHeteroHeap.hpp > > > 45 PSVirtualSpace* _young_vs; > > > 46 PSVirtualSpace* _old_vs; > > > - Can't we use 'AdjoiningVirtualSpaces::_low' and '_high' instead? > > > If it is not the case, adding high(),low() may result in confusion > > > between > > > AdjoiningVirtualSpaces::high() and low(). So use other name for > > > current HeteroVirtualSpaces::high()/low(). > > > > > >> AdjoiningVirtualSpaces contains two adjacent virtual spaces in the > > >> same > > reserved space and separated by a boundary. That?s why the name 'high' > > and 'low'. > > >> The class I added - HeteroVirtualSpaces, are not adjacent and do > > >> not > > share same reserved space. So I have names them '_young_vs' and > '_old_vs'. > > But from outside, i.e, users of VirtualSpaces, they call high() and > > low() to access these spaces. So I have not changed the function names. > > > > > -----Original Message----- > > > From: sangheon.kim at oracle.com [mailto:sangheon.kim at oracle.com] > > > Sent: Monday, September 17, 2018 11:32 AM > > > To: Kharbas, Kishor ; Thomas St?fe > > > ; Thomas Schatzl > > > > > Cc: hotspot-gc-dev at openjdk.java.net; Hotspot dev runtime > > runtime-dev at openjdk.java.net>; Viswanathan, Sandhya > > > ; Aundhe, Shirish > > > > > > Subject: Re: RFR(M): 8204908: NVDIMM for POGC and G1GC - > > > ReserveSpace.cpp changes are mostly eliminated/no collector specific > > code. > > > > > > Hi Kishor, > > > > > > > > > On 9/4/18 10:41 PM, Kharbas, Kishor wrote: > > > > Hi, > > > > I have uploaded implementation for parallel scavenge at > > > > http://cr.openjdk.java.net/~kkharbas/8204908/webrev-8204908.00 > > > > Majority of the implementation is handled in two new files - > > > adjoiningGenerationsForHeteroHeap.cpp and > > psFileBackedVirtualspace.cpp. > > > Would love to hear your feedback and suggestions. > > > Sorry for late review. > > > > > > ---------------- > > > General comments. > > > > > > 1. Looks like this patch is not based on latest repository, as it > > > fails with - Wreorder issue. > > > > > > 2. I see many wrong alignment location of having only 2 spaces after > > > new line that is continued. > > > e.g. > > > a->b(c, > > > __d, xxxx); // 2 spaces > > > this should be > > > a->b(c, > > > _____d, xxx); // 'd' should locate under 'c' as those are in the same > context. > > > > > > 3. I see assertion failures with below options combinations. There > > > could be more... Please run jtreg tests before posting webrev. > > > -XX:+UseLargePages -XX:+UseSHM -version -XX:+UseLargePages > > > -XX:+UseNUMA -version -XX:+UseLargePages - XX:AllocateHeapAt=. > > > -version > > > > > > ========================= > > > src/hotspot/share/gc/parallel/adjoiningGenerations.cpp > > > - Copyright update > > > > > > 43?? _virtual_spaces(new AdjoiningVirtualSpaces(old_young_rs, > > > policy->min_old_size(), > > > 44?????????????????? policy->min_young_size(), alignment) ) { > > > - Wrong alignment? > > > > > > > > > ========================= > > > src/hotspot/share/gc/parallel/adjoiningGenerations.hpp > > > - Copyright update > > > > > > 62?? AdjoiningGenerations(); > > > - Why we need this ctor? > > > > > > > > > ========================= > > > /src/hotspot/share/gc/parallel/adjoiningVirtualSpaces.hpp > > > - Copyright update > > > > > > ? 88?? virtual PSVirtualSpace* high() { return _high; } > > > ? 89?? virtual PSVirtualSpace* low()? { return _low; } > > > - Those methods don't need 'virtual' specifier as high()/low() > > > methods are only used at ctor of AdjoiningGenerations. The other > > > calling site is ctor of AdjoiningGenerationsForHeteroHeap but it is > > > used another type of > > > high()/low() which returns _young_vs or _old_vs. > > > > > > > > > ========================= > > > src/hotspot/share/gc/parallel/parallelScavengeHeap.cpp > > > - no comments. > > > > > > > > > ========================= > > > src/hotspot/share/gc/parallel/psOldGen.cpp > > > ========================= > > > src/hotspot/share/gc/parallel/psOldGen.hpp > > > ? 32 #include "gc/parallel/psFileBackedVirtualspace.hpp" > > > - Include this header file at cpp seems better rather than hpp. > > > > > > ========================= > > > src/hotspot/share/gc/parallel/adjoiningGenerationsForHeteroHeap.cpp > > > ?? 1 /* > > > ?? 2 * Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved. > > > ?? 3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE > > HEADER. > > > - Wrong alignment. The second/below star should locate under first > > > line > > star. > > > - Similarily there are more wrong alignment locations. > > > ? . Line 49, 50 > > > ? . 52, 53 > > > ? . 54, 55 > > > ? . 64, 65 > > > ? . 67, 68 69 70 > > > ? . 72, 73 74 75 76 > > > ? . 79, 80 > > > ? . 81, 82 > > > ? . 85, 86 > > > ? . 87, 88 > > > ? . 106, 107 108 109 > > > ? . 114, 115 > > > ? . 166, 167 168 > > > ? . 178, 179 180 > > > ? . 186, 187 188 > > > ? . 218, 219 220 > > > ? . 230, 231 232 > > > ? . 239, 240 241 > > > > > > > > > ? 59?? // Initialize the virtual spaces.? Then pass the > > > ? 60?? // a virtual to each generation for initialization of the > > > - Then pass 'the a' virtual to each generation. 'the a'? > > > > > > ? 64?? (static_cast > > > (_virtual_spaces))->initialize(max_old_byte_si > > > ze > > > , > > > init_old_byte_size, > > > ? 65???? init_young_byte_size); > > > - Just making 'initilize' method as 'virtual' seems better. > > > > > > ? 39 > > > > > > AdjoiningGenerationsForHeteroHeap::AdjoiningGenerationsForHeteroHeap( > > > ReservedSpace > > > old_young_rs, size_t total_size_limit, > > > ? 40?? GenerationSizer* policy, size_t alignment) : > > > _total_size_limit(total_size_limit) { > > > - Wrong alirnment at line 40. 'Generation' should be under > 'ReservedSpace' > > > at line 39. > > > > > > ? 49?? _virtual_spaces = new HeteroVirtualSpaces(old_young_rs, > > > min_old_byte_size,? 70???? (static_cast > > > (_virtual_spaces))->max_young_size()); > > > ? 75???? (static_cast > > > (_virtual_spaces))->max_old_size(), > > > - Instead assigning at line 49 and then cast to > > > HeteroVirtualSpaces*, create/initialize HeteroVirtualSpaces, get > > > max_young/old_size() and the assign to _virtual_spaces would be better > alternative. > > > > > > ?109?? _min_young_byte_size(min_yg_byte_size), > > > _max_total_size(max_total_size) { > > > ?110?? _max_old_byte_size = _max_total_size - _min_young_byte_size; > > > ?111?? _max_young_byte_size = _max_total_size - _min_old_byte_size; > > > - _max_old_byte_size and _max_young_byte_size can move to > > > initialization list. > > > > > > ?117?? // This the reserved space exclusively for old generation. > > > ?122?? // This the reserved space exclusively for young generation. > > > - Missing 'is'? i.e. // This *is* the reserved space exclusively for > > > old generation. > > > > > > ?131???? vm_exit_during_initialization("Could not reserve enough space > for " > > > ?132?????? "object heap"); > > > - 'object heap' can stay same line. Or changing align is necessary. > > > > > > ?137???? vm_exit_during_initialization("Could not reserve enough space > for " > > > ?138?????? "object heap"); > > > - 'object heap' can stay same line. Or changing align is necessary. > > > > > > ?152?? if (uncommitted_in_old > 0) { > > > - This condition seems redundant as uncommitted_in_old is type of > size_t. > > > > > > ?159?? if (bytes_needed == 0) { > > > ?160???? return true; > > > ?161?? } > > > - This condition, can move to inside of 'if (uncommitted_in_old > 0)' > > > condition because if uncommitted_in_old is zero, there's no way > > > bytes_needed to be zero - bytes_needed is guaranteed not to be zero > > > from caller site, so safe to move between line 154 and 155. The > > > condition to return true is 'uncommitted_size == bytes_needed && > > > success of expand_by()'. > > > > > > ?176???? bool ret = _young_vs->shrink_by(shrink_size); > > > ?177???? assert(ret, "We should be able to shrink young space"); > > > - I would prefer to add more conditions below if we fails to shrink. > > > i.e. just assert seems not enough. > > > > > > ?189?? _old_vs->expand_by(bytes_to_add_in_old); > > > - Check the return value of expand_by(). > > > > > > ?211?? if (bytes_needed == 0) { > > > ?212???? return true; > > > ?213?? } > > > - Same as 'adjust_boundary_down()' > > > > > > ?244?? DEBUG_ONLY(size_t total_size_after = > > > _young_vs->reserved_size() > > > + _old_vs->reserved_size()); > > > ?245?? assert(total_size_after == total_size_before, "should be > > > equal"); > > > - Why adjust_boundary_up() doesn't have this kind of check? > > > > > > > > > ========================= > > > src/hotspot/share/gc/parallel/adjoiningGenerationsForHeteroHeap.hpp > > > ?? 1 /* > > > ?? 2 * Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved. > > > ?? 3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE > > HEADER. > > > ?? 4 * > > > - Wrong alignment. The second/below star should locate under first > > > line > > star. > > > > > > ? 37?? size_t total_size_limit() { > > > ? 38???? return _total_size_limit; > > > ? 39?? } > > > - I don't think we need this but if you prefer to have, add 'const'. > > > > > > ? 45???? PSVirtualSpace*??? _young_vs; > > > ? 46???? PSVirtualSpace*??? _old_vs; > > > - Can't we use 'AdjoiningVirtualSpaces::_low' and '_high' instead? > > > If it is not the case, adding high(),low() may result in confusion > > > between > > > AdjoiningVirtualSpaces::high() and low(). So use other name for > > > current HeteroVirtualSpaces::high()/low(). > > > > > > ? 55???? HeteroVirtualSpaces(ReservedSpace rs, > > > ? 56?????? size_t min_old_byte_size, > > > ? 57?????? size_t min_young_byte_size, size_t max_total_size, > > > ? 58?????? size_t alignment); > > > - Wrong alignment. Line 56 ~ 58 should be same as the location of > > > 'ReservedSpace' at line 55. > > > > > > ? 67???? size_t max_young_size() { return _max_young_byte_size; } > > > ? 68???? size_t max_old_size() { return _max_old_byte_size; } > > > - 'const'? > > > > > > ? 70???? void initialize(size_t initial_old_reserved_size, size_t > > > init_low_byte_size, > > > ? 71?????? size_t init_high_byte_size); > > > - Wrong alignment > > > > > > ? 82?? size_t reserved_byte_size(); > > > - 'const'? > > > > > > ========================= > > > ?src/hotspot/share/gc/parallel/psFileBackedVirtualspace.cpp > > > ?? 1 /* > > > ?? 2 * Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved. > > > - Wrong alignment. The second/below star should locate under first > > > line > > star. > > > > > > ? 36?? if (_fd == -1) { > > > ? 37???? return; > > > ? 38?? } > > > - I prefer to have 'initialize()' method to handle when fails to > > > create the heap file. Current implementation seems easy to call > > > additional > > 'initialize()'. > > > ? Ctor of PSVirtualSpace doesn't have any failure path(e.g. > > > allocation), so no further check is needed around line 18:psOldGen.cpp. > > > But yours is different, so something should be checked. > > > > > > ? 33?? _mapping_succeeded = false; > > > ? 34?? _file_path = path; > > > ? 46?? _mapping_succeeded = true; > > > ? 47?? _special = true; > > > - Can move to initialization list with proper initial value. > > > > > > ? 55?? assert(special(), "_special should always be true for > > > PSFileBackedVirtualSpace"); > > > ? 66?? assert(special(), "_special should always be true for > > > PSFileBackedVirtualSpace"); > > > - For these 2, can we have more specific message? e.g. to include > > > meaning of expand or shrink. > > > > > > > > > ========================= > > > ?src/hotspot/share/gc/parallel/psFileBackedVirtualspace.hpp > > > ?? 1 /* > > > ?? 2 * Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved. > > > - Wrong alignment. The second/below star should locate under first > > > line > > star. > > > > > > ? 37?? PSFileBackedVirtualSpace(ReservedSpace rs, const char* > > > file_path); > > > ? 38?? bool?? expand_by(size_t bytes); > > > - Just recommendation to add new line between 37 and 38. > > > > > > ? 43 #endif // > > SHARE_VM_GC_PARALLEL_PSFILEBACKEDVIRTUALSPACE_HPP > > > - Last line is not terminated with a newline. > > > > > > > > > Thanks, > > > Sangheon > > > > > > > > > > > > > > I will post G1 GC implementation in a few days. > > > > > > > > Thanks > > > > Kishor. > > > > > > > >> -----Original Message----- > > > >> From: sangheon.kim at oracle.com > [mailto:sangheon.kim at oracle.com] > > > >> Sent: Thursday, August 30, 2018 4:06 PM > > > >> To: Kharbas, Kishor ; Thomas St?fe > > > >> ; Thomas Schatzl > > > > > > >> Cc: hotspot-gc-dev at openjdk.java.net; Hotspot dev runtime > > > >> ; Viswanathan, Sandhya > > > >> ; Aundhe, Shirish > > > >> > > > >> Subject: Re: RFR(M): 8204908: NVDIMM for POGC and G1GC - > > > >> ReserveSpace.cpp changes are mostly eliminated/no collector > > > >> specific > > > code. > > > >> > > > >> Hi Kishor, > > > >> > > > >> On 8/30/18 11:55 AM, Kharbas, Kishor wrote: > > > >>> Hi Sangheon, > > > >>> > > > >>> So far I don?t see a need to do so. I will post my > > > >>> implementation code > > > >> today or tomorrow, if we see a need or any simplification by > > > >> changing classes in VirtualSpace.hpp, we can discuss that. > > > >> Okay. > > > >> > > > >> Thanks, > > > >> Sangheon > > > >> > > > >> > > > >>> Regards, > > > >>> -Kishor > > > >>> > > > >>>> -----Original Message----- > > > >>>> From: sangheon.kim at oracle.com > > [mailto:sangheon.kim at oracle.com] > > > >>>> Sent: Wednesday, August 29, 2018 10:17 AM > > > >>>> To: Kharbas, Kishor ; Thomas St?fe > > > >>>> ; Thomas Schatzl > > > >> > > > >>>> Cc: hotspot-gc-dev at openjdk.java.net; Hotspot dev runtime > > > >>>> ; Viswanathan, Sandhya > > > >>>> ; Aundhe, Shirish > > > >>>> > > > >>>> Subject: Re: RFR(M): 8204908: NVDIMM for POGC and G1GC - > > > >>>> ReserveSpace.cpp changes are mostly eliminated/no collector > > > >>>> specific > > > >> code. > > > >>>> Hi Kishor, > > > >>>> > > > >>>> On 8/29/18 9:52 AM, Kharbas, Kishor wrote: > > > >>>>> Hi Sangheon, > > > >>>>> > > > >>>>> Thanks for reviewing the design. > > > >>>>> Since the collectors do not use them for heap memory, I did > > > >>>>> not have to override VirtualSpace > > > >>>> Sorry, I meant ReservedSpace and its friends at > > > >>>> share/memory/virtualspace.hpp. > > > >>>> > > > >>>> Thanks, > > > >>>> Sangheon > > > >>>> > > > >>>> > > > >>>>> -Kishor > > > >>>>>> -----Original Message----- > > > >>>>>> From: sangheon.kim at oracle.com > > > [mailto:sangheon.kim at oracle.com] > > > >>>>>> Sent: Tuesday, August 28, 2018 2:38 PM > > > >>>>>> To: Kharbas, Kishor ; Thomas St?fe > > > >>>>>> ; Thomas Schatzl > > > >>>> > > > >>>>>> Cc: hotspot-gc-dev at openjdk.java.net; Hotspot dev runtime > > > >>>>>> ; Viswanathan, > Sandhya > > > >>>>>> ; Aundhe, Shirish > > > >>>>>> > > > >>>>>> Subject: Re: RFR(M): 8204908: NVDIMM for POGC and G1GC - > > > >>>>>> ReserveSpace.cpp changes are mostly eliminated/no collector > > > >>>>>> specific > > > >>>> code. > > > >>>>>> Hi Kishor, > > > >>>>>> > > > >>>>>> On 8/21/18 10:57 AM, Kharbas, Kishor wrote: > > > >>>>>>> Hi All, > > > >>>>>>> > > > >>>>>>> Thank you for your valuable comments and feedback in this > > > >>>>>>> thread so > > > >>>> far. > > > >>>>>>> After taken in all the comments and reading thoroughly > > > >>>>>>> through the code, I > > > >>>>>> feel that the complexity added to virtualSpace.cpp is due to > > > >>>>>> lack of abstraction in VirtualSpace and at GC level. NV-DIMM > > > >>>>>> size and file paths are being passed all the way to OS calls. > > > >>>>>>> So I went back to the drawing board and created a high level > > > >>>>>>> design to remove all the pain points of current implementation. > > > >>>>>>> I have uploaded the design at > > > >>>>>>> > > > >> > > > > > > http://cr.openjdk.java.net/~kkharbas/8202286/Design%20for%20JEP%20JDK > > > >>>>>> - > > > >>>>>>> 8202286.pdf I would love to hear your feedback and > suggestions. > > > >>>>>>> Once we get confidence in the design, I will work on the > > > >> implementation. > > > >>>>>> The design looks good to me. > > > >>>>>> If you are planning to change VirtualSpace at > > > >>>>>> share/memory/virtualspace.hpp, including it on the design > > > >>>>>> document would be helpful too. > > > >>>>>> > > > >>>>>> Probably folks involved in the previous discussions would say > > > >>>>>> whether the design well covers their concerns or not. > > > >>>>>> > > > >>>>>> Thanks, > > > >>>>>> Sangheon > > > >>>>>> > > > >>>>>> > > > >>>>>>> PS: > > > >>>>>>> 1. Vinay has transitioned to another team in Intel, so he > > > >>>>>>> won't be able to > > > >>>>>> continue on this jep. > > > >>>>>>> 2. If you feel this should be part of JEP discussion thread, > > > >>>>>>> we can take it > > > >>>>>> there. > > > >>>>>>> Thanks and regards, > > > >>>>>>> Kishor > > > >>>>>>> > > > >>>>>>>> -----Original Message----- > > > >>>>>>>> From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] > > > >>>>>>>> Sent: Friday, June 22, 2018 9:25 AM > > > >>>>>>>> To: Thomas Schatzl > > > >>>>>>>> Cc: Awasthi, Vinay K ; Paul Su > > > >>>>>>>> ; hotspot-gc-dev at openjdk.java.net; > > > Hotspot > > > >>>> dev > > > >>>>>>>> runtime ; > > Viswanathan, > > > >>>>>> Sandhya > > > >>>>>>>> ; Aundhe, Shirish > > > >>>>>>>> ; Kharbas, Kishor > > > >>>>>>>> > > > >>>>>>>> Subject: Re: RFR(M): 8204908: NVDIMM for POGC and G1GC - > > > >>>>>>>> ReserveSpace.cpp changes are mostly eliminated/no collector > > > >>>>>>>> specific > > > >>>>>> code. > > > >>>>>>>> Hi Thomas, > > > >>>>>>>> > > > >>>>>>>> On Fri, Jun 22, 2018 at 4:44 PM, Thomas Schatzl > > > >>>>>>>> wrote: > > > >>>>>>>>> Hi Thomas, > > > >>>>>>>>> > > > >>>>>>>>> On Tue, 2018-06-19 at 13:40 +0200, Thomas St?fe wrote: > > > >>>>>>>>>> Hi Vinay, > > > >>>>>>>>>> > > > >>>>>>>>>> On Mon, Jun 18, 2018 at 6:47 PM, Awasthi, Vinay K > > > >>>>>>>>>> wrote: > > > >>>>>>>>>>> Hi Thomas, > > > >>>>>>>>>>> > > > >>>>>>>>>>> Os::commit_memory calls map_memory_to_file which is > > same > > > >> as > > > >>>>>>>>>>> os::reserve_memory. > > > >>>>>>>>>>> > > > >>>>>>>>>>> I am failing to see why os::reserve_memory can call > > > >>>>>>>>>>> map_memory_to_file (i.e. tie it to mmap) but > > > commit_memory > > > >>>>>> can't... > > > >>>>>>>>>>> Before this patch, commit_memory never dealt with > > > >>>>>>>>>>> incrementally committing pages to device so there has to > > > >>>>>>>>>>> be a way to pass file descriptor and offset. Windows has > > > >>>>>>>>>>> no such capability to manage incremental commits. All > > > >>>>>>>>>>> other OSes do and that is why map_memory_to_file is used > > > >>>>>>>>>>> (which > > > by > > > >>>>>>>>>>> the way also works on Windows). > > > >>>>>>>>>> AIX uses System V shared memory by default, which follows > > > >>>>>>>>>> a different allocation scheme (semantics more like > > > >>>>>>>>>> Windows > > > >>>>>> VirtualAlloc... > > > >>>>>>>>>> calls). > > > >>>>>>>>>> > > > >>>>>>>>>> But my doubts are not limited to that one, see my earlier > > > >>>>>>>>>> replies and those of others. It really makes sense to > > > >>>>>>>>>> step back one step and discuss the JEP first. > > > >>>>>>>>>> > > > >>>>>>>>> There is already a discussion thread as David mentioned > > > >>>>>>>>> (http://mail.op > > > >>>>>>>>> enjdk.java.net/pipermail/hotspot-gc-dev/2018- > > > May/022092.html) > > > >>>> that > > > >>>>>>>>> so far nobody answered to. > > > >>>>>>>>> > > > >>>>>>>> Ah, I thought he wanted to have the JEP discussed in the > > > >>>>>>>> comments section of the JEP itself. > > > >>>>>>>> > > > >>>>>>>>> I believe discussion about contents the JEP and the > > > >>>>>>>>> implementation should be separate. > > > >>>>>>>>> > > > >>>>>>>> makes sense. > > > >>>>>>>> > > > >>>>>>>>> So far what I gathered from the responses to the > > > >>>>>>>>> implementation, the proposed idea itself is not the issue > > > >>>>>>>>> here (allowing the use of NVDIMM memory for parts of the > > > >>>>>>>>> heap for allowing the use of larger heaps to improve > > > >>>>>>>>> overall performance; I am not saying that the text doesn't > > > >>>>>>>>> need a bit more work :) ), but rather how an > > > >>>>>>>>> implementation of this JEP > > > should proceed. > > > >>>>>>>> I have no problem with adding NVDIMM support. I think it is > > > >>>>>>>> a cool > > > >>>>>> feature. > > > >>>>>>>> Also, the awkwardness off the memory management > abstraction > > > >> layer > > > >>>>>>>> in hotspot has always been a sore point with me (I > > > >>>>>>>> originally implemented the AIX mm layer in os_aix.cpp, and > > > >>>>>>>> those are painful memories). So, I have a lot of sympathy > > > >>>>>>>> for Vinays > > > struggles. > > > >>>>>>>> Unfortunately not much time atm, but I will respond to your > > mail. > > > >>>>>>>> > > > >>>>>>>>> Let's discuss the non-implementation stuff in that thread. > > > >>>>>>>>> Vinay or I can repost the proposal email if you do not > > > >>>>>>>>> have it any more so that answers will be in-thread. > > > >>>>>>>>> > > > >>>>>>>> Okay, sounds good. > > > >>>>>>>> > > > >>>>>>>> Thanks, > > > >>>>>>>> Thomas > > > >>>>>>>> > > > >>>>>>>> (one of us should change his name to make this less > > > >>>>>>>> confusing > > > >>>>>>>> :-) > > > >>>>>>>> > > > >>>>>>>>> Thanks, > > > >>>>>>>>> Thomas > > > >>>>>>>>> From kishor.kharbas at intel.com Thu Oct 4 02:08:19 2018 From: kishor.kharbas at intel.com (Kharbas, Kishor) Date: Thu, 4 Oct 2018 02:08:19 +0000 Subject: RFR(M): 8211425: Allocation of old generation of java heap on alternate memory devices - G1 GC Message-ID: Hi all, Requesting review of the patch for allocating old generation of g1 gc on alternate memory devices such nv-dimm. The design of this implementation is explained on the bug page - https://bugs.openjdk.java.net/browse/JDK-8211425 Please follow the parent issue here : https://bugs.openjdk.java.net/browse/JDK-8202286. Webrev: http://cr.openjdk.java.net/~kkharbas/8211425/webrev.00 Testing : Passed tier1_gc and tier1_runtime jtret tests. I added extra options "-XX:+UnlockExperimentalVMOptions -XX:AllocateOldGenAt=/home/Kishor" to each test. There are some tests which I had to exclude since they won't work with this flag. Example: tests in ConcMarkSweepGC which does not support this flag, tests requiring CompressedOops to be enabled, etc. Thanks Kishor -------------- next part -------------- An HTML attachment was scrubbed... URL: From rbruno at gsd.inesc-id.pt Thu Oct 4 12:06:03 2018 From: rbruno at gsd.inesc-id.pt (Rodrigo Bruno) Date: Thu, 4 Oct 2018 13:06:03 +0100 Subject: RFR: bug: Timely Reducing Unused Committed Memory In-Reply-To: <9bbf4a13-8490-d918-3cda-5c34acd16fa6@oracle.com> References: <8af5c2468f5133bc1ba2517bf52c3078af516977.camel@oracle.com> <65063e5f31702649e12b485b570ecab9ebd32d01.camel@oracle.com> <66c536399b2f218af0d85bd3bced44f204e3dbf4.camel@oracle.com> <2821cf70-c4d5-016f-b976-5ecc850f377c@oracle.com> <57b10708-b870-cb98-0df0-fab650f431e4@oracle.com> <3927596f-8847-ba95-d4c6-3c5f3656330c@oracle.com> <9bbf4a13-8490-d918-3cda-5c34acd16fa6@oracle.com> Message-ID: Hi Stefan, Stefan Johansson escreveu no dia quarta, 3/10/2018 ?(s) 21:18: > > We discussed this a bit further today and I'll try to summarize what we > currently think is the way forward and what a first version should include: > * Periodic concurrent cycles (which after JDK-6490394 will uncommit > memory) are on by default, but will only trigger if no other GC has > occurred during the interval. The length of the interval should be > controlled by a flag and if set to -1, the periodic GCs will be turned > off. The name should probably be something other than GCFrequency, like > PeriodicGCInterval or so. This will be similar to what currently is > called CMSTriggerInterval for CMS. > * You can also control the periodic GCs by specifying a system average > load threshold, which the current avg load must be under for a periodic > GC to occur. This value will not be consider by default, and I think we > should try to come up with at more describing name than MaxLoadGC. > * The code checking if a periodic GC should trigger will be added to the > already existing G1YoungRemSetSamplingThread. This thread currently is > pretty idle and there are ideas around removing it, but if that is done > we can move the idle-checking code then. > * If you want the periodic GC to be a full collection, there will be a > flag to control this. > > Those were the key parts in our discussion, and I think Thomas plans to > update his webrev in a few days or so. Does this still sound like a good > starting point? > > Sounds great for me! cheers, rodrigo > And also, other users having opinions are very welcome :) > > Thanks, > Stefan > > > > -- > > Ruslan > > CEO @ Jelastic > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan.johansson at oracle.com Thu Oct 4 12:12:21 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 4 Oct 2018 14:12:21 +0200 Subject: RFR (S): 6490394: G1: Allow heap shrinking / memory unmapping after reclaiming regions during Remark In-Reply-To: <10549c6a3761fe3a2ec76a1dc3b001fa708520f6.camel@oracle.com> References: <10549c6a3761fe3a2ec76a1dc3b001fa708520f6.camel@oracle.com> Message-ID: <510a418e-ce30-6289-a1e4-dc44459f64c5@oracle.com> On 2018-10-02 17:57, Thomas Schatzl wrote: > Hi all, > > allow me to provide a fix for the oldest g1-specific open issue on > the bug tracker (from 2006 :)): resize the heap (eg. uncommit) > according to existing sizing policies at remark time. > > It does exactly what the subject of the CR says. > > The feature is enabled by default - I did not see any reason to disable > it as the functionality seems to actually be what would be expected of > the concurrent cycle (as it mirrors the functionality of a full gc). > > Needs the change for JDK-8071913 currently out for review applied to > work (i.e. not crash after uncommit). > > The current work for "JDK-8204089: Timely Reducing Unused Committed > Memory" also depends on this change. > > CR: > https://bugs.openjdk.java.net/browse/JDK-6490394 > Webrev: > http://cr.openjdk.java.net/~tschatzl/6490394/webrev/ Looks great! Thanks, Stefan > Testing: > hs-tier1-5, jdk-tier1-3, perf testing ongoing but not expecting a > difference > > Thanks, > Thomas > > > From synytskyy at jelastic.com Thu Oct 4 12:27:10 2018 From: synytskyy at jelastic.com (Ruslan Synytsky) Date: Thu, 4 Oct 2018 15:27:10 +0300 Subject: RFR: bug: Timely Reducing Unused Committed Memory In-Reply-To: References: <8af5c2468f5133bc1ba2517bf52c3078af516977.camel@oracle.com> <65063e5f31702649e12b485b570ecab9ebd32d01.camel@oracle.com> <66c536399b2f218af0d85bd3bced44f204e3dbf4.camel@oracle.com> <2821cf70-c4d5-016f-b976-5ecc850f377c@oracle.com> <57b10708-b870-cb98-0df0-fab650f431e4@oracle.com> <3927596f-8847-ba95-d4c6-3c5f3656330c@oracle.com> <9bbf4a13-8490-d918-3cda-5c34acd16fa6@oracle.com> Message-ID: On Thu, 4 Oct 2018 at 15:06, Rodrigo Bruno wrote: > Hi Stefan, > > > Stefan Johansson escreveu no dia quarta, > 3/10/2018 ?(s) 21:18: > >> >> We discussed this a bit further today and I'll try to summarize what we >> currently think is the way forward and what a first version should >> include: >> * Periodic concurrent cycles (which after JDK-6490394 will uncommit >> memory) are on by default, but will only trigger if no other GC has >> occurred during the interval. The length of the interval should be >> controlled by a flag and if set to -1, the periodic GCs will be turned >> off. The name should probably be something other than GCFrequency, like >> PeriodicGCInterval or so. This will be similar to what currently is >> called CMSTriggerInterval for CMS. >> * You can also control the periodic GCs by specifying a system average >> load threshold, which the current avg load must be under for a periodic >> GC to occur. This value will not be consider by default, and I think we >> should try to come up with at more describing name than MaxLoadGC. >> * The code checking if a periodic GC should trigger will be added to the >> already existing G1YoungRemSetSamplingThread. This thread currently is >> pretty idle and there are ideas around removing it, but if that is done >> we can move the idle-checking code then. >> * If you want the periodic GC to be a full collection, there will be a >> flag to control this. >> >> Those were the key parts in our discussion, and I think Thomas plans to >> update his webrev in a few days or so. Does this still sound like a good >> starting point? >> >> > Sounds great for me! > No objections on my side too. Regards > > cheers, > rodrigo > > >> And also, other users having opinions are very welcome :) >> >> Thanks, >> Stefan >> > >> > -- >> > Ruslan >> > CEO @ Jelastic >> > -- Ruslan CEO @ Jelastic -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.osterlund at oracle.com Thu Oct 4 13:53:40 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 4 Oct 2018 15:53:40 +0200 Subject: RFR: 8211718: Supporting multiple concurrent OopStorage iterators Message-ID: Hi, OopStorage has an artificial restriction of having only one concurrent iterator at a time. This restriction should be removed. The solution is to turn the bool that tracks concurrent iterations, into an int. For some background, the reason for the restriction has historic reasons, from a time when opening OopStorage for concurrent iteration was done using a lock-free protocol between GC worker threads, which made it tricky. Since then, the model was simplified to just let the ParState RAII object open and close the concurrent iterations. With the simplified model, there is no need for such a restriction. I talked to Kim, and long-term there are more plans in this area, such as removing weak_oops_do, and then unifying the iterators. But for now, I would like to just stop disallowing multiple concurrent iterations as one small step along the way. I updated the relevant OopStorage documentation to reflect the removal of this restriction. Webrev: http://cr.openjdk.java.net/~eosterlund/8211718/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8211718 Thanks, /Erik -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Thu Oct 4 15:39:26 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 04 Oct 2018 17:39:26 +0200 Subject: RFR (S): 6490394: G1: Allow heap shrinking / memory unmapping after reclaiming regions during Remark In-Reply-To: <510a418e-ce30-6289-a1e4-dc44459f64c5@oracle.com> References: <10549c6a3761fe3a2ec76a1dc3b001fa708520f6.camel@oracle.com> <510a418e-ce30-6289-a1e4-dc44459f64c5@oracle.com> Message-ID: Hi, On Thu, 2018-10-04 at 14:12 +0200, Stefan Johansson wrote: > > On 2018-10-02 17:57, Thomas Schatzl wrote: > > Hi all, > > > > allow me to provide a fix for the oldest g1-specific open issue > > on the bug tracker (from 2006 :)): resize the heap (eg. uncommit) > > according to existing sizing policies at remark time. > > > > It does exactly what the subject of the CR says. > > > > The feature is enabled by default - I did not see any reason to > > disable it as the functionality seems to actually be what would be > > expected of the concurrent cycle (as it mirrors the functionality > > of a full gc). > > > > Needs the change for JDK-8071913 currently out for review applied > > to work (i.e. not crash after uncommit). > > > > The current work for "JDK-8204089: Timely Reducing Unused Committed > > Memory" also depends on this change. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-6490394 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/6490394/webrev/ > > Looks great! Thanks for your review. Thanks, Thomas From thomas.schatzl at oracle.com Thu Oct 4 15:40:07 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 04 Oct 2018 17:40:07 +0200 Subject: RFR (S): 8211388: Make OtherRegionsTable independent of the region it is for In-Reply-To: References: <13515ce5470f232d34dfb1d232624770fe4dff28.camel@oracle.com> Message-ID: <800696c40c113ea5c66c8bcb9c2fb8006184fff5.camel@oracle.com> Hi, On Wed, 2018-10-03 at 12:37 +0200, Stefan Johansson wrote: > > On 2018-10-02 17:58, Thomas Schatzl wrote: > > Hi, > > > > the OtherRegionsTable is a set of a set of cards that comprises > > the remembered set. > > > > It stores the HeapRegion it collects cards for to do filtering in > > the FromCardCache. The set of cards should not have knowledge about > > what region it is for, from an abstraction POV it should just be a > > dumb container of cards. > > > > Move out the HeapRegion id from OtherRegionsTable (the "set of > > cards") into the enclosing HeapRegionRememberedSet (managing set of > > cards), separating the FCC functionality from the raw data storage. > > > > This will make code that assigns a HeapRegionRememberedSet to > > multiple regions in the future a bit simpler too. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8211388 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8211388/webrev/ > > Nice cleanup Thomas, I agree that the old abstraction was a bit > confusing and I've been looking a doing a similar cleanup myself. thanks for your review. Thomas From thomas.schatzl at oracle.com Thu Oct 4 15:40:50 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 04 Oct 2018 17:40:50 +0200 Subject: RFR (M): 8071913: Filter out entries to free/uncommitted regions during iteration In-Reply-To: References: <809796367d72a91d192bfdb6fdb05617a7506914.camel@oracle.com> Message-ID: <349a27b0e1b55bf8663004825fb96fab98454540.camel@oracle.com> Hi, On Wed, 2018-10-03 at 11:12 +0200, Stefan Johansson wrote: > > On 2018-10-02 13:28, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this change that makes sure that > > remembered set entries to free or uncommitted regions are not > > touched during GC? > > > > In order to implement uncommitting regions at remark we need to > > make sure that these regions and their corresponding side data > > structures are not touched by GC any more. > > > > This change adds a simple filter in the remembered set iterator > > plus some careful changes in various locations of the code to avoid > > just that. > > [...] > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8071913 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8071913/webrev/ > > Looks good, I agree that going with this simpler approach is a good > first step. thanks for your review. Thomas From leo.korinth at oracle.com Thu Oct 4 16:20:15 2018 From: leo.korinth at oracle.com (Leo Korinth) Date: Thu, 4 Oct 2018 18:20:15 +0200 Subject: RFR: 8201436: Replace oop_ps_push_contents with oop_iterate and closure Message-ID: <7b9c2549-4979-8a91-b29b-143d2865d733@oracle.com> Hi! This is the first enhancement in a series of three to remove Parallel GC specific behaviour from shared code. I will post all three enhancements to the mailing list. JDK-8201436 Replace oop_ps_push_contents with oop_iterate and closure https://bugs.openjdk.java.net/browse/JDK-8201436 JDK-8211446 Replace oop_pc_follow_contents with oop_iterate and closure https://bugs.openjdk.java.net/browse/JDK-8211446 JDK-8211447 Replace oop_pc_update_pointers with oop_iterate and closure https://bugs.openjdk.java.net/browse/JDK-8211447 The webrevs are to be applied in the same order as in the above list although logically they are quite independent of each other. The aim is to remove Parallel GC specific behaviour by removing oop_ps_push_contents, oop_pc_follow_contents and oop_pc_update_pointers from the *Klass hierarchy in oops/*Klass.hpp. Instead of using the old dynamic dispatch on the Klass object, the generic oop_iterate* (used by all other GCs) is used instead. Improvements include: - removing GC specific implementation from shared code. Better separation, less #ifdefs - reusing the same iterators everywhere, using the same mechanism to iterate in the different GCs - reduction in code Performance: Most of the benchmarks looks good, they show about the same performance before and after the changes. I have had serious problems with the Derby benchmark that did not like the way the generic reference handling was visiting objects. Changing the order in oop_oop_iterate_reverse introduced regressions in G1. Therefore I have added new oop_oop_iterate_reverse specialisations to speed up Parallel (reduce regression) without harming G1. Ideas of things to do after this commit series: - minor improvements in oop_oop_iterate_reverse (removing return of size that is not used anywhere) - remove dynamic dispatch call to check if klass is of typeArray "kind" (!obj->klass()->is_typeArray_klass()). The empty action on type arrays is already handled and this extra check seems mostly confusing and possibly a small unnecessary cost. - possibly move creation of push closure out of push_contents, and by that win some performance (the new closure now has to set a reference processor) Enhancement: https://bugs.openjdk.java.net/browse/JDK-8201436 Webrev: http://cr.openjdk.java.net/~lkorinth/8201436/00/ Testing: - all patches build on linux. - mach5 (linux, windows and mac) tier1,tier2,tier3,tier4,tier5 has passed with all three patches applied - aurora tests have been run and seem to indicate no change in general, and only a small regression in Derby. The last (Derby) run showed a young collection regression of 1.02% on Linux and 2.76% improvement on windows, but including experience from earlier runs, my estimate is that it is in reality a bit worse and that we have a _slight_ but acceptable regression in Derby. Thanks, Leo From leo.korinth at oracle.com Thu Oct 4 16:27:45 2018 From: leo.korinth at oracle.com (Leo Korinth) Date: Thu, 4 Oct 2018 18:27:45 +0200 Subject: RFR: 8211446: Replace oop_pc_follow_contents with oop_iterate and closure Message-ID: Hi! This is the is the second enhancement in a series of three to remove Parallel GC specific behaviour from shared code. I will post all three enhancements to the mailing list. For a description on what is changed see my earlier "RFR: 8201436: Replace oop_ps_push_contents with oop_iterate and closure" Enhancement: https://bugs.openjdk.java.net/browse/JDK-8211446 Webrev: http://cr.openjdk.java.net/~lkorinth/8211446/00/ Testing: - all patches build on linux after the latest rebase. - mach5 (linux, windows and mac) tier1,tier2,tier3,tier4,tier5 has passed with all three patches applied - aurora tests have been run and seem to indicate no change in general on this webrev in the patch series. Thanks, Leo From leo.korinth at oracle.com Thu Oct 4 16:31:59 2018 From: leo.korinth at oracle.com (Leo Korinth) Date: Thu, 4 Oct 2018 18:31:59 +0200 Subject: RFR: 8211447: Replace oop_pc_update_pointers with oop_iterate and closure Message-ID: <401fe6c9-dd64-5c50-4faa-2f51925546a7@oracle.com> Hi! This is the is the third enhancement in a series of three to remove Parallel GC specific behaviour from shared code. I will post all three enhancements to the mailing list. For a description on what is changed see my earlier "RFR: 8201436: Replace oop_ps_push_contents with oop_iterate and closure" Enhancement: https://bugs.openjdk.java.net/browse/JDK-8211447 Webrev: http://cr.openjdk.java.net/~lkorinth/8211447/00/ Testing: - all patches build on linux after the latest rebase. - mach5 (linux, windows and mac) tier1,tier2,tier3,tier4,tier5 has passed with all three patches applied - aurora tests have been run and seem to indicate no change in general on this webrev in the patch series. Thanks, Leo From kim.barrett at oracle.com Thu Oct 4 23:27:52 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 4 Oct 2018 19:27:52 -0400 Subject: RFR: 8211718: Supporting multiple concurrent OopStorage iterators In-Reply-To: References: Message-ID: <1F14027E-3B77-4847-942C-20665BF29FA6@oracle.com> > On Oct 4, 2018, at 9:53 AM, Erik ?sterlund wrote: > > Hi, > > OopStorage has an artificial restriction of having only one concurrent iterator at a time. This restriction should be removed. > > The solution is to turn the bool that tracks concurrent iterations, into an int. > > For some background, the reason for the restriction has historic reasons, from a time when opening OopStorage for concurrent iteration was done using a lock-free protocol between GC worker threads, which made it tricky. Since then, the model was simplified to just let the ParState RAII object open and close the concurrent iterations. With the simplified model, there is no need for such a restriction. > > I talked to Kim, and long-term there are more plans in this area, such as removing weak_oops_do, and then unifying the iterators. But for now, I would like to just stop disallowing multiple concurrent iterations as one small step along the way. > > I updated the relevant OopStorage documentation to reflect the removal of this restriction. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8211718/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8211718 > > Thanks, > /Erik Looks good, so far as it goes. However, maybe some renames are in order: _concurrent_iteration_active => _concurrent_iteration_state or _concurrent_iteration_active => _concurrent_iteration_count update_iteration_state => update_[concurrent_]iteration_count And perhaps an assertion in the update function that it didn?t get updated to negative. From per.liden at oracle.com Fri Oct 5 06:18:51 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 5 Oct 2018 08:18:51 +0200 Subject: RFR: 8211718: Supporting multiple concurrent OopStorage iterators In-Reply-To: <1F14027E-3B77-4847-942C-20665BF29FA6@oracle.com> References: <1F14027E-3B77-4847-942C-20665BF29FA6@oracle.com> Message-ID: <5015fefa-ea3f-d823-fb65-84dc51cd379e@oracle.com> Hi, On 10/05/2018 01:27 AM, Kim Barrett wrote: >> On Oct 4, 2018, at 9:53 AM, Erik ?sterlund wrote: >> >> Hi, >> >> OopStorage has an artificial restriction of having only one concurrent iterator at a time. This restriction should be removed. >> >> The solution is to turn the bool that tracks concurrent iterations, into an int. >> >> For some background, the reason for the restriction has historic reasons, from a time when opening OopStorage for concurrent iteration was done using a lock-free protocol between GC worker threads, which made it tricky. Since then, the model was simplified to just let the ParState RAII object open and close the concurrent iterations. With the simplified model, there is no need for such a restriction. >> >> I talked to Kim, and long-term there are more plans in this area, such as removing weak_oops_do, and then unifying the iterators. But for now, I would like to just stop disallowing multiple concurrent iterations as one small step along the way. >> >> I updated the relevant OopStorage documentation to reflect the removal of this restriction. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8211718/webrev.00/ Looks good. > > Looks good, so far as it goes. However, maybe some renames are in order: > > _concurrent_iteration_active => _concurrent_iteration_state > or > _concurrent_iteration_active => _concurrent_iteration_count > update_iteration_state => update_[concurrent_]iteration_count > > And perhaps an assertion in the update function that it didn?t get updated to negative. I agree, _concurrent_iteration_count and update_concurrent_iteration_count sounds like good new names. cheers, Per From erik.osterlund at oracle.com Fri Oct 5 10:18:16 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 5 Oct 2018 12:18:16 +0200 Subject: RFR: 8211718: Supporting multiple concurrent OopStorage iterators In-Reply-To: <5015fefa-ea3f-d823-fb65-84dc51cd379e@oracle.com> References: <1F14027E-3B77-4847-942C-20665BF29FA6@oracle.com> <5015fefa-ea3f-d823-fb65-84dc51cd379e@oracle.com> Message-ID: <5BB73A68.9070501@oracle.com> Hi Per and Kim, Thank you both for the reviews. The renaming was applied as suggested (including comments). Incremental: http://cr.openjdk.java.net/~eosterlund/8211718/webrev.00_01/ Full: http://cr.openjdk.java.net/~eosterlund/8211718/webrev.01/ Thanks, /Erik On 2018-10-05 08:18, Per Liden wrote: > Hi, > > On 10/05/2018 01:27 AM, Kim Barrett wrote: >>> On Oct 4, 2018, at 9:53 AM, Erik ?sterlund >>> wrote: >>> >>> Hi, >>> >>> OopStorage has an artificial restriction of having only one >>> concurrent iterator at a time. This restriction should be removed. >>> >>> The solution is to turn the bool that tracks concurrent iterations, >>> into an int. >>> >>> For some background, the reason for the restriction has historic >>> reasons, from a time when opening OopStorage for concurrent >>> iteration was done using a lock-free protocol between GC worker >>> threads, which made it tricky. Since then, the model was simplified >>> to just let the ParState RAII object open and close the concurrent >>> iterations. With the simplified model, there is no need for such a >>> restriction. >>> >>> I talked to Kim, and long-term there are more plans in this area, >>> such as removing weak_oops_do, and then unifying the iterators. But >>> for now, I would like to just stop disallowing multiple concurrent >>> iterations as one small step along the way. >>> >>> I updated the relevant OopStorage documentation to reflect the >>> removal of this restriction. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8211718/webrev.00/ > > Looks good. > >> >> Looks good, so far as it goes. However, maybe some renames are in >> order: >> >> _concurrent_iteration_active => _concurrent_iteration_state >> or >> _concurrent_iteration_active => _concurrent_iteration_count >> update_iteration_state => update_[concurrent_]iteration_count >> >> And perhaps an assertion in the update function that it didn?t get >> updated to negative. > > I agree, _concurrent_iteration_count and > update_concurrent_iteration_count sounds like good new names. > > cheers, > Per From per.liden at oracle.com Fri Oct 5 10:29:41 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 5 Oct 2018 12:29:41 +0200 Subject: RFR: 8211718: Supporting multiple concurrent OopStorage iterators In-Reply-To: <5BB73A68.9070501@oracle.com> References: <1F14027E-3B77-4847-942C-20665BF29FA6@oracle.com> <5015fefa-ea3f-d823-fb65-84dc51cd379e@oracle.com> <5BB73A68.9070501@oracle.com> Message-ID: <421a0ed7-aec5-85c5-8b3a-34d6ce7bcdaf@oracle.com> Looks good! /Per On 10/05/2018 12:18 PM, Erik ?sterlund wrote: > Hi Per and Kim, > > Thank you both for the reviews. The renaming was applied as suggested > (including comments). > > Incremental: > http://cr.openjdk.java.net/~eosterlund/8211718/webrev.00_01/ > > Full: > http://cr.openjdk.java.net/~eosterlund/8211718/webrev.01/ > > Thanks, > /Erik > > On 2018-10-05 08:18, Per Liden wrote: >> Hi, >> >> On 10/05/2018 01:27 AM, Kim Barrett wrote: >>>> On Oct 4, 2018, at 9:53 AM, Erik ?sterlund >>>> wrote: >>>> >>>> Hi, >>>> >>>> OopStorage has an artificial restriction of having only one >>>> concurrent iterator at a time. This restriction should be removed. >>>> >>>> The solution is to turn the bool that tracks concurrent iterations, >>>> into an int. >>>> >>>> For some background, the reason for the restriction has historic >>>> reasons, from a time when opening OopStorage for concurrent >>>> iteration was done using a lock-free protocol between GC worker >>>> threads, which made it tricky. Since then, the model was simplified >>>> to just let the ParState RAII object open and close the concurrent >>>> iterations. With the simplified model, there is no need for such a >>>> restriction. >>>> >>>> I talked to Kim, and long-term there are more plans in this area, >>>> such as removing weak_oops_do, and then unifying the iterators. But >>>> for now, I would like to just stop disallowing multiple concurrent >>>> iterations as one small step along the way. >>>> >>>> I updated the relevant OopStorage documentation to reflect the >>>> removal of this restriction. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8211718/webrev.00/ >> >> Looks good. >> >>> >>> Looks good, so far as it goes.? However, maybe some renames are in >>> order: >>> >>> _concurrent_iteration_active => _concurrent_iteration_state >>> or >>> _concurrent_iteration_active => _concurrent_iteration_count >>> update_iteration_state => update_[concurrent_]iteration_count >>> >>> And perhaps an assertion in the update function that it didn?t get >>> updated to negative. >> >> I agree, _concurrent_iteration_count and >> update_concurrent_iteration_count sounds like good new names. >> >> cheers, >> Per > From erik.osterlund at oracle.com Fri Oct 5 11:23:08 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Fri, 5 Oct 2018 13:23:08 +0200 Subject: RFR: 8211718: Supporting multiple concurrent OopStorage iterators In-Reply-To: <421a0ed7-aec5-85c5-8b3a-34d6ce7bcdaf@oracle.com> References: <1F14027E-3B77-4847-942C-20665BF29FA6@oracle.com> <5015fefa-ea3f-d823-fb65-84dc51cd379e@oracle.com> <5BB73A68.9070501@oracle.com> <421a0ed7-aec5-85c5-8b3a-34d6ce7bcdaf@oracle.com> Message-ID: <28B4729B-2D26-4F25-9E0F-5E6D89B5FF35@oracle.com> Hi Per, Thanks for the review! /Erik > On 5 Oct 2018, at 12:29, Per Liden wrote: > > Looks good! > > /Per > >> On 10/05/2018 12:18 PM, Erik ?sterlund wrote: >> Hi Per and Kim, >> Thank you both for the reviews. The renaming was applied as suggested (including comments). >> Incremental: >> http://cr.openjdk.java.net/~eosterlund/8211718/webrev.00_01/ >> Full: >> http://cr.openjdk.java.net/~eosterlund/8211718/webrev.01/ >> Thanks, >> /Erik >>> On 2018-10-05 08:18, Per Liden wrote: >>> Hi, >>> >>> On 10/05/2018 01:27 AM, Kim Barrett wrote: >>>>> On Oct 4, 2018, at 9:53 AM, Erik ?sterlund wrote: >>>>> >>>>> Hi, >>>>> >>>>> OopStorage has an artificial restriction of having only one concurrent iterator at a time. This restriction should be removed. >>>>> >>>>> The solution is to turn the bool that tracks concurrent iterations, into an int. >>>>> >>>>> For some background, the reason for the restriction has historic reasons, from a time when opening OopStorage for concurrent iteration was done using a lock-free protocol between GC worker threads, which made it tricky. Since then, the model was simplified to just let the ParState RAII object open and close the concurrent iterations. With the simplified model, there is no need for such a restriction. >>>>> >>>>> I talked to Kim, and long-term there are more plans in this area, such as removing weak_oops_do, and then unifying the iterators. But for now, I would like to just stop disallowing multiple concurrent iterations as one small step along the way. >>>>> >>>>> I updated the relevant OopStorage documentation to reflect the removal of this restriction. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8211718/webrev.00/ >>> >>> Looks good. >>> >>>> >>>> Looks good, so far as it goes. However, maybe some renames are in order: >>>> >>>> _concurrent_iteration_active => _concurrent_iteration_state >>>> or >>>> _concurrent_iteration_active => _concurrent_iteration_count >>>> update_iteration_state => update_[concurrent_]iteration_count >>>> >>>> And perhaps an assertion in the update function that it didn?t get updated to negative. >>> >>> I agree, _concurrent_iteration_count and update_concurrent_iteration_count sounds like good new names. >>> >>> cheers, >>> Per From vladimir.kozlov at oracle.com Fri Oct 5 16:05:54 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 5 Oct 2018 09:05:54 -0700 Subject: RFR: 8208686: [AOT] JVMTI ResourceExhausted event repeated for same allocation In-Reply-To: References: <910A4A3C-3EEF-4167-84D5-9819C83D6FC1@oracle.com> <64cbe730-5c9a-a04d-9eee-a56abfbb8e07@oracle.com> <015e8416-a948-fdc5-46db-8b5d80ba52e8@oracle.com> <5F46B759-4AF0-4EA2-B2B8-F076DD350F5A@oracle.com> <7FD94ACE-4CF4-488C-9A2B-C440429B915A@oracle.com> Message-ID: <407cc96c-c846-83f8-a924-9df6705a031f@oracle.com> Looks good. Thanks, Vladimir On 10/5/18 8:22 AM, Doug Simon wrote: > Hi, > > Testing found a bug in the original webrev. Namely, when clearing out a pending exception and returning null in the JVMCI ..._or_null stubs, the JavaThread::_vm_result field was not being set to NULL. I've addressed this in v2 of the webrev: > > Relative diff for bug fix: > > ----------------------------------------------------------------------------------------------- > -// Manages a scope in which a failed heap allocation will throw an exception. > -// The pending exception is cleared when leaving the scope. > +// Manages a scope for a JVMCI runtime call that attempts a heap allocation. > +// If there is a pending exception upon closing the scope and the runtime > +// call is of the variety where allocation failure returns NULL without an > +// exception, the following action is taken: > +// 1. The pending exception is cleared > +// 2. NULL is written to JavaThread::_vm_result > +// 3. Checks that an OutOfMemoryError is Universe::out_of_memory_error_retry(). > class RetryableAllocationMark: public StackObj { > private: > JavaThread* _thread; > public: > RetryableAllocationMark(JavaThread* thread, bool activate) { > if (activate) { > - assert(thread->in_retryable_allocation(), "retryable allocation scope is non-reentrant"); > + assert(!thread->in_retryable_allocation(), "retryable allocation scope is non-reentrant"); > _thread = thread; > _thread->set_in_retryable_allocation(true); > } else { > @@ -136,6 +141,7 @@ > ResourceMark rm; > fatal("Unexpected exception in scope of retryable allocation: " INTPTR_FORMAT " of type %s", p2i(ex), ex->klass()->external_name()); > } > + _thread->set_vm_result(NULL); > } > } > } > ----------------------------------------------------------------------------------------------- > > I also took the opportunity to factor out negative array length checking: > > ----------------------------------------------------------------------------------------------- > diff -r 4d36f5998a8b src/hotspot/share/oops/arrayKlass.cpp > --- a/src/hotspot/share/oops/arrayKlass.cpp Fri Oct 05 17:04:06 2018 +0200 > +++ b/src/hotspot/share/oops/arrayKlass.cpp Fri Oct 05 17:17:17 2018 +0200 > @@ -130,9 +130,6 @@ > } > > objArrayOop ArrayKlass::allocate_arrayArray(int n, int length, TRAPS) { > - if (length < 0) { > - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); > - } > check_array_allocation_length(length, arrayOopDesc::max_array_length(T_ARRAY), CHECK_0); > int size = objArrayOopDesc::object_size(length); > Klass* k = array_klass(n+dimension(), CHECK_0); > diff -r 4d36f5998a8b src/hotspot/share/oops/instanceKlass.cpp > --- a/src/hotspot/share/oops/instanceKlass.cpp Fri Oct 05 17:04:06 2018 +0200 > +++ b/src/hotspot/share/oops/instanceKlass.cpp Fri Oct 05 17:17:17 2018 +0200 > @@ -1201,9 +1201,6 @@ > } > > objArrayOop InstanceKlass::allocate_objArray(int n, int length, TRAPS) { > - if (length < 0) { > - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); > - } > check_array_allocation_length(length, arrayOopDesc::max_array_length(T_OBJECT), CHECK_NULL); > int size = objArrayOopDesc::object_size(length); > Klass* ak = array_klass(n, CHECK_NULL); > diff -r 4d36f5998a8b src/hotspot/share/oops/klass.cpp > --- a/src/hotspot/share/oops/klass.cpp Fri Oct 05 17:04:06 2018 +0200 > +++ b/src/hotspot/share/oops/klass.cpp Fri Oct 05 17:17:17 2018 +0200 > @@ -620,6 +620,8 @@ > } else { > THROW_OOP(Universe::out_of_memory_error_retry()); > } > + } else if (length < 0) { > + THROW_MSG(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); > } > } > > diff -r 4d36f5998a8b src/hotspot/share/oops/klass.hpp > --- a/src/hotspot/share/oops/klass.hpp Fri Oct 05 17:04:06 2018 +0200 > +++ b/src/hotspot/share/oops/klass.hpp Fri Oct 05 17:17:17 2018 +0200 > @@ -514,7 +514,7 @@ > virtual Klass* array_klass_impl(bool or_null, int rank, TRAPS); > virtual Klass* array_klass_impl(bool or_null, TRAPS); > > - // Error handling when length > max_length > + // Error handling when length > max_length or length < 0 > static void check_array_allocation_length(int length, int max_length, TRAPS); > > void set_vtable_length(int len) { _vtable_len= len; } > diff -r 4d36f5998a8b src/hotspot/share/oops/objArrayKlass.cpp > --- a/src/hotspot/share/oops/objArrayKlass.cpp Fri Oct 05 17:04:06 2018 +0200 > +++ b/src/hotspot/share/oops/objArrayKlass.cpp Fri Oct 05 17:17:17 2018 +0200 > @@ -170,14 +170,10 @@ > } > > objArrayOop ObjArrayKlass::allocate(int length, TRAPS) { > - if (length >= 0) { > - check_array_allocation_length(length, arrayOopDesc::max_array_length(T_OBJECT), CHECK_0); > - int size = objArrayOopDesc::object_size(length); > - return (objArrayOop)Universe::heap()->array_allocate(this, size, length, > - /* do_zero */ true, THREAD); > - } else { > - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); > - } > + check_array_allocation_length(length, arrayOopDesc::max_array_length(T_OBJECT), CHECK_0); > + int size = objArrayOopDesc::object_size(length); > + return (objArrayOop)Universe::heap()->array_allocate(this, size, length, > + /* do_zero */ true, THREAD); > } > > static int multi_alloc_counter = 0; > diff -r 4d36f5998a8b src/hotspot/share/oops/typeArrayKlass.cpp > --- a/src/hotspot/share/oops/typeArrayKlass.cpp Fri Oct 05 17:04:06 2018 +0200 > +++ b/src/hotspot/share/oops/typeArrayKlass.cpp Fri Oct 05 17:17:17 2018 +0200 > @@ -99,14 +99,10 @@ > > typeArrayOop TypeArrayKlass::allocate_common(int length, bool do_zero, TRAPS) { > assert(log2_element_size() >= 0, "bad scale"); > - if (length >= 0) { > - check_array_allocation_length(length, max_length(), CHECK_NULL); > - size_t size = typeArrayOopDesc::object_size(layout_helper(), length); > - return (typeArrayOop)Universe::heap()->array_allocate(this, (int)size, length, > - do_zero, CHECK_NULL); > - } else { > - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); > - } > + check_array_allocation_length(length, max_length(), CHECK_NULL); > + size_t size = typeArrayOopDesc::object_size(layout_helper(), length); > + return (typeArrayOop)Universe::heap()->array_allocate(this, (int)size, length, > + do_zero, CHECK_NULL); > } > > oop TypeArrayKlass::multi_allocate(int rank, jint* last_size, TRAPS) { > ----------------------------------------------------------------------------------------------- > > Please confirm review these new changes: > > http://cr.openjdk.java.net/~dnsimon/8208686v2 > > -Doug > > >> On 4 Oct 2018, at 00:20, Doug Simon wrote: >> >> Thanks for the reviews Serguei and Vladimir. >> >> Unless I hear objections in the next 24 hours, I'll push this webrev. >> >> -Doug >> >>> On 3 Oct 2018, at 03:14, serguei.spitsyn at oracle.com wrote: >>> >>> Hi Doug, >>> >>> The JVMTI related part looks good to me. >>> Thank you for fixing it! >>> >>> Thanks, >>> Serguei >>> >>> On 10/2/18 1:11 AM, Doug Simon wrote: >>>> It would be great to get some input from the non-compilers teams on this RFR. >>>> >>>> -Doug >>>> >>>>> On 28 Sep 2018, at 19:51, Vladimir Kozlov wrote: >>>>> >>>>> To let you know, me and Tom R. did review these changes and agreed that it is the least intrusive changes for Hotspot shared code. >>>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>> On 9/25/18 8:11 AM, Daniel D. Daugherty wrote: >>>>>> Adding serviceability-dev at ... since this is JVM/TI... >>>>>> Dan >>>>>> On 9/25/18 10:48 AM, Doug Simon wrote: >>>>>>> A major design point of Graal is to treat allocations as non-side effecting to give more freedom to the optimizer by reducing the number of distinct FrameStates that need to be managed. When failing an allocation, Graal will deoptimize to the last side effecting instruction before the allocation. This mean the VM code for heap allocation will potentially be executed twice, once from Graal compiled code and then again in the interpreter. While this is perfectly fine according to the JVM specification, it can cause confusing behavior for JVMTI based tools. They will receive 2 ResourceExhausted events for a single allocation. Furthermore, the first ResourceExhausted event (on the Graal allocation slow path) might denote a bytecode instruction that performs no allocation, making it hard to debug the memory failure. >>>>>>> >>>>>>> The proposed solution is to add an extra set of JVMCI VM runtime calls for allocation. These entry points will attempt the allocation and upon failure, >>>>>>> skip side-effects such as posting JVMTI events or handling -XX:OnOutOfMemoryError. The compiled code using these entry points is expected deoptmize on null. >>>>>>> >>>>>>> The path from these new entry points to where allocation can fail goes through quite a bit of VM code. One could modify all these paths by: >>>>>>> * Returning null instead of throwing an exception on failure. >>>>>>> * Adding a `bool null_on_fail` argument to all relevant methods. >>>>>>> * Adding extra null checking where necessary after each call to these methods when `null_on_fail == true`. >>>>>>> This represents a significant number of changes. >>>>>>> >>>>>>> Instead, the proposed solution introduces a new _in_retryable_allocation thread-local. This way, only the entry points and allocation routines that raise an exception need to be modified. Failure is communicated back to the new entry points by throwing a special pre-allocated OOME object (i.e., Universe::out_of_memory_error_retry()) which must not propagate back to Java code. Use of this object is not strictly necessary; it is introduced to highlight/document the special allocation mode. >>>>>>> >>>>>>> The proposed solution is at http://cr.openjdk.java.net/~dnsimon/8208686. >>>>>>> THE JBS bug is: https://bugs.openjdk.java.net/browse/JDK-8208686 >>>>>>> >>>>>>> -Doug >>> >> > From rkennke at redhat.com Fri Oct 5 16:20:49 2018 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 5 Oct 2018 18:20:49 +0200 Subject: RFR: JDK-8211270: GC abstraction to get real object and headers size In-Reply-To: References: <1b75ccd7-ba1a-a183-f338-18d3c651827e@redhat.com> Message-ID: <5689e478-6f2f-d7ea-127e-689108364074@redhat.com> Any opinions on that one? Roman > Duh. > > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8211270/webrev.00/ > > Thanks, > Roman > > Am 28.09.18 um 12:11 schrieb Roman Kennke: >> Shenandoah GC uses one extra word per object. Some places need to know >> the real object or header size (fwd ptr plus object). >> >> The affected code paths are WB_GetObjectSize(), >> JvmtiEnv::GetObjectSize() and the PLAB constructor. The patch also adds >> an assert later in PLAB that uses the header_size(). >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8211270 >> Webrev: >> file:///home/rkennke/src/openjdk/webrev/JDK-8211270/webrev.00/ >> >> Can I please get reviews? >> >> Thanks, >> Roman >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From doug.simon at oracle.com Fri Oct 5 15:22:18 2018 From: doug.simon at oracle.com (Doug Simon) Date: Fri, 5 Oct 2018 17:22:18 +0200 Subject: RFR: 8208686: [AOT] JVMTI ResourceExhausted event repeated for same allocation In-Reply-To: <7FD94ACE-4CF4-488C-9A2B-C440429B915A@oracle.com> References: <910A4A3C-3EEF-4167-84D5-9819C83D6FC1@oracle.com> <64cbe730-5c9a-a04d-9eee-a56abfbb8e07@oracle.com> <015e8416-a948-fdc5-46db-8b5d80ba52e8@oracle.com> <5F46B759-4AF0-4EA2-B2B8-F076DD350F5A@oracle.com> <7FD94ACE-4CF4-488C-9A2B-C440429B915A@oracle.com> Message-ID: Hi, Testing found a bug in the original webrev. Namely, when clearing out a pending exception and returning null in the JVMCI ..._or_null stubs, the JavaThread::_vm_result field was not being set to NULL. I've addressed this in v2 of the webrev: Relative diff for bug fix: ----------------------------------------------------------------------------------------------- -// Manages a scope in which a failed heap allocation will throw an exception. -// The pending exception is cleared when leaving the scope. +// Manages a scope for a JVMCI runtime call that attempts a heap allocation. +// If there is a pending exception upon closing the scope and the runtime +// call is of the variety where allocation failure returns NULL without an +// exception, the following action is taken: +// 1. The pending exception is cleared +// 2. NULL is written to JavaThread::_vm_result +// 3. Checks that an OutOfMemoryError is Universe::out_of_memory_error_retry(). class RetryableAllocationMark: public StackObj { private: JavaThread* _thread; public: RetryableAllocationMark(JavaThread* thread, bool activate) { if (activate) { - assert(thread->in_retryable_allocation(), "retryable allocation scope is non-reentrant"); + assert(!thread->in_retryable_allocation(), "retryable allocation scope is non-reentrant"); _thread = thread; _thread->set_in_retryable_allocation(true); } else { @@ -136,6 +141,7 @@ ResourceMark rm; fatal("Unexpected exception in scope of retryable allocation: " INTPTR_FORMAT " of type %s", p2i(ex), ex->klass()->external_name()); } + _thread->set_vm_result(NULL); } } } ----------------------------------------------------------------------------------------------- I also took the opportunity to factor out negative array length checking: ----------------------------------------------------------------------------------------------- diff -r 4d36f5998a8b src/hotspot/share/oops/arrayKlass.cpp --- a/src/hotspot/share/oops/arrayKlass.cpp Fri Oct 05 17:04:06 2018 +0200 +++ b/src/hotspot/share/oops/arrayKlass.cpp Fri Oct 05 17:17:17 2018 +0200 @@ -130,9 +130,6 @@ } objArrayOop ArrayKlass::allocate_arrayArray(int n, int length, TRAPS) { - if (length < 0) { - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); - } check_array_allocation_length(length, arrayOopDesc::max_array_length(T_ARRAY), CHECK_0); int size = objArrayOopDesc::object_size(length); Klass* k = array_klass(n+dimension(), CHECK_0); diff -r 4d36f5998a8b src/hotspot/share/oops/instanceKlass.cpp --- a/src/hotspot/share/oops/instanceKlass.cpp Fri Oct 05 17:04:06 2018 +0200 +++ b/src/hotspot/share/oops/instanceKlass.cpp Fri Oct 05 17:17:17 2018 +0200 @@ -1201,9 +1201,6 @@ } objArrayOop InstanceKlass::allocate_objArray(int n, int length, TRAPS) { - if (length < 0) { - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); - } check_array_allocation_length(length, arrayOopDesc::max_array_length(T_OBJECT), CHECK_NULL); int size = objArrayOopDesc::object_size(length); Klass* ak = array_klass(n, CHECK_NULL); diff -r 4d36f5998a8b src/hotspot/share/oops/klass.cpp --- a/src/hotspot/share/oops/klass.cpp Fri Oct 05 17:04:06 2018 +0200 +++ b/src/hotspot/share/oops/klass.cpp Fri Oct 05 17:17:17 2018 +0200 @@ -620,6 +620,8 @@ } else { THROW_OOP(Universe::out_of_memory_error_retry()); } + } else if (length < 0) { + THROW_MSG(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); } } diff -r 4d36f5998a8b src/hotspot/share/oops/klass.hpp --- a/src/hotspot/share/oops/klass.hpp Fri Oct 05 17:04:06 2018 +0200 +++ b/src/hotspot/share/oops/klass.hpp Fri Oct 05 17:17:17 2018 +0200 @@ -514,7 +514,7 @@ virtual Klass* array_klass_impl(bool or_null, int rank, TRAPS); virtual Klass* array_klass_impl(bool or_null, TRAPS); - // Error handling when length > max_length + // Error handling when length > max_length or length < 0 static void check_array_allocation_length(int length, int max_length, TRAPS); void set_vtable_length(int len) { _vtable_len= len; } diff -r 4d36f5998a8b src/hotspot/share/oops/objArrayKlass.cpp --- a/src/hotspot/share/oops/objArrayKlass.cpp Fri Oct 05 17:04:06 2018 +0200 +++ b/src/hotspot/share/oops/objArrayKlass.cpp Fri Oct 05 17:17:17 2018 +0200 @@ -170,14 +170,10 @@ } objArrayOop ObjArrayKlass::allocate(int length, TRAPS) { - if (length >= 0) { - check_array_allocation_length(length, arrayOopDesc::max_array_length(T_OBJECT), CHECK_0); - int size = objArrayOopDesc::object_size(length); - return (objArrayOop)Universe::heap()->array_allocate(this, size, length, - /* do_zero */ true, THREAD); - } else { - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); - } + check_array_allocation_length(length, arrayOopDesc::max_array_length(T_OBJECT), CHECK_0); + int size = objArrayOopDesc::object_size(length); + return (objArrayOop)Universe::heap()->array_allocate(this, size, length, + /* do_zero */ true, THREAD); } static int multi_alloc_counter = 0; diff -r 4d36f5998a8b src/hotspot/share/oops/typeArrayKlass.cpp --- a/src/hotspot/share/oops/typeArrayKlass.cpp Fri Oct 05 17:04:06 2018 +0200 +++ b/src/hotspot/share/oops/typeArrayKlass.cpp Fri Oct 05 17:17:17 2018 +0200 @@ -99,14 +99,10 @@ typeArrayOop TypeArrayKlass::allocate_common(int length, bool do_zero, TRAPS) { assert(log2_element_size() >= 0, "bad scale"); - if (length >= 0) { - check_array_allocation_length(length, max_length(), CHECK_NULL); - size_t size = typeArrayOopDesc::object_size(layout_helper(), length); - return (typeArrayOop)Universe::heap()->array_allocate(this, (int)size, length, - do_zero, CHECK_NULL); - } else { - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); - } + check_array_allocation_length(length, max_length(), CHECK_NULL); + size_t size = typeArrayOopDesc::object_size(layout_helper(), length); + return (typeArrayOop)Universe::heap()->array_allocate(this, (int)size, length, + do_zero, CHECK_NULL); } oop TypeArrayKlass::multi_allocate(int rank, jint* last_size, TRAPS) { ----------------------------------------------------------------------------------------------- Please confirm review these new changes: http://cr.openjdk.java.net/~dnsimon/8208686v2 -Doug > On 4 Oct 2018, at 00:20, Doug Simon wrote: > > Thanks for the reviews Serguei and Vladimir. > > Unless I hear objections in the next 24 hours, I'll push this webrev. > > -Doug > >> On 3 Oct 2018, at 03:14, serguei.spitsyn at oracle.com wrote: >> >> Hi Doug, >> >> The JVMTI related part looks good to me. >> Thank you for fixing it! >> >> Thanks, >> Serguei >> >> On 10/2/18 1:11 AM, Doug Simon wrote: >>> It would be great to get some input from the non-compilers teams on this RFR. >>> >>> -Doug >>> >>>> On 28 Sep 2018, at 19:51, Vladimir Kozlov wrote: >>>> >>>> To let you know, me and Tom R. did review these changes and agreed that it is the least intrusive changes for Hotspot shared code. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 9/25/18 8:11 AM, Daniel D. Daugherty wrote: >>>>> Adding serviceability-dev at ... since this is JVM/TI... >>>>> Dan >>>>> On 9/25/18 10:48 AM, Doug Simon wrote: >>>>>> A major design point of Graal is to treat allocations as non-side effecting to give more freedom to the optimizer by reducing the number of distinct FrameStates that need to be managed. When failing an allocation, Graal will deoptimize to the last side effecting instruction before the allocation. This mean the VM code for heap allocation will potentially be executed twice, once from Graal compiled code and then again in the interpreter. While this is perfectly fine according to the JVM specification, it can cause confusing behavior for JVMTI based tools. They will receive 2 ResourceExhausted events for a single allocation. Furthermore, the first ResourceExhausted event (on the Graal allocation slow path) might denote a bytecode instruction that performs no allocation, making it hard to debug the memory failure. >>>>>> >>>>>> The proposed solution is to add an extra set of JVMCI VM runtime calls for allocation. These entry points will attempt the allocation and upon failure, >>>>>> skip side-effects such as posting JVMTI events or handling -XX:OnOutOfMemoryError. The compiled code using these entry points is expected deoptmize on null. >>>>>> >>>>>> The path from these new entry points to where allocation can fail goes through quite a bit of VM code. One could modify all these paths by: >>>>>> * Returning null instead of throwing an exception on failure. >>>>>> * Adding a `bool null_on_fail` argument to all relevant methods. >>>>>> * Adding extra null checking where necessary after each call to these methods when `null_on_fail == true`. >>>>>> This represents a significant number of changes. >>>>>> >>>>>> Instead, the proposed solution introduces a new _in_retryable_allocation thread-local. This way, only the entry points and allocation routines that raise an exception need to be modified. Failure is communicated back to the new entry points by throwing a special pre-allocated OOME object (i.e., Universe::out_of_memory_error_retry()) which must not propagate back to Java code. Use of this object is not strictly necessary; it is introduced to highlight/document the special allocation mode. >>>>>> >>>>>> The proposed solution is at http://cr.openjdk.java.net/~dnsimon/8208686. >>>>>> THE JBS bug is: https://bugs.openjdk.java.net/browse/JDK-8208686 >>>>>> >>>>>> -Doug >> > From tom.rodriguez at oracle.com Fri Oct 5 17:27:08 2018 From: tom.rodriguez at oracle.com (Tom Rodriguez) Date: Fri, 5 Oct 2018 10:27:08 -0700 Subject: RFR: 8208686: [AOT] JVMTI ResourceExhausted event repeated for same allocation In-Reply-To: References: <910A4A3C-3EEF-4167-84D5-9819C83D6FC1@oracle.com> <64cbe730-5c9a-a04d-9eee-a56abfbb8e07@oracle.com> <015e8416-a948-fdc5-46db-8b5d80ba52e8@oracle.com> <5F46B759-4AF0-4EA2-B2B8-F076DD350F5A@oracle.com> <7FD94ACE-4CF4-488C-9A2B-C440429B915A@oracle.com> Message-ID: Looks good. tom Doug Simon wrote on 10/5/18 8:22 AM: > Hi, > > Testing found a bug in the original webrev. Namely, when clearing out a pending exception and returning null in the JVMCI ..._or_null stubs, the JavaThread::_vm_result field was not being set to NULL. I've addressed this in v2 of the webrev: > > Relative diff for bug fix: > > ----------------------------------------------------------------------------------------------- > -// Manages a scope in which a failed heap allocation will throw an exception. > -// The pending exception is cleared when leaving the scope. > +// Manages a scope for a JVMCI runtime call that attempts a heap allocation. > +// If there is a pending exception upon closing the scope and the runtime > +// call is of the variety where allocation failure returns NULL without an > +// exception, the following action is taken: > +// 1. The pending exception is cleared > +// 2. NULL is written to JavaThread::_vm_result > +// 3. Checks that an OutOfMemoryError is Universe::out_of_memory_error_retry(). > class RetryableAllocationMark: public StackObj { > private: > JavaThread* _thread; > public: > RetryableAllocationMark(JavaThread* thread, bool activate) { > if (activate) { > - assert(thread->in_retryable_allocation(), "retryable allocation scope is non-reentrant"); > + assert(!thread->in_retryable_allocation(), "retryable allocation scope is non-reentrant"); > _thread = thread; > _thread->set_in_retryable_allocation(true); > } else { > @@ -136,6 +141,7 @@ > ResourceMark rm; > fatal("Unexpected exception in scope of retryable allocation: " INTPTR_FORMAT " of type %s", p2i(ex), ex->klass()->external_name()); > } > + _thread->set_vm_result(NULL); > } > } > } > ----------------------------------------------------------------------------------------------- > > I also took the opportunity to factor out negative array length checking: > > ----------------------------------------------------------------------------------------------- > diff -r 4d36f5998a8b src/hotspot/share/oops/arrayKlass.cpp > --- a/src/hotspot/share/oops/arrayKlass.cpp Fri Oct 05 17:04:06 2018 +0200 > +++ b/src/hotspot/share/oops/arrayKlass.cpp Fri Oct 05 17:17:17 2018 +0200 > @@ -130,9 +130,6 @@ > } > > objArrayOop ArrayKlass::allocate_arrayArray(int n, int length, TRAPS) { > - if (length < 0) { > - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); > - } > check_array_allocation_length(length, arrayOopDesc::max_array_length(T_ARRAY), CHECK_0); > int size = objArrayOopDesc::object_size(length); > Klass* k = array_klass(n+dimension(), CHECK_0); > diff -r 4d36f5998a8b src/hotspot/share/oops/instanceKlass.cpp > --- a/src/hotspot/share/oops/instanceKlass.cpp Fri Oct 05 17:04:06 2018 +0200 > +++ b/src/hotspot/share/oops/instanceKlass.cpp Fri Oct 05 17:17:17 2018 +0200 > @@ -1201,9 +1201,6 @@ > } > > objArrayOop InstanceKlass::allocate_objArray(int n, int length, TRAPS) { > - if (length < 0) { > - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); > - } > check_array_allocation_length(length, arrayOopDesc::max_array_length(T_OBJECT), CHECK_NULL); > int size = objArrayOopDesc::object_size(length); > Klass* ak = array_klass(n, CHECK_NULL); > diff -r 4d36f5998a8b src/hotspot/share/oops/klass.cpp > --- a/src/hotspot/share/oops/klass.cpp Fri Oct 05 17:04:06 2018 +0200 > +++ b/src/hotspot/share/oops/klass.cpp Fri Oct 05 17:17:17 2018 +0200 > @@ -620,6 +620,8 @@ > } else { > THROW_OOP(Universe::out_of_memory_error_retry()); > } > + } else if (length < 0) { > + THROW_MSG(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); > } > } > > diff -r 4d36f5998a8b src/hotspot/share/oops/klass.hpp > --- a/src/hotspot/share/oops/klass.hpp Fri Oct 05 17:04:06 2018 +0200 > +++ b/src/hotspot/share/oops/klass.hpp Fri Oct 05 17:17:17 2018 +0200 > @@ -514,7 +514,7 @@ > virtual Klass* array_klass_impl(bool or_null, int rank, TRAPS); > virtual Klass* array_klass_impl(bool or_null, TRAPS); > > - // Error handling when length > max_length > + // Error handling when length > max_length or length < 0 > static void check_array_allocation_length(int length, int max_length, TRAPS); > > void set_vtable_length(int len) { _vtable_len= len; } > diff -r 4d36f5998a8b src/hotspot/share/oops/objArrayKlass.cpp > --- a/src/hotspot/share/oops/objArrayKlass.cpp Fri Oct 05 17:04:06 2018 +0200 > +++ b/src/hotspot/share/oops/objArrayKlass.cpp Fri Oct 05 17:17:17 2018 +0200 > @@ -170,14 +170,10 @@ > } > > objArrayOop ObjArrayKlass::allocate(int length, TRAPS) { > - if (length >= 0) { > - check_array_allocation_length(length, arrayOopDesc::max_array_length(T_OBJECT), CHECK_0); > - int size = objArrayOopDesc::object_size(length); > - return (objArrayOop)Universe::heap()->array_allocate(this, size, length, > - /* do_zero */ true, THREAD); > - } else { > - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); > - } > + check_array_allocation_length(length, arrayOopDesc::max_array_length(T_OBJECT), CHECK_0); > + int size = objArrayOopDesc::object_size(length); > + return (objArrayOop)Universe::heap()->array_allocate(this, size, length, > + /* do_zero */ true, THREAD); > } > > static int multi_alloc_counter = 0; > diff -r 4d36f5998a8b src/hotspot/share/oops/typeArrayKlass.cpp > --- a/src/hotspot/share/oops/typeArrayKlass.cpp Fri Oct 05 17:04:06 2018 +0200 > +++ b/src/hotspot/share/oops/typeArrayKlass.cpp Fri Oct 05 17:17:17 2018 +0200 > @@ -99,14 +99,10 @@ > > typeArrayOop TypeArrayKlass::allocate_common(int length, bool do_zero, TRAPS) { > assert(log2_element_size() >= 0, "bad scale"); > - if (length >= 0) { > - check_array_allocation_length(length, max_length(), CHECK_NULL); > - size_t size = typeArrayOopDesc::object_size(layout_helper(), length); > - return (typeArrayOop)Universe::heap()->array_allocate(this, (int)size, length, > - do_zero, CHECK_NULL); > - } else { > - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); > - } > + check_array_allocation_length(length, max_length(), CHECK_NULL); > + size_t size = typeArrayOopDesc::object_size(layout_helper(), length); > + return (typeArrayOop)Universe::heap()->array_allocate(this, (int)size, length, > + do_zero, CHECK_NULL); > } > > oop TypeArrayKlass::multi_allocate(int rank, jint* last_size, TRAPS) { > ----------------------------------------------------------------------------------------------- > > Please confirm review these new changes: > > http://cr.openjdk.java.net/~dnsimon/8208686v2 > > -Doug > > >> On 4 Oct 2018, at 00:20, Doug Simon wrote: >> >> Thanks for the reviews Serguei and Vladimir. >> >> Unless I hear objections in the next 24 hours, I'll push this webrev. >> >> -Doug >> >>> On 3 Oct 2018, at 03:14, serguei.spitsyn at oracle.com wrote: >>> >>> Hi Doug, >>> >>> The JVMTI related part looks good to me. >>> Thank you for fixing it! >>> >>> Thanks, >>> Serguei >>> >>> On 10/2/18 1:11 AM, Doug Simon wrote: >>>> It would be great to get some input from the non-compilers teams on this RFR. >>>> >>>> -Doug >>>> >>>>> On 28 Sep 2018, at 19:51, Vladimir Kozlov wrote: >>>>> >>>>> To let you know, me and Tom R. did review these changes and agreed that it is the least intrusive changes for Hotspot shared code. >>>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>> On 9/25/18 8:11 AM, Daniel D. Daugherty wrote: >>>>>> Adding serviceability-dev at ... since this is JVM/TI... >>>>>> Dan >>>>>> On 9/25/18 10:48 AM, Doug Simon wrote: >>>>>>> A major design point of Graal is to treat allocations as non-side effecting to give more freedom to the optimizer by reducing the number of distinct FrameStates that need to be managed. When failing an allocation, Graal will deoptimize to the last side effecting instruction before the allocation. This mean the VM code for heap allocation will potentially be executed twice, once from Graal compiled code and then again in the interpreter. While this is perfectly fine according to the JVM specification, it can cause confusing behavior for JVMTI based tools. They will receive 2 ResourceExhausted events for a single allocation. Furthermore, the first ResourceExhausted event (on the Graal allocation slow path) might denote a bytecode instruction that performs no allocation, making it hard to debug the memory failure. >>>>>>> >>>>>>> The proposed solution is to add an extra set of JVMCI VM runtime calls for allocation. These entry points will attempt the allocation and upon failure, >>>>>>> skip side-effects such as posting JVMTI events or handling -XX:OnOutOfMemoryError. The compiled code using these entry points is expected deoptmize on null. >>>>>>> >>>>>>> The path from these new entry points to where allocation can fail goes through quite a bit of VM code. One could modify all these paths by: >>>>>>> * Returning null instead of throwing an exception on failure. >>>>>>> * Adding a `bool null_on_fail` argument to all relevant methods. >>>>>>> * Adding extra null checking where necessary after each call to these methods when `null_on_fail == true`. >>>>>>> This represents a significant number of changes. >>>>>>> >>>>>>> Instead, the proposed solution introduces a new _in_retryable_allocation thread-local. This way, only the entry points and allocation routines that raise an exception need to be modified. Failure is communicated back to the new entry points by throwing a special pre-allocated OOME object (i.e., Universe::out_of_memory_error_retry()) which must not propagate back to Java code. Use of this object is not strictly necessary; it is introduced to highlight/document the special allocation mode. >>>>>>> >>>>>>> The proposed solution is at http://cr.openjdk.java.net/~dnsimon/8208686. >>>>>>> THE JBS bug is: https://bugs.openjdk.java.net/browse/JDK-8208686 >>>>>>> >>>>>>> -Doug >>> >> > From doug.simon at oracle.com Fri Oct 5 17:58:39 2018 From: doug.simon at oracle.com (Doug Simon) Date: Fri, 5 Oct 2018 19:58:39 +0200 Subject: RFR: 8208686: [AOT] JVMTI ResourceExhausted event repeated for same allocation In-Reply-To: <407cc96c-c846-83f8-a924-9df6705a031f@oracle.com> References: <910A4A3C-3EEF-4167-84D5-9819C83D6FC1@oracle.com> <64cbe730-5c9a-a04d-9eee-a56abfbb8e07@oracle.com> <015e8416-a948-fdc5-46db-8b5d80ba52e8@oracle.com> <5F46B759-4AF0-4EA2-B2B8-F076DD350F5A@oracle.com> <7FD94ACE-4CF4-488C-9A2B-C440429B915A@oracle.com> <407cc96c-c846-83f8-a924-9df6705a031f@oracle.com> Message-ID: Thanks for re-reviewing. -Doug > On 5 Oct 2018, at 18:05, Vladimir Kozlov wrote: > > Looks good. > > Thanks, > Vladimir > > On 10/5/18 8:22 AM, Doug Simon wrote: >> Hi, >> Testing found a bug in the original webrev. Namely, when clearing out a pending exception and returning null in the JVMCI ..._or_null stubs, the JavaThread::_vm_result field was not being set to NULL. I've addressed this in v2 of the webrev: >> Relative diff for bug fix: >> ----------------------------------------------------------------------------------------------- >> -// Manages a scope in which a failed heap allocation will throw an exception. >> -// The pending exception is cleared when leaving the scope. >> +// Manages a scope for a JVMCI runtime call that attempts a heap allocation. >> +// If there is a pending exception upon closing the scope and the runtime >> +// call is of the variety where allocation failure returns NULL without an >> +// exception, the following action is taken: >> +// 1. The pending exception is cleared >> +// 2. NULL is written to JavaThread::_vm_result >> +// 3. Checks that an OutOfMemoryError is Universe::out_of_memory_error_retry(). >> class RetryableAllocationMark: public StackObj { >> private: >> JavaThread* _thread; >> public: >> RetryableAllocationMark(JavaThread* thread, bool activate) { >> if (activate) { >> - assert(thread->in_retryable_allocation(), "retryable allocation scope is non-reentrant"); >> + assert(!thread->in_retryable_allocation(), "retryable allocation scope is non-reentrant"); >> _thread = thread; >> _thread->set_in_retryable_allocation(true); >> } else { >> @@ -136,6 +141,7 @@ >> ResourceMark rm; >> fatal("Unexpected exception in scope of retryable allocation: " INTPTR_FORMAT " of type %s", p2i(ex), ex->klass()->external_name()); >> } >> + _thread->set_vm_result(NULL); >> } >> } >> } >> ----------------------------------------------------------------------------------------------- >> I also took the opportunity to factor out negative array length checking: >> ----------------------------------------------------------------------------------------------- >> diff -r 4d36f5998a8b src/hotspot/share/oops/arrayKlass.cpp >> --- a/src/hotspot/share/oops/arrayKlass.cpp Fri Oct 05 17:04:06 2018 +0200 >> +++ b/src/hotspot/share/oops/arrayKlass.cpp Fri Oct 05 17:17:17 2018 +0200 >> @@ -130,9 +130,6 @@ >> } >> objArrayOop ArrayKlass::allocate_arrayArray(int n, int length, TRAPS) { >> - if (length < 0) { >> - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); >> - } >> check_array_allocation_length(length, arrayOopDesc::max_array_length(T_ARRAY), CHECK_0); >> int size = objArrayOopDesc::object_size(length); >> Klass* k = array_klass(n+dimension(), CHECK_0); >> diff -r 4d36f5998a8b src/hotspot/share/oops/instanceKlass.cpp >> --- a/src/hotspot/share/oops/instanceKlass.cpp Fri Oct 05 17:04:06 2018 +0200 >> +++ b/src/hotspot/share/oops/instanceKlass.cpp Fri Oct 05 17:17:17 2018 +0200 >> @@ -1201,9 +1201,6 @@ >> } >> objArrayOop InstanceKlass::allocate_objArray(int n, int length, TRAPS) { >> - if (length < 0) { >> - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); >> - } >> check_array_allocation_length(length, arrayOopDesc::max_array_length(T_OBJECT), CHECK_NULL); >> int size = objArrayOopDesc::object_size(length); >> Klass* ak = array_klass(n, CHECK_NULL); >> diff -r 4d36f5998a8b src/hotspot/share/oops/klass.cpp >> --- a/src/hotspot/share/oops/klass.cpp Fri Oct 05 17:04:06 2018 +0200 >> +++ b/src/hotspot/share/oops/klass.cpp Fri Oct 05 17:17:17 2018 +0200 >> @@ -620,6 +620,8 @@ >> } else { >> THROW_OOP(Universe::out_of_memory_error_retry()); >> } >> + } else if (length < 0) { >> + THROW_MSG(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); >> } >> } >> diff -r 4d36f5998a8b src/hotspot/share/oops/klass.hpp >> --- a/src/hotspot/share/oops/klass.hpp Fri Oct 05 17:04:06 2018 +0200 >> +++ b/src/hotspot/share/oops/klass.hpp Fri Oct 05 17:17:17 2018 +0200 >> @@ -514,7 +514,7 @@ >> virtual Klass* array_klass_impl(bool or_null, int rank, TRAPS); >> virtual Klass* array_klass_impl(bool or_null, TRAPS); >> - // Error handling when length > max_length >> + // Error handling when length > max_length or length < 0 >> static void check_array_allocation_length(int length, int max_length, TRAPS); >> void set_vtable_length(int len) { _vtable_len= len; } >> diff -r 4d36f5998a8b src/hotspot/share/oops/objArrayKlass.cpp >> --- a/src/hotspot/share/oops/objArrayKlass.cpp Fri Oct 05 17:04:06 2018 +0200 >> +++ b/src/hotspot/share/oops/objArrayKlass.cpp Fri Oct 05 17:17:17 2018 +0200 >> @@ -170,14 +170,10 @@ >> } >> objArrayOop ObjArrayKlass::allocate(int length, TRAPS) { >> - if (length >= 0) { >> - check_array_allocation_length(length, arrayOopDesc::max_array_length(T_OBJECT), CHECK_0); >> - int size = objArrayOopDesc::object_size(length); >> - return (objArrayOop)Universe::heap()->array_allocate(this, size, length, >> - /* do_zero */ true, THREAD); >> - } else { >> - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); >> - } >> + check_array_allocation_length(length, arrayOopDesc::max_array_length(T_OBJECT), CHECK_0); >> + int size = objArrayOopDesc::object_size(length); >> + return (objArrayOop)Universe::heap()->array_allocate(this, size, length, >> + /* do_zero */ true, THREAD); >> } >> static int multi_alloc_counter = 0; >> diff -r 4d36f5998a8b src/hotspot/share/oops/typeArrayKlass.cpp >> --- a/src/hotspot/share/oops/typeArrayKlass.cpp Fri Oct 05 17:04:06 2018 +0200 >> +++ b/src/hotspot/share/oops/typeArrayKlass.cpp Fri Oct 05 17:17:17 2018 +0200 >> @@ -99,14 +99,10 @@ >> typeArrayOop TypeArrayKlass::allocate_common(int length, bool do_zero, TRAPS) { >> assert(log2_element_size() >= 0, "bad scale"); >> - if (length >= 0) { >> - check_array_allocation_length(length, max_length(), CHECK_NULL); >> - size_t size = typeArrayOopDesc::object_size(layout_helper(), length); >> - return (typeArrayOop)Universe::heap()->array_allocate(this, (int)size, length, >> - do_zero, CHECK_NULL); >> - } else { >> - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); >> - } >> + check_array_allocation_length(length, max_length(), CHECK_NULL); >> + size_t size = typeArrayOopDesc::object_size(layout_helper(), length); >> + return (typeArrayOop)Universe::heap()->array_allocate(this, (int)size, length, >> + do_zero, CHECK_NULL); >> } >> oop TypeArrayKlass::multi_allocate(int rank, jint* last_size, TRAPS) { >> ----------------------------------------------------------------------------------------------- >> Please confirm review these new changes: >> http://cr.openjdk.java.net/~dnsimon/8208686v2 >> -Doug >>> On 4 Oct 2018, at 00:20, Doug Simon wrote: >>> >>> Thanks for the reviews Serguei and Vladimir. >>> >>> Unless I hear objections in the next 24 hours, I'll push this webrev. >>> >>> -Doug >>> >>>> On 3 Oct 2018, at 03:14, serguei.spitsyn at oracle.com wrote: >>>> >>>> Hi Doug, >>>> >>>> The JVMTI related part looks good to me. >>>> Thank you for fixing it! >>>> >>>> Thanks, >>>> Serguei >>>> >>>> On 10/2/18 1:11 AM, Doug Simon wrote: >>>>> It would be great to get some input from the non-compilers teams on this RFR. >>>>> >>>>> -Doug >>>>> >>>>>> On 28 Sep 2018, at 19:51, Vladimir Kozlov wrote: >>>>>> >>>>>> To let you know, me and Tom R. did review these changes and agreed that it is the least intrusive changes for Hotspot shared code. >>>>>> >>>>>> Thanks, >>>>>> Vladimir >>>>>> >>>>>> On 9/25/18 8:11 AM, Daniel D. Daugherty wrote: >>>>>>> Adding serviceability-dev at ... since this is JVM/TI... >>>>>>> Dan >>>>>>> On 9/25/18 10:48 AM, Doug Simon wrote: >>>>>>>> A major design point of Graal is to treat allocations as non-side effecting to give more freedom to the optimizer by reducing the number of distinct FrameStates that need to be managed. When failing an allocation, Graal will deoptimize to the last side effecting instruction before the allocation. This mean the VM code for heap allocation will potentially be executed twice, once from Graal compiled code and then again in the interpreter. While this is perfectly fine according to the JVM specification, it can cause confusing behavior for JVMTI based tools. They will receive 2 ResourceExhausted events for a single allocation. Furthermore, the first ResourceExhausted event (on the Graal allocation slow path) might denote a bytecode instruction that performs no allocation, making it hard to debug the memory failure. >>>>>>>> >>>>>>>> The proposed solution is to add an extra set of JVMCI VM runtime calls for allocation. These entry points will attempt the allocation and upon failure, >>>>>>>> skip side-effects such as posting JVMTI events or handling -XX:OnOutOfMemoryError. The compiled code using these entry points is expected deoptmize on null. >>>>>>>> >>>>>>>> The path from these new entry points to where allocation can fail goes through quite a bit of VM code. One could modify all these paths by: >>>>>>>> * Returning null instead of throwing an exception on failure. >>>>>>>> * Adding a `bool null_on_fail` argument to all relevant methods. >>>>>>>> * Adding extra null checking where necessary after each call to these methods when `null_on_fail == true`. >>>>>>>> This represents a significant number of changes. >>>>>>>> >>>>>>>> Instead, the proposed solution introduces a new _in_retryable_allocation thread-local. This way, only the entry points and allocation routines that raise an exception need to be modified. Failure is communicated back to the new entry points by throwing a special pre-allocated OOME object (i.e., Universe::out_of_memory_error_retry()) which must not propagate back to Java code. Use of this object is not strictly necessary; it is introduced to highlight/document the special allocation mode. >>>>>>>> >>>>>>>> The proposed solution is at http://cr.openjdk.java.net/~dnsimon/8208686. >>>>>>>> THE JBS bug is: https://bugs.openjdk.java.net/browse/JDK-8208686 >>>>>>>> >>>>>>>> -Doug >>>> >>> From doug.simon at oracle.com Fri Oct 5 17:58:56 2018 From: doug.simon at oracle.com (Doug Simon) Date: Fri, 5 Oct 2018 19:58:56 +0200 Subject: RFR: 8208686: [AOT] JVMTI ResourceExhausted event repeated for same allocation In-Reply-To: References: <910A4A3C-3EEF-4167-84D5-9819C83D6FC1@oracle.com> <64cbe730-5c9a-a04d-9eee-a56abfbb8e07@oracle.com> <015e8416-a948-fdc5-46db-8b5d80ba52e8@oracle.com> <5F46B759-4AF0-4EA2-B2B8-F076DD350F5A@oracle.com> <7FD94ACE-4CF4-488C-9A2B-C440429B915A@oracle.com> Message-ID: Thanks for the review. -Doug > On 5 Oct 2018, at 19:27, Tom Rodriguez wrote: > > Looks good. > > tom > > Doug Simon wrote on 10/5/18 8:22 AM: >> Hi, >> Testing found a bug in the original webrev. Namely, when clearing out a pending exception and returning null in the JVMCI ..._or_null stubs, the JavaThread::_vm_result field was not being set to NULL. I've addressed this in v2 of the webrev: >> Relative diff for bug fix: >> ----------------------------------------------------------------------------------------------- >> -// Manages a scope in which a failed heap allocation will throw an exception. >> -// The pending exception is cleared when leaving the scope. >> +// Manages a scope for a JVMCI runtime call that attempts a heap allocation. >> +// If there is a pending exception upon closing the scope and the runtime >> +// call is of the variety where allocation failure returns NULL without an >> +// exception, the following action is taken: >> +// 1. The pending exception is cleared >> +// 2. NULL is written to JavaThread::_vm_result >> +// 3. Checks that an OutOfMemoryError is Universe::out_of_memory_error_retry(). >> class RetryableAllocationMark: public StackObj { >> private: >> JavaThread* _thread; >> public: >> RetryableAllocationMark(JavaThread* thread, bool activate) { >> if (activate) { >> - assert(thread->in_retryable_allocation(), "retryable allocation scope is non-reentrant"); >> + assert(!thread->in_retryable_allocation(), "retryable allocation scope is non-reentrant"); >> _thread = thread; >> _thread->set_in_retryable_allocation(true); >> } else { >> @@ -136,6 +141,7 @@ >> ResourceMark rm; >> fatal("Unexpected exception in scope of retryable allocation: " INTPTR_FORMAT " of type %s", p2i(ex), ex->klass()->external_name()); >> } >> + _thread->set_vm_result(NULL); >> } >> } >> } >> ----------------------------------------------------------------------------------------------- >> I also took the opportunity to factor out negative array length checking: >> ----------------------------------------------------------------------------------------------- >> diff -r 4d36f5998a8b src/hotspot/share/oops/arrayKlass.cpp >> --- a/src/hotspot/share/oops/arrayKlass.cpp Fri Oct 05 17:04:06 2018 +0200 >> +++ b/src/hotspot/share/oops/arrayKlass.cpp Fri Oct 05 17:17:17 2018 +0200 >> @@ -130,9 +130,6 @@ >> } >> objArrayOop ArrayKlass::allocate_arrayArray(int n, int length, TRAPS) { >> - if (length < 0) { >> - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); >> - } >> check_array_allocation_length(length, arrayOopDesc::max_array_length(T_ARRAY), CHECK_0); >> int size = objArrayOopDesc::object_size(length); >> Klass* k = array_klass(n+dimension(), CHECK_0); >> diff -r 4d36f5998a8b src/hotspot/share/oops/instanceKlass.cpp >> --- a/src/hotspot/share/oops/instanceKlass.cpp Fri Oct 05 17:04:06 2018 +0200 >> +++ b/src/hotspot/share/oops/instanceKlass.cpp Fri Oct 05 17:17:17 2018 +0200 >> @@ -1201,9 +1201,6 @@ >> } >> objArrayOop InstanceKlass::allocate_objArray(int n, int length, TRAPS) { >> - if (length < 0) { >> - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); >> - } >> check_array_allocation_length(length, arrayOopDesc::max_array_length(T_OBJECT), CHECK_NULL); >> int size = objArrayOopDesc::object_size(length); >> Klass* ak = array_klass(n, CHECK_NULL); >> diff -r 4d36f5998a8b src/hotspot/share/oops/klass.cpp >> --- a/src/hotspot/share/oops/klass.cpp Fri Oct 05 17:04:06 2018 +0200 >> +++ b/src/hotspot/share/oops/klass.cpp Fri Oct 05 17:17:17 2018 +0200 >> @@ -620,6 +620,8 @@ >> } else { >> THROW_OOP(Universe::out_of_memory_error_retry()); >> } >> + } else if (length < 0) { >> + THROW_MSG(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); >> } >> } >> diff -r 4d36f5998a8b src/hotspot/share/oops/klass.hpp >> --- a/src/hotspot/share/oops/klass.hpp Fri Oct 05 17:04:06 2018 +0200 >> +++ b/src/hotspot/share/oops/klass.hpp Fri Oct 05 17:17:17 2018 +0200 >> @@ -514,7 +514,7 @@ >> virtual Klass* array_klass_impl(bool or_null, int rank, TRAPS); >> virtual Klass* array_klass_impl(bool or_null, TRAPS); >> - // Error handling when length > max_length >> + // Error handling when length > max_length or length < 0 >> static void check_array_allocation_length(int length, int max_length, TRAPS); >> void set_vtable_length(int len) { _vtable_len= len; } >> diff -r 4d36f5998a8b src/hotspot/share/oops/objArrayKlass.cpp >> --- a/src/hotspot/share/oops/objArrayKlass.cpp Fri Oct 05 17:04:06 2018 +0200 >> +++ b/src/hotspot/share/oops/objArrayKlass.cpp Fri Oct 05 17:17:17 2018 +0200 >> @@ -170,14 +170,10 @@ >> } >> objArrayOop ObjArrayKlass::allocate(int length, TRAPS) { >> - if (length >= 0) { >> - check_array_allocation_length(length, arrayOopDesc::max_array_length(T_OBJECT), CHECK_0); >> - int size = objArrayOopDesc::object_size(length); >> - return (objArrayOop)Universe::heap()->array_allocate(this, size, length, >> - /* do_zero */ true, THREAD); >> - } else { >> - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); >> - } >> + check_array_allocation_length(length, arrayOopDesc::max_array_length(T_OBJECT), CHECK_0); >> + int size = objArrayOopDesc::object_size(length); >> + return (objArrayOop)Universe::heap()->array_allocate(this, size, length, >> + /* do_zero */ true, THREAD); >> } >> static int multi_alloc_counter = 0; >> diff -r 4d36f5998a8b src/hotspot/share/oops/typeArrayKlass.cpp >> --- a/src/hotspot/share/oops/typeArrayKlass.cpp Fri Oct 05 17:04:06 2018 +0200 >> +++ b/src/hotspot/share/oops/typeArrayKlass.cpp Fri Oct 05 17:17:17 2018 +0200 >> @@ -99,14 +99,10 @@ >> typeArrayOop TypeArrayKlass::allocate_common(int length, bool do_zero, TRAPS) { >> assert(log2_element_size() >= 0, "bad scale"); >> - if (length >= 0) { >> - check_array_allocation_length(length, max_length(), CHECK_NULL); >> - size_t size = typeArrayOopDesc::object_size(layout_helper(), length); >> - return (typeArrayOop)Universe::heap()->array_allocate(this, (int)size, length, >> - do_zero, CHECK_NULL); >> - } else { >> - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), err_msg("%d", length)); >> - } >> + check_array_allocation_length(length, max_length(), CHECK_NULL); >> + size_t size = typeArrayOopDesc::object_size(layout_helper(), length); >> + return (typeArrayOop)Universe::heap()->array_allocate(this, (int)size, length, >> + do_zero, CHECK_NULL); >> } >> oop TypeArrayKlass::multi_allocate(int rank, jint* last_size, TRAPS) { >> ----------------------------------------------------------------------------------------------- >> Please confirm review these new changes: >> http://cr.openjdk.java.net/~dnsimon/8208686v2 >> -Doug >>> On 4 Oct 2018, at 00:20, Doug Simon wrote: >>> >>> Thanks for the reviews Serguei and Vladimir. >>> >>> Unless I hear objections in the next 24 hours, I'll push this webrev. >>> >>> -Doug >>> >>>> On 3 Oct 2018, at 03:14, serguei.spitsyn at oracle.com wrote: >>>> >>>> Hi Doug, >>>> >>>> The JVMTI related part looks good to me. >>>> Thank you for fixing it! >>>> >>>> Thanks, >>>> Serguei >>>> >>>> On 10/2/18 1:11 AM, Doug Simon wrote: >>>>> It would be great to get some input from the non-compilers teams on this RFR. >>>>> >>>>> -Doug >>>>> >>>>>> On 28 Sep 2018, at 19:51, Vladimir Kozlov wrote: >>>>>> >>>>>> To let you know, me and Tom R. did review these changes and agreed that it is the least intrusive changes for Hotspot shared code. >>>>>> >>>>>> Thanks, >>>>>> Vladimir >>>>>> >>>>>> On 9/25/18 8:11 AM, Daniel D. Daugherty wrote: >>>>>>> Adding serviceability-dev at ... since this is JVM/TI... >>>>>>> Dan >>>>>>> On 9/25/18 10:48 AM, Doug Simon wrote: >>>>>>>> A major design point of Graal is to treat allocations as non-side effecting to give more freedom to the optimizer by reducing the number of distinct FrameStates that need to be managed. When failing an allocation, Graal will deoptimize to the last side effecting instruction before the allocation. This mean the VM code for heap allocation will potentially be executed twice, once from Graal compiled code and then again in the interpreter. While this is perfectly fine according to the JVM specification, it can cause confusing behavior for JVMTI based tools. They will receive 2 ResourceExhausted events for a single allocation. Furthermore, the first ResourceExhausted event (on the Graal allocation slow path) might denote a bytecode instruction that performs no allocation, making it hard to debug the memory failure. >>>>>>>> >>>>>>>> The proposed solution is to add an extra set of JVMCI VM runtime calls for allocation. These entry points will attempt the allocation and upon failure, >>>>>>>> skip side-effects such as posting JVMTI events or handling -XX:OnOutOfMemoryError. The compiled code using these entry points is expected deoptmize on null. >>>>>>>> >>>>>>>> The path from these new entry points to where allocation can fail goes through quite a bit of VM code. One could modify all these paths by: >>>>>>>> * Returning null instead of throwing an exception on failure. >>>>>>>> * Adding a `bool null_on_fail` argument to all relevant methods. >>>>>>>> * Adding extra null checking where necessary after each call to these methods when `null_on_fail == true`. >>>>>>>> This represents a significant number of changes. >>>>>>>> >>>>>>>> Instead, the proposed solution introduces a new _in_retryable_allocation thread-local. This way, only the entry points and allocation routines that raise an exception need to be modified. Failure is communicated back to the new entry points by throwing a special pre-allocated OOME object (i.e., Universe::out_of_memory_error_retry()) which must not propagate back to Java code. Use of this object is not strictly necessary; it is introduced to highlight/document the special allocation mode. >>>>>>>> >>>>>>>> The proposed solution is at http://cr.openjdk.java.net/~dnsimon/8208686. >>>>>>>> THE JBS bug is: https://bugs.openjdk.java.net/browse/JDK-8208686 >>>>>>>> >>>>>>>> -Doug >>>> >>> From serguei.spitsyn at oracle.com Fri Oct 5 19:56:12 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 5 Oct 2018 12:56:12 -0700 Subject: RFR: 8208686: [AOT] JVMTI ResourceExhausted event repeated for same allocation In-Reply-To: <407cc96c-c846-83f8-a924-9df6705a031f@oracle.com> References: <910A4A3C-3EEF-4167-84D5-9819C83D6FC1@oracle.com> <64cbe730-5c9a-a04d-9eee-a56abfbb8e07@oracle.com> <015e8416-a948-fdc5-46db-8b5d80ba52e8@oracle.com> <5F46B759-4AF0-4EA2-B2B8-F076DD350F5A@oracle.com> <7FD94ACE-4CF4-488C-9A2B-C440429B915A@oracle.com> <407cc96c-c846-83f8-a924-9df6705a031f@oracle.com> Message-ID: +1 Thanks, Serguei On 10/5/18 9:05 AM, Vladimir Kozlov wrote: > Looks good. > > Thanks, > Vladimir > > On 10/5/18 8:22 AM, Doug Simon wrote: >> Hi, >> >> Testing found a bug in the original webrev. Namely, when clearing out >> a pending exception and returning null in the JVMCI ..._or_null >> stubs, the JavaThread::_vm_result field was not being set to NULL. >> I've addressed this in v2 of the webrev: >> >> Relative diff for bug fix: >> >> ----------------------------------------------------------------------------------------------- >> >> -// Manages a scope in which a failed heap allocation will throw an >> exception. >> -// The pending exception is cleared when leaving the scope. >> +// Manages a scope for a JVMCI runtime call that attempts a heap >> allocation. >> +// If there is a pending exception upon closing the scope and the >> runtime >> +// call is of the variety where allocation failure returns NULL >> without an >> +// exception, the following action is taken: >> +//?? 1. The pending exception is cleared >> +//?? 2. NULL is written to JavaThread::_vm_result >> +//?? 3. Checks that an OutOfMemoryError is >> Universe::out_of_memory_error_retry(). >> ? class RetryableAllocationMark: public StackObj { >> ?? private: >> ??? JavaThread* _thread; >> ?? public: >> ??? RetryableAllocationMark(JavaThread* thread, bool activate) { >> ????? if (activate) { >> -????? assert(thread->in_retryable_allocation(), "retryable >> allocation scope is non-reentrant"); >> +????? assert(!thread->in_retryable_allocation(), "retryable >> allocation scope is non-reentrant"); >> ??????? _thread = thread; >> ??????? _thread->set_in_retryable_allocation(true); >> ????? } else { >> @@ -136,6 +141,7 @@ >> ??????????? ResourceMark rm; >> ??????????? fatal("Unexpected exception in scope of retryable >> allocation: " INTPTR_FORMAT " of type %s", p2i(ex), >> ex->klass()->external_name()); >> ????????? } >> +??????? _thread->set_vm_result(NULL); >> ??????? } >> ????? } >> ??? } >> ----------------------------------------------------------------------------------------------- >> >> >> I also took the opportunity to factor out negative array length >> checking: >> >> ----------------------------------------------------------------------------------------------- >> >> diff -r 4d36f5998a8b src/hotspot/share/oops/arrayKlass.cpp >> --- a/src/hotspot/share/oops/arrayKlass.cpp??? Fri Oct 05 17:04:06 >> 2018 +0200 >> +++ b/src/hotspot/share/oops/arrayKlass.cpp??? Fri Oct 05 17:17:17 >> 2018 +0200 >> @@ -130,9 +130,6 @@ >> ? } >> ? ? objArrayOop ArrayKlass::allocate_arrayArray(int n, int length, >> TRAPS) { >> -? if (length < 0) { >> - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), >> err_msg("%d", length)); >> -? } >> ??? check_array_allocation_length(length, >> arrayOopDesc::max_array_length(T_ARRAY), CHECK_0); >> ??? int size = objArrayOopDesc::object_size(length); >> ??? Klass* k = array_klass(n+dimension(), CHECK_0); >> diff -r 4d36f5998a8b src/hotspot/share/oops/instanceKlass.cpp >> --- a/src/hotspot/share/oops/instanceKlass.cpp??? Fri Oct 05 17:04:06 >> 2018 +0200 >> +++ b/src/hotspot/share/oops/instanceKlass.cpp??? Fri Oct 05 17:17:17 >> 2018 +0200 >> @@ -1201,9 +1201,6 @@ >> ? } >> ? ? objArrayOop InstanceKlass::allocate_objArray(int n, int length, >> TRAPS) { >> -? if (length < 0)? { >> - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), >> err_msg("%d", length)); >> -? } >> ??? check_array_allocation_length(length, >> arrayOopDesc::max_array_length(T_OBJECT), CHECK_NULL); >> ??? int size = objArrayOopDesc::object_size(length); >> ??? Klass* ak = array_klass(n, CHECK_NULL); >> diff -r 4d36f5998a8b src/hotspot/share/oops/klass.cpp >> --- a/src/hotspot/share/oops/klass.cpp??? Fri Oct 05 17:04:06 2018 +0200 >> +++ b/src/hotspot/share/oops/klass.cpp??? Fri Oct 05 17:17:17 2018 +0200 >> @@ -620,6 +620,8 @@ >> ????? } else { >> ??????? THROW_OOP(Universe::out_of_memory_error_retry()); >> ????? } >> +? } else if (length < 0) { >> + THROW_MSG(vmSymbols::java_lang_NegativeArraySizeException(), >> err_msg("%d", length)); >> ??? } >> ? } >> ? diff -r 4d36f5998a8b src/hotspot/share/oops/klass.hpp >> --- a/src/hotspot/share/oops/klass.hpp??? Fri Oct 05 17:04:06 2018 +0200 >> +++ b/src/hotspot/share/oops/klass.hpp??? Fri Oct 05 17:17:17 2018 +0200 >> @@ -514,7 +514,7 @@ >> ??? virtual Klass* array_klass_impl(bool or_null, int rank, TRAPS); >> ??? virtual Klass* array_klass_impl(bool or_null, TRAPS); >> ? -? // Error handling when length > max_length >> +? // Error handling when length > max_length or length < 0 >> ??? static void check_array_allocation_length(int length, int >> max_length, TRAPS); >> ? ??? void set_vtable_length(int len) { _vtable_len= len; } >> diff -r 4d36f5998a8b src/hotspot/share/oops/objArrayKlass.cpp >> --- a/src/hotspot/share/oops/objArrayKlass.cpp??? Fri Oct 05 17:04:06 >> 2018 +0200 >> +++ b/src/hotspot/share/oops/objArrayKlass.cpp??? Fri Oct 05 17:17:17 >> 2018 +0200 >> @@ -170,14 +170,10 @@ >> ? } >> ? ? objArrayOop ObjArrayKlass::allocate(int length, TRAPS) { >> -? if (length >= 0) { >> -??? check_array_allocation_length(length, >> arrayOopDesc::max_array_length(T_OBJECT), CHECK_0); >> -??? int size = objArrayOopDesc::object_size(length); >> -??? return (objArrayOop)Universe::heap()->array_allocate(this, size, >> length, >> -???????????????????????????????????????????????????????? /* do_zero >> */ true, THREAD); >> -? } else { >> - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), >> err_msg("%d", length)); >> -? } >> +? check_array_allocation_length(length, >> arrayOopDesc::max_array_length(T_OBJECT), CHECK_0); >> +? int size = objArrayOopDesc::object_size(length); >> +? return (objArrayOop)Universe::heap()->array_allocate(this, size, >> length, >> +?????????????????????????????????????????????????????? /* do_zero */ >> true, THREAD); >> ? } >> ? ? static int multi_alloc_counter = 0; >> diff -r 4d36f5998a8b src/hotspot/share/oops/typeArrayKlass.cpp >> --- a/src/hotspot/share/oops/typeArrayKlass.cpp??? Fri Oct 05 >> 17:04:06 2018 +0200 >> +++ b/src/hotspot/share/oops/typeArrayKlass.cpp??? Fri Oct 05 >> 17:17:17 2018 +0200 >> @@ -99,14 +99,10 @@ >> ? ? typeArrayOop TypeArrayKlass::allocate_common(int length, bool >> do_zero, TRAPS) { >> ??? assert(log2_element_size() >= 0, "bad scale"); >> -? if (length >= 0) { >> -??? check_array_allocation_length(length, max_length(), CHECK_NULL); >> -??? size_t size = typeArrayOopDesc::object_size(layout_helper(), >> length); >> -??? return (typeArrayOop)Universe::heap()->array_allocate(this, >> (int)size, length, >> - do_zero, CHECK_NULL); >> -? } else { >> - THROW_MSG_0(vmSymbols::java_lang_NegativeArraySizeException(), >> err_msg("%d", length)); >> -? } >> +? check_array_allocation_length(length, max_length(), CHECK_NULL); >> +? size_t size = typeArrayOopDesc::object_size(layout_helper(), length); >> +? return (typeArrayOop)Universe::heap()->array_allocate(this, >> (int)size, length, >> + do_zero, CHECK_NULL); >> ? } >> ? ? oop TypeArrayKlass::multi_allocate(int rank, jint* last_size, >> TRAPS) { >> ----------------------------------------------------------------------------------------------- >> >> >> Please confirm review these new changes: >> >> http://cr.openjdk.java.net/~dnsimon/8208686v2 >> >> -Doug >> >> >>> On 4 Oct 2018, at 00:20, Doug Simon wrote: >>> >>> Thanks for the reviews Serguei and Vladimir. >>> >>> Unless I hear objections in the next 24 hours, I'll push this webrev. >>> >>> -Doug >>> >>>> On 3 Oct 2018, at 03:14, serguei.spitsyn at oracle.com wrote: >>>> >>>> Hi Doug, >>>> >>>> The JVMTI related part looks good to me. >>>> Thank you for fixing it! >>>> >>>> Thanks, >>>> Serguei >>>> >>>> On 10/2/18 1:11 AM, Doug Simon wrote: >>>>> It would be great to get some input from the non-compilers teams >>>>> on this RFR. >>>>> >>>>> -Doug >>>>> >>>>>> On 28 Sep 2018, at 19:51, Vladimir Kozlov >>>>>> wrote: >>>>>> >>>>>> To let you know, me and Tom R. did review these changes and >>>>>> agreed that it is the least intrusive changes for Hotspot shared >>>>>> code. >>>>>> >>>>>> Thanks, >>>>>> Vladimir >>>>>> >>>>>> On 9/25/18 8:11 AM, Daniel D. Daugherty wrote: >>>>>>> Adding serviceability-dev at ... since this is JVM/TI... >>>>>>> Dan >>>>>>> On 9/25/18 10:48 AM, Doug Simon wrote: >>>>>>>> A major design point of Graal is to treat allocations as >>>>>>>> non-side effecting to give more freedom to the optimizer by >>>>>>>> reducing the number of distinct FrameStates that need to be >>>>>>>> managed. When failing an allocation, Graal will deoptimize to >>>>>>>> the last side effecting instruction before the allocation. This >>>>>>>> mean the VM code for heap allocation will potentially be >>>>>>>> executed twice, once from Graal compiled code and then again in >>>>>>>> the interpreter. While this is perfectly fine according to the >>>>>>>> JVM specification, it can cause confusing behavior for JVMTI >>>>>>>> based tools. They will receive 2 ResourceExhausted events for a >>>>>>>> single allocation. Furthermore, the first ResourceExhausted >>>>>>>> event (on the Graal allocation slow path) might denote a >>>>>>>> bytecode instruction that performs no allocation, making it >>>>>>>> hard to debug the memory failure. >>>>>>>> >>>>>>>> The proposed solution is to add an extra set of JVMCI VM >>>>>>>> runtime calls for allocation. These entry points will attempt >>>>>>>> the allocation and upon failure, >>>>>>>> skip side-effects such as posting JVMTI events or handling >>>>>>>> -XX:OnOutOfMemoryError. The compiled code using these entry >>>>>>>> points is expected deoptmize on null. >>>>>>>> >>>>>>>> The path from these new entry points to where allocation can >>>>>>>> fail goes through quite a bit of VM code. One could modify all >>>>>>>> these paths by: >>>>>>>> * Returning null instead of throwing an exception on failure. >>>>>>>> * Adding a `bool null_on_fail` argument to all relevant methods. >>>>>>>> * Adding extra null checking where necessary after each call to >>>>>>>> these methods when `null_on_fail == true`. >>>>>>>> This represents a significant number of changes. >>>>>>>> >>>>>>>> Instead, the proposed solution introduces a new >>>>>>>> _in_retryable_allocation thread-local. This way, only the entry >>>>>>>> points and allocation routines that raise an exception need to >>>>>>>> be modified. Failure is communicated back to the new entry >>>>>>>> points by throwing a special pre-allocated OOME object (i.e., >>>>>>>> Universe::out_of_memory_error_retry()) which must not propagate >>>>>>>> back to Java code. Use of this object is not strictly >>>>>>>> necessary; it is introduced to highlight/document the special >>>>>>>> allocation mode. >>>>>>>> >>>>>>>> The proposed solution is at >>>>>>>> http://cr.openjdk.java.net/~dnsimon/8208686. >>>>>>>> THE JBS bug is: https://bugs.openjdk.java.net/browse/JDK-8208686 >>>>>>>> >>>>>>>> -Doug >>>> >>> >> From kim.barrett at oracle.com Fri Oct 5 20:40:32 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 5 Oct 2018 16:40:32 -0400 Subject: RFR: 8211718: Supporting multiple concurrent OopStorage iterators In-Reply-To: <5BB73A68.9070501@oracle.com> References: <1F14027E-3B77-4847-942C-20665BF29FA6@oracle.com> <5015fefa-ea3f-d823-fb65-84dc51cd379e@oracle.com> <5BB73A68.9070501@oracle.com> Message-ID: <523FF4AA-C400-4669-B633-EFF86983AC19@oracle.com> > On Oct 5, 2018, at 6:18 AM, Erik ?sterlund wrote: > > Hi Per and Kim, > > Thank you both for the reviews. The renaming was applied as suggested (including comments). > > Incremental: > http://cr.openjdk.java.net/~eosterlund/8211718/webrev.00_01/ > > Full: > http://cr.openjdk.java.net/~eosterlund/8211718/webrev.01/ > > Thanks, > /Erik Looks good. From erik.osterlund at oracle.com Fri Oct 5 21:17:39 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Fri, 5 Oct 2018 23:17:39 +0200 Subject: RFR: 8211718: Supporting multiple concurrent OopStorage iterators In-Reply-To: <523FF4AA-C400-4669-B633-EFF86983AC19@oracle.com> References: <1F14027E-3B77-4847-942C-20665BF29FA6@oracle.com> <5015fefa-ea3f-d823-fb65-84dc51cd379e@oracle.com> <5BB73A68.9070501@oracle.com> <523FF4AA-C400-4669-B633-EFF86983AC19@oracle.com> Message-ID: Hi Kim, Thanks for the review. /Erik On 5 Oct 2018, at 22:40, Kim Barrett wrote: >> On Oct 5, 2018, at 6:18 AM, Erik ?sterlund wrote: >> >> Hi Per and Kim, >> >> Thank you both for the reviews. The renaming was applied as suggested (including comments). >> >> Incremental: >> http://cr.openjdk.java.net/~eosterlund/8211718/webrev.00_01/ >> >> Full: >> http://cr.openjdk.java.net/~eosterlund/8211718/webrev.01/ >> >> Thanks, >> /Erik > > Looks good. > From stefan.johansson at oracle.com Mon Oct 8 12:58:53 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 8 Oct 2018 14:58:53 +0200 Subject: RFR: 8201436: Replace oop_ps_push_contents with oop_iterate and closure In-Reply-To: <7b9c2549-4979-8a91-b29b-143d2865d733@oracle.com> References: <7b9c2549-4979-8a91-b29b-143d2865d733@oracle.com> Message-ID: Hi Leo, Thanks for taking on this task. On 2018-10-04 18:20, Leo Korinth wrote: > Hi! > > This is the first enhancement in a series of three to remove Parallel GC > specific behaviour from shared code. I will post all three enhancements > to the mailing list. > > JDK-8201436 Replace oop_ps_push_contents with oop_iterate and closure > ??????????? https://bugs.openjdk.java.net/browse/JDK-8201436 > JDK-8211446 Replace oop_pc_follow_contents with oop_iterate and closure > ??????????? https://bugs.openjdk.java.net/browse/JDK-8211446 > JDK-8211447 Replace oop_pc_update_pointers with oop_iterate and closure > ??????????? https://bugs.openjdk.java.net/browse/JDK-8211447 > > The webrevs are to be applied in the same order as in the above list > although logically they are quite independent of each other. > > The aim is to remove Parallel GC specific behaviour by removing > oop_ps_push_contents, oop_pc_follow_contents and oop_pc_update_pointers > from the *Klass hierarchy in oops/*Klass.hpp. Instead of using the old > dynamic dispatch on the Klass object, the generic oop_iterate* (used by > all other GCs) is used instead. > > Improvements include: > - removing GC specific implementation from shared code. Better > separation, less #ifdefs > - reusing the same iterators everywhere, using the same mechanism to > iterate in the different GCs > - reduction in code > > Performance: > Most of the benchmarks looks good, they show about the same performance > before and after the changes. I have had serious problems with the Derby > benchmark that did not like the way the generic reference handling was > visiting objects. Changing the order in oop_oop_iterate_reverse > introduced regressions in G1. Therefore I have added new > oop_oop_iterate_reverse specialisations to speed up Parallel (reduce > regression) without harming G1. > > Ideas of things to do after this commit series: > - minor improvements in oop_oop_iterate_reverse (removing return of size > that is not used anywhere) > - remove dynamic dispatch call to check if klass is of typeArray "kind" > (!obj->klass()->is_typeArray_klass()). The empty action on type arrays > is already handled and this extra check seems mostly confusing and > possibly a small unnecessary cost. > - possibly move creation of push closure out of push_contents, and by > that win some performance (the new closure now has to set a reference > processor) > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8201436 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/8201436/00/ The change looks good. Thanks, Stefan > > Testing: > - all patches build on linux. > - mach5 (linux, windows and mac) tier1,tier2,tier3,tier4,tier5 has > passed with all three patches applied > - aurora tests have been run and seem to indicate no change in general, > and only a small regression in Derby. The last (Derby) run showed a > young collection regression of 1.02% on Linux and 2.76% improvement on > windows, but including experience from earlier runs, my estimate is that > it is in reality a bit worse and that we have a _slight_ but acceptable > regression in Derby. > > Thanks, Leo From stefan.johansson at oracle.com Mon Oct 8 14:02:24 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 8 Oct 2018 16:02:24 +0200 Subject: RFR: 8211446: Replace oop_pc_follow_contents with oop_iterate and closure In-Reply-To: References: Message-ID: <2c61fa2d-2c08-bf98-08c4-6be34f3c6bca@oracle.com> Hi Leo, On 2018-10-04 18:27, Leo Korinth wrote: > Hi! > > This is the is the second enhancement in a series of three to remove > Parallel GC specific behaviour from shared code. I will post all three > enhancements to the mailing list. > > For a description on what is changed see my earlier "RFR: 8201436: > Replace oop_ps_push_contents with oop_iterate and closure" > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8211446 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/8211446/00/ The change looks good in general but the closures here should not be prefixed with PS but rather PC, since they are used by the parallel compact algorithm. Some additional comments: src/hotspot/share/gc/parallel/psCompactionManager.inline.hpp The new versions of follow_class_loader and follow_contents have been moved to the end of the file, is there any reason for that otherwise I would like them to be in the old location right with the other follow_*-functions. Also the indentation of the closures should be fixed, use 2 space instead of 4. --- src/hotspot/share/gc/parallel/psParallelCompact.cpp PSReferenceProcessor should be called PCReferenceProcessor and I also think you should remove the default value in the constructor. --- Thanks, Stefan > > Testing: > - all patches build on linux after the latest rebase. > - mach5 (linux, windows and mac) tier1,tier2,tier3,tier4,tier5 has > passed with all three patches applied > - aurora tests have been run and seem to indicate no change in general > on this webrev in the patch series. > > Thanks, Leo From thomas.schatzl at oracle.com Mon Oct 8 14:06:13 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 08 Oct 2018 16:06:13 +0200 Subject: RFR: bug: Timely Reducing Unused Committed Memory In-Reply-To: References: <8af5c2468f5133bc1ba2517bf52c3078af516977.camel@oracle.com> <65063e5f31702649e12b485b570ecab9ebd32d01.camel@oracle.com> <66c536399b2f218af0d85bd3bced44f204e3dbf4.camel@oracle.com> <2821cf70-c4d5-016f-b976-5ecc850f377c@oracle.com> <57b10708-b870-cb98-0df0-fab650f431e4@oracle.com> <3927596f-8847-ba95-d4c6-3c5f3656330c@oracle.com> <9bbf4a13-8490-d918-3cda-5c34acd16fa6@oracle.com> Message-ID: Hi all, here's the promised patch based on latest discussions: http://cr.openjdk.java.net/~tschatzl/jelastic/pgc-webrev.3/webrev/ It is based on "6490394: G1: Allow heap shrinking / memory unmapping after reclaiming regions during Remark", that is out for review now. As for the options, they were named as follows: G1PeriodicGCInterval: Number of milliseconds after a previous GC to wait before triggering a periodic gc. A value of zero disables periodically enforced gc cycles. (Note that I just found out when copy&pasting this description from the g1_globals file that the text I there needs fixing - I will do that asap :)) G1PeriodicGCInvokesConcurrent: Determines the kind of periodic GC. Set to true to have G1 perform a concurrent GC as periodic GC, otherwise use a STW Full GC. G1PeriodicGCSystemLoadThreshold: Maximum recent system wide system load at which G1 triggers a periodic GC. A load above this value cancels a given periodic GCs. A value of zero disables this check. The policy has been implemented so that any reason to not start a periodic collection right now will cancel the current attempt - another attempt will be made after G1PeriodicGCInterval ms. This is different to suppress the periodic gc until the next time the conditions are met. >From a default values POV, periodic gcs will be on by default with an interval of 5mins. This should be fairly non-intrusive for all but exceptional cases. Please check whether all this still fits your use case :) Thanks, Thomas On Thu, 2018-10-04 at 15:27 +0300, Ruslan Synytsky wrote: > > > On Thu, 4 Oct 2018 at 15:06, Rodrigo Bruno > wrote: > > Hi Stefan, > > > > > > Stefan Johansson escreveu no dia > > quarta, 3/10/2018 ?(s) 21:18: > > > We discussed this a bit further today and I'll try to summarize > > > what we currently think is the way forward and what a first > > > version should include: > > > * Periodic concurrent cycles (which after JDK-6490394 will > > > uncommit memory) are on by default, but will only trigger if no > > > other GC has occurred during the interval. The length of the > > > interval should be controlled by a flag and if set to -1, the > > > periodic GCs will be turned off. The name should probably be > > > something other than GCFrequency, like PeriodicGCInterval or so. > > > This will be similar to what currently is called > > > CMSTriggerInterval for CMS. > > > * You can also control the periodic GCs by specifying a system > > > average load threshold, which the current avg load must be under > > > for a periodic GC to occur. This value will not be consider by > > > default, and I think we should try to come up with at more > > > describing name than MaxLoadGC. > > > * The code checking if a periodic GC should trigger will be added > > > to the already existing G1YoungRemSetSamplingThread. This thread > > > currently is pretty idle and there are ideas around removing it, > > > but if that is done we can move the idle-checking code then. > > > * If you want the periodic GC to be a full collection, there will > > > be a flag to control this. > > > > > > Those were the key parts in our discussion, and I think Thomas > > > plans to update his webrev in a few days or so. Does this still > > > sound like a good starting point? > > > > > > > Sounds great for me! > > > > No objections on my side too. > Regards > > cheers, > > rodrigo > > > > > And also, other users having opinions are very welcome :) > > > > > > Thanks, > > > Stefan > > > > > > > > -- > > > > Ruslan > > > > CEO @ Jelastic > > From stefan.johansson at oracle.com Mon Oct 8 14:07:15 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 8 Oct 2018 16:07:15 +0200 Subject: RFR: 8211447: Replace oop_pc_update_pointers with oop_iterate and closure In-Reply-To: <401fe6c9-dd64-5c50-4faa-2f51925546a7@oracle.com> References: <401fe6c9-dd64-5c50-4faa-2f51925546a7@oracle.com> Message-ID: <8844b37f-4398-3ad9-a094-7092fd79e398@oracle.com> On 2018-10-04 18:31, Leo Korinth wrote: > Hi! > > This is the is the third enhancement in a series of three to remove > Parallel GC specific behaviour from shared code. I will post all three > enhancements to the mailing list. > > For a description on what is changed see my earlier "RFR: 8201436: > Replace oop_ps_push_contents with oop_iterate and closure" > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8211447 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/8211447/00/ The change looks good, but like I said for the last patch. Please use the prefix PC instead of PS for the closure. Thanks, Stefan > > Testing: > - all patches build on linux after the latest rebase. > - mach5 (linux, windows and mac) tier1,tier2,tier3,tier4,tier5 has > passed with all three patches applied > - aurora tests have been run and seem to indicate no change in general > on this webrev in the patch series. > > Thanks, Leo From thomas.schatzl at oracle.com Mon Oct 8 14:58:22 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 08 Oct 2018 16:58:22 +0200 Subject: RFR (XS): 8211853: Avoid additional duplicate work when a reference in the task queue has already been evacuated Message-ID: Hi all, can I have reviews for this tiny change that removes G1 doing some duplicate work? Due to benign races we can sometimes get references that have already been evacuated from the task queue. Current code in G1ParScanThreadState::do_oop_evac() needlessly applies follow-up calculations (remembered set, humonguous liveness) on this already evacuated reference - that all has been done the first time around already. Just exit the method if we detect such a task. CR: https://bugs.openjdk.java.net/browse/JDK-8211853 Webrev: http://cr.openjdk.java.net/~tschatzl/8211853/webrev/ Testing: local jtreg run of gc passes, some benchmarks Thanks, Thomas From shade at redhat.com Mon Oct 8 16:29:04 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 8 Oct 2018 18:29:04 +0200 Subject: RFR: JDK-8211279: Verify missing object equals barriers In-Reply-To: References: <935559b9-6df2-e337-97b5-5807cf57a154@redhat.com> Message-ID: On 10/03/2018 03:25 PM, Roman Kennke wrote: > This addresses all the issues mentioned above: > http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.02/ I think this is a cute way to get things going. *) I always get a bit uneasy seeing the casts to (void*) like here: 413 static bool equals(oop o1, oop o2) { return (void*)o1 == (void*)o2; } ...but I think this comparison is well-defined in C++ with the semantics we want. *) So, if CHECK_UNHANDLED_OOPS is not defined, this declaration is inaccessible, and asserts that use it would fail to compile? I think this ifdef is not needed here, but rather it should be around the asserts in oop::operator== and !=? 133 #ifdef CHECK_UNHANDLED_OOPS 134 virtual bool oop_equals_operator_allowed() { return true; } 135 #endif 136 ... 58 bool oop::operator==(const oop o) const { 59 assert(BarrierSet::barrier_set()->oop_equals_operator_allowed(), "Not allowed"); 60 return obj() == o.obj(); 61 } Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Mon Oct 8 16:32:25 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 8 Oct 2018 18:32:25 +0200 Subject: RFR: JDK-8211279: Verify missing object equals barriers In-Reply-To: References: <935559b9-6df2-e337-97b5-5807cf57a154@redhat.com> Message-ID: <04ad0da1-3e16-bd73-bc47-f3b62c52c91d@redhat.com> Hi Aleksey, Thanks for reviewing! > On 10/03/2018 03:25 PM, Roman Kennke wrote: >> This addresses all the issues mentioned above: >> http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.02/ > > I think this is a cute way to get things going. > > *) I always get a bit uneasy seeing the casts to (void*) like here: > > 413 static bool equals(oop o1, oop o2) { return (void*)o1 == (void*)o2; } > > ...but I think this comparison is well-defined in C++ with the semantics we want. What would you suggest? reinterpret_cast(..) better? > *) So, if CHECK_UNHANDLED_OOPS is not defined, this declaration is inaccessible, and asserts that > use it would fail to compile? I think this ifdef is not needed here, but rather it should be around > the asserts in oop::operator== and !=? > > 133 #ifdef CHECK_UNHANDLED_OOPS > 134 virtual bool oop_equals_operator_allowed() { return true; } > 135 #endif > 136 > > ... > > 58 bool oop::operator==(const oop o) const { > 59 assert(BarrierSet::barrier_set()->oop_equals_operator_allowed(), "Not allowed"); > 60 return obj() == o.obj(); > 61 } It already *is* around the whole (oop class) declaration in oopsHierarchy.hpp, that's the only user of the check so I put this under #ifdef CHECK_UNHANDLED_OOPS too, but I can remove this #ifdef if you prefer. Let me know? Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Mon Oct 8 16:34:46 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 8 Oct 2018 18:34:46 +0200 Subject: RFR: JDK-8211279: Verify missing object equals barriers In-Reply-To: <04ad0da1-3e16-bd73-bc47-f3b62c52c91d@redhat.com> References: <935559b9-6df2-e337-97b5-5807cf57a154@redhat.com> <04ad0da1-3e16-bd73-bc47-f3b62c52c91d@redhat.com> Message-ID: <3826dcb4-1a99-7ecb-ed6f-dd80f86ca1ca@redhat.com> On 10/08/2018 06:32 PM, Roman Kennke wrote: >> *) I always get a bit uneasy seeing the casts to (void*) like here: >> >> 413 static bool equals(oop o1, oop o2) { return (void*)o1 == (void*)o2; } >> >> ...but I think this comparison is well-defined in C++ with the semantics we want. > > What would you suggest? reinterpret_cast(..) better? No, from my reading of C++ spec casting to void* is safe here. Just thinking out loud. >> *) So, if CHECK_UNHANDLED_OOPS is not defined, this declaration is inaccessible, and asserts that >> use it would fail to compile? I think this ifdef is not needed here, but rather it should be around >> the asserts in oop::operator== and !=? >> >> 133 #ifdef CHECK_UNHANDLED_OOPS >> 134 virtual bool oop_equals_operator_allowed() { return true; } >> 135 #endif >> 136 > > It already *is* around the whole (oop class) declaration in > oopsHierarchy.hpp, that's the only user of the check so I put this under > #ifdef CHECK_UNHANDLED_OOPS too, but I can remove this #ifdef if you > prefer. Let me know? Ah! Missed that. No, this is fine then. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Mon Oct 8 16:46:13 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 8 Oct 2018 18:46:13 +0200 Subject: RFR: JDK-8211270: GC abstraction to get real object and headers size In-Reply-To: <5689e478-6f2f-d7ea-127e-689108364074@redhat.com> References: <1b75ccd7-ba1a-a183-f338-18d3c651827e@redhat.com> <5689e478-6f2f-d7ea-127e-689108364074@redhat.com> Message-ID: <3e3fbeff-679e-6934-04b8-24b44ca42dac@redhat.com> On 10/05/2018 06:20 PM, Roman Kennke wrote: >> Webrev: >> http://cr.openjdk.java.net/~rkennke/JDK-8211270/webrev.00/ Do we really need the abstractions for header sizes? I think all usages need the object size, and header sizes can be just picked up from the usual places. No GC, including Shenandoah is changing the header size? Trying to understand the use case, looking at Shenandoah's diff here: https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/gc/shared/plab.cpp.sdiff.html ...it seems what we really want is to mix in oop_extra_words into plab.cpp accessors. Because doing ShenandoahHeap::obj_header_size() { return obj_header_size + oop_extra_words; } seems wrong down the line -- it would _not_ be the header size. Cheers, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From synytskyy at jelastic.com Mon Oct 8 19:09:41 2018 From: synytskyy at jelastic.com (Ruslan Synytsky) Date: Mon, 8 Oct 2018 22:09:41 +0300 Subject: RFR: bug: Timely Reducing Unused Committed Memory In-Reply-To: References: <8af5c2468f5133bc1ba2517bf52c3078af516977.camel@oracle.com> <65063e5f31702649e12b485b570ecab9ebd32d01.camel@oracle.com> <66c536399b2f218af0d85bd3bced44f204e3dbf4.camel@oracle.com> <2821cf70-c4d5-016f-b976-5ecc850f377c@oracle.com> <57b10708-b870-cb98-0df0-fab650f431e4@oracle.com> <3927596f-8847-ba95-d4c6-3c5f3656330c@oracle.com> <9bbf4a13-8490-d918-3cda-5c34acd16fa6@oracle.com> Message-ID: Hi Thomas, thank you for the patch. On Mon, 8 Oct 2018 at 17:06, Thomas Schatzl wrote: > Hi all, > > here's the promised patch based on latest discussions: > > http://cr.openjdk.java.net/~tschatzl/jelastic/pgc-webrev.3/webrev/ > > It is based on "6490394: G1: Allow heap shrinking / memory unmapping > after reclaiming regions during Remark", that is out for review now. > > As for the options, they were named as follows: > > G1PeriodicGCInterval: Number of milliseconds after a previous GC to > wait before triggering a periodic gc. A value of zero disables > periodically enforced gc cycles. > (Note that I just found out when copy&pasting this description from the > g1_globals file that the text I there needs fixing - I will do that > asap :)) > > G1PeriodicGCInvokesConcurrent: Determines the kind of periodic GC. Set > to true to have G1 perform a concurrent GC as periodic GC, otherwise > use a STW Full GC. > Is concurrent enabled by default? If not, then why not? > > > G1PeriodicGCSystemLoadThreshold: Maximum recent system wide system load > at which G1 triggers a periodic GC. A load above this value cancels a > given periodic GCs. A value of zero disables this check. > Maybe a clear mention of loadavg in the description can help to avoid potential confusions around system load definition. > The policy has been implemented so that any reason to not start a > periodic collection right now will cancel the current attempt - another > attempt will be made after G1PeriodicGCInterval ms. This is different > to suppress the periodic gc until the next time the conditions are met. > > From a default values POV, periodic gcs will be on by default with an > interval of 5mins. This should be fairly non-intrusive for all but > exceptional cases. > > Please check whether all this still fits your use case :) > Looks good for me. Thanks again. Btw, we have got an approved session at OracleCode about JVM elasticity https://oracle.rainfocus.com/widget/oracle/oow18/catalogcodeone18?search=elastic%2Bjvm. I'm planning to bring up topics about general benefits, available options with different GCs and JVMs, specifics of running elastic JVM in containers, elaboration of possible further improvements. Please join the session for a live talk. Regards > > Thanks, > Thomas > > On Thu, 2018-10-04 at 15:27 +0300, Ruslan Synytsky wrote: > > > > > > On Thu, 4 Oct 2018 at 15:06, Rodrigo Bruno > > wrote: > > > Hi Stefan, > > > > > > > > > Stefan Johansson escreveu no dia > > > quarta, 3/10/2018 ?(s) 21:18: > > > > We discussed this a bit further today and I'll try to summarize > > > > what we currently think is the way forward and what a first > > > > version should include: > > > > * Periodic concurrent cycles (which after JDK-6490394 will > > > > uncommit memory) are on by default, but will only trigger if no > > > > other GC has occurred during the interval. The length of the > > > > interval should be controlled by a flag and if set to -1, the > > > > periodic GCs will be turned off. The name should probably be > > > > something other than GCFrequency, like PeriodicGCInterval or so. > > > > This will be similar to what currently is called > > > > CMSTriggerInterval for CMS. > > > > * You can also control the periodic GCs by specifying a system > > > > average load threshold, which the current avg load must be under > > > > for a periodic GC to occur. This value will not be consider by > > > > default, and I think we should try to come up with at more > > > > describing name than MaxLoadGC. > > > > * The code checking if a periodic GC should trigger will be added > > > > to the already existing G1YoungRemSetSamplingThread. This thread > > > > currently is pretty idle and there are ideas around removing it, > > > > but if that is done we can move the idle-checking code then. > > > > * If you want the periodic GC to be a full collection, there will > > > > be a flag to control this. > > > > > > > > Those were the key parts in our discussion, and I think Thomas > > > > plans to update his webrev in a few days or so. Does this still > > > > sound like a good starting point? > > > > > > > > > > Sounds great for me! > > > > > > > No objections on my side too. > > Regards > > > cheers, > > > rodrigo > > > > > > > And also, other users having opinions are very welcome :) > > > > > > > > Thanks, > > > > Stefan > > > > > > > > > > -- > > > > > Ruslan > > > > > CEO @ Jelastic > > > > > > > -- Ruslan CEO @ Jelastic -------------- next part -------------- An HTML attachment was scrubbed... URL: From rkennke at redhat.com Mon Oct 8 20:13:46 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 8 Oct 2018 22:13:46 +0200 Subject: RFR: JDK-8211279: Verify missing object equals barriers In-Reply-To: <3826dcb4-1a99-7ecb-ed6f-dd80f86ca1ca@redhat.com> References: <935559b9-6df2-e337-97b5-5807cf57a154@redhat.com> <04ad0da1-3e16-bd73-bc47-f3b62c52c91d@redhat.com> <3826dcb4-1a99-7ecb-ed6f-dd80f86ca1ca@redhat.com> Message-ID: <1f7a6191-cb83-5326-d959-495651ad656f@redhat.com> Thanks, Aleksey, for the review. I've sent the patch through jdk/submit, and it came back PASSED. Per, Erik, everybody else, what do you think about the last changeset? Can you live with it? Thanks, Roman > On 10/08/2018 06:32 PM, Roman Kennke wrote: >>> *) I always get a bit uneasy seeing the casts to (void*) like here: >>> >>> 413 static bool equals(oop o1, oop o2) { return (void*)o1 == (void*)o2; } >>> >>> ...but I think this comparison is well-defined in C++ with the semantics we want. >> >> What would you suggest? reinterpret_cast(..) better? > > No, from my reading of C++ spec casting to void* is safe here. Just thinking out loud. > >>> *) So, if CHECK_UNHANDLED_OOPS is not defined, this declaration is inaccessible, and asserts that >>> use it would fail to compile? I think this ifdef is not needed here, but rather it should be around >>> the asserts in oop::operator== and !=? >>> >>> 133 #ifdef CHECK_UNHANDLED_OOPS >>> 134 virtual bool oop_equals_operator_allowed() { return true; } >>> 135 #endif >>> 136 >> >> It already *is* around the whole (oop class) declaration in >> oopsHierarchy.hpp, that's the only user of the check so I put this under >> #ifdef CHECK_UNHANDLED_OOPS too, but I can remove this #ifdef if you >> prefer. Let me know? > > Ah! Missed that. No, this is fine then. > > -Aleksey > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From hohensee at amazon.com Mon Oct 8 23:49:33 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 8 Oct 2018 23:49:33 +0000 Subject: RFR: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions Message-ID: <8C241BB9-2089-491B-A27E-57C4F43FBA9E@amazon.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8196989 CSR: https://bugs.openjdk.java.net/browse/JDK-8196991 Webrev: http://cr.openjdk.java.net/~phh/8196989/webrev.02/ As requested, I split the jstat counter update off from the MXBean update. This is the MXBean update. The jstat counter RFE is https://bugs.openjdk.java.net/browse/JDK-8210965 and its CSR is https://bugs.openjdk.java.net/browse/JDK-8210966. The MXBean CSR is in draft state, I?d greatly appreciate review and sign-off. It?s been suggested that we add another pool to represent the free region set, but doing so would be incompatible with existing MXBean use invariants for all GCs. These are: 1. The sum of the pools? MemoryUsage.max properties is the total reserved heap size. 2. The sum of the pools? MemoryUsage.committed properties is the total committed size. 3. The sum of the pools? MemoryUsage.used properties is the total size of the memory containing objects, live and dead-and-yet-to-be-collected, as the case might be, plus intentional gaps between them. 4. The total free space is (sum of the max properties ? sum of the used properties). 5. The total uncommitted space is (sum of the max properties ? sum of the committed properties). 6. The total committed free space is (2) ? (3). To keep invariants 1, 2 and 3, the free region pool?s ?max? property should be ?undefined? (i.e., -1). The intuitive, to me, ?used? property value would be the total free space, but that would violate invariant 4 above. Defining the ?committed? property as the total committed free space would violate invariants 2 and 6. The patch passes the submit repo, hotspot tier1, and, separately, the serviceability, jfr, and gc jtreg tests. I?m uncertain how to construct a test that checks for valid MXBean content: the existing tests don?t. Any such test will be fragile due to possible future Hotspot changes that affect the values, and to run-to-run variability. I?ve done by-hand comparisons between the old and new MXBean content using the SwingSet2 demo, including using App CDS, and the numbers look reasonable. The guts of the change are in G1MonitoringSupport::recalculate_sizes, initialize_serviceability, memory_managers, memory_pools, and G1MonitoringScope. I also defined TraceConcMemoryManagerStats to track the concurrent cycle in a way analogous to TraceCMSMemoryManagerStats. The changes to the includes in g1FullGCOopClosures.inline.hpp and g1HeapVerifier.cpp are to satisfy compiler complaints. I changed the 3rd argument to the G1MonitoringScope constructor to a mixed_gc flag, and use accessor methods instead of direct field accesses when accessor methods exist. I believe I?ve minimized the latter. I updated the copyright date to 2018 in memoryService.hpp because I neglected to do so in my previous G1 MXBean patch. Thanks, Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Tue Oct 9 08:57:03 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 09 Oct 2018 10:57:03 +0200 Subject: RFR: 8201436: Replace oop_ps_push_contents with oop_iterate and closure In-Reply-To: <7b9c2549-4979-8a91-b29b-143d2865d733@oracle.com> References: <7b9c2549-4979-8a91-b29b-143d2865d733@oracle.com> Message-ID: Hi Leo, On Thu, 2018-10-04 at 18:20 +0200, Leo Korinth wrote: > Hi! > > This is the first enhancement in a series of three to remove Parallel > GC specific behaviour from shared code. I will post all three > enhancements to the mailing list. Thanks for doing this cleanup. > > JDK-8201436 Replace oop_ps_push_contents with oop_iterate and closure > https://bugs.openjdk.java.net/browse/JDK-8201436 > JDK-8211446 Replace oop_pc_follow_contents with oop_iterate and > closure > https://bugs.openjdk.java.net/browse/JDK-8211446 > JDK-8211447 Replace oop_pc_update_pointers with oop_iterate and > closure > https://bugs.openjdk.java.net/browse/JDK-8211447 > > The webrevs are to be applied in the same order as in the above list > although logically they are quite independent of each other. Looks good. Thomas From thomas.schatzl at oracle.com Tue Oct 9 09:00:30 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 09 Oct 2018 11:00:30 +0200 Subject: RFR: 8211446: Replace oop_pc_follow_contents with oop_iterate and closure In-Reply-To: References: Message-ID: Hi, On Thu, 2018-10-04 at 18:27 +0200, Leo Korinth wrote: > Hi! > > This is the is the second enhancement in a series of three to remove > Parallel GC specific behaviour from shared code. I will post all > three enhancements to the mailing list. > > For a description on what is changed see my earlier "RFR: 8201436: > Replace oop_ps_push_contents with oop_iterate and closure" > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8211446 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/8211446/00/ Some minor issues: - psCompactionManager.inline.hpp:96: remove two of the newlines - psCompactionManager.inline.hpp: please fix indentation of PSMarkAndPushClosure and PSIterateMarkAndPushClosure class declarations. - the "pc" vs. "ps" prefixes Stefan already mentioned. Thanks, Thomas From thomas.schatzl at oracle.com Tue Oct 9 09:06:34 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 09 Oct 2018 11:06:34 +0200 Subject: RFR: 8211447: Replace oop_pc_update_pointers with oop_iterate and closure In-Reply-To: <401fe6c9-dd64-5c50-4faa-2f51925546a7@oracle.com> References: <401fe6c9-dd64-5c50-4faa-2f51925546a7@oracle.com> Message-ID: <8804f31ec81f5e27457c560ef908d78799d3ec6d.camel@oracle.com> Hi, On Thu, 2018-10-04 at 18:31 +0200, Leo Korinth wrote: > Hi! > > This is the is the third enhancement in a series of three to remove > Parallel GC specific behaviour from shared code. I will post all > three enhancements to the mailing list. > > For a description on what is changed see my earlier "RFR: 8201436: > Replace oop_ps_push_contents with oop_iterate and closure" > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8211447 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/8211447/00/ - psClosure.inline.hpp:103: superfluous "protected" keyword (probably pre-existing) - I *think* psClosure.inline.hpp should be called "pcClosure.inline.hpp" - i.e. please fix the already mentioned "ps" vs. "pc" naming issue. :) - maybe fix the spacing of the "public" keyword in instanceClassLoaderKlass.hpp:55 (pre-existing issue); actually it can be removed. Thanks, Thomas From leo.korinth at oracle.com Tue Oct 9 12:12:16 2018 From: leo.korinth at oracle.com (Leo Korinth) Date: Tue, 9 Oct 2018 14:12:16 +0200 Subject: RFR: 8211447: Replace oop_pc_update_pointers with oop_iterate and closure In-Reply-To: <8804f31ec81f5e27457c560ef908d78799d3ec6d.camel@oracle.com> References: <401fe6c9-dd64-5c50-4faa-2f51925546a7@oracle.com> <8804f31ec81f5e27457c560ef908d78799d3ec6d.camel@oracle.com> Message-ID: Hi, and thanks for reviewing! On 09/10/2018 11:06, Thomas Schatzl wrote: > Hi, > > On Thu, 2018-10-04 at 18:31 +0200, Leo Korinth wrote: >> Hi! >> >> This is the is the third enhancement in a series of three to remove >> Parallel GC specific behaviour from shared code. I will post all >> three enhancements to the mailing list. >> >> For a description on what is changed see my earlier "RFR: 8201436: >> Replace oop_ps_push_contents with oop_iterate and closure" >> >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8211447 >> >> Webrev: >> http://cr.openjdk.java.net/~lkorinth/8211447/00/ > > - psClosure.inline.hpp:103: superfluous "protected" keyword (probably > pre-existing) > > - I *think* psClosure.inline.hpp should be called > "pcClosure.inline.hpp" - i.e. please fix the already mentioned "ps" vs. > "pc" naming issue. :) psClosure.inline.hpp is changed in the first patch for JDK-8201436 and not for this patch (actually extracted from psScavenge.inline.hpp to psClosure.inline.hpp without being changed to fix include logic, but that _is hard_ to see in my webrev). I have neither created nor renamed the closures, just moved them. Also, they seem to be used from the scavenge part if I read the code correctly, so I think they are correctly named to begin with. I guess you confused this change in my earlier patch (PS related) with this RFR right? I will try to incorporate all your other changes together with Stefan's improvements and create new webrevs! Thanks, Leo > > - maybe fix the spacing of the "public" keyword in > instanceClassLoaderKlass.hpp:55 (pre-existing issue); actually it can > be removed. > > Thanks, > Thomas > > From leo.korinth at oracle.com Tue Oct 9 13:34:25 2018 From: leo.korinth at oracle.com (Leo Korinth) Date: Tue, 9 Oct 2018 15:34:25 +0200 Subject: RFR: 8201436: Replace oop_ps_push_contents with oop_iterate and closure In-Reply-To: References: <7b9c2549-4979-8a91-b29b-143d2865d733@oracle.com> Message-ID: <32d69a42-53dc-2625-b497-5b03a39f1b4e@oracle.com> Thank you Stefan and Thomas for your reviews. /Leo On 09/10/2018 10:57, Thomas Schatzl wrote: > Hi Leo, > > On Thu, 2018-10-04 at 18:20 +0200, Leo Korinth wrote: >> Hi! >> >> This is the first enhancement in a series of three to remove Parallel >> GC specific behaviour from shared code. I will post all three >> enhancements to the mailing list. > > Thanks for doing this cleanup. > >> >> JDK-8201436 Replace oop_ps_push_contents with oop_iterate and closure >> https://bugs.openjdk.java.net/browse/JDK-8201436 >> JDK-8211446 Replace oop_pc_follow_contents with oop_iterate and >> closure >> https://bugs.openjdk.java.net/browse/JDK-8211446 >> JDK-8211447 Replace oop_pc_update_pointers with oop_iterate and >> closure >> https://bugs.openjdk.java.net/browse/JDK-8211447 >> >> The webrevs are to be applied in the same order as in the above list >> although logically they are quite independent of each other. > > Looks good. > > Thomas > From leo.korinth at oracle.com Tue Oct 9 13:52:19 2018 From: leo.korinth at oracle.com (Leo Korinth) Date: Tue, 9 Oct 2018 15:52:19 +0200 Subject: RFR: 8211446: Replace oop_pc_follow_contents with oop_iterate and closure In-Reply-To: References: Message-ID: <082ab861-5370-f7b6-f28c-0e60b7bae412@oracle.com> Hi! Here are new incremental and full webrevs that tries to improve the code according to suggestions from Stefan and Thomas. incremental: http://cr.openjdk.java.net/~lkorinth/8211446/00-01/ full: http://cr.openjdk.java.net/~lkorinth/8211446/01/ Thanks, Leo On 09/10/2018 11:00, Thomas Schatzl wrote: > Hi, > > On Thu, 2018-10-04 at 18:27 +0200, Leo Korinth wrote: >> Hi! >> >> This is the is the second enhancement in a series of three to remove >> Parallel GC specific behaviour from shared code. I will post all >> three enhancements to the mailing list. >> >> For a description on what is changed see my earlier "RFR: 8201436: >> Replace oop_ps_push_contents with oop_iterate and closure" >> >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8211446 >> >> Webrev: >> http://cr.openjdk.java.net/~lkorinth/8211446/00/ > > Some minor issues: > > - psCompactionManager.inline.hpp:96: remove two of the newlines > > - psCompactionManager.inline.hpp: please fix indentation of > PSMarkAndPushClosure and PSIterateMarkAndPushClosure class > declarations. > > - the "pc" vs. "ps" prefixes Stefan already mentioned. > > Thanks, > Thomas > From leo.korinth at oracle.com Tue Oct 9 14:26:04 2018 From: leo.korinth at oracle.com (Leo Korinth) Date: Tue, 9 Oct 2018 16:26:04 +0200 Subject: RFR: 8211447: Replace oop_pc_update_pointers with oop_iterate and closure In-Reply-To: References: <401fe6c9-dd64-5c50-4faa-2f51925546a7@oracle.com> <8804f31ec81f5e27457c560ef908d78799d3ec6d.camel@oracle.com> Message-ID: <474b84b5-4b6b-fa6b-bccf-2d8846bcb51c@oracle.com> Hi! Here are new incremental and full webrevs that tries to improve the code according to suggestions from Stefan and Thomas. I did *not* change the name to pcClosure.inline.hpp as I think that was a mistake (see my last e-mail). incremental: http://cr.openjdk.java.net/~lkorinth/8211447/00-01/ full: http://cr.openjdk.java.net/~lkorinth/8211447/01/ Thanks, Leo On 09/10/2018 14:12, Leo Korinth wrote: > Hi, and thanks for reviewing! > > On 09/10/2018 11:06, Thomas Schatzl wrote: >> Hi, >> >> On Thu, 2018-10-04 at 18:31 +0200, Leo Korinth wrote: >>> Hi! >>> >>> This is the is the third enhancement in a series of three to remove >>> Parallel GC specific behaviour from shared code. I will post all >>> three enhancements to the mailing list. >>> >>> For a description on what is changed see my earlier "RFR: 8201436: >>> Replace oop_ps_push_contents with oop_iterate and closure" >>> >>> Enhancement: >>> https://bugs.openjdk.java.net/browse/JDK-8211447 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~lkorinth/8211447/00/ >> >> - psClosure.inline.hpp:103: superfluous "protected" keyword (probably >> pre-existing) >> >> - I *think* psClosure.inline.hpp should be called >> "pcClosure.inline.hpp" - i.e. please fix the already mentioned "ps" vs. >> "pc" naming issue. :) > > psClosure.inline.hpp is changed in the first patch for JDK-8201436 and > not for this patch (actually extracted from psScavenge.inline.hpp to > psClosure.inline.hpp without being changed to fix include logic, but > that _is hard_ to see in my webrev). I have neither created nor renamed > the closures, just moved them. Also, they seem to be used from the > scavenge part if I read the code correctly, so I think they are > correctly named to begin with. I guess you confused this change in my > earlier patch (PS related) with this RFR right? > > I will try to incorporate all your other changes together with Stefan's > improvements and create new webrevs! > > Thanks, > Leo > >> >> - maybe fix the spacing of the "public" keyword in >> instanceClassLoaderKlass.hpp:55 (pre-existing issue); actually it can >> be removed. >> >> Thanks, >> ?? Thomas >> >> From rkennke at redhat.com Tue Oct 9 14:39:21 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 9 Oct 2018 16:39:21 +0200 Subject: RFR: JDK-8211270: GC abstraction to get real object and headers size In-Reply-To: <3e3fbeff-679e-6934-04b8-24b44ca42dac@redhat.com> References: <1b75ccd7-ba1a-a183-f338-18d3c651827e@redhat.com> <5689e478-6f2f-d7ea-127e-689108364074@redhat.com> <3e3fbeff-679e-6934-04b8-24b44ca42dac@redhat.com> Message-ID: Hey Aleksey, > On 10/05/2018 06:20 PM, Roman Kennke wrote: >>> Webrev: >>> http://cr.openjdk.java.net/~rkennke/JDK-8211270/webrev.00/ > > Do we really need the abstractions for header sizes? I think all usages need the object size, and > header sizes can be just picked up from the usual places. No GC, including Shenandoah is changing > the header size? > > Trying to understand the use case, looking at Shenandoah's diff here: > > https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/gc/shared/plab.cpp.sdiff.html > > ...it seems what we really want is to mix in oop_extra_words into plab.cpp accessors. Because doing > ShenandoahHeap::obj_header_size() { return obj_header_size + oop_extra_words; } seems wrong down the > line -- it would _not_ be the header size. It seems that the PLAB stuff does indeed require a different kind of abstraction, and most likely something that can be shared with, e.g., TLAB in the form of a virtual version of CollectedHeap::min_fill_size() or such. Let's take this part out of the patch: http://cr.openjdk.java.net/~rkennke/JDK-8211270/webrev.01/ Good now? Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Tue Oct 9 14:40:45 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 9 Oct 2018 16:40:45 +0200 Subject: RFR: JDK-8211270: GC abstraction to get real object and headers size In-Reply-To: References: <1b75ccd7-ba1a-a183-f338-18d3c651827e@redhat.com> <5689e478-6f2f-d7ea-127e-689108364074@redhat.com> <3e3fbeff-679e-6934-04b8-24b44ca42dac@redhat.com> Message-ID: <228274e5-3706-6719-5f08-cf0a9227254d@redhat.com> On 10/09/2018 04:39 PM, Roman Kennke wrote: > It seems that the PLAB stuff does indeed require a different kind of > abstraction, and most likely something that can be shared with, e.g., > TLAB in the form of a virtual version of CollectedHeap::min_fill_size() > or such. Let's take this part out of the patch: > > http://cr.openjdk.java.net/~rkennke/JDK-8211270/webrev.01/ Yeah, this looks good to me. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From stefan.johansson at oracle.com Tue Oct 9 14:55:21 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 9 Oct 2018 16:55:21 +0200 Subject: RFR: 8211447: Replace oop_pc_update_pointers with oop_iterate and closure In-Reply-To: <474b84b5-4b6b-fa6b-bccf-2d8846bcb51c@oracle.com> References: <401fe6c9-dd64-5c50-4faa-2f51925546a7@oracle.com> <8804f31ec81f5e27457c560ef908d78799d3ec6d.camel@oracle.com> <474b84b5-4b6b-fa6b-bccf-2d8846bcb51c@oracle.com> Message-ID: <211f1080-d7af-3e34-6400-c175b3c17030@oracle.com> Looks good, StefanJ On 2018-10-09 16:26, Leo Korinth wrote: > Hi! > > Here are new incremental and full webrevs that tries to improve the code > according to suggestions from Stefan and Thomas. > > I did *not* change the name to pcClosure.inline.hpp as I think that was > a mistake (see my last e-mail). > > incremental: > http://cr.openjdk.java.net/~lkorinth/8211447/00-01/ > full: > http://cr.openjdk.java.net/~lkorinth/8211447/01/ > > Thanks, > Leo > > On 09/10/2018 14:12, Leo Korinth wrote: >> Hi, and thanks for reviewing! >> >> On 09/10/2018 11:06, Thomas Schatzl wrote: >>> Hi, >>> >>> On Thu, 2018-10-04 at 18:31 +0200, Leo Korinth wrote: >>>> Hi! >>>> >>>> This is the is the third enhancement in a series of three to remove >>>> Parallel GC specific behaviour from shared code. I will post all >>>> three enhancements to the mailing list. >>>> >>>> For a description on what is changed see my earlier "RFR: 8201436: >>>> Replace oop_ps_push_contents with oop_iterate and closure" >>>> >>>> Enhancement: >>>> https://bugs.openjdk.java.net/browse/JDK-8211447 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~lkorinth/8211447/00/ >>> >>> - psClosure.inline.hpp:103: superfluous "protected" keyword (probably >>> pre-existing) >>> >>> - I *think* psClosure.inline.hpp should be called >>> "pcClosure.inline.hpp" - i.e. please fix the already mentioned "ps" vs. >>> "pc" naming issue. :) >> >> psClosure.inline.hpp is changed in the first patch for JDK-8201436 and >> not for this patch (actually extracted from psScavenge.inline.hpp to >> psClosure.inline.hpp without being changed to fix include logic, but >> that _is hard_ to see in my webrev). I have neither created nor >> renamed the closures, just moved them. Also, they seem to be used from >> the scavenge part if I read the code correctly, so I think they are >> correctly named to begin with. I guess you confused this change in my >> earlier patch (PS related) with this RFR right? >> >> I will try to incorporate all your other changes together with >> Stefan's improvements and create new webrevs! >> >> Thanks, >> Leo >> >>> >>> - maybe fix the spacing of the "public" keyword in >>> instanceClassLoaderKlass.hpp:55 (pre-existing issue); actually it can >>> be removed. >>> >>> Thanks, >>> ?? Thomas >>> >>> From stefan.johansson at oracle.com Tue Oct 9 14:55:39 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 9 Oct 2018 16:55:39 +0200 Subject: RFR: 8211446: Replace oop_pc_follow_contents with oop_iterate and closure In-Reply-To: <082ab861-5370-f7b6-f28c-0e60b7bae412@oracle.com> References: <082ab861-5370-f7b6-f28c-0e60b7bae412@oracle.com> Message-ID: <34cf5aeb-28af-61e4-e95f-69eb092d211b@oracle.com> Looks good, StefanJ On 2018-10-09 15:52, Leo Korinth wrote: > Hi! > > Here are new incremental and full webrevs that tries to improve the code > according to suggestions from Stefan and Thomas. > > incremental: > http://cr.openjdk.java.net/~lkorinth/8211446/00-01/ > full: > http://cr.openjdk.java.net/~lkorinth/8211446/01/ > > Thanks, > Leo > > On 09/10/2018 11:00, Thomas Schatzl wrote: >> Hi, >> >> On Thu, 2018-10-04 at 18:27 +0200, Leo Korinth wrote: >>> Hi! >>> >>> This is the is the second enhancement in a series of three to remove >>> Parallel GC specific behaviour from shared code. I will post all >>> three enhancements to the mailing list. >>> >>> For a description on what is changed see my earlier "RFR: 8201436: >>> Replace oop_ps_push_contents with oop_iterate and closure" >>> >>> Enhancement: >>> https://bugs.openjdk.java.net/browse/JDK-8211446 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~lkorinth/8211446/00/ >> >> Some minor issues: >> >> - psCompactionManager.inline.hpp:96: remove two of the newlines >> >> - psCompactionManager.inline.hpp: please fix indentation of >> PSMarkAndPushClosure and PSIterateMarkAndPushClosure class >> declarations. >> >> - the "pc" vs. "ps" prefixes Stefan already mentioned. >> >> Thanks, >> ?? Thomas >> From igor.ignatyev at oracle.com Tue Oct 9 17:23:55 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 9 Oct 2018 10:23:55 -0700 Subject: RFR(S) : 8157728 : Covert GCTimer_test to GTest Message-ID: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html > 450 lines changed: 238 ins; 211 del; 1 mod; Hi all, could you please review this small (and hopefully trivial) patch which converts internal GCTimer_test to gtest? webrev: http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8157728 testing: - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants - build w/ precompiled-headers enabled and disabled PS the patch has been originally created by Kirill Zh, but hasn't been sent out for official review Thanks, -- Igor From rkennke at redhat.com Tue Oct 9 20:16:26 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 9 Oct 2018 22:16:26 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve Message-ID: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> TLABs need to be made parseable before GC. This means to fill remaining space with a dummy object, so that there remain no holes in the heap, and the heap becomes contiguously filled with objects. However, since we cannot (generally) fill holes with arbitrarily small objects, we need an 'allocation reserve' at the end of TLABs (and PLABs). Current PLAB and TLAB reserve assumes a specific object size. GCs like Shenandoah need to allocate a little more space per object and thus require some abstraction to get the minimum filler object size / allocation reserve. This change proposes an API in CollectedHeap to determine the minimum filler object size. It also proposes some consolidation between TLAB and PLAB in this respect. plab.cpp has this logic to calculate the reserve: AlignmentReserve = oopDesc::header_size() > MinObjAlignment ? align_object_size(arrayOopDesc::header_size(T_INT)) : 0; The idea is that if obj-alignment is larger/equal than min-object-size, then we don't need a reserve, because we can always make the filler fit. Otherwise, we choose the min-array-size (3 words). The reason for the latter eludes me: it should be enough to choose the minimum object size (2 words). TLAB always uses min-array-size, and does not take MinObjAlignment into account. I propose to add two methods to CollectedHeap: virtual size_t min_dummy_object_word_size() const; to return the minimum possible filler object size. GCs can override this to return a different size (e.g. Shenandoah would return one extra word here). And then: size_t tlab_alloc_reserve() const; to calculate the allocation reserve, using the logic from PLAB that takes MinObjAlignment into account. This uses min_dummy_object_word_size() and would be used by both TLAB and PLAB. Bug: https://bugs.openjdk.java.net/browse/JDK-8211955 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.00/ What do you think? Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From jcbeyler at google.com Tue Oct 9 20:35:47 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 9 Oct 2018 13:35:47 -0700 Subject: RFR (S) 8211950: Deprecate the check if a JVMTI collector is present assertion Message-ID: Hi all, I'm sending this to the GC list but I'm not sure as it relates to the JVMTI heap sampling notifications. Let me know what you think. Could I get a review for this: Webrev: http://cr.openjdk.java.net/~jcbeyler/8211950/webrev.00/ Associated bug: https://bugs.openjdk.java.net/browse/JDK-8211950 Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Tue Oct 9 20:47:03 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 9 Oct 2018 13:47:03 -0700 Subject: RFR(S) : 8157728 : Covert GCTimer_test to GTest In-Reply-To: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> References: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> Message-ID: Hi Igor, I was looking at this and it looks good to me (not a reviewer of course) but I noticed a few nits: - http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/src/hotspot/share/gc/shared/gcTimer.cpp.udiff.html - Seems like you could remove a few more lines there because now you have empty lines at the end of the file - http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/test/hotspot/gtest/gc/shared/test_gcTimer.cpp.html - Should the copyright year 2001 be here? - Indentation is a bit off: - GCTimerTest has 2 spaces for public and 4 for methods - TimePartitionPhasesIteratorTest has 1 and 2 - Tests afterwards have 2 - The code could be simplified if we removed the TimePartitionPhasesIteratorTest entirely and just put it in an anonymous namespace - hotspot/gtest/utilities/test_bitMap_search.cpp uses anonymous namespaces so gtest does support it Then lines such as: 219 EXPECT_NO_FATAL_FAILURE(TimePartitionPhasesIteratorTest::validate_gc_phase(iter.next(), 1, "SubPhase3", 15, 16)); become: 219 EXPECT_NO_FATAL_FAILURE(validate_gc_phase(iter.next(), 1, "SubPhase3", 15, 16)); Apart from that, it looks good to me :) Jc On Tue, Oct 9, 2018 at 10:24 AM Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html > > 450 lines changed: 238 ins; 211 del; 1 mod; > > Hi all, > > could you please review this small (and hopefully trivial) patch which > converts internal GCTimer_test to gtest? > > webrev: > http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8157728 > testing: > - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 > in product and fastdebug variants > - build w/ precompiled-headers enabled and disabled > > PS the patch has been originally created by Kirill Zh, but hasn't been > sent out for official review > > Thanks, > -- Igor > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.ignatyev at oracle.com Tue Oct 9 21:03:46 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 9 Oct 2018 14:03:46 -0700 Subject: RFR(S) : 8157728 : Covert GCTimer_test to GTest In-Reply-To: References: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> Message-ID: <5D55535E-A01E-42DA-A64A-64BCE828FD30@oracle.com> Hi JC, thanks for reviewing it! > On Oct 9, 2018, at 1:47 PM, JC Beyler wrote: > > Hi Igor, > > I was looking at this and it looks good to me (not a reviewer of course) but I noticed a few nits: > > - http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/src/hotspot/share/gc/shared/gcTimer.cpp.udiff.html > - Seems like you could remove a few more lines there because now you have empty lines at the end of the file fixed > > - http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/test/hotspot/gtest/gc/shared/test_gcTimer.cpp.html > - Should the copyright year 2001 be here? yes, as this code was originally introduced in 2001 as part of gcTimer.cpp > > - Indentation is a bit off: > - GCTimerTest has 2 spaces for public and 4 for methods > - TimePartitionPhasesIteratorTest has 1 and 2 > - Tests afterwards have 2 dough, that's what happens when you copy-paste someone's else code... fixed. now GCTimerTest and TimePartitionPhasesIteratorTest have 1 for public and 2 for methods, all tests have 2. > > - The code could be simplified if we removed the TimePartitionPhasesIteratorTest entirely and just put it in an anonymous namespace > - hotspot/gtest/utilities/test_bitMap_search.cpp uses anonymous namespaces so gtest does support it > Then lines such as: > 219 EXPECT_NO_FATAL_FAILURE(TimePartitionPhasesIteratorTest::validate_gc_phase(iter.next(), 1, "SubPhase3", 15, 16)); > become: > 219 EXPECT_NO_FATAL_FAILURE(validate_gc_phase(iter.next(), 1, "SubPhase3", 15, 16)); unfortunately we can't do it, TimePartitionPhasesIteratorTest (as well as GCTimerTest) is needed b/c it's a friend of TimeInstant (Ticks), so it can use private c-tor TimeInstant(jlong). -- Igor > Apart from that, it looks good to me :) > Jc > > > On Tue, Oct 9, 2018 at 10:24 AM Igor Ignatyev > wrote: > http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html > > 450 lines changed: 238 ins; 211 del; 1 mod; > > Hi all, > > could you please review this small (and hopefully trivial) patch which converts internal GCTimer_test to gtest? > > webrev: http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8157728 > testing: > - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants > - build w/ precompiled-headers enabled and disabled > > PS the patch has been originally created by Kirill Zh, but hasn't been sent out for official review > > Thanks, > -- Igor > > > > -- > > Thanks, > Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Tue Oct 9 21:11:43 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 9 Oct 2018 14:11:43 -0700 Subject: RFR(S) : 8157728 : Covert GCTimer_test to GTest In-Reply-To: <5D55535E-A01E-42DA-A64A-64BCE828FD30@oracle.com> References: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> <5D55535E-A01E-42DA-A64A-64BCE828FD30@oracle.com> Message-ID: Good point on the friend class, looks good to me then :) Jc On Tue, Oct 9, 2018 at 2:03 PM Igor Ignatyev wrote: > Hi JC, > > thanks for reviewing it! > > On Oct 9, 2018, at 1:47 PM, JC Beyler wrote: > > Hi Igor, > > I was looking at this and it looks good to me (not a reviewer of course) > but I noticed a few nits: > > - > http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/src/hotspot/share/gc/shared/gcTimer.cpp.udiff.html > - Seems like you could remove a few more lines there because now you > have empty lines at the end of the file > > fixed > > > - > http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/test/hotspot/gtest/gc/shared/test_gcTimer.cpp.html > - Should the copyright year 2001 be here? > > yes, as this code was originally introduced in 2001 as part of gcTimer.cpp > > > - Indentation is a bit off: > - GCTimerTest has 2 spaces for public and 4 for methods > - TimePartitionPhasesIteratorTest has 1 and 2 > - Tests afterwards have 2 > > dough, that's what happens when you copy-paste someone's else code... > fixed. now GCTimerTest and TimePartitionPhasesIteratorTest have 1 for > public and 2 for methods, all tests have 2. > > > - The code could be simplified if we removed the TimePartitionPhasesIteratorTest > entirely and just put it in an anonymous namespace > > - hotspot/gtest/utilities/test_bitMap_search.cpp uses anonymous > namespaces so gtest does support it > Then lines such as: > > 219 EXPECT_NO_FATAL_FAILURE(TimePartitionPhasesIteratorTest::validate_gc_phase(iter.next(), 1, "SubPhase3", 15, 16)); > > become: > > 219 EXPECT_NO_FATAL_FAILURE(validate_gc_phase(iter.next(), 1, "SubPhase3", 15, 16)); > > unfortunately we can't do it, TimePartitionPhasesIteratorTest (as well as > GCTimerTest) is needed b/c it's a friend of TimeInstant (Ticks), so it can > use private c-tor TimeInstant(jlong). > > -- Igor > > Apart from that, it looks good to me :) > > Jc > > > > On Tue, Oct 9, 2018 at 10:24 AM Igor Ignatyev > wrote: > >> http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html >> > 450 lines changed: 238 ins; 211 del; 1 mod; >> >> Hi all, >> >> could you please review this small (and hopefully trivial) patch which >> converts internal GCTimer_test to gtest? >> >> webrev: >> http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html >> JBS: https://bugs.openjdk.java.net/browse/JDK-8157728 >> testing: >> - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 >> in product and fastdebug variants >> - build w/ precompiled-headers enabled and disabled >> >> PS the patch has been originally created by Kirill Zh, but hasn't been >> sent out for official review >> >> Thanks, >> -- Igor >> > > > > -- > > Thanks, > Jc > > > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From hohensee at amazon.com Tue Oct 9 22:46:04 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Tue, 9 Oct 2018 22:46:04 +0000 Subject: RFR (S) 8211950: Deprecate the check if a JVMTI collector is present assertion In-Reply-To: References: Message-ID: <3C881693-CB85-4AFB-A4BC-EC5690632B91@amazon.com> Looks good. Paul From: hotspot-gc-dev on behalf of JC Beyler Date: Tuesday, October 9, 2018 at 4:37 PM To: "hotspot-gc-dev at openjdk.java.net" Subject: RFR (S) 8211950: Deprecate the check if a JVMTI collector is present assertion Hi all, I'm sending this to the GC list but I'm not sure as it relates to the JVMTI heap sampling notifications. Let me know what you think. Could I get a review for this: Webrev: http://cr.openjdk.java.net/~jcbeyler/8211950/webrev.00/ Associated bug: https://bugs.openjdk.java.net/browse/JDK-8211950 Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From zgu at redhat.com Tue Oct 9 22:51:41 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 9 Oct 2018 18:51:41 -0400 Subject: RFR: JDK-8211279: Verify missing object equals barriers In-Reply-To: References: <935559b9-6df2-e337-97b5-5807cf57a154@redhat.com> Message-ID: <2af3e21b-8c09-b16c-b042-4454165e8854@redhat.com> Looks good to me. Thanks, -Zhengyu On 10/03/2018 09:25 AM, Roman Kennke wrote: > Hi Per, > >> On 10/01/2018 02:48 PM, Roman Kennke wrote: >>> Hi Per, >>> >>> >>>>> GCs like Shenandoah require an extra barrier for comparing objects >>>>> (oop==oop). It is easy to forget or overlook this. It would be very >>>>> useful to have a flag to turn on extra verification that catches >>>>> missing >>>>> object equality barriers. >>>>> >>>>> This change inserts an assert into == and != operators for the oop >>>>> class >>>>> in oopsHierarchy.hpp. This only gets compiled in fastdebug builds (when >>>>> CHECK_UNHANDLED_OOPS is on). >>>>> >>>>> It also adds a develop flag VerifyObjectEquals that is used to turn on >>>>> this verification. >>>>> >>>>> It also adds a method oopDesc::unsafe_equals(..) to be used in cases >>>>> where you know what what you are doing, and really want to use >>>>> direct == >>>>> comparison without using barriers. This is used in e.g. >>>>> ReferenceProcessor or all over the place in ShenandoahGC. >>>>> >>>>> The change also fixes a couple of places where oops are compared to >>>>> non-oops like Universe::non_oop_word() to use the oop==void* operator >>>>> instead, so those don't falsely trip the verification. >>>>> >>>>> It doesn't make sense to turn this check on if you're not running a GC >>>>> that needs it, unless you want to go fix all the oop==oop in the GC >>>>> itself. >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8211279 >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.00/ >>>>> >>>>> What do you think? >>>> >>>> So this means we would have a verification option that, when enabled, >>>> always crashes the VM unless you run Shenandoah? That doesn't sound >>>> quite right to me. This should just be a noop when not using Shenandoah, >>>> don't you think? >>> >>> >>> Hmm, right. Let's add some BarrierSet-infrastructure to handle this, and >>> remove the option (it would be a GC-'private' option). It would probably >>> have looked slightly better to do this in BarrierSet::Access, next to >>> the Access::equals() API, but I don't feel like adding tons of >>> boilerplate just to add this. (Infact, this is a big red warning signal >>> regarding the Access API...) >>> >>> http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.01/ >>> >>> How does this look now? >> >> >> src/hotspot/share/oops/oop.hpp >> ------------------------------ >> >> ?157?? inline static bool unsafe_equals(oop o1, oop o2) { >> ?158???? return (void*) o1 == (void*) o2; >> ?159?? } >> >> I think this should be called oopDesc::equals_raw() to follow the naming >> convention we have for these types of functions. Also, it should do: >> >> ? return RawAccess<>::equals(o1, o2); >> >> Also, please make it a one-liner to better match the look of >> oopDesc::equals(). >> >> >> src/hotspot/share/gc/shared/referenceProcessor.cpp >> -------------------------------------------------- >> >> ?477?? while (! oopDesc::unsafe_equals(next, obj)) { >> >> Stray white-space, please remove. >> >> >> src/hotspot/share/gc/shared/referenceProcessor.hpp >> -------------------------------------------------- >> >> ?152???? assert(! oopDesc::unsafe_equals(_current_discovered, >> _first_seen), "cyclic ref_list found"); >> >> Stray white-space, please remove. >> >> >> src/hotspot/share/oops/accessBackend.hpp >> ---------------------------------------- >> >> ?413?? static bool equals(oop o1, oop o2) { return (void*) o1 == (void*) >> o2; } >> >> Stray white-spaces, please make that "(void*)o1 == (void*)o2". >> >> >> src/hotspot/share/gc/shared/barrierSet.hpp >> ------------------------------------------ >> >> ?134?? virtual void verify_equals(oop o1, oop o2) { } >> >> I'm thinking this should be: >> >> ? virtual bool oop_equals_operator_allowed() { return true; } >> >> And let oop::operator==(...) do: >> >> ? assert(BarrierSet::barrier_set()->oop_equals_operator_allowed(), "Not >> allowed"); >> >> >> Erik, can you live with this, or do you have any better ideas here? >> I'm not ecstatic about having a new function on BarrierSet just for >> this. Should we just make oop::operator==() private and fix all the >> places where it's used? One could also argue the oop::operator==() _is_ >> the raw equals and that we should be allowed to use it. Any other ideas? > > > This addresses all the issues mentioned above: > http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.02/ > > It does not implement Erik's suggestion, because I think it's inferior. > I'm open to discussions though. > > Roman > From zgu at redhat.com Tue Oct 9 23:07:47 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 9 Oct 2018 19:07:47 -0400 Subject: RFR: JDK-8211270: GC abstraction to get real object and headers size In-Reply-To: References: <1b75ccd7-ba1a-a183-f338-18d3c651827e@redhat.com> <5689e478-6f2f-d7ea-127e-689108364074@redhat.com> <3e3fbeff-679e-6934-04b8-24b44ca42dac@redhat.com> Message-ID: Looks good to me. Thanks, -Zhengyu On 10/09/2018 10:39 AM, Roman Kennke wrote: > Hey Aleksey, > >> On 10/05/2018 06:20 PM, Roman Kennke wrote: >>>> Webrev: >>>> http://cr.openjdk.java.net/~rkennke/JDK-8211270/webrev.00/ >> >> Do we really need the abstractions for header sizes? I think all usages need the object size, and >> header sizes can be just picked up from the usual places. No GC, including Shenandoah is changing >> the header size? >> >> Trying to understand the use case, looking at Shenandoah's diff here: >> >> https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/gc/shared/plab.cpp.sdiff.html >> >> ...it seems what we really want is to mix in oop_extra_words into plab.cpp accessors. Because doing >> ShenandoahHeap::obj_header_size() { return obj_header_size + oop_extra_words; } seems wrong down the >> line -- it would _not_ be the header size. > > It seems that the PLAB stuff does indeed require a different kind of > abstraction, and most likely something that can be shared with, e.g., > TLAB in the form of a virtual version of CollectedHeap::min_fill_size() > or such. Let's take this part out of the patch: > > http://cr.openjdk.java.net/~rkennke/JDK-8211270/webrev.01/ > > Good now? > > Roman > From david.holmes at oracle.com Wed Oct 10 01:49:08 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 10 Oct 2018 11:49:08 +1000 Subject: RFR(S) : 8157728 : Covert GCTimer_test to GTest In-Reply-To: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> References: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> Message-ID: Hi Igor, Not a review - I fixed the typo in the bug synopsis: Covert -> Convert :) Please ensure you commit with corrected synopsis. Thanks, David On 10/10/2018 3:23 AM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html >> 450 lines changed: 238 ins; 211 del; 1 mod; > > Hi all, > > could you please review this small (and hopefully trivial) patch which converts internal GCTimer_test to gtest? > > webrev: http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8157728 > testing: > - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants > - build w/ precompiled-headers enabled and disabled > > PS the patch has been originally created by Kirill Zh, but hasn't been sent out for official review > > Thanks, > -- Igor > > From per.liden at oracle.com Wed Oct 10 06:02:27 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 10 Oct 2018 08:02:27 +0200 Subject: RFR: JDK-8211279: Verify missing object equals barriers In-Reply-To: <1f7a6191-cb83-5326-d959-495651ad656f@redhat.com> References: <935559b9-6df2-e337-97b5-5807cf57a154@redhat.com> <04ad0da1-3e16-bd73-bc47-f3b62c52c91d@redhat.com> <3826dcb4-1a99-7ecb-ed6f-dd80f86ca1ca@redhat.com> <1f7a6191-cb83-5326-d959-495651ad656f@redhat.com> Message-ID: <6867d473-0e8d-b058-4d5f-3c630ad55966@oracle.com> Hi Roman, On 10/08/2018 10:13 PM, Roman Kennke wrote: > Thanks, Aleksey, for the review. > > I've sent the patch through jdk/submit, and it came back PASSED. > > Per, Erik, everybody else, what do you think about the last changeset? > Can you live with it? I can live with it. cheers, Per > > Thanks, > Roman > > >> On 10/08/2018 06:32 PM, Roman Kennke wrote: >>>> *) I always get a bit uneasy seeing the casts to (void*) like here: >>>> >>>> 413 static bool equals(oop o1, oop o2) { return (void*)o1 == (void*)o2; } >>>> >>>> ...but I think this comparison is well-defined in C++ with the semantics we want. >>> >>> What would you suggest? reinterpret_cast(..) better? >> >> No, from my reading of C++ spec casting to void* is safe here. Just thinking out loud. >> >>>> *) So, if CHECK_UNHANDLED_OOPS is not defined, this declaration is inaccessible, and asserts that >>>> use it would fail to compile? I think this ifdef is not needed here, but rather it should be around >>>> the asserts in oop::operator== and !=? >>>> >>>> 133 #ifdef CHECK_UNHANDLED_OOPS >>>> 134 virtual bool oop_equals_operator_allowed() { return true; } >>>> 135 #endif >>>> 136 >>> >>> It already *is* around the whole (oop class) declaration in >>> oopsHierarchy.hpp, that's the only user of the check so I put this under >>> #ifdef CHECK_UNHANDLED_OOPS too, but I can remove this #ifdef if you >>> prefer. Let me know? >> >> Ah! Missed that. No, this is fine then. >> >> -Aleksey >> > From rkennke at redhat.com Wed Oct 10 06:24:22 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 10 Oct 2018 08:24:22 +0200 Subject: RFR: JDK-8211279: Verify missing object equals barriers In-Reply-To: <6867d473-0e8d-b058-4d5f-3c630ad55966@oracle.com> References: <935559b9-6df2-e337-97b5-5807cf57a154@redhat.com> <04ad0da1-3e16-bd73-bc47-f3b62c52c91d@redhat.com> <3826dcb4-1a99-7ecb-ed6f-dd80f86ca1ca@redhat.com> <1f7a6191-cb83-5326-d959-495651ad656f@redhat.com> <6867d473-0e8d-b058-4d5f-3c630ad55966@oracle.com> Message-ID: > On 10/08/2018 10:13 PM, Roman Kennke wrote: >> Thanks, Aleksey, for the review. >> >> I've sent the patch through jdk/submit, and it came back PASSED. >> >> Per, Erik, everybody else, what do you think about the last changeset? >> Can you live with it? > > I can live with it. > > cheers, > Per Thanks for the review, Per! Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed Oct 10 06:25:35 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 10 Oct 2018 08:25:35 +0200 Subject: RFR: JDK-8211270: GC abstraction to get real object and headers size In-Reply-To: References: <1b75ccd7-ba1a-a183-f338-18d3c651827e@redhat.com> <5689e478-6f2f-d7ea-127e-689108364074@redhat.com> <3e3fbeff-679e-6934-04b8-24b44ca42dac@redhat.com> Message-ID: <938651d9-76bb-6e4c-c13d-9a796b70bcd8@redhat.com> Zhengyu, thanks for reviewing! Roman > Looks good to me. > > Thanks, > > -Zhengyu > > On 10/09/2018 10:39 AM, Roman Kennke wrote: >> Hey Aleksey, >> >>> On 10/05/2018 06:20 PM, Roman Kennke wrote: >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~rkennke/JDK-8211270/webrev.00/ >>> >>> Do we really need the abstractions for header sizes? I think all >>> usages need the object size, and >>> header sizes can be just picked up from the usual places. No GC, >>> including Shenandoah is changing >>> the header size? >>> >>> Trying to understand the use case, looking at Shenandoah's diff here: >>> >>> https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/gc/shared/plab.cpp.sdiff.html >>> >>> >>> ...it seems what we really want is to mix in oop_extra_words into >>> plab.cpp accessors. Because doing >>> ShenandoahHeap::obj_header_size() { return obj_header_size + >>> oop_extra_words; } seems wrong down the >>> line -- it would _not_ be the header size. >> >> It seems that the PLAB stuff does indeed require a different kind of >> abstraction, and most likely something that can be shared with, e.g., >> TLAB in the form of a virtual version of CollectedHeap::min_fill_size() >> or such. Let's take this part out of the patch: >> >> http://cr.openjdk.java.net/~rkennke/JDK-8211270/webrev.01/ >> >> Good now? >> >> Roman >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed Oct 10 06:25:55 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 10 Oct 2018 08:25:55 +0200 Subject: RFR: JDK-8211279: Verify missing object equals barriers In-Reply-To: <2af3e21b-8c09-b16c-b042-4454165e8854@redhat.com> References: <935559b9-6df2-e337-97b5-5807cf57a154@redhat.com> <2af3e21b-8c09-b16c-b042-4454165e8854@redhat.com> Message-ID: <60212510-7305-6935-5565-247ec7a6ea30@redhat.com> Thanks for reviewing, Zhengyu! Roman > Looks good to me. > > Thanks, > > -Zhengyu > > On 10/03/2018 09:25 AM, Roman Kennke wrote: >> Hi Per, >> >>> On 10/01/2018 02:48 PM, Roman Kennke wrote: >>>> Hi Per, >>>> >>>> >>>>>> GCs like Shenandoah require an extra barrier for comparing objects >>>>>> (oop==oop). It is easy to forget or overlook this. It would be very >>>>>> useful to have a flag to turn on extra verification that catches >>>>>> missing >>>>>> object equality barriers. >>>>>> >>>>>> This change inserts an assert into == and != operators for the oop >>>>>> class >>>>>> in oopsHierarchy.hpp. This only gets compiled in fastdebug builds >>>>>> (when >>>>>> CHECK_UNHANDLED_OOPS is on). >>>>>> >>>>>> It also adds a develop flag VerifyObjectEquals that is used to >>>>>> turn on >>>>>> this verification. >>>>>> >>>>>> It also adds a method oopDesc::unsafe_equals(..) to be used in cases >>>>>> where you know what what you are doing, and really want to use >>>>>> direct == >>>>>> comparison without using barriers. This is used in e.g. >>>>>> ReferenceProcessor or all over the place in ShenandoahGC. >>>>>> >>>>>> The change also fixes a couple of places where oops are compared to >>>>>> non-oops like Universe::non_oop_word() to use the oop==void* operator >>>>>> instead, so those don't falsely trip the verification. >>>>>> >>>>>> It doesn't make sense to turn this check on if you're not running >>>>>> a GC >>>>>> that needs it, unless you want to go fix all the oop==oop in the GC >>>>>> itself. >>>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8211279 >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.00/ >>>>>> >>>>>> What do you think? >>>>> >>>>> So this means we would have a verification option that, when enabled, >>>>> always crashes the VM unless you run Shenandoah? That doesn't sound >>>>> quite right to me. This should just be a noop when not using >>>>> Shenandoah, >>>>> don't you think? >>>> >>>> >>>> Hmm, right. Let's add some BarrierSet-infrastructure to handle this, >>>> and >>>> remove the option (it would be a GC-'private' option). It would >>>> probably >>>> have looked slightly better to do this in BarrierSet::Access, next to >>>> the Access::equals() API, but I don't feel like adding tons of >>>> boilerplate just to add this. (Infact, this is a big red warning signal >>>> regarding the Access API...) >>>> >>>> http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.01/ >>>> >>>> How does this look now? >>> >>> >>> src/hotspot/share/oops/oop.hpp >>> ------------------------------ >>> >>> ??157?? inline static bool unsafe_equals(oop o1, oop o2) { >>> ??158???? return (void*) o1 == (void*) o2; >>> ??159?? } >>> >>> I think this should be called oopDesc::equals_raw() to follow the naming >>> convention we have for these types of functions. Also, it should do: >>> >>> ?? return RawAccess<>::equals(o1, o2); >>> >>> Also, please make it a one-liner to better match the look of >>> oopDesc::equals(). >>> >>> >>> src/hotspot/share/gc/shared/referenceProcessor.cpp >>> -------------------------------------------------- >>> >>> ??477?? while (! oopDesc::unsafe_equals(next, obj)) { >>> >>> Stray white-space, please remove. >>> >>> >>> src/hotspot/share/gc/shared/referenceProcessor.hpp >>> -------------------------------------------------- >>> >>> ??152???? assert(! oopDesc::unsafe_equals(_current_discovered, >>> _first_seen), "cyclic ref_list found"); >>> >>> Stray white-space, please remove. >>> >>> >>> src/hotspot/share/oops/accessBackend.hpp >>> ---------------------------------------- >>> >>> ??413?? static bool equals(oop o1, oop o2) { return (void*) o1 == >>> (void*) >>> o2; } >>> >>> Stray white-spaces, please make that "(void*)o1 == (void*)o2". >>> >>> >>> src/hotspot/share/gc/shared/barrierSet.hpp >>> ------------------------------------------ >>> >>> ??134?? virtual void verify_equals(oop o1, oop o2) { } >>> >>> I'm thinking this should be: >>> >>> ?? virtual bool oop_equals_operator_allowed() { return true; } >>> >>> And let oop::operator==(...) do: >>> >>> ?? assert(BarrierSet::barrier_set()->oop_equals_operator_allowed(), "Not >>> allowed"); >>> >>> >>> Erik, can you live with this, or do you have any better ideas here? >>> I'm not ecstatic about having a new function on BarrierSet just for >>> this. Should we just make oop::operator==() private and fix all the >>> places where it's used? One could also argue the oop::operator==() _is_ >>> the raw equals and that we should be allowed to use it. Any other ideas? >> >> >> This addresses all the issues mentioned above: >> http://cr.openjdk.java.net/~rkennke/JDK-8211279/webrev.02/ >> >> It does not implement Erik's suggestion, because I think it's inferior. >> I'm open to discussions though. >> >> Roman >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From erik.osterlund at oracle.com Wed Oct 10 08:11:34 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 10 Oct 2018 10:11:34 +0200 Subject: RFR (S) 8211950: Deprecate the check if a JVMTI collector is present assertion In-Reply-To: References: Message-ID: <5BBDB436.8000102@oracle.com> Hi JC Looks good. Thanks, /Erik On 2018-10-09 22:35, JC Beyler wrote: > Hi all, > > I'm sending this to the GC list but I'm not sure as it relates to the > JVMTI heap sampling notifications. Let me know what you think. > > Could I get a review for this: > Webrev: http://cr.openjdk.java.net/~jcbeyler/8211950/webrev.00/ > > Associated bug: https://bugs.openjdk.java.net/browse/JDK-8211950 > > Thanks, > Jc From per.liden at oracle.com Wed Oct 10 08:12:49 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 10 Oct 2018 10:12:49 +0200 Subject: RFR (S) 8211950: Deprecate the check if a JVMTI collector is present assertion In-Reply-To: References: Message-ID: Hi JC, On 10/09/2018 10:35 PM, JC Beyler wrote: > Hi all, > > I'm sending this to the GC list but I'm not sure as it relates to the > JVMTI heap sampling notifications. Let me know what you think. Since it's touching code in gc/shared/ it's good to include the GC list. > > Could I get a review for this: > Webrev: http://cr.openjdk.java.net/~jcbeyler/8211950/webrev.00/ > Looks good! /Per > Associated bug: https://bugs.openjdk.java.net/browse/JDK-8211950 > > Thanks, > Jc From erik.osterlund at oracle.com Wed Oct 10 08:24:48 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 10 Oct 2018 10:24:48 +0200 Subject: RFR: JDK-8211270: GC abstraction to get real object and headers size In-Reply-To: References: <1b75ccd7-ba1a-a183-f338-18d3c651827e@redhat.com> <5689e478-6f2f-d7ea-127e-689108364074@redhat.com> <3e3fbeff-679e-6934-04b8-24b44ca42dac@redhat.com> Message-ID: <5BBDB750.7090302@oracle.com> Hi Roman, I think this looks good for now. But I thought I should still mention an idea I have had since before even setting food in HotSpot. I have always thought of "cells" and "objects" as different things. Cells, to me, are chunks of memory allocated by an allocator (possibly with its own header), and objects are the things we initialize into these cells. It just happens to be that cells and objects have had the same size up until now, and we never initialize anything else than objects inside of our cells. With the introduction of Shenandoah, cells and objects are once again different, to the extent that we need these type of APIs. The cell header is at a negative offset, and the object header at the base pointer. The cell size includes the cell header, the object size does not. So to me, it is not obvious when we are asking for the cell size as opposed to the object size without being explicit about it. In your changes, you make the object size the cell size, trying to keep them the same, and not expose the cell abstraction. For example, the whitebox API when asking for the object size, always gets the cell size. But it's not obvious to me that you always want to know the cell size and never the object size. I suppose now the distinction can be made by simply not calling this API that slaps on the cell header size. So for me, introducing a distinction between objects and cells, so we always know which one of the two we are referring to, would make things less confusing and more precise. Having said all this, I don't know exactly what I would like that to look like, and I don't want to block anything based on a vague idea. So I'm okay with doing things this way now, and perhaps we think a bit about this until another day, and see if we can come up with something even better eventually. Thanks, /Erik On 2018-10-09 16:39, Roman Kennke wrote: > Hey Aleksey, > >> On 10/05/2018 06:20 PM, Roman Kennke wrote: >>>> Webrev: >>>> http://cr.openjdk.java.net/~rkennke/JDK-8211270/webrev.00/ >> Do we really need the abstractions for header sizes? I think all usages need the object size, and >> header sizes can be just picked up from the usual places. No GC, including Shenandoah is changing >> the header size? >> >> Trying to understand the use case, looking at Shenandoah's diff here: >> >> https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/gc/shared/plab.cpp.sdiff.html >> >> ...it seems what we really want is to mix in oop_extra_words into plab.cpp accessors. Because doing >> ShenandoahHeap::obj_header_size() { return obj_header_size + oop_extra_words; } seems wrong down the >> line -- it would _not_ be the header size. > It seems that the PLAB stuff does indeed require a different kind of > abstraction, and most likely something that can be shared with, e.g., > TLAB in the form of a virtual version of CollectedHeap::min_fill_size() > or such. Let's take this part out of the patch: > > http://cr.openjdk.java.net/~rkennke/JDK-8211270/webrev.01/ > > Good now? > > Roman > From rkennke at redhat.com Wed Oct 10 08:29:43 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 10 Oct 2018 10:29:43 +0200 Subject: RFR: JDK-8211270: GC abstraction to get real object and headers size In-Reply-To: <5BBDB750.7090302@oracle.com> References: <1b75ccd7-ba1a-a183-f338-18d3c651827e@redhat.com> <5689e478-6f2f-d7ea-127e-689108364074@redhat.com> <3e3fbeff-679e-6934-04b8-24b44ca42dac@redhat.com> <5BBDB750.7090302@oracle.com> Message-ID: <34490016-d489-f758-e6f8-5dfd2b21cff5@redhat.com> Hi Erik, Your reasoning makes sense. A simple solution (for this particular patch) would be to rename obj_size() to cell_size() in CollectedHeap. But of course, the other APIs would have to be made consistent with this change too, so... > Hi Roman, > > I think this looks good for now. But I thought I should still mention an > idea I have had since before even setting food in HotSpot. I have always > thought of "cells" and "objects" as different things. Cells, to me, are > chunks of memory allocated by an allocator (possibly with its own > header), and objects are the things we initialize into these cells. It > just happens to be that cells and objects have had the same size up > until now, and we never initialize anything else than objects inside of > our cells. With the introduction of Shenandoah, cells and objects are > once again different, to the extent that we need these type of APIs. The > cell header is at a negative offset, and the object header at the base > pointer. The cell size includes the cell header, the object size does not. > > So to me, it is not obvious when we are asking for the cell size as > opposed to the object size without being explicit about it. In your > changes, you make the object size the cell size, trying to keep them the > same, and not expose the cell abstraction. For example, the whitebox API > when asking for the object size, always gets the cell size. But it's not > obvious to me that you always want to know the cell size and never the > object size. I suppose now the distinction can be made by simply not > calling this API that slaps on the cell header size. > > So for me, introducing a distinction between objects and cells, so we > always know which one of the two we are referring to, would make things > less confusing and more precise. > > Having said all this, I don't know exactly what I would like that to > look like, and I don't want to block anything based on a vague idea. So > I'm okay with doing things this way now, and perhaps we think a bit > about this until another day, and see if we can come up with something > even better eventually. > > Thanks, > /Erik > > On 2018-10-09 16:39, Roman Kennke wrote: >> Hey Aleksey, >> >>> On 10/05/2018 06:20 PM, Roman Kennke wrote: >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~rkennke/JDK-8211270/webrev.00/ >>> Do we really need the abstractions for header sizes? I think all >>> usages need the object size, and >>> header sizes can be just picked up from the usual places. No GC, >>> including Shenandoah is changing >>> the header size? >>> >>> Trying to understand the use case, looking at Shenandoah's diff here: >>> >>> https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/gc/shared/plab.cpp.sdiff.html >>> >>> >>> ...it seems what we really want is to mix in oop_extra_words into >>> plab.cpp accessors. Because doing >>> ShenandoahHeap::obj_header_size() { return obj_header_size + >>> oop_extra_words; } seems wrong down the >>> line -- it would _not_ be the header size. >> It seems that the PLAB stuff does indeed require a different kind of >> abstraction, and most likely something that can be shared with, e.g., >> TLAB in the form of a virtual version of CollectedHeap::min_fill_size() >> or such. Let's take this part out of the patch: >> >> http://cr.openjdk.java.net/~rkennke/JDK-8211270/webrev.01/ >> >> Good now? >> >> Roman >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From erik.osterlund at oracle.com Wed Oct 10 08:37:12 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 10 Oct 2018 10:37:12 +0200 Subject: RFR: JDK-8211270: GC abstraction to get real object and headers size In-Reply-To: <34490016-d489-f758-e6f8-5dfd2b21cff5@redhat.com> References: <1b75ccd7-ba1a-a183-f338-18d3c651827e@redhat.com> <5689e478-6f2f-d7ea-127e-689108364074@redhat.com> <3e3fbeff-679e-6934-04b8-24b44ca42dac@redhat.com> <5BBDB750.7090302@oracle.com> <34490016-d489-f758-e6f8-5dfd2b21cff5@redhat.com> Message-ID: <5BBDBA38.3090504@oracle.com> Hi Roman, I think you can keep it as obj for now, as the concept of a cell is not yet introduced. We can sort it out later instead when we know better what we want to do. Thanks, /Erik On 2018-10-10 10:29, Roman Kennke wrote: > Hi Erik, > > Your reasoning makes sense. A simple solution (for this particular > patch) would be to rename obj_size() to cell_size() in CollectedHeap. > But of course, the other APIs would have to be made consistent with this > change too, so... > >> Hi Roman, >> >> I think this looks good for now. But I thought I should still mention an >> idea I have had since before even setting food in HotSpot. I have always >> thought of "cells" and "objects" as different things. Cells, to me, are >> chunks of memory allocated by an allocator (possibly with its own >> header), and objects are the things we initialize into these cells. It >> just happens to be that cells and objects have had the same size up >> until now, and we never initialize anything else than objects inside of >> our cells. With the introduction of Shenandoah, cells and objects are >> once again different, to the extent that we need these type of APIs. The >> cell header is at a negative offset, and the object header at the base >> pointer. The cell size includes the cell header, the object size does not. >> >> So to me, it is not obvious when we are asking for the cell size as >> opposed to the object size without being explicit about it. In your >> changes, you make the object size the cell size, trying to keep them the >> same, and not expose the cell abstraction. For example, the whitebox API >> when asking for the object size, always gets the cell size. But it's not >> obvious to me that you always want to know the cell size and never the >> object size. I suppose now the distinction can be made by simply not >> calling this API that slaps on the cell header size. >> >> So for me, introducing a distinction between objects and cells, so we >> always know which one of the two we are referring to, would make things >> less confusing and more precise. >> >> Having said all this, I don't know exactly what I would like that to >> look like, and I don't want to block anything based on a vague idea. So >> I'm okay with doing things this way now, and perhaps we think a bit >> about this until another day, and see if we can come up with something >> even better eventually. >> >> Thanks, >> /Erik >> >> On 2018-10-09 16:39, Roman Kennke wrote: >>> Hey Aleksey, >>> >>>> On 10/05/2018 06:20 PM, Roman Kennke wrote: >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8211270/webrev.00/ >>>> Do we really need the abstractions for header sizes? I think all >>>> usages need the object size, and >>>> header sizes can be just picked up from the usual places. No GC, >>>> including Shenandoah is changing >>>> the header size? >>>> >>>> Trying to understand the use case, looking at Shenandoah's diff here: >>>> >>>> https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/gc/shared/plab.cpp.sdiff.html >>>> >>>> >>>> ...it seems what we really want is to mix in oop_extra_words into >>>> plab.cpp accessors. Because doing >>>> ShenandoahHeap::obj_header_size() { return obj_header_size + >>>> oop_extra_words; } seems wrong down the >>>> line -- it would _not_ be the header size. >>> It seems that the PLAB stuff does indeed require a different kind of >>> abstraction, and most likely something that can be shared with, e.g., >>> TLAB in the form of a virtual version of CollectedHeap::min_fill_size() >>> or such. Let's take this part out of the patch: >>> >>> http://cr.openjdk.java.net/~rkennke/JDK-8211270/webrev.01/ >>> >>> Good now? >>> >>> Roman >>> From rkennke at redhat.com Wed Oct 10 08:55:49 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 10 Oct 2018 10:55:49 +0200 Subject: RFR: JDK-8211270: GC abstraction to get real object and headers size In-Reply-To: <5BBDBA38.3090504@oracle.com> References: <1b75ccd7-ba1a-a183-f338-18d3c651827e@redhat.com> <5689e478-6f2f-d7ea-127e-689108364074@redhat.com> <3e3fbeff-679e-6934-04b8-24b44ca42dac@redhat.com> <5BBDB750.7090302@oracle.com> <34490016-d489-f758-e6f8-5dfd2b21cff5@redhat.com> <5BBDBA38.3090504@oracle.com> Message-ID: <6caee627-a4f6-f159-556a-6faed1aba0a7@redhat.com> Yes. Thank you! It would indeed be useful. It could be used in the various allocation and size paths, as well as in the filler-object-handling (which is just a variation of alloc and size). We used to have oop_extra_words() API in Shenandoah (until now). This could be renamed to: virtual size_t cell_extra_words() { return 0; } and: size_t cell_words(oop obj) { return obj->size() + cell_extra_words(); } and then all the allocation and size paths could use those two methods. E.g. the TLAB alloc path in interpreter would emit one extra 'add cell_extra_words()' instruction to dynamically extend the alloc-size (if it's != 0). That being said, what we have now (including "JDK-8211955: GC abstraction for LAB reserve" if you could review that) works for me, and it hides the concept of 'cells' from the rest of the runtime, and keeps it completely at the discretion of the GC (with the exception of this patch, which exposes the cell-size to whitebox and jvmti). I'll push the change as-is if jdk/submit comes back clean. Thanks for reviewing and your input! Roman > Hi Roman, > > I think you can keep it as obj for now, as the concept of a cell is not > yet introduced. > We can sort it out later instead when we know better what we want to do. > > Thanks, > /Erik > > On 2018-10-10 10:29, Roman Kennke wrote: >> Hi Erik, >> >> Your reasoning makes sense. A simple solution (for this particular >> patch) would be to rename obj_size() to cell_size() in CollectedHeap. >> But of course, the other APIs would have to be made consistent with this >> change too, so... >> >>> Hi Roman, >>> >>> I think this looks good for now. But I thought I should still mention an >>> idea I have had since before even setting food in HotSpot. I have always >>> thought of "cells" and "objects" as different things. Cells, to me, are >>> chunks of memory allocated by an allocator (possibly with its own >>> header), and objects are the things we initialize into these cells. It >>> just happens to be that cells and objects have had the same size up >>> until now, and we never initialize anything else than objects inside of >>> our cells. With the introduction of Shenandoah, cells and objects are >>> once again different, to the extent that we need these type of APIs. The >>> cell header is at a negative offset, and the object header at the base >>> pointer. The cell size includes the cell header, the object size does >>> not. >>> >>> So to me, it is not obvious when we are asking for the cell size as >>> opposed to the object size without being explicit about it. In your >>> changes, you make the object size the cell size, trying to keep them the >>> same, and not expose the cell abstraction. For example, the whitebox API >>> when asking for the object size, always gets the cell size. But it's not >>> obvious to me that you always want to know the cell size and never the >>> object size. I suppose now the distinction can be made by simply not >>> calling this API that slaps on the cell header size. >>> >>> So for me, introducing a distinction between objects and cells, so we >>> always know which one of the two we are referring to, would make things >>> less confusing and more precise. >>> >>> Having said all this, I don't know exactly what I would like that to >>> look like, and I don't want to block anything based on a vague idea. So >>> I'm okay with doing things this way now, and perhaps we think a bit >>> about this until another day, and see if we can come up with something >>> even better eventually. >>> >>> Thanks, >>> /Erik >>> >>> On 2018-10-09 16:39, Roman Kennke wrote: >>>> Hey Aleksey, >>>> >>>>> On 10/05/2018 06:20 PM, Roman Kennke wrote: >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8211270/webrev.00/ >>>>> Do we really need the abstractions for header sizes? I think all >>>>> usages need the object size, and >>>>> header sizes can be just picked up from the usual places. No GC, >>>>> including Shenandoah is changing >>>>> the header size? >>>>> >>>>> Trying to understand the use case, looking at Shenandoah's diff here: >>>>> >>>>> https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/gc/shared/plab.cpp.sdiff.html >>>>> >>>>> >>>>> >>>>> ...it seems what we really want is to mix in oop_extra_words into >>>>> plab.cpp accessors. Because doing >>>>> ShenandoahHeap::obj_header_size() { return obj_header_size + >>>>> oop_extra_words; } seems wrong down the >>>>> line -- it would _not_ be the header size. >>>> It seems that the PLAB stuff does indeed require a different kind of >>>> abstraction, and most likely something that can be shared with, e.g., >>>> TLAB in the form of a virtual version of CollectedHeap::min_fill_size() >>>> or such. Let's take this part out of the patch: >>>> >>>> http://cr.openjdk.java.net/~rkennke/JDK-8211270/webrev.01/ >>>> >>>> Good now? >>>> >>>> Roman >>>> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Wed Oct 10 09:30:51 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 10 Oct 2018 11:30:51 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> Message-ID: <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> On 10/09/2018 10:16 PM, Roman Kennke wrote: > virtual size_t min_dummy_object_word_size() const; I think the proper nomenclature would be min_filler_object_size_words()? We can even drop "words" from here. > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.00/ That looks much cleaner than PLAB/TLAB hacks! Looks good to me. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed Oct 10 09:43:54 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 10 Oct 2018 11:43:54 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> Message-ID: <8243705E-5843-4907-A449-DFE588D1E564@redhat.com> Am 10. Oktober 2018 11:30:51 MESZ schrieb Aleksey Shipilev : >On 10/09/2018 10:16 PM, Roman Kennke wrote: >> virtual size_t min_dummy_object_word_size() const; > >I think the proper nomenclature would be >min_filler_object_size_words()? We can even drop "words" >from here. Yeah maybe. I chose 'dummy' because it relates closely to the fill_with_dummy_object() right above. And I put 'words' there to avoid the usual confusion 'is it bytes, or is it words?' >> Webrev: >> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.00/ > >That looks much cleaner than PLAB/TLAB hacks! Looks good to me. Cool, thanks for reviewing! I'll post an updated webrev with min_filler_object_size() changed when I'm back from doctor appt, unless we agree on something better in the meantime. Can you do me a favour and test this on a 32bits build? I suspect that the alignment stuff is entirely different there... Cheers, Roman From shade at redhat.com Wed Oct 10 11:32:27 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 10 Oct 2018 13:32:27 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: <8243705E-5843-4907-A449-DFE588D1E564@redhat.com> References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <8243705E-5843-4907-A449-DFE588D1E564@redhat.com> Message-ID: On 10/10/2018 11:43 AM, Roman Kennke wrote: > Can you do me a favour and test this on a 32bits build? I suspect that the alignment stuff is > entirely different there... x86_32 build is fine. x86_32 tier1_gc returns with lots Epsilon test failures, but I think those failures are present in upstream, I'll handle them separately. Failures are like this: # Internal Error (/home/shade/jdk-jdk/src/hotspot/share/gc/shared/space.cpp:595), pid=17562, tid=17564 # assert(is_aligned(obj) && is_aligned(new_top)) failed: checking alignment --------------- T H R E A D --------------- Current thread (0xf5815800): JavaThread "Unknown thread" [_thread_in_vm, id=17564, stack(0xf59c3000,0xf5a14000)] Stack: [0xf59c3000,0xf5a14000], sp=0xf5a12a80, free space=318k Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0x157c991] VMError::report_and_die(int, char const*, char const*, char*, Thread*, unsigned char*, void*, void*, char const*, int, unsigned int)+0x241 V [libjvm.so+0x157d762] VMError::report_and_die(Thread*, void*, char const*, int, char const*, char const*, char*)+0x32 V [libjvm.so+0x951471] report_vm_error(char const*, int, char const*, char const*, ...)+0x81 V [libjvm.so+0x138125f] ContiguousSpace::par_allocate(unsigned int)+0x7f V [libjvm.so+0xa615ac] EpsilonHeap::allocate_work(unsigned int)+0x1c V [libjvm.so+0xa61bde] EpsilonHeap::allocate_new_tlab(unsigned int, unsigned int, unsigned int*)+0x1be V [libjvm.so+0x1081935] MemAllocator::allocate_inside_tlab_slow(MemAllocator::Allocation&) const+0x4a5 V [libjvm.so+0x1082509] MemAllocator::allocate() const+0x129 V [libjvm.so+0x84c485] CollectedHeap::class_allocate(Klass*, int, Thread*)+0x35 V [libjvm.so+0xc54916] InstanceMirrorKlass::allocate_instance(Klass*, Thread*)+0x96 V [libjvm.so+0xca514c] java_lang_Class::create_mirror(Klass*, Handle, Handle, Handle, Thread*)+0x14c V [libjvm.so+0xca5b55] java_lang_Class::fixup_mirror(Klass*, Thread*)+0x55 V [libjvm.so+0x14fc0e7] Universe::fixup_mirrors(Thread*)+0xc7 V [libjvm.so+0x144e8d5] SystemDictionary::resolve_preloaded_classes(Thread*)+0x125 V [libjvm.so+0x144ecba] SystemDictionary::initialize(Thread*)+0x21a V [libjvm.so+0x1503ea3] Universe::genesis(Thread*)+0x363 V [libjvm.so+0x15048f5] universe2_init()+0x25 V [libjvm.so+0xc3c038] init_globals()+0xa8 V [libjvm.so+0x14bb310] Threads::create_vm(JavaVMInitArgs*, bool*)+0x2c0 V [libjvm.so+0xdb4545] JNI_CreateJavaVM+0x95 C [libjli.so+0x3806] JavaMain+0x86 C [libpthread.so.0+0x627a] start_thread+0xda -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed Oct 10 11:36:36 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 10 Oct 2018 13:36:36 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <8243705E-5843-4907-A449-DFE588D1E564@redhat.com> Message-ID: <49ac2e6d-830a-83e0-ca1b-e22abdbcacae@redhat.com> Are you sure those asserts are present without the patch? Because it does look like it might be related to my changes... Roman > On 10/10/2018 11:43 AM, Roman Kennke wrote: >> Can you do me a favour and test this on a 32bits build? I suspect that the alignment stuff is >> entirely different there... > x86_32 build is fine. x86_32 tier1_gc returns with lots Epsilon test failures, but I think those > failures are present in upstream, I'll handle them separately. Failures are like this: > > # Internal Error (/home/shade/jdk-jdk/src/hotspot/share/gc/shared/space.cpp:595), pid=17562, tid=17564 > # assert(is_aligned(obj) && is_aligned(new_top)) failed: checking alignment > > --------------- T H R E A D --------------- > > Current thread (0xf5815800): JavaThread "Unknown thread" [_thread_in_vm, id=17564, > stack(0xf59c3000,0xf5a14000)] > > Stack: [0xf59c3000,0xf5a14000], sp=0xf5a12a80, free space=318k > Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native > code) > V [libjvm.so+0x157c991] VMError::report_and_die(int, char const*, char const*, char*, Thread*, > unsigned char*, void*, void*, char const*, int, unsigned int)+0x241 > V [libjvm.so+0x157d762] VMError::report_and_die(Thread*, void*, char const*, int, char const*, > char const*, char*)+0x32 > V [libjvm.so+0x951471] report_vm_error(char const*, int, char const*, char const*, ...)+0x81 > V [libjvm.so+0x138125f] ContiguousSpace::par_allocate(unsigned int)+0x7f > V [libjvm.so+0xa615ac] EpsilonHeap::allocate_work(unsigned int)+0x1c > V [libjvm.so+0xa61bde] EpsilonHeap::allocate_new_tlab(unsigned int, unsigned int, unsigned int*)+0x1be > V [libjvm.so+0x1081935] MemAllocator::allocate_inside_tlab_slow(MemAllocator::Allocation&) const+0x4a5 > V [libjvm.so+0x1082509] MemAllocator::allocate() const+0x129 > V [libjvm.so+0x84c485] CollectedHeap::class_allocate(Klass*, int, Thread*)+0x35 > V [libjvm.so+0xc54916] InstanceMirrorKlass::allocate_instance(Klass*, Thread*)+0x96 > V [libjvm.so+0xca514c] java_lang_Class::create_mirror(Klass*, Handle, Handle, Handle, Thread*)+0x14c > V [libjvm.so+0xca5b55] java_lang_Class::fixup_mirror(Klass*, Thread*)+0x55 > V [libjvm.so+0x14fc0e7] Universe::fixup_mirrors(Thread*)+0xc7 > V [libjvm.so+0x144e8d5] SystemDictionary::resolve_preloaded_classes(Thread*)+0x125 > V [libjvm.so+0x144ecba] SystemDictionary::initialize(Thread*)+0x21a > V [libjvm.so+0x1503ea3] Universe::genesis(Thread*)+0x363 > V [libjvm.so+0x15048f5] universe2_init()+0x25 > V [libjvm.so+0xc3c038] init_globals()+0xa8 > V [libjvm.so+0x14bb310] Threads::create_vm(JavaVMInitArgs*, bool*)+0x2c0 > V [libjvm.so+0xdb4545] JNI_CreateJavaVM+0x95 > C [libjli.so+0x3806] JavaMain+0x86 > C [libpthread.so.0+0x627a] start_thread+0xda > > > > -Aleksey > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Wed Oct 10 11:41:49 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 10 Oct 2018 13:41:49 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: <49ac2e6d-830a-83e0-ca1b-e22abdbcacae@redhat.com> References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <8243705E-5843-4907-A449-DFE588D1E564@redhat.com> <49ac2e6d-830a-83e0-ca1b-e22abdbcacae@redhat.com> Message-ID: On 10/10/2018 01:36 PM, Roman Kennke wrote: > Are you sure those asserts are present without the patch? Because it > does look like it might be related to my changes... Yes, I have tested without the patch too, same asserts. They can hide the failures introduced by your patch, so we can fix Epsilon first. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed Oct 10 13:21:05 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 10 Oct 2018 15:21:05 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> Message-ID: <2aa7c91e-9b1e-3129-cd7a-d1fa50da422e@redhat.com> Here's the updated webrev with name changed: Incremental: http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.01.diff/ Full: http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.01/ Better? Let's wait for the Epsilon test fix... Roman > On 10/09/2018 10:16 PM, Roman Kennke wrote: >> virtual size_t min_dummy_object_word_size() const; > > I think the proper nomenclature would be min_filler_object_size_words()? We can even drop "words" > from here. > >> Webrev: >> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.00/ > > That looks much cleaner than PLAB/TLAB hacks! Looks good to me. > > -Aleksey > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Wed Oct 10 14:43:47 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 10 Oct 2018 16:43:47 +0200 Subject: RFR (XS) 8212005: Epsilon elastic TLAB sizing may cause misalignment Message-ID: <6be8a6b8-6cd1-1685-d2b0-60fece602451@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8212005 I thought we handle it properly, but we are not. This causes tier1_gc failures in x86_32 now. Fix: http://cr.openjdk.java.net/~shade/8212005/webrev.01/ Testing: Linux {x86_64, x86_32} gc/epsilon, jdk-submit (running) Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed Oct 10 14:45:50 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 10 Oct 2018 16:45:50 +0200 Subject: RFR (XS) 8212005: Epsilon elastic TLAB sizing may cause misalignment In-Reply-To: <6be8a6b8-6cd1-1685-d2b0-60fece602451@redhat.com> References: <6be8a6b8-6cd1-1685-d2b0-60fece602451@redhat.com> Message-ID: Looks sensible! Thanks, Roman > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212005 > > I thought we handle it properly, but we are not. This causes tier1_gc failures in x86_32 now. > > Fix: > http://cr.openjdk.java.net/~shade/8212005/webrev.01/ > > Testing: Linux {x86_64, x86_32} gc/epsilon, jdk-submit (running) > > Thanks, > -Aleksey > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Wed Oct 10 15:36:29 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 10 Oct 2018 17:36:29 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: <2aa7c91e-9b1e-3129-cd7a-d1fa50da422e@redhat.com> References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <2aa7c91e-9b1e-3129-cd7a-d1fa50da422e@redhat.com> Message-ID: <07b75752-0430-68d0-a0b3-5bdf8a4b1d1d@redhat.com> On 10/10/2018 03:21 PM, Roman Kennke wrote: > Incremental: > http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.01.diff/ > Full: > http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.01/ I think you were right with "dummy" name symmetry, but I have not strong opinion either. In plab.cpp, this header is not needed anymore? 30 #include "oops/arrayOop.hpp" ...and maybe not even this one? 31 #include "oops/oop.inline.hpp" > Let's wait for the Epsilon test fix... FWIW, the running with candidate fix for JDK-8212005 applied passes x86_32 tier1_gc. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From igor.ignatyev at oracle.com Wed Oct 10 18:33:22 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 10 Oct 2018 11:33:22 -0700 Subject: RFR(S) : 8157728 : Covert GCTimer_test to GTest In-Reply-To: References: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> Message-ID: <32170658-7540-402A-ABB7-93C25E1D0A0F@oracle.com> Hi David, thanks for spotting the typo, I've fixed the commit message. still looking for a Reviewer though. Cheers, -- Igor > On Oct 9, 2018, at 6:49 PM, David Holmes wrote: > > Hi Igor, > > Not a review - I fixed the typo in the bug synopsis: Covert -> Convert :) > > Please ensure you commit with corrected synopsis. > > Thanks, > David > > On 10/10/2018 3:23 AM, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html >>> 450 lines changed: 238 ins; 211 del; 1 mod; >> Hi all, >> could you please review this small (and hopefully trivial) patch which converts internal GCTimer_test to gtest? >> webrev: http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html >> JBS: https://bugs.openjdk.java.net/browse/JDK-8157728 >> testing: >> - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants >> - build w/ precompiled-headers enabled and disabled >> PS the patch has been originally created by Kirill Zh, but hasn't been sent out for official review >> Thanks, >> -- Igor >> From jcbeyler at google.com Wed Oct 10 20:15:25 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 10 Oct 2018 13:15:25 -0700 Subject: RFR (XS) 8212025: Remove collector_present variable from ThreadHeapSampler Message-ID: Hi all, Here is a tiny webrev to remove the _collectors_present member that is now unused. I forgot to remove it when I did the fix for JDK-8211950. Webrev: http://cr.openjdk.java.net/~jcbeyler/8212025/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8212025 My apologies, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From rkennke at redhat.com Wed Oct 10 21:08:41 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 10 Oct 2018 23:08:41 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: <07b75752-0430-68d0-a0b3-5bdf8a4b1d1d@redhat.com> References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <2aa7c91e-9b1e-3129-cd7a-d1fa50da422e@redhat.com> <07b75752-0430-68d0-a0b3-5bdf8a4b1d1d@redhat.com> Message-ID: <376d3148-6492-ada6-3514-fd0779e6786d@redhat.com> Hi Aleksey, >> Incremental: >> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.01.diff/ >> Full: >> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.01/ > > I think you were right with "dummy" name symmetry, but I have not strong opinion either. Haha, ok. Let's change it back, but without words, ok? > In plab.cpp, this header is not needed anymore? > 30 #include "oops/arrayOop.hpp" Right. Fixed. > ...and maybe not even this one? > 31 #include "oops/oop.inline.hpp" Still needed elsewhere I think. >> Let's wait for the Epsilon test fix... > > FWIW, the running with candidate fix for JDK-8212005 applied passes x86_32 tier1_gc. Ok, cool. Here's the updated webrevs: Incremental: http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.02.diff/ Full: http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.02/ Good now? Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From sangheon.kim at oracle.com Thu Oct 11 06:16:09 2018 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Wed, 10 Oct 2018 23:16:09 -0700 Subject: RFR(M): 8204908: NVDIMM for POGC and G1GC - ReserveSpace.cpp changes are mostly eliminated/no collector specific code. In-Reply-To: References: <5df6d080894cfad5e6486a00f28b6ccfc5ca633f.camel@oracle.com> <87e5373f-a2e0-083e-6421-0209f519dca5@oracle.com> <177b91b2-c0ed-d7ad-84ab-ce479f07474a@oracle.com> <0b6a7684-3120-ce80-2069-1a0d163b14ba@oracle.com> Message-ID: <95cec811-1ec4-14ed-fa7f-c51f2bea0471@oracle.com> Hi Kishor, On 10/3/18 6:24 PM, Kharbas, Kishor wrote: > Hi, > I have an update to the webrev which addresses comments from Sangheon during an offline discussion. > > The JEP has been moved to an enhancement, consequently bug id '8204908' has been closed. I have created a new issue for this implementation -https://bugs.openjdk.java.net/browse/JDK-8211424. > Check the comments in main issue -https://bugs.openjdk.java.net/browse/JDK-8202286. > Any suggestion on whether I continue on this thread or start new one. Not only the bug id has changed, but also the design, implementation. Thanks for the explanation. > Full:http://cr.openjdk.java.net/~kkharbas/8211424/webrev-8204908.03 > Incremental :http://cr.openjdk.java.net/~kkharbas/8211424/webrev-8204908.03_to_02 webrev.03 looks okay in general. A few minor nits: src/hotspot/share/gc/parallel/adjoiningGenerationsForHeteroHeap.cpp ?104 _max_old_byte_size(_max_total_size - _min_young_byte_size), ?105 _max_young_byte_size(_max_total_size - _min_old_byte_size), ?106 _max_total_size(max_total_size) { - line 104 and 105 will use uninitialized value of _max_total_size. I couldn't catch it before. One style comment is that adjoiningGenerationsForHeteroHeap.cpp uses both high()/low() and young_vs()/old_vs() which makes me confused. I would prefer not to add young_vs()/old_vs(), but I don't have strong opinion on this. But using only one(high() or young_vs()) instead of mixed use of high() or young_vs() at ctor seems better. > Testing : Passed tier1_gc and tier1_runtime jtret tests. > I added extra options "-XX:+UnlockExperimentalVMOptions -XX:AllocateOldGenAt=/home/Kishor" to each test. There are some tests which I had to exclude since they won't work with this flag. Example: tests in ConcMarkSweepGC which does not support this flag, tests requiring CompressedOops to be enabled, etc. Thanks for testing. Could you also test without AllocateOldGetAt option enabled? As this option will be disabled as a default, modified codes also need verifications. Thanks, Sangheon > Thanks, > Kishor > >> -----Original Message----- >> From: Kharbas, Kishor >> Sent: Wednesday, September 19, 2018 3:57 PM >> To:sangheon.kim at oracle.com; Thomas St?fe >> ; Thomas Schatzl >> >> Cc:hotspot-gc-dev at openjdk.java.net; Hotspot dev runtime > runtime-dev at openjdk.java.net>; Viswanathan, Sandhya >> ; Aundhe, Shirish >> ; Kharbas, Kishor >> Subject: RE: RFR(M): 8204908: NVDIMM for POGC and G1GC - >> ReserveSpace.cpp changes are mostly eliminated/no collector specific code. >> >> Hi, >> I have an small update to the patch to disable UseCompressedOops. >> http://cr.openjdk.java.net/~kkharbas/8204908/webrev-8204908.02/ >> http://cr.openjdk.java.net/~kkharbas/8204908/webrev-8204908.02_to_01/ >> >> Regards, >> Kishor >> >>> -----Original Message----- >>> From: Kharbas, Kishor >>> Sent: Wednesday, September 19, 2018 9:35 AM >>> To: 'sangheon.kim at oracle.com'; Thomas >> St?fe >>> ; Thomas Schatzl >> >>> Cc:hotspot-gc-dev at openjdk.java.net; Hotspot dev runtime >> runtime-dev at openjdk.java.net>; Viswanathan, Sandhya >>> ; Aundhe, Shirish >>> ; Kharbas, Kishor >>> Subject: RE: RFR(M): 8204908: NVDIMM for POGC and G1GC - >>> ReserveSpace.cpp changes are mostly eliminated/no collector specific >> code. >>> Thanks Sangheon, >>> >>> I have uploaded the updated patch at, >>> http://cr.openjdk.java.net/~kkharbas/8204908/webrev-8204908.01/ >>> http://cr.openjdk.java.net/~kkharbas/8204908/webrev- >> 8204908.01_to_00/ >>> I have fixed all the indentation and format related comments, others I >>> have pasted here below with my comments inline. >>> >>> ============================= >>>> src/hotspot/share/gc/parallel/adjoiningGenerations.hpp >>>> - Copyright update >>>> >>>> 62 AdjoiningGenerations(); >>>> - Why we need this ctor? >>>> >>>>> I need this default ctor for constructing >>> adjoiningGenerationsForHeteroHeap, since I don't want to call the non- >>> default constructor of adjoiningGenerations. >>> >>>> ========================= >>>> /src/hotspot/share/gc/parallel/adjoiningVirtualSpaces.hpp >>>> - Copyright update >>>> >>>> 88 virtual PSVirtualSpace* high() { return _high; } >>>> 89 virtual PSVirtualSpace* low() { return _low; } >>>> - Those methods don't need 'virtual' specifier as high()/low() >>>> methods are only used at ctor of AdjoiningGenerations. The other >>>> calling site is ctor of AdjoiningGenerationsForHeteroHeap but it is >>>> used another type of >>>> high()/low() which returns _young_vs or _old_vs. >>>> >>>>> I feel overriding these methods is a good idea from design >>>>> standpoint; >>> code changes in future would take benefit of this. >>> >>> ======================== >>>> src/hotspot/share/gc/parallel/adjoiningGenerationsForHeteroHeap.hpp >>>> 45 PSVirtualSpace* _young_vs; >>>> 46 PSVirtualSpace* _old_vs; >>>> - Can't we use 'AdjoiningVirtualSpaces::_low' and '_high' instead? >>>> If it is not the case, adding high(),low() may result in confusion >>>> between >>>> AdjoiningVirtualSpaces::high() and low(). So use other name for >>>> current HeteroVirtualSpaces::high()/low(). >>>> >>>>> AdjoiningVirtualSpaces contains two adjacent virtual spaces in the >>>>> same >>> reserved space and separated by a boundary. That?s why the name 'high' >>> and 'low'. >>>>> The class I added - HeteroVirtualSpaces, are not adjacent and do >>>>> not >>> share same reserved space. So I have names them '_young_vs' and >> '_old_vs'. >>> But from outside, i.e, users of VirtualSpaces, they call high() and >>> low() to access these spaces. So I have not changed the function names. >>> >>>> -----Original Message----- >>>> From:sangheon.kim at oracle.com [mailto:sangheon.kim at oracle.com] >>>> Sent: Monday, September 17, 2018 11:32 AM >>>> To: Kharbas, Kishor; Thomas St?fe >>>> ; Thomas Schatzl >>> >>>> Cc:hotspot-gc-dev at openjdk.java.net; Hotspot dev runtime >>> runtime-dev at openjdk.java.net>; Viswanathan, Sandhya >>>> ; Aundhe, Shirish >>>> >>>> Subject: Re: RFR(M): 8204908: NVDIMM for POGC and G1GC - >>>> ReserveSpace.cpp changes are mostly eliminated/no collector specific >>> code. >>>> Hi Kishor, >>>> >>>> >>>> On 9/4/18 10:41 PM, Kharbas, Kishor wrote: >>>>> Hi, >>>>> I have uploaded implementation for parallel scavenge at >>>>> http://cr.openjdk.java.net/~kkharbas/8204908/webrev-8204908.00 >>>>> Majority of the implementation is handled in two new files - >>>> adjoiningGenerationsForHeteroHeap.cpp and >>> psFileBackedVirtualspace.cpp. >>>> Would love to hear your feedback and suggestions. >>>> Sorry for late review. >>>> >>>> ---------------- >>>> General comments. >>>> >>>> 1. Looks like this patch is not based on latest repository, as it >>>> fails with - Wreorder issue. >>>> >>>> 2. I see many wrong alignment location of having only 2 spaces after >>>> new line that is continued. >>>> e.g. >>>> a->b(c, >>>> __d, xxxx); // 2 spaces >>>> this should be >>>> a->b(c, >>>> _____d, xxx); // 'd' should locate under 'c' as those are in the same >> context. >>>> 3. I see assertion failures with below options combinations. There >>>> could be more... Please run jtreg tests before posting webrev. >>>> -XX:+UseLargePages -XX:+UseSHM -version -XX:+UseLargePages >>>> -XX:+UseNUMA -version -XX:+UseLargePages - XX:AllocateHeapAt=. >>>> -version >>>> >>>> ========================= >>>> src/hotspot/share/gc/parallel/adjoiningGenerations.cpp >>>> - Copyright update >>>> >>>> 43?? _virtual_spaces(new AdjoiningVirtualSpaces(old_young_rs, >>>> policy->min_old_size(), >>>> 44?????????????????? policy->min_young_size(), alignment) ) { >>>> - Wrong alignment? >>>> >>>> >>>> ========================= >>>> src/hotspot/share/gc/parallel/adjoiningGenerations.hpp >>>> - Copyright update >>>> >>>> 62?? AdjoiningGenerations(); >>>> - Why we need this ctor? >>>> >>>> >>>> ========================= >>>> /src/hotspot/share/gc/parallel/adjoiningVirtualSpaces.hpp >>>> - Copyright update >>>> >>>> ? 88?? virtual PSVirtualSpace* high() { return _high; } >>>> ? 89?? virtual PSVirtualSpace* low()? { return _low; } >>>> - Those methods don't need 'virtual' specifier as high()/low() >>>> methods are only used at ctor of AdjoiningGenerations. The other >>>> calling site is ctor of AdjoiningGenerationsForHeteroHeap but it is >>>> used another type of >>>> high()/low() which returns _young_vs or _old_vs. >>>> >>>> >>>> ========================= >>>> src/hotspot/share/gc/parallel/parallelScavengeHeap.cpp >>>> - no comments. >>>> >>>> >>>> ========================= >>>> src/hotspot/share/gc/parallel/psOldGen.cpp >>>> ========================= >>>> src/hotspot/share/gc/parallel/psOldGen.hpp >>>> ? 32 #include "gc/parallel/psFileBackedVirtualspace.hpp" >>>> - Include this header file at cpp seems better rather than hpp. >>>> >>>> ========================= >>>> src/hotspot/share/gc/parallel/adjoiningGenerationsForHeteroHeap.cpp >>>> ?? 1 /* >>>> ?? 2 * Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved. >>>> ?? 3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE >>> HEADER. >>>> - Wrong alignment. The second/below star should locate under first >>>> line >>> star. >>>> - Similarily there are more wrong alignment locations. >>>> ? . Line 49, 50 >>>> ? . 52, 53 >>>> ? . 54, 55 >>>> ? . 64, 65 >>>> ? . 67, 68 69 70 >>>> ? . 72, 73 74 75 76 >>>> ? . 79, 80 >>>> ? . 81, 82 >>>> ? . 85, 86 >>>> ? . 87, 88 >>>> ? . 106, 107 108 109 >>>> ? . 114, 115 >>>> ? . 166, 167 168 >>>> ? . 178, 179 180 >>>> ? . 186, 187 188 >>>> ? . 218, 219 220 >>>> ? . 230, 231 232 >>>> ? . 239, 240 241 >>>> >>>> >>>> ? 59?? // Initialize the virtual spaces.? Then pass the >>>> ? 60?? // a virtual to each generation for initialization of the >>>> - Then pass 'the a' virtual to each generation. 'the a'? >>>> >>>> ? 64?? (static_cast >>>> (_virtual_spaces))->initialize(max_old_byte_si >>>> ze >>>> , >>>> init_old_byte_size, >>>> ? 65???? init_young_byte_size); >>>> - Just making 'initilize' method as 'virtual' seems better. >>>> >>>> ? 39 >>>> >> AdjoiningGenerationsForHeteroHeap::AdjoiningGenerationsForHeteroHeap( >>>> ReservedSpace >>>> old_young_rs, size_t total_size_limit, >>>> ? 40?? GenerationSizer* policy, size_t alignment) : >>>> _total_size_limit(total_size_limit) { >>>> - Wrong alirnment at line 40. 'Generation' should be under >> 'ReservedSpace' >>>> at line 39. >>>> >>>> ? 49?? _virtual_spaces = new HeteroVirtualSpaces(old_young_rs, >>>> min_old_byte_size,? 70???? (static_cast >>>> (_virtual_spaces))->max_young_size()); >>>> ? 75???? (static_cast >>>> (_virtual_spaces))->max_old_size(), >>>> - Instead assigning at line 49 and then cast to >>>> HeteroVirtualSpaces*, create/initialize HeteroVirtualSpaces, get >>>> max_young/old_size() and the assign to _virtual_spaces would be better >> alternative. >>>> ?109?? _min_young_byte_size(min_yg_byte_size), >>>> _max_total_size(max_total_size) { >>>> ?110?? _max_old_byte_size = _max_total_size - _min_young_byte_size; >>>> ?111?? _max_young_byte_size = _max_total_size - _min_old_byte_size; >>>> - _max_old_byte_size and _max_young_byte_size can move to >>>> initialization list. >>>> >>>> ?117?? // This the reserved space exclusively for old generation. >>>> ?122?? // This the reserved space exclusively for young generation. >>>> - Missing 'is'? i.e. // This *is* the reserved space exclusively for >>>> old generation. >>>> >>>> ?131???? vm_exit_during_initialization("Could not reserve enough space >> for " >>>> ?132?????? "object heap"); >>>> - 'object heap' can stay same line. Or changing align is necessary. >>>> >>>> ?137???? vm_exit_during_initialization("Could not reserve enough space >> for " >>>> ?138?????? "object heap"); >>>> - 'object heap' can stay same line. Or changing align is necessary. >>>> >>>> ?152?? if (uncommitted_in_old > 0) { >>>> - This condition seems redundant as uncommitted_in_old is type of >> size_t. >>>> ?159?? if (bytes_needed == 0) { >>>> ?160???? return true; >>>> ?161?? } >>>> - This condition, can move to inside of 'if (uncommitted_in_old > 0)' >>>> condition because if uncommitted_in_old is zero, there's no way >>>> bytes_needed to be zero - bytes_needed is guaranteed not to be zero >>>> from caller site, so safe to move between line 154 and 155. The >>>> condition to return true is 'uncommitted_size == bytes_needed && >>>> success of expand_by()'. >>>> >>>> ?176???? bool ret = _young_vs->shrink_by(shrink_size); >>>> ?177???? assert(ret, "We should be able to shrink young space"); >>>> - I would prefer to add more conditions below if we fails to shrink. >>>> i.e. just assert seems not enough. >>>> >>>> ?189?? _old_vs->expand_by(bytes_to_add_in_old); >>>> - Check the return value of expand_by(). >>>> >>>> ?211?? if (bytes_needed == 0) { >>>> ?212???? return true; >>>> ?213?? } >>>> - Same as 'adjust_boundary_down()' >>>> >>>> ?244?? DEBUG_ONLY(size_t total_size_after = >>>> _young_vs->reserved_size() >>>> + _old_vs->reserved_size()); >>>> ?245?? assert(total_size_after == total_size_before, "should be >>>> equal"); >>>> - Why adjust_boundary_up() doesn't have this kind of check? >>>> >>>> >>>> ========================= >>>> src/hotspot/share/gc/parallel/adjoiningGenerationsForHeteroHeap.hpp >>>> ?? 1 /* >>>> ?? 2 * Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved. >>>> ?? 3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE >>> HEADER. >>>> ?? 4 * >>>> - Wrong alignment. The second/below star should locate under first >>>> line >>> star. >>>> ? 37?? size_t total_size_limit() { >>>> ? 38???? return _total_size_limit; >>>> ? 39?? } >>>> - I don't think we need this but if you prefer to have, add 'const'. >>>> >>>> ? 45???? PSVirtualSpace*??? _young_vs; >>>> ? 46???? PSVirtualSpace*??? _old_vs; >>>> - Can't we use 'AdjoiningVirtualSpaces::_low' and '_high' instead? >>>> If it is not the case, adding high(),low() may result in confusion >>>> between >>>> AdjoiningVirtualSpaces::high() and low(). So use other name for >>>> current HeteroVirtualSpaces::high()/low(). >>>> >>>> ? 55???? HeteroVirtualSpaces(ReservedSpace rs, >>>> ? 56?????? size_t min_old_byte_size, >>>> ? 57?????? size_t min_young_byte_size, size_t max_total_size, >>>> ? 58?????? size_t alignment); >>>> - Wrong alignment. Line 56 ~ 58 should be same as the location of >>>> 'ReservedSpace' at line 55. >>>> >>>> ? 67???? size_t max_young_size() { return _max_young_byte_size; } >>>> ? 68???? size_t max_old_size() { return _max_old_byte_size; } >>>> - 'const'? >>>> >>>> ? 70???? void initialize(size_t initial_old_reserved_size, size_t >>>> init_low_byte_size, >>>> ? 71?????? size_t init_high_byte_size); >>>> - Wrong alignment >>>> >>>> ? 82?? size_t reserved_byte_size(); >>>> - 'const'? >>>> >>>> ========================= >>>> ?src/hotspot/share/gc/parallel/psFileBackedVirtualspace.cpp >>>> ?? 1 /* >>>> ?? 2 * Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved. >>>> - Wrong alignment. The second/below star should locate under first >>>> line >>> star. >>>> ? 36?? if (_fd == -1) { >>>> ? 37???? return; >>>> ? 38?? } >>>> - I prefer to have 'initialize()' method to handle when fails to >>>> create the heap file. Current implementation seems easy to call >>>> additional >>> 'initialize()'. >>>> ? Ctor of PSVirtualSpace doesn't have any failure path(e.g. >>>> allocation), so no further check is needed around line 18:psOldGen.cpp. >>>> But yours is different, so something should be checked. >>>> >>>> ? 33?? _mapping_succeeded = false; >>>> ? 34?? _file_path = path; >>>> ? 46?? _mapping_succeeded = true; >>>> ? 47?? _special = true; >>>> - Can move to initialization list with proper initial value. >>>> >>>> ? 55?? assert(special(), "_special should always be true for >>>> PSFileBackedVirtualSpace"); >>>> ? 66?? assert(special(), "_special should always be true for >>>> PSFileBackedVirtualSpace"); >>>> - For these 2, can we have more specific message? e.g. to include >>>> meaning of expand or shrink. >>>> >>>> >>>> ========================= >>>> ?src/hotspot/share/gc/parallel/psFileBackedVirtualspace.hpp >>>> ?? 1 /* >>>> ?? 2 * Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved. >>>> - Wrong alignment. The second/below star should locate under first >>>> line >>> star. >>>> ? 37?? PSFileBackedVirtualSpace(ReservedSpace rs, const char* >>>> file_path); >>>> ? 38?? bool?? expand_by(size_t bytes); >>>> - Just recommendation to add new line between 37 and 38. >>>> >>>> ? 43 #endif // >>> SHARE_VM_GC_PARALLEL_PSFILEBACKEDVIRTUALSPACE_HPP >>>> - Last line is not terminated with a newline. >>>> >>>> >>>> Thanks, >>>> Sangheon >>>> >>>> >>>>> I will post G1 GC implementation in a few days. >>>>> >>>>> Thanks >>>>> Kishor. >>>>> >>>>>> -----Original Message----- >>>>>> From:sangheon.kim at oracle.com >> [mailto:sangheon.kim at oracle.com] >>>>>> Sent: Thursday, August 30, 2018 4:06 PM >>>>>> To: Kharbas, Kishor; Thomas St?fe >>>>>> ; Thomas Schatzl >>>> >>>>>> Cc:hotspot-gc-dev at openjdk.java.net; Hotspot dev runtime >>>>>> ; Viswanathan, Sandhya >>>>>> ; Aundhe, Shirish >>>>>> >>>>>> Subject: Re: RFR(M): 8204908: NVDIMM for POGC and G1GC - >>>>>> ReserveSpace.cpp changes are mostly eliminated/no collector >>>>>> specific >>>> code. >>>>>> Hi Kishor, >>>>>> >>>>>> On 8/30/18 11:55 AM, Kharbas, Kishor wrote: >>>>>>> Hi Sangheon, >>>>>>> >>>>>>> So far I don?t see a need to do so. I will post my >>>>>>> implementation code >>>>>> today or tomorrow, if we see a need or any simplification by >>>>>> changing classes in VirtualSpace.hpp, we can discuss that. >>>>>> Okay. >>>>>> >>>>>> Thanks, >>>>>> Sangheon >>>>>> >>>>>> >>>>>>> Regards, >>>>>>> -Kishor >>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From:sangheon.kim at oracle.com >>> [mailto:sangheon.kim at oracle.com] >>>>>>>> Sent: Wednesday, August 29, 2018 10:17 AM >>>>>>>> To: Kharbas, Kishor; Thomas St?fe >>>>>>>> ; Thomas Schatzl >>>>>> >>>>>>>> Cc:hotspot-gc-dev at openjdk.java.net; Hotspot dev runtime >>>>>>>> ; Viswanathan, Sandhya >>>>>>>> ; Aundhe, Shirish >>>>>>>> >>>>>>>> Subject: Re: RFR(M): 8204908: NVDIMM for POGC and G1GC - >>>>>>>> ReserveSpace.cpp changes are mostly eliminated/no collector >>>>>>>> specific >>>>>> code. >>>>>>>> Hi Kishor, >>>>>>>> >>>>>>>> On 8/29/18 9:52 AM, Kharbas, Kishor wrote: >>>>>>>>> Hi Sangheon, >>>>>>>>> >>>>>>>>> Thanks for reviewing the design. >>>>>>>>> Since the collectors do not use them for heap memory, I did >>>>>>>>> not have to override VirtualSpace >>>>>>>> Sorry, I meant ReservedSpace and its friends at >>>>>>>> share/memory/virtualspace.hpp. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Sangheon >>>>>>>> >>>>>>>> >>>>>>>>> -Kishor >>>>>>>>>> -----Original Message----- >>>>>>>>>> From:sangheon.kim at oracle.com >>>> [mailto:sangheon.kim at oracle.com] >>>>>>>>>> Sent: Tuesday, August 28, 2018 2:38 PM >>>>>>>>>> To: Kharbas, Kishor; Thomas St?fe >>>>>>>>>> ; Thomas Schatzl >>>>>>>> >>>>>>>>>> Cc:hotspot-gc-dev at openjdk.java.net; Hotspot dev runtime >>>>>>>>>> ; Viswanathan, >> Sandhya >>>>>>>>>> ; Aundhe, Shirish >>>>>>>>>> >>>>>>>>>> Subject: Re: RFR(M): 8204908: NVDIMM for POGC and G1GC - >>>>>>>>>> ReserveSpace.cpp changes are mostly eliminated/no collector >>>>>>>>>> specific >>>>>>>> code. >>>>>>>>>> Hi Kishor, >>>>>>>>>> >>>>>>>>>> On 8/21/18 10:57 AM, Kharbas, Kishor wrote: >>>>>>>>>>> Hi All, >>>>>>>>>>> >>>>>>>>>>> Thank you for your valuable comments and feedback in this >>>>>>>>>>> thread so >>>>>>>> far. >>>>>>>>>>> After taken in all the comments and reading thoroughly >>>>>>>>>>> through the code, I >>>>>>>>>> feel that the complexity added to virtualSpace.cpp is due to >>>>>>>>>> lack of abstraction in VirtualSpace and at GC level. NV-DIMM >>>>>>>>>> size and file paths are being passed all the way to OS calls. >>>>>>>>>>> So I went back to the drawing board and created a high level >>>>>>>>>>> design to remove all the pain points of current implementation. >>>>>>>>>>> I have uploaded the design at >>>>>>>>>>> >> http://cr.openjdk.java.net/~kkharbas/8202286/Design%20for%20JEP%20JDK >>>>>>>>>> - >>>>>>>>>>> 8202286.pdf I would love to hear your feedback and >> suggestions. >>>>>>>>>>> Once we get confidence in the design, I will work on the >>>>>> implementation. >>>>>>>>>> The design looks good to me. >>>>>>>>>> If you are planning to change VirtualSpace at >>>>>>>>>> share/memory/virtualspace.hpp, including it on the design >>>>>>>>>> document would be helpful too. >>>>>>>>>> >>>>>>>>>> Probably folks involved in the previous discussions would say >>>>>>>>>> whether the design well covers their concerns or not. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Sangheon >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> PS: >>>>>>>>>>> 1. Vinay has transitioned to another team in Intel, so he >>>>>>>>>>> won't be able to >>>>>>>>>> continue on this jep. >>>>>>>>>>> 2. If you feel this should be part of JEP discussion thread, >>>>>>>>>>> we can take it >>>>>>>>>> there. >>>>>>>>>>> Thanks and regards, >>>>>>>>>>> Kishor >>>>>>>>>>> >>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>> From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] >>>>>>>>>>>> Sent: Friday, June 22, 2018 9:25 AM >>>>>>>>>>>> To: Thomas Schatzl >>>>>>>>>>>> Cc: Awasthi, Vinay K; Paul Su >>>>>>>>>>>> ;hotspot-gc-dev at openjdk.java.net; >>>> Hotspot >>>>>>>> dev >>>>>>>>>>>> runtime; >>> Viswanathan, >>>>>>>>>> Sandhya >>>>>>>>>>>> ; Aundhe, Shirish >>>>>>>>>>>> ; Kharbas, Kishor >>>>>>>>>>>> >>>>>>>>>>>> Subject: Re: RFR(M): 8204908: NVDIMM for POGC and G1GC - >>>>>>>>>>>> ReserveSpace.cpp changes are mostly eliminated/no collector >>>>>>>>>>>> specific >>>>>>>>>> code. >>>>>>>>>>>> Hi Thomas, >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Jun 22, 2018 at 4:44 PM, Thomas Schatzl >>>>>>>>>>>> wrote: >>>>>>>>>>>>> Hi Thomas, >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, 2018-06-19 at 13:40 +0200, Thomas St?fe wrote: >>>>>>>>>>>>>> Hi Vinay, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, Jun 18, 2018 at 6:47 PM, Awasthi, Vinay K >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> Hi Thomas, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Os::commit_memory calls map_memory_to_file which is >>> same >>>>>> as >>>>>>>>>>>>>>> os::reserve_memory. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I am failing to see why os::reserve_memory can call >>>>>>>>>>>>>>> map_memory_to_file (i.e. tie it to mmap) but >>>> commit_memory >>>>>>>>>> can't... >>>>>>>>>>>>>>> Before this patch, commit_memory never dealt with >>>>>>>>>>>>>>> incrementally committing pages to device so there has to >>>>>>>>>>>>>>> be a way to pass file descriptor and offset. Windows has >>>>>>>>>>>>>>> no such capability to manage incremental commits. All >>>>>>>>>>>>>>> other OSes do and that is why map_memory_to_file is used >>>>>>>>>>>>>>> (which >>>> by >>>>>>>>>>>>>>> the way also works on Windows). >>>>>>>>>>>>>> AIX uses System V shared memory by default, which follows >>>>>>>>>>>>>> a different allocation scheme (semantics more like >>>>>>>>>>>>>> Windows >>>>>>>>>> VirtualAlloc... >>>>>>>>>>>>>> calls). >>>>>>>>>>>>>> >>>>>>>>>>>>>> But my doubts are not limited to that one, see my earlier >>>>>>>>>>>>>> replies and those of others. It really makes sense to >>>>>>>>>>>>>> step back one step and discuss the JEP first. >>>>>>>>>>>>>> >>>>>>>>>>>>> There is already a discussion thread as David mentioned >>>>>>>>>>>>> (http://mail.op >>>>>>>>>>>>> enjdk.java.net/pipermail/hotspot-gc-dev/2018- >>>> May/022092.html) >>>>>>>> that >>>>>>>>>>>>> so far nobody answered to. >>>>>>>>>>>>> >>>>>>>>>>>> Ah, I thought he wanted to have the JEP discussed in the >>>>>>>>>>>> comments section of the JEP itself. >>>>>>>>>>>> >>>>>>>>>>>>> I believe discussion about contents the JEP and the >>>>>>>>>>>>> implementation should be separate. >>>>>>>>>>>>> >>>>>>>>>>>> makes sense. >>>>>>>>>>>> >>>>>>>>>>>>> So far what I gathered from the responses to the >>>>>>>>>>>>> implementation, the proposed idea itself is not the issue >>>>>>>>>>>>> here (allowing the use of NVDIMM memory for parts of the >>>>>>>>>>>>> heap for allowing the use of larger heaps to improve >>>>>>>>>>>>> overall performance; I am not saying that the text doesn't >>>>>>>>>>>>> need a bit more work :) ), but rather how an >>>>>>>>>>>>> implementation of this JEP >>>> should proceed. >>>>>>>>>>>> I have no problem with adding NVDIMM support. I think it is >>>>>>>>>>>> a cool >>>>>>>>>> feature. >>>>>>>>>>>> Also, the awkwardness off the memory management >> abstraction >>>>>> layer >>>>>>>>>>>> in hotspot has always been a sore point with me (I >>>>>>>>>>>> originally implemented the AIX mm layer in os_aix.cpp, and >>>>>>>>>>>> those are painful memories). So, I have a lot of sympathy >>>>>>>>>>>> for Vinays >>>> struggles. >>>>>>>>>>>> Unfortunately not much time atm, but I will respond to your >>> mail. >>>>>>>>>>>>> Let's discuss the non-implementation stuff in that thread. >>>>>>>>>>>>> Vinay or I can repost the proposal email if you do not >>>>>>>>>>>>> have it any more so that answers will be in-thread. >>>>>>>>>>>>> >>>>>>>>>>>> Okay, sounds good. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Thomas >>>>>>>>>>>> >>>>>>>>>>>> (one of us should change his name to make this less >>>>>>>>>>>> confusing >>>>>>>>>>>> :-) >>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Thomas >>>>>>>>>>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From per.liden at oracle.com Thu Oct 11 07:00:54 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 11 Oct 2018 09:00:54 +0200 Subject: RFR (XS) 8212025: Remove collector_present variable from ThreadHeapSampler In-Reply-To: References: Message-ID: Looks good. /Per On 10/10/2018 10:15 PM, JC Beyler wrote: > Hi all, > > Here is a tiny webrev to remove the _collectors_present member that is > now unused. I forgot to remove it when I did the fix for JDK-8211950. > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8212025/webrev.00/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8212025 > > My apologies, > Jc From per.liden at oracle.com Thu Oct 11 07:45:05 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 11 Oct 2018 09:45:05 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: <376d3148-6492-ada6-3514-fd0779e6786d@redhat.com> References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <2aa7c91e-9b1e-3129-cd7a-d1fa50da422e@redhat.com> <07b75752-0430-68d0-a0b3-5bdf8a4b1d1d@redhat.com> <376d3148-6492-ada6-3514-fd0779e6786d@redhat.com> Message-ID: Hi, On 10/10/2018 11:08 PM, Roman Kennke wrote: > Hi Aleksey, > >>> Incremental: >>> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.01.diff/ >>> Full: >>> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.01/ >> >> I think you were right with "dummy" name symmetry, but I have not strong opinion either. > > Haha, ok. Let's change it back, but without words, ok? > >> In plab.cpp, this header is not needed anymore? >> 30 #include "oops/arrayOop.hpp" > > Right. Fixed. > >> ...and maybe not even this one? >> 31 #include "oops/oop.inline.hpp" > > Still needed elsewhere I think. > >>> Let's wait for the Epsilon test fix... >> >> FWIW, the running with candidate fix for JDK-8212005 applied passes x86_32 tier1_gc. > > Ok, cool. Here's the updated webrevs: > Incremental: > http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.02.diff/ > Full: > http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.02/ > > Good now? I'm not sure I understand why we need yet another abstraction for this. I'm thinking the stuff you did in JDK-8211270 should be enough? We already have CollectedHeap::min_fill_size() to answer the question what the min filler size is, so adding a new function doesn't make sense to me. What am I missing? cheers, Per From shade at redhat.com Thu Oct 11 07:46:08 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 11 Oct 2018 09:46:08 +0200 Subject: RFR (XS) 8212005: Epsilon elastic TLAB sizing may cause misalignment In-Reply-To: References: <6be8a6b8-6cd1-1685-d2b0-60fece602451@redhat.com> Message-ID: <68b86d15-64d5-72e8-8042-47140acb062c@redhat.com> Thanks! jdk-submit came back clean. Any other reviews? -Aleksey On 10/10/2018 04:45 PM, Roman Kennke wrote: > Looks sensible! > > Thanks, > Roman > > >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8212005 >> >> I thought we handle it properly, but we are not. This causes tier1_gc failures in x86_32 now. >> >> Fix: >> http://cr.openjdk.java.net/~shade/8212005/webrev.01/ >> >> Testing: Linux {x86_64, x86_32} gc/epsilon, jdk-submit (running) >> >> Thanks, >> -Aleksey >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Thu Oct 11 08:37:43 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 11 Oct 2018 10:37:43 +0200 Subject: RFR (XS) 8212005: Epsilon elastic TLAB sizing may cause misalignment In-Reply-To: <6be8a6b8-6cd1-1685-d2b0-60fece602451@redhat.com> References: <6be8a6b8-6cd1-1685-d2b0-60fece602451@redhat.com> Message-ID: Hi Aleksey, On Wed, 2018-10-10 at 16:43 +0200, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212005 > > I thought we handle it properly, but we are not. This causes tier1_gc > failures in x86_32 now. > > Fix: > http://cr.openjdk.java.net/~shade/8212005/webrev.01/ > > Testing: Linux {x86_64, x86_32} gc/epsilon, jdk-submit (running) looks good. One minor nit: the test seems to be very similar to gc/TestObjectAlignment, maybe it can be changed slightly (e.g. not bailing out immediately on 32 bit systems) for the same purpose. Otoh we may not have Epsilon gc on rotation in our tests, so the test might never be run with Epsilon explicitly. Your call. Thanks, Thomas From thomas.schatzl at oracle.com Thu Oct 11 08:40:34 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 11 Oct 2018 10:40:34 +0200 Subject: RFR (XS) 8212025: Remove collector_present variable from ThreadHeapSampler In-Reply-To: References: Message-ID: <1219dadb859369ccb2862cb552656b15c0b20056.camel@oracle.com> Hi, On Wed, 2018-10-10 at 13:15 -0700, JC Beyler wrote: > Hi all, > > Here is a tiny webrev to remove the _collectors_present member that > is now unused. I forgot to remove it when I did the fix for JDK- > 8211950. > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8212025/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8212025 > > My apologies, > Jc np, we are always happy about code improvements like this :) Ship it. Thomas From shade at redhat.com Thu Oct 11 08:40:32 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 11 Oct 2018 10:40:32 +0200 Subject: RFR (XS) 8212005: Epsilon elastic TLAB sizing may cause misalignment In-Reply-To: References: <6be8a6b8-6cd1-1685-d2b0-60fece602451@redhat.com> Message-ID: <3242c70d-7259-2ba9-450a-9a43f4ccb4a4@redhat.com> On 10/11/2018 10:37 AM, Thomas Schatzl wrote: >> Fix: >> http://cr.openjdk.java.net/~shade/8212005/webrev.01/ > > looks good. Thanks! > One minor nit: the test seems to be very similar to > gc/TestObjectAlignment, maybe it can be changed slightly (e.g. not > bailing out immediately on 32 bit systems) for the same purpose. > > Otoh we may not have Epsilon gc on rotation in our tests, so the test > might never be run with Epsilon explicitly. Yeah, I'd prefer to keep this targeted/regression test for Epsilon explicitly. It should have been there from day 1, but I lost it somehow. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Thu Oct 11 08:49:31 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 11 Oct 2018 10:49:31 +0200 Subject: RFR: 8211447: Replace oop_pc_update_pointers with oop_iterate and closure In-Reply-To: <474b84b5-4b6b-fa6b-bccf-2d8846bcb51c@oracle.com> References: <401fe6c9-dd64-5c50-4faa-2f51925546a7@oracle.com> <8804f31ec81f5e27457c560ef908d78799d3ec6d.camel@oracle.com> <474b84b5-4b6b-fa6b-bccf-2d8846bcb51c@oracle.com> Message-ID: Hi, On Tue, 2018-10-09 at 16:26 +0200, Leo Korinth wrote: > Hi! > > Here are new incremental and full webrevs that tries to improve the > code according to suggestions from Stefan and Thomas. > > I did *not* change the name to pcClosure.inline.hpp as I think that > was a mistake (see my last e-mail). Okay, thanks for pointing this out. > incremental: > http://cr.openjdk.java.net/~lkorinth/8211447/00-01/ > full: > http://cr.openjdk.java.net/~lkorinth/8211447/01/ psClosure.inline.hpp:101: extra space at the start of the "public" visibility declaration. I do not need to see a re-review for that. Ship it! :) Thomas From thomas.schatzl at oracle.com Thu Oct 11 08:50:32 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 11 Oct 2018 10:50:32 +0200 Subject: RFR: 8211446: Replace oop_pc_follow_contents with oop_iterate and closure In-Reply-To: <082ab861-5370-f7b6-f28c-0e60b7bae412@oracle.com> References: <082ab861-5370-f7b6-f28c-0e60b7bae412@oracle.com> Message-ID: <9a714bd9cdcf24bc36b0ae7d585bbb160554646c.camel@oracle.com> Hi Leo, On Tue, 2018-10-09 at 15:52 +0200, Leo Korinth wrote: > Hi! > > Here are new incremental and full webrevs that tries to improve the > code according to suggestions from Stefan and Thomas. > > incremental: > http://cr.openjdk.java.net/~lkorinth/8211446/00-01/ > full: > http://cr.openjdk.java.net/~lkorinth/8211446/01/ > looks good. Thomas From leo.korinth at oracle.com Thu Oct 11 09:21:02 2018 From: leo.korinth at oracle.com (Leo Korinth) Date: Thu, 11 Oct 2018 11:21:02 +0200 Subject: RFR: 8211447: Replace oop_pc_update_pointers with oop_iterate and closure In-Reply-To: References: <401fe6c9-dd64-5c50-4faa-2f51925546a7@oracle.com> <8804f31ec81f5e27457c560ef908d78799d3ec6d.camel@oracle.com> <474b84b5-4b6b-fa6b-bccf-2d8846bcb51c@oracle.com> Message-ID: <1a335a99-8e4a-7836-fa80-c7b3c05b79f9@oracle.com> Thank you Stefan and Thomas for your reviews! /Leo On 11/10/2018 10:49, Thomas Schatzl wrote: > Hi, > > On Tue, 2018-10-09 at 16:26 +0200, Leo Korinth wrote: >> Hi! >> >> Here are new incremental and full webrevs that tries to improve the >> code according to suggestions from Stefan and Thomas. >> >> I did *not* change the name to pcClosure.inline.hpp as I think that >> was a mistake (see my last e-mail). > > Okay, thanks for pointing this out. > >> incremental: >> http://cr.openjdk.java.net/~lkorinth/8211447/00-01/ >> full: >> http://cr.openjdk.java.net/~lkorinth/8211447/01/ > > psClosure.inline.hpp:101: extra space at the start of the "public" > visibility declaration. > > I do not need to see a re-review for that. Ship it! :) > > Thomas > > From leo.korinth at oracle.com Thu Oct 11 09:22:06 2018 From: leo.korinth at oracle.com (Leo Korinth) Date: Thu, 11 Oct 2018 11:22:06 +0200 Subject: RFR: 8211446: Replace oop_pc_follow_contents with oop_iterate and closure In-Reply-To: <9a714bd9cdcf24bc36b0ae7d585bbb160554646c.camel@oracle.com> References: <082ab861-5370-f7b6-f28c-0e60b7bae412@oracle.com> <9a714bd9cdcf24bc36b0ae7d585bbb160554646c.camel@oracle.com> Message-ID: Thank you Stefan and Thomas for your reviews! /Leo On 11/10/2018 10:50, Thomas Schatzl wrote: > Hi Leo, > > On Tue, 2018-10-09 at 15:52 +0200, Leo Korinth wrote: >> Hi! >> >> Here are new incremental and full webrevs that tries to improve the >> code according to suggestions from Stefan and Thomas. >> >> incremental: >> http://cr.openjdk.java.net/~lkorinth/8211446/00-01/ >> full: >> http://cr.openjdk.java.net/~lkorinth/8211446/01/ >> > > looks good. > > Thomas > > From rkennke at redhat.com Thu Oct 11 09:24:50 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 11 Oct 2018 11:24:50 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <2aa7c91e-9b1e-3129-cd7a-d1fa50da422e@redhat.com> <07b75752-0430-68d0-a0b3-5bdf8a4b1d1d@redhat.com> <376d3148-6492-ada6-3514-fd0779e6786d@redhat.com> Message-ID: Hi Per, > Hi, > > On 10/10/2018 11:08 PM, Roman Kennke wrote: >> Hi Aleksey, >> >>>> Incremental: >>>> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.01.diff/ >>>> Full: >>>> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.01/ >>> >>> I think you were right with "dummy" name symmetry, but I have not >>> strong opinion either. >> >> Haha, ok. Let's change it back, but without words, ok? >> >>> In plab.cpp, this header is not needed anymore? >>> ?? 30 #include "oops/arrayOop.hpp" >> >> Right. Fixed. >> >>> ...and maybe not even this one? >>> ?? 31 #include "oops/oop.inline.hpp" >> >> Still needed elsewhere I think. >> >>>> Let's wait for the Epsilon test fix... >>> >>> FWIW, the running with candidate fix for JDK-8212005 applied passes >>> x86_32 tier1_gc. >> >> Ok, cool. Here's the updated webrevs: >> Incremental: >> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.02.diff/ >> Full: >> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.02/ >> >> Good now? > > I'm not sure I understand why we need yet another abstraction for this. > I'm thinking the stuff you did in JDK-8211270 should be enough? We > already have CollectedHeap::min_fill_size() to answer the question what > the min filler size is, so adding a new function doesn't make sense to > me. What am I missing? min_fill_size() is static and cannot be overridden by GC. obj_size() from JDK-8211270 could be used if we'd pass a specific object. I guess we could make a better abstraction like what Erik suggested in the other thread: virtual size_t cell_size(size_t obj_size) { return obj_size; } and this could be called by JVMT and whitebox (JDK-8211270) with obj->size(), and by min_fill_size() with the static minimum obj size, and the GC would translate it to the cell size. The obj_size() from JDK-8211270 would then be removed and we'd have a single abstraction for this. Want me to make that change? Do you agree with the rest of this change? Because there's some significant changes how TLAB and PLAB calculate their alloc reserves in there too. Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Thu Oct 11 09:48:25 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 11 Oct 2018 11:48:25 +0200 Subject: RFR (XS) 8212005: Epsilon elastic TLAB sizing may cause misalignment In-Reply-To: <6be8a6b8-6cd1-1685-d2b0-60fece602451@redhat.com> References: <6be8a6b8-6cd1-1685-d2b0-60fece602451@redhat.com> Message-ID: Hi, On 10/10/2018 04:43 PM, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212005 > > I thought we handle it properly, but we are not. This causes tier1_gc failures in x86_32 now. > > Fix: > http://cr.openjdk.java.net/~shade/8212005/webrev.01/ I think you want to make sure that _max_tlab_size also ends up being aligned to MinObjectAlignment here: 50 _max_tlab_size = MIN2(CollectedHeap::max_tlab_size(), EpsilonMaxTLABSize / HeapWordSize); Otherwise you risk allocating a TLAB larger than _max_tlab_size here: 214 // Always honor boundaries 215 size = MAX2(min_size, MIN2(_max_tlab_size, size)); 216 217 // Always honor alignment 218 size = align_up(size, MinObjAlignment); Right? cheers, Per > > Testing: Linux {x86_64, x86_32} gc/epsilon, jdk-submit (running) > > Thanks, > -Aleksey > From shade at redhat.com Thu Oct 11 10:00:33 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 11 Oct 2018 12:00:33 +0200 Subject: RFR (XS) 8212005: Epsilon elastic TLAB sizing may cause misalignment In-Reply-To: References: <6be8a6b8-6cd1-1685-d2b0-60fece602451@redhat.com> Message-ID: <5e81caa3-2dec-2b5b-c352-92102c56fa16@redhat.com> On 10/11/2018 11:48 AM, Per Liden wrote: > I think you want to make sure that _max_tlab_size also ends up being aligned to MinObjectAlignment > here: > > ? 50?? _max_tlab_size = MIN2(CollectedHeap::max_tlab_size(), EpsilonMaxTLABSize / HeapWordSize); > > Otherwise you risk allocating a TLAB larger than _max_tlab_size here: > > ?214?? // Always honor boundaries > ?215?? size = MAX2(min_size, MIN2(_max_tlab_size, size)); > ?216 > ?217?? // Always honor alignment > ?218?? size = align_up(size, MinObjAlignment); > > Right? Right. Or this means that EpsilonMaxTLABSize has to be additionally checked for alignment before use. If EpsilonMaxTLABSize is proper, then size is always aligned too, even if it gets capped by _max_tlab_size, right? Unfortunately, I have already pushed the fix, and can follow up on this in a separate RFE. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Thu Oct 11 10:03:39 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 11 Oct 2018 12:03:39 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <2aa7c91e-9b1e-3129-cd7a-d1fa50da422e@redhat.com> <07b75752-0430-68d0-a0b3-5bdf8a4b1d1d@redhat.com> <376d3148-6492-ada6-3514-fd0779e6786d@redhat.com> Message-ID: <465b7cf0-7783-12b0-b121-a4c319328da9@redhat.com> Like this: Incremental: http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.03.diff/ Full: http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.03/ Some notes: - I was thinking to change min_fill_size() to use cell_size() instead, but it's used elsewhere as filler-object-size (as opposed to filler-cell-size) - I was also thinking to have min_dummy_object_size() use min_fill_size(), but min_fill_size() returns an aligned size, which is not what we want there. - We could keep non-virtual obj_size(oop obj) (or rename it to cell_size(oop obj) ) and make it use cell_size() instead as little helper for JVMTI and whitebox, which now do the same thing 2x. WDYT? How do you like this? Roman Am 11.10.18 um 09:45 schrieb Per Liden: > Hi, > > On 10/10/2018 11:08 PM, Roman Kennke wrote: >> Hi Aleksey, >> >>>> Incremental: >>>> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.01.diff/ >>>> Full: >>>> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.01/ >>> >>> I think you were right with "dummy" name symmetry, but I have not >>> strong opinion either. >> >> Haha, ok. Let's change it back, but without words, ok? >> >>> In plab.cpp, this header is not needed anymore? >>> ?? 30 #include "oops/arrayOop.hpp" >> >> Right. Fixed. >> >>> ...and maybe not even this one? >>> ?? 31 #include "oops/oop.inline.hpp" >> >> Still needed elsewhere I think. >> >>>> Let's wait for the Epsilon test fix... >>> >>> FWIW, the running with candidate fix for JDK-8212005 applied passes >>> x86_32 tier1_gc. >> >> Ok, cool. Here's the updated webrevs: >> Incremental: >> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.02.diff/ >> Full: >> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.02/ >> >> Good now? > > I'm not sure I understand why we need yet another abstraction for this. > I'm thinking the stuff you did in JDK-8211270 should be enough? We > already have CollectedHeap::min_fill_size() to answer the question what > the min filler size is, so adding a new function doesn't make sense to > me. What am I missing? > > cheers, > Per -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Thu Oct 11 10:03:58 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 11 Oct 2018 12:03:58 +0200 Subject: RFR (XS) 8212005: Epsilon elastic TLAB sizing may cause misalignment In-Reply-To: <5e81caa3-2dec-2b5b-c352-92102c56fa16@redhat.com> References: <6be8a6b8-6cd1-1685-d2b0-60fece602451@redhat.com> <5e81caa3-2dec-2b5b-c352-92102c56fa16@redhat.com> Message-ID: On 10/11/2018 12:00 PM, Aleksey Shipilev wrote: > On 10/11/2018 11:48 AM, Per Liden wrote: >> I think you want to make sure that _max_tlab_size also ends up being aligned to MinObjectAlignment >> here: >> >> ? 50?? _max_tlab_size = MIN2(CollectedHeap::max_tlab_size(), EpsilonMaxTLABSize / HeapWordSize); >> >> Otherwise you risk allocating a TLAB larger than _max_tlab_size here: >> >> ?214?? // Always honor boundaries >> ?215?? size = MAX2(min_size, MIN2(_max_tlab_size, size)); >> ?216 >> ?217?? // Always honor alignment >> ?218?? size = align_up(size, MinObjAlignment); >> >> Right? > > Right. Or this means that EpsilonMaxTLABSize has to be additionally checked for alignment before > use. If EpsilonMaxTLABSize is proper, then size is always aligned too, even if it gets capped by > _max_tlab_size, right? Right. > > Unfortunately, I have already pushed the fix, and can follow up on this in a separate RFE. Ok. /Per From per.liden at oracle.com Thu Oct 11 10:34:46 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 11 Oct 2018 12:34:46 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <2aa7c91e-9b1e-3129-cd7a-d1fa50da422e@redhat.com> <07b75752-0430-68d0-a0b3-5bdf8a4b1d1d@redhat.com> <376d3148-6492-ada6-3514-fd0779e6786d@redhat.com> Message-ID: <9467830c-0983-fc12-7f3a-1446e5b85362@oracle.com> Hi, On 10/11/2018 11:24 AM, Roman Kennke wrote: [...] >>> Ok, cool. Here's the updated webrevs: >>> Incremental: >>> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.02.diff/ >>> Full: >>> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.02/ >>> >>> Good now? >> >> I'm not sure I understand why we need yet another abstraction for this. >> I'm thinking the stuff you did in JDK-8211270 should be enough? We >> already have CollectedHeap::min_fill_size() to answer the question what >> the min filler size is, so adding a new function doesn't make sense to >> me. What am I missing? I see now that I missed the second half of the JDK-8211270 mail thread. > > min_fill_size() is static and cannot be overridden by GC. obj_size() > from JDK-8211270 could be used if we'd pass a specific object. I guess > we could make a better abstraction like what Erik suggested in the other > thread: > > virtual size_t cell_size(size_t obj_size) { return obj_size; } I don't think we should introduced the concept of cell in this patch. That's a much larger concept/change that needs to be carefully thought out first. For example, it's not obvious to me if you consider the Brooks pointer to be an extension of the object header, and hence part of the object and included in obj->size(), of if it's a separate thing. If it's a separate thing, then a cell concept might prove useful. But if it's a separate thing, then JDK-8211270 now incorrectly made JvmtiEnv::GetObjectSize() and WB_GetObjectSize() return the cell size instead of the object size. I assume you thought these things through? What's the story for Shenandoah here? > > and this could be called by JVMT and whitebox (JDK-8211270) with > obj->size(), and by min_fill_size() with the static minimum obj size, > and the GC would translate it to the cell size. The obj_size() from > JDK-8211270 would then be removed and we'd have a single abstraction for > this. Want me to make that change? > > Do you agree with the rest of this change? Because there's some > significant changes how TLAB and PLAB calculate their alloc reserves in > there too. I think we need to think through the basics above first. Generally, having PLAB call Universe::heap()->tlab_alloc_reserve() looks wrong to me, since a PLAB is not a TLAB. If we really want to share this code I think we should try to find some other way. Maybe just give tlab_alloc_reserve() a better name, but at the same time I'm not even sure this belongs in CollectedHeap. cheers, Per From rkennke at redhat.com Thu Oct 11 10:54:03 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 11 Oct 2018 12:54:03 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: <9467830c-0983-fc12-7f3a-1446e5b85362@oracle.com> References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <2aa7c91e-9b1e-3129-cd7a-d1fa50da422e@redhat.com> <07b75752-0430-68d0-a0b3-5bdf8a4b1d1d@redhat.com> <376d3148-6492-ada6-3514-fd0779e6786d@redhat.com> <9467830c-0983-fc12-7f3a-1446e5b85362@oracle.com> Message-ID: <9c425de5-387f-6d7d-6ee2-cf6160a0d46f@redhat.com> Hi Per, > On 10/11/2018 11:24 AM, Roman Kennke wrote: > [...] >>>> Ok, cool. Here's the updated webrevs: >>>> Incremental: >>>> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.02.diff/ >>>> Full: >>>> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.02/ >>>> >>>> Good now? >>> >>> I'm not sure I understand why we need yet another abstraction for this. >>> I'm thinking the stuff you did in JDK-8211270 should be enough? We >>> already have CollectedHeap::min_fill_size() to answer the question what >>> the min filler size is, so adding a new function doesn't make sense to >>> me. What am I missing? > > I see now that I missed the second half of the JDK-8211270 mail thread. ;-) >> min_fill_size() is static and cannot be overridden by GC. obj_size() >> from JDK-8211270 could be used if we'd pass a specific object. I guess >> we could make a better abstraction like what Erik suggested in the other >> thread: >> >> virtual size_t cell_size(size_t obj_size) { return obj_size; } > > I don't think we should introduced the concept of cell in this patch. > That's a much larger concept/change that needs to be carefully thought > out first. Yeah. It should at least involve the allocation path. But let's drop it for now. > For example, it's not obvious to me if you consider the Brooks pointer > to be an extension of the object header, and hence part of the object > and included in obj->size(), of if it's a separate thing. If it's a > separate thing, then a cell concept might prove useful. But if it's a > separate thing, then JDK-8211270 now incorrectly made > JvmtiEnv::GetObjectSize() and WB_GetObjectSize() return the cell size > instead of the object size. > > I assume you thought these things through? What's the story for > Shenandoah here? We never made the distinction between cell and object size. Practically speaking, we treated the fwd pointer as part of the object, but made it invisible to the rest of the runtime, i.e. it's GC specific. We wanted to be honest about reporting the space that each object occupies in the heap, hence the changes to WB and JVMTI. >> and this could be called by JVMT and whitebox (JDK-8211270) with >> obj->size(), and by min_fill_size() with the static minimum obj size, >> and the GC would translate it to the cell size. The obj_size() from >> JDK-8211270 would then be removed and we'd have a single abstraction for >> this. Want me to make that change? >> >> Do you agree with the rest of this change? Because there's some >> significant changes how TLAB and PLAB calculate their alloc reserves in >> there too. > > I think we need to think through the basics above first. > > Generally, having PLAB call Universe::heap()->tlab_alloc_reserve() looks > wrong to me, since a PLAB is not a TLAB. If we really want to share this > code I think we should try to find some other way. Maybe just give > tlab_alloc_reserve() a better name, but at the same time I'm not even > sure this belongs in CollectedHeap. We can change it to lab_alloc_reserve() instead. I think it should be shared between TLAB and PLAB because the concept is the same thing, and relies on the same properties: we need some reserve at the end in order to be able to fill it completely, and with Shenandoah this is GC specific. And since all the other code that deals with filling TLABs and PLABs resides in CollectedHeap, this naturally belongs there too, right next to CollectedHeap::fill_with_dummy_object(..) which is also used by both TLAB and PLAB. (BTW: some explanation why we care about PLAB at all: Shenandoah uses a sort of GCLAB for the GC workers to evacuate objects from one region to another. Java threads also use this for evacuations from write-barriers. We did use TLAB for this in the past, but this was - and still is - cumbersome because of the thight coupling of TLAB with Thread. For example TLAB needs to know about its offset in Thread. That's why we eventually switched to PLAB: this can be much easier used in GCThreadLocalData structure. It fits our purpose very well. We could have invented yet another LAB structure though...) How do we want to go forward? Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Thu Oct 11 12:28:33 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 11 Oct 2018 14:28:33 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: <9c425de5-387f-6d7d-6ee2-cf6160a0d46f@redhat.com> References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <2aa7c91e-9b1e-3129-cd7a-d1fa50da422e@redhat.com> <07b75752-0430-68d0-a0b3-5bdf8a4b1d1d@redhat.com> <376d3148-6492-ada6-3514-fd0779e6786d@redhat.com> <9467830c-0983-fc12-7f3a-1446e5b85362@oracle.com> <9c425de5-387f-6d7d-6ee2-cf6160a0d46f@redhat.com> Message-ID: <13396609-82b6-be14-f1fe-cc168b8db95d@oracle.com> Hi Roman, On 10/11/2018 12:54 PM, Roman Kennke wrote: [...] >> For example, it's not obvious to me if you consider the Brooks pointer >> to be an extension of the object header, and hence part of the object >> and included in obj->size(), of if it's a separate thing. If it's a >> separate thing, then a cell concept might prove useful. But if it's a >> separate thing, then JDK-8211270 now incorrectly made >> JvmtiEnv::GetObjectSize() and WB_GetObjectSize() return the cell size >> instead of the object size. >> >> I assume you thought these things through? What's the story for >> Shenandoah here? > > We never made the distinction between cell and object size. Practically > speaking, we treated the fwd pointer as part of the object, but made it > invisible to the rest of the runtime, i.e. it's GC specific. We wanted > to be honest about reporting the space that each object occupies in the > heap, hence the changes to WB and JVMTI. Were these two functions the only places where this was an issue, or do you have more patches lined up affecting shared code with similar problems? It would help to know to better understand if this is a small-ish problem that we can live with for now or if we need to do something bigger here, like start thinking about if we need the cell concept or something else. I can't help but feel that we're piece by piece building a mess here, that will take a long time to clean up, instead of having a clear and well thought out plan from the beginning. I hope I'm wrong, of course. > >>> and this could be called by JVMT and whitebox (JDK-8211270) with >>> obj->size(), and by min_fill_size() with the static minimum obj size, >>> and the GC would translate it to the cell size. The obj_size() from >>> JDK-8211270 would then be removed and we'd have a single abstraction for >>> this. Want me to make that change? >>> >>> Do you agree with the rest of this change? Because there's some >>> significant changes how TLAB and PLAB calculate their alloc reserves in >>> there too. >> >> I think we need to think through the basics above first. >> >> Generally, having PLAB call Universe::heap()->tlab_alloc_reserve() looks >> wrong to me, since a PLAB is not a TLAB. If we really want to share this >> code I think we should try to find some other way. Maybe just give >> tlab_alloc_reserve() a better name, but at the same time I'm not even >> sure this belongs in CollectedHeap. > > We can change it to lab_alloc_reserve() instead. I think it should be > shared between TLAB and PLAB because the concept is the same thing, and > relies on the same properties: we need some reserve at the end in order > to be able to fill it completely, and with Shenandoah this is GC > specific. And since all the other code that deals with filling TLABs and > PLABs resides in CollectedHeap, this naturally belongs there too, right > next to CollectedHeap::fill_with_dummy_object(..) which is also used by > both TLAB and PLAB. Maybe a stupid question, but why do you even need a Brooks pointer in your filler objects? Is not having that maybe a solution here? cheers, Per > > (BTW: some explanation why we care about PLAB at all: Shenandoah uses a > sort of GCLAB for the GC workers to evacuate objects from one region to > another. Java threads also use this for evacuations from write-barriers. > We did use TLAB for this in the past, but this was - and still is - > cumbersome because of the thight coupling of TLAB with Thread. For > example TLAB needs to know about its offset in Thread. That's why we > eventually switched to PLAB: this can be much easier used in > GCThreadLocalData structure. It fits our purpose very well. We could > have invented yet another LAB structure though...) > > How do we want to go forward? > > Roman > From rkennke at redhat.com Thu Oct 11 12:41:35 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 11 Oct 2018 14:41:35 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: <13396609-82b6-be14-f1fe-cc168b8db95d@oracle.com> References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <2aa7c91e-9b1e-3129-cd7a-d1fa50da422e@redhat.com> <07b75752-0430-68d0-a0b3-5bdf8a4b1d1d@redhat.com> <376d3148-6492-ada6-3514-fd0779e6786d@redhat.com> <9467830c-0983-fc12-7f3a-1446e5b85362@oracle.com> <9c425de5-387f-6d7d-6ee2-cf6160a0d46f@redhat.com> <13396609-82b6-be14-f1fe-cc168b8db95d@oracle.com> Message-ID: Hi Per, > On 10/11/2018 12:54 PM, Roman Kennke wrote: > [...] >>> For example, it's not obvious to me if you consider the Brooks pointer >>> to be an extension of the object header, and hence part of the object >>> and included in obj->size(), of if it's a separate thing. If it's a >>> separate thing, then a cell concept might prove useful. But if it's a >>> separate thing, then JDK-8211270 now incorrectly made >>> JvmtiEnv::GetObjectSize() and WB_GetObjectSize() return the cell size >>> instead of the object size. >>> >>> I assume you thought these things through? What's the story for >>> Shenandoah here? >> >> We never made the distinction between cell and object size. Practically >> speaking, we treated the fwd pointer as part of the object, but made it >> invisible to the rest of the runtime, i.e. it's GC specific. We wanted >> to be honest about reporting the space that each object occupies in the >> heap, hence the changes to WB and JVMTI. > > Were these two functions the only places where this was an issue, or do > you have more patches lined up affecting shared code with similar > problems? It would help to know to better understand if this is a > small-ish problem that we can live with for now or if we need to do > something bigger here, like start thinking about if we need the cell > concept or something else. > > I can't help but feel that we're piece by piece building a mess here, > that will take a long time to clean up, instead of having a clear and > well thought out plan from the beginning. I hope I'm wrong, of course. Those are the only functions where this matters. In-fact, the LAB reserve abstraction is the last one I have lined up. The remaining stuff that Shenandoah needs is a bunch of extra utility classes and C2 changes. >>>> and this could be called by JVMT and whitebox (JDK-8211270) with >>>> obj->size(), and by min_fill_size() with the static minimum obj size, >>>> and the GC would translate it to the cell size. The obj_size() from >>>> JDK-8211270 would then be removed and we'd have a single abstraction >>>> for >>>> this. Want me to make that change? >>>> >>>> Do you agree with the rest of this change? Because there's some >>>> significant changes how TLAB and PLAB calculate their alloc reserves in >>>> there too. >>> >>> I think we need to think through the basics above first. >>> >>> Generally, having PLAB call Universe::heap()->tlab_alloc_reserve() looks >>> wrong to me, since a PLAB is not a TLAB. If we really want to share this >>> code I think we should try to find some other way. Maybe just give >>> tlab_alloc_reserve() a better name, but at the same time I'm not even >>> sure this belongs in CollectedHeap. >> >> We can change it to lab_alloc_reserve() instead. I think it should be >> shared between TLAB and PLAB because the concept is the same thing, and >> relies on the same properties: we need some reserve at the end in order >> to be able to fill it completely, and with Shenandoah this is GC >> specific. And since all the other code that deals with filling TLABs and >> PLABs resides in CollectedHeap, this naturally belongs there too, right >> next to CollectedHeap::fill_with_dummy_object(..) which is also used by >> both TLAB and PLAB. > > Maybe a stupid question, but why do you even need a Brooks pointer in > your filler objects? Is not having that maybe a solution here? Filler objects are objects. Laying them out differently would make parsing the heap impossible. Thanks, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Thu Oct 11 14:54:31 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 11 Oct 2018 16:54:31 +0200 Subject: RFR: bug: Timely Reducing Unused Committed Memory In-Reply-To: References: <8af5c2468f5133bc1ba2517bf52c3078af516977.camel@oracle.com> <65063e5f31702649e12b485b570ecab9ebd32d01.camel@oracle.com> <66c536399b2f218af0d85bd3bced44f204e3dbf4.camel@oracle.com> <2821cf70-c4d5-016f-b976-5ecc850f377c@oracle.com> <57b10708-b870-cb98-0df0-fab650f431e4@oracle.com> <3927596f-8847-ba95-d4c6-3c5f3656330c@oracle.com> <9bbf4a13-8490-d918-3cda-5c34acd16fa6@oracle.com> Message-ID: <9c99d018799a7de321925f04def9ee5bfa451aab.camel@oracle.com> Hi, On Mon, 2018-10-08 at 22:09 +0300, Ruslan Synytsky wrote: > Hi Thomas, thank you for the patch. > > On Mon, 8 Oct 2018 at 17:06, Thomas Schatzl < > thomas.schatzl at oracle.com> wrote: > > Hi all, > > > > here's the promised patch based on latest discussions: > > > > http://cr.openjdk.java.net/~tschatzl/jelastic/pgc-webrev.3/webrev/ > > > > It is based on "6490394: G1: Allow heap shrinking / memory > > unmapping > > after reclaiming regions during Remark", that is out for review > > now. > > > > As for the options, they were named as follows: > > > > G1PeriodicGCInterval: Number of milliseconds after a previous GC to > > wait before triggering a periodic gc. A value of zero disables > > periodically enforced gc cycles. (Note that I just found out when > > copy&pasting this description from the g1_globals file that the > > text I there needs fixing - I will do thatasap :)) (Updated the webrev with all your comments) > > > > G1PeriodicGCInvokesConcurrent: Determines the kind of periodic GC. > > Set to true to have G1 perform a concurrent GC as periodic GC, > > otherwise use a STW Full GC. > > Is concurrent enabled by default? If not, then why not? > Concurrent is enabled by default as otherwise this change would probably accidentally disturb operation too much :) > > > > G1PeriodicGCSystemLoadThreshold: Maximum recent system wide system > > load at which G1 triggers a periodic GC. A load above this value > > cancels a given periodic GCs. A value of zero disables this check. > > Maybe a clear mention of loadavg in the description can help to avoid > potential confusions around system load definition. Done. > > > The policy has been implemented so that any reason to not start a > > periodic collection right now will cancel the current attempt - > > another attempt will be made after G1PeriodicGCInterval ms. This is > > different to suppress the periodic gc until the next time the > > conditions are met. > > > > From a default values POV, periodic gcs will be on by default with > > an interval of 5mins. This should be fairly non-intrusive for all > > but exceptional cases. > > > > Please check whether all this still fits your use case :) > > Looks good for me. Thanks again. I am currently testing the change a bit more, in particular I found that the periodic gcs mess up prediction (obviously because they are occurring during a time when the application is not very active, so the data they return is not very helpful for normal application behavior). I fixed that, but there may be some other details lurking that need fixing. Other than that everything looks good so far (performance, functional testing). Meanhwhile I will start updating the JEP again, and kick off the process (initially mostly internal). > > Btw, we have got an approved session at OracleCode about JVM > elasticity > https://oracle.rainfocus.com/widget/oracle/oow18/catalogcodeone18?search=elastic%2Bjvm > . I'm planning to bring up topics about general benefits, available > options with different GCs and JVMs, specifics of running elastic JVM > in containers, elaboration of possible further improvements. Please > join the session for a live talk. > Unfortunately I will not attend Code One this year, but I asked Per who does (he's giving some talk about ZGC if you are interested) to attend instead. I will certainly follow up on any recording there is :) Thanks, Thomas From rkennke at redhat.com Thu Oct 11 16:49:44 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 11 Oct 2018 18:49:44 +0200 Subject: RFR: JDK-8212054: Boilerplate to bind oopDesc::equals_raw() to actual raw implementation Message-ID: With JDK-8211279 I introduced oopDesc::equals_raw() and it calls RawAccess<>::equals(), however I did not realize that the boilerplace to actually bind this straight to the raw impl was also needed. So here it comes: Bug: https://bugs.openjdk.java.net/browse/JDK-8212054 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8212054/webrev.00/ Builds and runs hotspot/tier1 fine. Ok to push? Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Thu Oct 11 16:53:36 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 11 Oct 2018 18:53:36 +0200 Subject: RFR: JDK-8212053: A few more missing object equals barriers Message-ID: <2b4c871f-a241-d433-56ea-46b80ab48f6f@redhat.com> Testing and code analysis have revealed a couple more places where we need oopDesc::equals() or oopDesc::equals_raw(). Those should really be the last ones (notwithstanding new code of course): in compressed oops, we should use equals_raw() because it's only used to verify that we encoded/decoded the oop correctly a few lines above. It cannot have been changed yet. In vframe, we need the usual oopDesc::equals(..) because we may see different copies of the same object. Bug: https://bugs.openjdk.java.net/browse/JDK-8212053 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8212053/webrev.00/ Builds and runs hotspot/tier1 fine. Ok? Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Thu Oct 11 17:02:10 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 11 Oct 2018 19:02:10 +0200 Subject: RFR: JDK-8212053: A few more missing object equals barriers In-Reply-To: <2b4c871f-a241-d433-56ea-46b80ab48f6f@redhat.com> References: <2b4c871f-a241-d433-56ea-46b80ab48f6f@redhat.com> Message-ID: <77a928bf-c48f-22fd-921d-10fa3083ecc0@redhat.com> On 10/11/2018 06:53 PM, Roman Kennke wrote: > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8212053/webrev.00/ Looks good to me. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From erik.osterlund at oracle.com Thu Oct 11 17:50:16 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Thu, 11 Oct 2018 19:50:16 +0200 Subject: RFR: JDK-8212054: Boilerplate to bind oopDesc::equals_raw() to actual raw implementation In-Reply-To: References: Message-ID: Hi Roman, Looks good. Thanks, /Erik > On 11 Oct 2018, at 18:49, Roman Kennke wrote: > > With JDK-8211279 I introduced oopDesc::equals_raw() and it calls > RawAccess<>::equals(), however I did not realize that the boilerplace to > actually bind this straight to the raw impl was also needed. So here it > comes: > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212054 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8212054/webrev.00/ > > Builds and runs hotspot/tier1 fine. > > Ok to push? > > Roman > From rkennke at redhat.com Thu Oct 11 17:59:20 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 11 Oct 2018 19:59:20 +0200 Subject: RFR: JDK-8212054: Boilerplate to bind oopDesc::equals_raw() to actual raw implementation In-Reply-To: References: Message-ID: Thanks Erik! Roman > Hi Roman, > > Looks good. > > Thanks, > /Erik > >> On 11 Oct 2018, at 18:49, Roman Kennke wrote: >> >> With JDK-8211279 I introduced oopDesc::equals_raw() and it calls >> RawAccess<>::equals(), however I did not realize that the boilerplace to >> actually bind this straight to the raw impl was also needed. So here it >> comes: >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8212054 >> Webrev: >> http://cr.openjdk.java.net/~rkennke/JDK-8212054/webrev.00/ >> >> Builds and runs hotspot/tier1 fine. >> >> Ok to push? >> >> Roman >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Thu Oct 11 17:59:52 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 11 Oct 2018 19:59:52 +0200 Subject: RFR: JDK-8212054: Boilerplate to bind oopDesc::equals_raw() to actual raw implementation In-Reply-To: References: Message-ID: On 10/11/2018 06:49 PM, Roman Kennke wrote: > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8212054/webrev.00/ Looks good to me. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From jcbeyler at google.com Thu Oct 11 20:06:12 2018 From: jcbeyler at google.com (JC Beyler) Date: Thu, 11 Oct 2018 13:06:12 -0700 Subject: RFR (XS) 8212025: Remove collector_present variable from ThreadHeapSampler In-Reply-To: <1219dadb859369ccb2862cb552656b15c0b20056.camel@oracle.com> References: <1219dadb859369ccb2862cb552656b15c0b20056.camel@oracle.com> Message-ID: Thanks both for the reviews, I pushed it. Jc On Thu, Oct 11, 2018 at 1:40 AM Thomas Schatzl wrote: > Hi, > > On Wed, 2018-10-10 at 13:15 -0700, JC Beyler wrote: > > Hi all, > > > > Here is a tiny webrev to remove the _collectors_present member that > > is now unused. I forgot to remove it when I did the fix for JDK- > > 8211950. > > > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8212025/webrev.00/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8212025 > > > > My apologies, > > Jc > > np, we are always happy about code improvements like this :) > > Ship it. > > Thomas > > > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From manc at google.com Thu Oct 11 21:51:43 2018 From: manc at google.com (Man Cao) Date: Thu, 11 Oct 2018 14:51:43 -0700 Subject: UseGCOverheadLimit for G1 Message-ID: Hi all, I noticed that UseGCOverheadLimit does not really work for G1. I created an RFE: https://bugs.openjdk.java.net/browse/JDK-8212084 I also search JBS and mail archive, but didn't find any plan to implement UseGCOverheadLimit for G1. Is this a desirable feature to be implemented in the future? -Man -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Thu Oct 11 22:23:43 2018 From: jcbeyler at google.com (JC Beyler) Date: Thu, 11 Oct 2018 15:23:43 -0700 Subject: RFR (S) 8212083: Handle remaining gc/lock native code and fix two strings Message-ID: Hi all, I've cleaned up the gc/lock tests to use the ExceptionCheckingJniEnv wrapper and now we have all JNI calls in that test group covered from a "checking exceptions and returns of JNI methods". Let me know what you think: Webrev: http://cr.openjdk.java.net/~jcbeyler/8212083/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8212083 Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From hohensee at amazon.com Thu Oct 11 22:45:16 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Thu, 11 Oct 2018 22:45:16 +0000 Subject: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions In-Reply-To: <8C241BB9-2089-491B-A27E-57C4F43FBA9E@amazon.com> References: <8C241BB9-2089-491B-A27E-57C4F43FBA9E@amazon.com> Message-ID: Any takers? :) From: serviceability-dev on behalf of "Hohensee, Paul" Date: Monday, October 8, 2018 at 7:50 PM To: "hotspot-gc-dev at openjdk.java.net" , "serviceability-dev at openjdk.java.net" Subject: RFR: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions Bug: https://bugs.openjdk.java.net/browse/JDK-8196989 CSR: https://bugs.openjdk.java.net/browse/JDK-8196991 Webrev: http://cr.openjdk.java.net/~phh/8196989/webrev.02/ As requested, I split the jstat counter update off from the MXBean update. This is the MXBean update. The jstat counter RFE is https://bugs.openjdk.java.net/browse/JDK-8210965 and its CSR is https://bugs.openjdk.java.net/browse/JDK-8210966. The MXBean CSR is in draft state, I?d greatly appreciate review and sign-off. It?s been suggested that we add another pool to represent the free region set, but doing so would be incompatible with existing MXBean use invariants for all GCs. These are: 1. The sum of the pools? MemoryUsage.max properties is the total reserved heap size. 2. The sum of the pools? MemoryUsage.committed properties is the total committed size. 3. The sum of the pools? MemoryUsage.used properties is the total size of the memory containing objects, live and dead-and-yet-to-be-collected, as the case might be, plus intentional gaps between them. 4. The total free space is (sum of the max properties ? sum of the used properties). 5. The total uncommitted space is (sum of the max properties ? sum of the committed properties). 6. The total committed free space is (2) ? (3). To keep invariants 1, 2 and 3, the free region pool?s ?max? property should be ?undefined? (i.e., -1). The intuitive, to me, ?used? property value would be the total free space, but that would violate invariant 4 above. Defining the ?committed? property as the total committed free space would violate invariants 2 and 6. The patch passes the submit repo, hotspot tier1, and, separately, the serviceability, jfr, and gc jtreg tests. I?m uncertain how to construct a test that checks for valid MXBean content: the existing tests don?t. Any such test will be fragile due to possible future Hotspot changes that affect the values, and to run-to-run variability. I?ve done by-hand comparisons between the old and new MXBean content using the SwingSet2 demo, including using App CDS, and the numbers look reasonable. The guts of the change are in G1MonitoringSupport::recalculate_sizes, initialize_serviceability, memory_managers, memory_pools, and G1MonitoringScope. I also defined TraceConcMemoryManagerStats to track the concurrent cycle in a way analogous to TraceCMSMemoryManagerStats. The changes to the includes in g1FullGCOopClosures.inline.hpp and g1HeapVerifier.cpp are to satisfy compiler complaints. I changed the 3rd argument to the G1MonitoringScope constructor to a mixed_gc flag, and use accessor methods instead of direct field accesses when accessor methods exist. I believe I?ve minimized the latter. I updated the copyright date to 2018 in memoryService.hpp because I neglected to do so in my previous G1 MXBean patch. Thanks, Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From hohensee at amazon.com Fri Oct 12 00:03:54 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 12 Oct 2018 00:03:54 +0000 Subject: RFR: Backport 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results Message-ID: <9D007BB7-F6FE-4A24-81BC-6629B68DEF70@amazon.com> Please review a backport to jdk8u. Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.05/ JDK11 patch: http://hg.openjdk.java.net/jdk/jdk/rev/5d3c5af82654 The backport is slightly different from the JDK11 patch due to G1 refactoring, hence my request for new review. I?ll ask for jdk8u approval once the backport is reviewed. I backported two jtreg tests from JDK11, which pass. Also, all the hotspot gc jtreg tests pass as well as they do for jdk8u-dev. There was a CSR involved, https://bugs.openjdk.java.net/browse/JDK-8196719. Does that have to be re-approved for jdk8u as well, and if so, what?s the process? Thanks, Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Fri Oct 12 02:35:52 2018 From: jcbeyler at google.com (JC Beyler) Date: Thu, 11 Oct 2018 19:35:52 -0700 Subject: RFR: Backport 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: <9D007BB7-F6FE-4A24-81BC-6629B68DEF70@amazon.com> References: <9D007BB7-F6FE-4A24-81BC-6629B68DEF70@amazon.com> Message-ID: Hi Paul, The biggest thing I saw in this RFR was that the flags for the test: http://cr.openjdk.java.net/~phh/8195115/webrev.05/test/gc/g1/mixedgc/TestOldGenCollectionUsage.java.html were changed it seems: - the @requires are different for the backport (you accept null for JDK8 for GC and also removed the @requires vm.opt.MaxGCPauseMillis == "null") - the @run flags are different (-Xms/Xmx are 14m for the backport; they were 12 originally; there is a comment below in the backport saying this requires normally 12m though you ask for 14 in the @run) What are the reasons for these differences? Apart from that, the backport seemed ok but I'm not that well versed in the GC changes :) Jc On Thu, Oct 11, 2018 at 5:04 PM Hohensee, Paul wrote: > Please review a backport to jdk8u. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.05/ > > JDK11 patch: http://hg.openjdk.java.net/jdk/jdk/rev/5d3c5af82654 > > > > The backport is slightly different from the JDK11 patch due to G1 > refactoring, hence my request for new review. I?ll ask for jdk8u approval > once the backport is reviewed. > > > > I backported two jtreg tests from JDK11, which pass. Also, all the hotspot > gc jtreg tests pass as well as they do for jdk8u-dev. > > > > There was a CSR involved, https://bugs.openjdk.java.net/browse/JDK-8196719. > Does that have to be re-approved for jdk8u as well, and if so, what?s the > process? > > > > Thanks, > > > > Paul > > > > > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Fri Oct 12 07:10:43 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 12 Oct 2018 09:10:43 +0200 Subject: UseGCOverheadLimit for G1 In-Reply-To: References: Message-ID: Hi Man, On Thu, 2018-10-11 at 14:51 -0700, Man Cao wrote: > Hi all, > > I noticed that UseGCOverheadLimit does not really work for G1. > I created an RFE: https://bugs.openjdk.java.net/browse/JDK-8212084 > > I also search JBS and mail archive, but didn't find any plan to > implement UseGCOverheadLimit for G1. Is this a desirable feature to > be implemented in the future? > if it is something you need, feel free to implement. I see no particular reason why it shouldn't be supported. Thanks, Thomas From per.liden at oracle.com Fri Oct 12 07:14:07 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 12 Oct 2018 09:14:07 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <2aa7c91e-9b1e-3129-cd7a-d1fa50da422e@redhat.com> <07b75752-0430-68d0-a0b3-5bdf8a4b1d1d@redhat.com> <376d3148-6492-ada6-3514-fd0779e6786d@redhat.com> <9467830c-0983-fc12-7f3a-1446e5b85362@oracle.com> <9c425de5-387f-6d7d-6ee2-cf6160a0d46f@redhat.com> <13396609-82b6-be14-f1fe-cc168b8db95d@oracle.com> Message-ID: <45395b5d-216c-eb40-1e03-fed6167b4b4f@oracle.com> Hi Roman, On 10/11/2018 02:41 PM, Roman Kennke wrote: [...] > Those are the only functions where this matters. In-fact, the LAB > reserve abstraction is the last one I have lined up. The remaining stuff > that Shenandoah needs is a bunch of extra utility classes and C2 changes. Thanks for clarifying. That makes me much less worried. For C2, the strategy we had and the requirement we put on our selves when integrating ZGC was basically this: We really didn't want to screw things up in C2. If things went south, we wanted to just be able to disable building of ZGC and C2 would go back to do _exactly_ what it did before we merged the ZGC patch. As you maybe remember, we _almost_ got there. We had a very small set of places in C2 where we did change/add code, which was _not_ disabled when disabling ZGC. However, it was easy to prove that that code was safe/correct. This strategy turned out to work well, and it forced us to think hard about having abstractions in the right places, move as much code as possible into the BarrierSet, etc. I'd recommend you do the same for Shenandoah, that way the pressure on you (in case something breaks and results instability or a long bug trail) will be reduced a lot. If shit hits the fan, you can always disable building of Shenandoah until the problems are sorted out. [...] >>>> Generally, having PLAB call Universe::heap()->tlab_alloc_reserve() looks >>>> wrong to me, since a PLAB is not a TLAB. If we really want to share this >>>> code I think we should try to find some other way. Maybe just give >>>> tlab_alloc_reserve() a better name, but at the same time I'm not even >>>> sure this belongs in CollectedHeap. >>> >>> We can change it to lab_alloc_reserve() instead. I think it should be >>> shared between TLAB and PLAB because the concept is the same thing, and >>> relies on the same properties: we need some reserve at the end in order >>> to be able to fill it completely, and with Shenandoah this is GC >>> specific. And since all the other code that deals with filling TLABs and >>> PLABs resides in CollectedHeap, this naturally belongs there too, right >>> next to CollectedHeap::fill_with_dummy_object(..) which is also used by >>> both TLAB and PLAB. >> >> Maybe a stupid question, but why do you even need a Brooks pointer in >> your filler objects? Is not having that maybe a solution here? > > Filler objects are objects. Laying them out differently would make > parsing the heap impossible. Check. I thought some more about this. How about we rip out the filler-stuff from CollectedHeap, and put it in a Fill utility class. You could think of Fill as a cosine to the "class Copy" we have today. Something like: class Fill : public AllStatic { private: static int _min_size; public: static void set_min_size(int min_size) { _min_size = min_size; } static int min_size() { return _min_size; } static int min_reserve_size() { return _min_size > MinObjAlignment ? _min_size : 0; } static void with_objects(...); static void with_object(...); ... }; Fill::_min_size is initialized to oopDesc::header_size() by default. A collector that wants something different calls Fill::set_min_size(...) during start up. TLAB/PLAB would call Fill::min_reserve_size(). In my mind, I think this would put things where they belong. And as a bonus we would cut out some cruft from CollectedHeap. What do you think? I started doing a patch for this to see if it seems like a good approach. I hope to have something to show later today. cheers, Per From rkennke at redhat.com Fri Oct 12 08:02:12 2018 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 12 Oct 2018 10:02:12 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: <45395b5d-216c-eb40-1e03-fed6167b4b4f@oracle.com> References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <2aa7c91e-9b1e-3129-cd7a-d1fa50da422e@redhat.com> <07b75752-0430-68d0-a0b3-5bdf8a4b1d1d@redhat.com> <376d3148-6492-ada6-3514-fd0779e6786d@redhat.com> <9467830c-0983-fc12-7f3a-1446e5b85362@oracle.com> <9c425de5-387f-6d7d-6ee2-cf6160a0d46f@redhat.com> <13396609-82b6-be14-f1fe-cc168b8db95d@oracle.com> <45395b5d-216c-eb40-1e03-fed6167b4b4f@oracle.com> Message-ID: <4897b66b-e2b1-ffbf-8bc8-8c1ec2e0e81b@redhat.com> Hi Per, > On 10/11/2018 02:41 PM, Roman Kennke wrote: > [...] >> Those are the only functions where this matters. In-fact, the LAB >> reserve abstraction is the last one I have lined up. The remaining stuff >> that Shenandoah needs is a bunch of extra utility classes and C2 changes. > > Thanks for clarifying. That makes me much less worried. > > For C2, the strategy we had and the requirement we put on our selves > when integrating ZGC was basically this: > > We really didn't want to screw things up in C2. If things went south, we > wanted to just be able to disable building of ZGC and C2 would go back > to do _exactly_ what it did before we merged the ZGC patch. As you maybe > remember, we _almost_ got there. We had a very small set of places in C2 > where we did change/add code, which was _not_ disabled when disabling > ZGC. However, it was easy to prove that that code was safe/correct. This > strategy turned out to work well, and it forced us to think hard about > having abstractions in the right places, move as much code as possible > into the BarrierSet, etc. > > I'd recommend you do the same for Shenandoah, that way the pressure on > you (in case something breaks and results instability or a long bug > trail) will be reduced a lot. If shit hits the fan, you can always > disable building of Shenandoah until the problems are sorted out. Thanks for your suggestions and support! That's basically the approach we're following with Shenandoah too: 1. put as much stuff (that makes sense) behind a reasonable BarrierSetC2 interface 2. wrap the rest in #ifdef INCLUDE_SHENANDOAHGC and/or if (UseShenandoahGC) and justify/upstream stuff ahead of time that's not Shenandoah-specific and otherwise useful. > [...] >>>>> Generally, having PLAB call Universe::heap()->tlab_alloc_reserve() >>>>> looks >>>>> wrong to me, since a PLAB is not a TLAB. If we really want to share >>>>> this >>>>> code I think we should try to find some other way. Maybe just give >>>>> tlab_alloc_reserve() a better name, but at the same time I'm not even >>>>> sure this belongs in CollectedHeap. >>>> >>>> We can change it to lab_alloc_reserve() instead. I think it should be >>>> shared between TLAB and PLAB because the concept is the same thing, and >>>> relies on the same properties: we need some reserve at the end in order >>>> to be able to fill it completely, and with Shenandoah this is GC >>>> specific. And since all the other code that deals with filling TLABs >>>> and >>>> PLABs resides in CollectedHeap, this naturally belongs there too, right >>>> next to CollectedHeap::fill_with_dummy_object(..) which is also used by >>>> both TLAB and PLAB. >>> >>> Maybe a stupid question, but why do you even need a Brooks pointer in >>> your filler objects? Is not having that maybe a solution here? >> >> Filler objects are objects. Laying them out differently would make >> parsing the heap impossible. > > Check. > > I thought some more about this. How about we rip out the filler-stuff > from CollectedHeap, and put it in a Fill utility class. You could think > of Fill as a cosine to the "class Copy" we have today. Something like: > > class Fill : public AllStatic { > private: > ? static int _min_size; > > public: > ? static void set_min_size(int min_size) { > ??? _min_size = min_size; > ? } > > ? static int min_size() { > ??? return _min_size; > ? } > > ? static int min_reserve_size() { > ??? return _min_size > MinObjAlignment ? _min_size : 0; > ? } > > ? static void with_objects(...); > ? static void with_object(...); > ? ... > }; > > Fill::_min_size is initialized to oopDesc::header_size() by default. A > collector that wants something different calls Fill::set_min_size(...) > during start up. TLAB/PLAB would call Fill::min_reserve_size(). > > In my mind, I think this would put things where they belong. And as a > bonus we would cut out some cruft from CollectedHeap. What do you think? > I started doing a patch for this to see if it seems like a good > approach. I hope to have something to show later today. That seems like a good idea. Not sure if it should be AllStatic? Maybe put the abstractions (that are currently in CH, plus the one I proposed with this patch) into the Fill class? I'm wondering if I should push this patch ahead of this, and then we can build the Fill refactoring on top of it? Otherwise I'll just wait for you to come up with a proposal. Thanks & Cheers, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Fri Oct 12 08:57:25 2018 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 12 Oct 2018 10:57:25 +0200 Subject: RFC: How to deal with/abstract some offset-related Shenandoah changes? Message-ID: <27467623-88df-c77d-20a3-2e2e26a3ecee@redhat.com> Hello, I'd like to get some opinions on how should we deal with some changes that we need in Shenandoah, that really look a bit ugly, but that seem a little out of place in a GC interface too. Let me show you the code: https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/asm/assembler.cpp.udiff.html There are 3 blocks there: - The first deals with the aarch64 addressing bits. Only the lower 48bits are used for addressing, and thus the upper 16 are masked away. - the second block adjusts the offset by 8 bytes for Shenandoah when computing the normalized heap-based-offset for narrow-oops. This is needed to get combined decode+read-barrier instruction to work iirc. - the last block allows implicit null-checks on Shenandoah fwd ptr loads (now including the masking from block#1 above). I really am not sure how do deal with this. It does look like it would require a two-way abstraction: 1. CPU specific. i.e. move this method under src/cpu/$CPU and do the right thing there (possibly duplicating for platforms that do common stuff) 2. GC specific to deal with the -8 offset of Shenandoah However, I doubt that it would make the code any more easier to understand/read/etc. And how would the GC abstraction look like? bool BarrierSetAssembler::needs_explicit_null_check(..) ? And if GC says 'false' then don't check any of the other stuff? I'm a bit at a loss here. Any suggestions are appreciated. There is another place that deals with special offsets: https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/ci/ciInstanceKlass.cpp.udiff.html (look at get_canonical_holder() there) Please let me know what you think. Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Fri Oct 12 12:38:51 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Fri, 12 Oct 2018 08:38:51 -0400 Subject: RFR: JDK-8212053: A few more missing object equals barriers In-Reply-To: <2b4c871f-a241-d433-56ea-46b80ab48f6f@redhat.com> References: <2b4c871f-a241-d433-56ea-46b80ab48f6f@redhat.com> Message-ID: Good to me. Thanks, -Zhengyu On 10/11/2018 12:53 PM, Roman Kennke wrote: > Testing and code analysis have revealed a couple more places where we > need oopDesc::equals() or oopDesc::equals_raw(). Those should really be > the last ones (notwithstanding new code of course): > > in compressed oops, we should use equals_raw() because it's only used to > verify that we encoded/decoded the oop correctly a few lines above. It > cannot have been changed yet. In vframe, we need the usual > oopDesc::equals(..) because we may see different copies of the same object. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212053 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8212053/webrev.00/ > > Builds and runs hotspot/tier1 fine. > > Ok? > > Roman > From zgu at redhat.com Fri Oct 12 13:28:55 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Fri, 12 Oct 2018 09:28:55 -0400 Subject: RFR(XS) 8212074: Add method to peek the remaining tasks in task queues Message-ID: <4ada4955-27b6-aabb-cae2-5b626c9d4aca@redhat.com> Hi All, Shenandoah implemented an variant of termination protocol based on [1], which requires to peek into task queues for remaining tasks, so that it can wake just enough workers to steal tasks. The patch adds methods for this purpose, and allow subclasses of ParallelTaskTerminator to access it members. Bug: https://bugs.openjdk.java.net/browse/JDK-8212074 Webrev: http://cr.openjdk.java.net/~zgu/8212074/webrev.00/ Test: hotspot_gc on Linux 64 (fastdebug and release) Thanks, -Zhengyu [1] Understanding and improving JVM GC work stealing at the data center scale (http://dl.acm.org/citation.cfm?id=2926706) From hohensee at amazon.com Fri Oct 12 14:13:48 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 12 Oct 2018 14:13:48 +0000 Subject: Backport 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results Message-ID: <7565CED8-5174-4CB2-8AC1-F2544BD3B316@amazon.com> Thanks for the review, new webrev at http://cr.openjdk.java.net/~phh/8195115/webrev.06/. The different requires were an artifact of me trying to get TestOldGenCollectionUsage.java to run. Good catch. Jiangli changed the heap size to 14m to get it to work with CDS, see https://bugs.openjdk.java.net/browse/JDK-8210193. Paul From: JC Beyler Date: Thursday, October 11, 2018 at 10:37 PM To: "Hohensee, Paul" Cc: "hotspot-gc-dev at openjdk.java.net" , "serviceability-dev at openjdk.java.net" , "jdk8u-dev at openjdk.java.net" Subject: Re: RFR: Backport 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results Hi Paul, The biggest thing I saw in this RFR was that the flags for the test: http://cr.openjdk.java.net/~phh/8195115/webrev.05/test/gc/g1/mixedgc/TestOldGenCollectionUsage.java.html were changed it seems: - the @requires are different for the backport (you accept null for JDK8 for GC and also removed the @requires vm.opt.MaxGCPauseMillis == "null") - the @run flags are different (-Xms/Xmx are 14m for the backport; they were 12 originally; there is a comment below in the backport saying this requires normally 12m though you ask for 14 in the @run) What are the reasons for these differences? Apart from that, the backport seemed ok but I'm not that well versed in the GC changes :) Jc On Thu, Oct 11, 2018 at 5:04 PM Hohensee, Paul > wrote: Please review a backport to jdk8u. Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.05/ JDK11 patch: http://hg.openjdk.java.net/jdk/jdk/rev/5d3c5af82654 The backport is slightly different from the JDK11 patch due to G1 refactoring, hence my request for new review. I?ll ask for jdk8u approval once the backport is reviewed. I backported two jtreg tests from JDK11, which pass. Also, all the hotspot gc jtreg tests pass as well as they do for jdk8u-dev. There was a CSR involved, https://bugs.openjdk.java.net/browse/JDK-8196719. Does that have to be re-approved for jdk8u as well, and if so, what?s the process? Thanks, Paul -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From rkennke at redhat.com Fri Oct 12 14:56:21 2018 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 12 Oct 2018 16:56:21 +0200 Subject: RFR(XS): JDK-8212122: Allow ReferenceProcessor to always be MT processing Message-ID: Currently, in ReferenceProcessor, the mt_processing flag is adjusted for each phase depending on the number of workers: if it's 1 worker, then mt_processing is disabled, for more than 1 worker, mt_processing is enabled. Depending on the flag, ref-processing is either done through AbstractRefProcTaskExecutor, or directly in VMThread. However, we have a situation where we always want to execute via the executor, and never directly. Instead we decide in the executor whether or not we want to execute directly in the VMThread. A simple change allows a subclass of ReferenceProcessor to change this behavior. In case you're interested why we need to do this: Shenandoah requires (for each worker) a scope around reference processing. In current scheme, we'd need to put one scope around RP::process_discovered_reference() to cover the 1-threaded case, and one scope inside each worker to cover those. However, we cannot handle this sort of weirdly nested scoping. (If it was me, I'd rework this whole execution scheme to always call the MT config, and let RP figure out and execute directly in VMThread if it's only 1 worker, and not put this burden on the caller to set up both for 1-threaded and N-threaded execution. This is just an ugly mess.) Bug: https://bugs.openjdk.java.net/browse/JDK-8212122 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8212122/webrev.00/ Testing: passes hotspot/tier1 locally. Works well with Shenandoah too. Ok? Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From manc at google.com Fri Oct 12 17:47:07 2018 From: manc at google.com (Man Cao) Date: Fri, 12 Oct 2018 10:47:07 -0700 Subject: UseGCOverheadLimit for G1 In-Reply-To: References: Message-ID: Sounds good. I'll take a stab at it. Does initiating the AdaptiveSizePolicy class for G1, or create a subclass G1AdaptiveSizePolicy sound like an acceptable high-level approach? I know it is a non-goal to make UseAdaptiveSizePolicy work for G1, but most logic for UseGCOverheadLimit is in the AdaptiveSizePolicy class. This seems the approach that reuse most existing code and does not affect ParallelGC. A side story is that we have a Google-local patch since JDK6 that makes UseGCOverheadLimit work properly for CMS. UseAdaptiveSizePolicy doesn't work and is disabled by default for CMS, but UseGCOverheadLimit depends on logic in the CMSAdaptiveSizePolicy class (which was removed in JDK9) to properly collect major GC overhead. -Man On Fri, Oct 12, 2018 at 12:10 AM Thomas Schatzl wrote: > Hi Man, > > On Thu, 2018-10-11 at 14:51 -0700, Man Cao wrote: > > Hi all, > > > > I noticed that UseGCOverheadLimit does not really work for G1. > > I created an RFE: https://bugs.openjdk.java.net/browse/JDK-8212084 > > > > I also search JBS and mail archive, but didn't find any plan to > > implement UseGCOverheadLimit for G1. Is this a desirable feature to > > be implemented in the future? > > > > if it is something you need, feel free to implement. I see no > particular reason why it shouldn't be supported. > > Thanks, > Thomas > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hohensee at amazon.com Fri Oct 12 18:49:49 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 12 Oct 2018 18:49:49 +0000 Subject: RFR (S) 8212083: Handle remaining gc/lock native code and fix two strings In-Reply-To: References: Message-ID: <8365A415-F0A2-49DF-B9F8-ABDCF859AD23@amazon.com> Looks good. Paul From: hotspot-gc-dev on behalf of JC Beyler Date: Thursday, October 11, 2018 at 6:24 PM To: "hotspot-gc-dev at openjdk.java.net" Subject: RFR (S) 8212083: Handle remaining gc/lock native code and fix two strings Hi all, I've cleaned up the gc/lock tests to use the ExceptionCheckingJniEnv wrapper and now we have all JNI calls in that test group covered from a "checking exceptions and returns of JNI methods". Let me know what you think: Webrev: http://cr.openjdk.java.net/~jcbeyler/8212083/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8212083 Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Fri Oct 12 19:04:07 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 12 Oct 2018 15:04:07 -0400 Subject: RFR(XS): JDK-8212122: Allow ReferenceProcessor to always be MT processing In-Reply-To: References: Message-ID: <8BFBCB9D-6ADE-47E6-A134-618E836CE71E@oracle.com> > On Oct 12, 2018, at 10:56 AM, Roman Kennke wrote: > (If it was me, I'd rework this whole execution scheme to always call the > MT config, and let RP figure out and execute directly in VMThread if > it's only 1 worker, and not put this burden on the caller to set up both > for 1-threaded and N-threaded execution. This is just an ugly mess.) The ugliness here is, I think, largely a result of ParallelGC having it?s very own parallelization framework. There?s an RFE to make it use WorkGangs: https://bugs.openjdk.java.net/browse/JDK-8204951 Personally, I?d like to see that get worked on rather than piling on more cruft to work around that problem. And yes, I realize that?s a largish amount of work to take on in order to make progress on your immediate problem. So I?m not going to insist on such a path; you wouldn?t be the first person to kick this particular can down the road. But it sure would be nice? From sangheon.kim at oracle.com Fri Oct 12 20:26:47 2018 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Fri, 12 Oct 2018 13:26:47 -0700 Subject: RFR(XS): JDK-8212122: Allow ReferenceProcessor to always be MT processing In-Reply-To: References: Message-ID: Hi Roman, On 10/12/18 7:56 AM, Roman Kennke wrote: > Currently, in ReferenceProcessor, the mt_processing flag is adjusted for > each phase depending on the number of workers: if it's 1 worker, then > mt_processing is disabled, for more than 1 worker, mt_processing is > enabled. Depending on the flag, ref-processing is either done through > AbstractRefProcTaskExecutor, or directly in VMThread. However, we have a > situation where we always want to execute via the executor, and never > directly. Instead we decide in the executor whether or not we want to > execute directly in the VMThread. > > A simple change allows a subclass of ReferenceProcessor to change this > behavior. > > In case you're interested why we need to do this: Shenandoah requires > (for each worker) a scope around reference processing. In current > scheme, we'd need to put one scope around > RP::process_discovered_reference() to cover the 1-threaded case, and one > scope inside each worker to cover those. However, we cannot handle this > sort of weirdly nested scoping. > > (If it was me, I'd rework this whole execution scheme to always call the > MT config, and let RP figure out and execute directly in VMThread if > it's only 1 worker, and not put this burden on the caller to set up both > for 1-threaded and N-threaded execution. This is just an ugly mess.) I agree with you. In addition to Kim's comment which covered ParallelGC case, 1) For CMS, single vs. MT processing should be decided before calling RP? because of CMS specific implementation. For more detail, please refer comments at https://bugs.openjdk.java.net/browse/JDK-6938732 2) For G1, MT processing is set to true by default. But still caller has 2 paths as you described. I'm not sure whether single thread case is still needed there or not because RP is deciding to use single or multiple threads. Probably we can remove it unless an user really wants to force using single thread only. :) > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212122 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8212122/webrev.00/ Looks good to me. Thanks, Sangheon > > Testing: passes hotspot/tier1 locally. Works well with Shenandoah too. > > Ok? > > Roman > From rkennke at redhat.com Sat Oct 13 09:35:09 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sat, 13 Oct 2018 11:35:09 +0200 Subject: RFR(XS): JDK-8212122: Allow ReferenceProcessor to always be MT processing In-Reply-To: <8BFBCB9D-6ADE-47E6-A134-618E836CE71E@oracle.com> References: <8BFBCB9D-6ADE-47E6-A134-618E836CE71E@oracle.com> Message-ID: Hi Kim, >> On Oct 12, 2018, at 10:56 AM, Roman Kennke wrote: >> (If it was me, I'd rework this whole execution scheme to always call the >> MT config, and let RP figure out and execute directly in VMThread if >> it's only 1 worker, and not put this burden on the caller to set up both >> for 1-threaded and N-threaded execution. This is just an ugly mess.) > > The ugliness here is, I think, largely a result of ParallelGC having it?s very > own parallelization framework. There?s an RFE to make it use WorkGangs: > https://bugs.openjdk.java.net/browse/JDK-8204951 Yes, that may be true. > Personally, I?d like to see that get worked on rather than piling on more > cruft to work around that problem. Right. But the tiny patch can hardly be called 'piling on more cruft'. I may be willing to work on the above mentioned issue, but currently I cannot afford it. I'd like to get Shenandoah into JDK12, and can't afford to sidetrack myself with rewriting the interaction between ReferenceProcessor, ParallelGC and CMS. Would you agree to the proposed change for now? As I said, I might pick up the above mentioned bug later (feel free to assign it to me so that it doesn't get lost). Thanks, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Sat Oct 13 09:38:23 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sat, 13 Oct 2018 11:38:23 +0200 Subject: RFR(XS): JDK-8212122: Allow ReferenceProcessor to always be MT processing In-Reply-To: References: Message-ID: >> Currently, in ReferenceProcessor, the mt_processing flag is adjusted for >> each phase depending on the number of workers: if it's 1 worker, then >> mt_processing is disabled, for more than 1 worker, mt_processing is >> enabled. Depending on the flag, ref-processing is either done through >> AbstractRefProcTaskExecutor, or directly in VMThread. However, we have a >> situation where we always want to execute via the executor, and never >> directly. Instead we decide in the executor whether or not we want to >> execute directly in the VMThread. >> >> A simple change allows a subclass of ReferenceProcessor to change this >> behavior. >> >> In case you're interested why we need to do this: Shenandoah requires >> (for each worker) a scope around reference processing. In current >> scheme, we'd need to put one scope around >> RP::process_discovered_reference() to cover the 1-threaded case, and one >> scope inside each worker to cover those. However, we cannot handle this >> sort of weirdly nested scoping. >> >> (If it was me, I'd rework this whole execution scheme to always call the >> MT config, and let RP figure out and execute directly in VMThread if >> it's only 1 worker, and not put this burden on the caller to set up both >> for 1-threaded and N-threaded execution. This is just an ugly mess.) > I agree with you. > In addition to Kim's comment which covered ParallelGC case, > 1) For CMS, single vs. MT processing should be decided before calling > RP? because of CMS specific implementation. For more detail, please > refer comments at https://bugs.openjdk.java.net/browse/JDK-6938732 > 2) For G1, MT processing is set to true by default. But still caller has > 2 paths as you described. I'm not sure whether single thread case is > still needed there or not because RP is deciding to use single or > multiple threads. Probably we can remove it unless an user really wants > to force using single thread only. :) The problem is the (rather new) ergonomics code in RP that scales number of threads to spin up based on the amount of work to do. And if it ends up being only 1 thread, then it goes down the single-threaded-path. Otherwise I wouldn't actually need this change... (that might be another approach to this problem: don't actually change _mt_processing flag at all, and call the executor even with 1 thread. In Shenandoah, we do this and it works fine, and we special-case the 1-thread-path by not spinning up the workers and execute it in the calling thread (which is the VMThread). Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From synytskyy at jelastic.com Sat Oct 13 12:21:47 2018 From: synytskyy at jelastic.com (Ruslan Synytsky) Date: Sat, 13 Oct 2018 15:21:47 +0300 Subject: RFR: bug: Timely Reducing Unused Committed Memory In-Reply-To: <9c99d018799a7de321925f04def9ee5bfa451aab.camel@oracle.com> References: <8af5c2468f5133bc1ba2517bf52c3078af516977.camel@oracle.com> <65063e5f31702649e12b485b570ecab9ebd32d01.camel@oracle.com> <66c536399b2f218af0d85bd3bced44f204e3dbf4.camel@oracle.com> <2821cf70-c4d5-016f-b976-5ecc850f377c@oracle.com> <57b10708-b870-cb98-0df0-fab650f431e4@oracle.com> <3927596f-8847-ba95-d4c6-3c5f3656330c@oracle.com> <9bbf4a13-8490-d918-3cda-5c34acd16fa6@oracle.com> <9c99d018799a7de321925f04def9ee5bfa451aab.camel@oracle.com> Message-ID: On Thu, 11 Oct 2018 at 17:55, Thomas Schatzl wrote: > Hi, > > On Mon, 2018-10-08 at 22:09 +0300, Ruslan Synytsky wrote: > > Hi Thomas, thank you for the patch. > > > > On Mon, 8 Oct 2018 at 17:06, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > Hi all, > > > > > > here's the promised patch based on latest discussions: > > > > > > http://cr.openjdk.java.net/~tschatzl/jelastic/pgc-webrev.3/webrev/ > > > > > > It is based on "6490394: G1: Allow heap shrinking / memory > > > unmapping > > > after reclaiming regions during Remark", that is out for review > > > now. > > > > > > As for the options, they were named as follows: > > > > > > G1PeriodicGCInterval: Number of milliseconds after a previous GC to > > > wait before triggering a periodic gc. A value of zero disables > > > periodically enforced gc cycles. (Note that I just found out when > > > copy&pasting this description from the g1_globals file that the > > > text I there needs fixing - I will do thatasap :)) > > (Updated the webrev with all your comments) > > > > > > > G1PeriodicGCInvokesConcurrent: Determines the kind of periodic GC. > > > Set to true to have G1 perform a concurrent GC as periodic GC, > > > otherwise use a STW Full GC. > > > > Is concurrent enabled by default? If not, then why not? > > > > Concurrent is enabled by default as otherwise this change would > probably accidentally disturb operation too much :) > > > > > > > G1PeriodicGCSystemLoadThreshold: Maximum recent system wide system > > > load at which G1 triggers a periodic GC. A load above this value > > > cancels a given periodic GCs. A value of zero disables this check. > > > > Maybe a clear mention of loadavg in the description can help to avoid > > potential confusions around system load definition. > > Done. > > > > > > The policy has been implemented so that any reason to not start a > > > periodic collection right now will cancel the current attempt - > > > another attempt will be made after G1PeriodicGCInterval ms. This is > > > different to suppress the periodic gc until the next time the > > > conditions are met. > > > > > > From a default values POV, periodic gcs will be on by default with > > > an interval of 5mins. This should be fairly non-intrusive for all > > > but exceptional cases. > > > > > > Please check whether all this still fits your use case :) > > > > Looks good for me. Thanks again. > > I am currently testing the change a bit more, in particular I found > that the periodic gcs mess up prediction (obviously because they are > occurring during a time when the application is not very active, so the > data they return is not very helpful for normal application behavior). > I fixed that, but there may be some other details lurking that need > fixing. Other than that everything looks good so far (performance, > functional testing). > > Meanhwhile I will start updating the JEP again, and kick off the > process (initially mostly internal). > > > > > Btw, we have got an approved session at OracleCode about JVM > > elasticity > > > https://oracle.rainfocus.com/widget/oracle/oow18/catalogcodeone18?search=elastic%2Bjvm > > . I'm planning to bring up topics about general benefits, available > > options with different GCs and JVMs, specifics of running elastic JVM > > in containers, elaboration of possible further improvements. Please > > join the session for a live talk. > > > > Unfortunately I will not attend Code One this year, but I asked Per who > does (he's giving some talk about ZGC if you are interested) to attend > instead. I will certainly follow up on any recording there is :) Hi Thomas, yes, I?m definitely interested. How can I contact Per directly? As I would like to mention ZGC specifics related to the elasticity at my session. Thanks > > > Thanks, > Thomas > > > -- Ruslan CEO @ Jelastic -- Ruslan CEO @ Jelastic -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Mon Oct 15 08:47:24 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 15 Oct 2018 10:47:24 +0200 Subject: RFR (S) 8212083: Handle remaining gc/lock native code and fix two strings In-Reply-To: References: Message-ID: Hi, On Thu, 2018-10-11 at 15:23 -0700, JC Beyler wrote: > Hi all, > > I've cleaned up the gc/lock tests to use the ExceptionCheckingJniEnv > wrapper and now we have all JNI calls in that test group covered from > a "checking exceptions and returns of JNI methods". > > Let me know what you think: > Webrev: http://cr.openjdk.java.net/~jcbeyler/8212083/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8212083 > looks good to me as far as I understand the code. One nit: could you fix the #include declarations: they should be ordered lexicographically. Thanks, Thomas From thomas.schatzl at oracle.com Mon Oct 15 09:01:44 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 15 Oct 2018 11:01:44 +0200 Subject: UseGCOverheadLimit for G1 In-Reply-To: References: Message-ID: <9a6fb903181b6f13ee91a7733761b89283246c43.camel@oracle.com> Hi Man, On Fri, 2018-10-12 at 10:47 -0700, Man Cao wrote: > Sounds good. I'll take a stab at it. > > Does initiating the AdaptiveSizePolicy class for G1, or create a > subclass G1AdaptiveSizePolicy sound like an acceptable high-level > approach? > I know it is a non-goal to make UseAdaptiveSizePolicy work for G1, > but most logic for UseGCOverheadLimit is in the AdaptiveSizePolicy > class. > This seems the approach that reuse most existing code and does not > affect ParallelGC. > > A side story is that we have a Google-local patch since JDK6 that > makes UseGCOverheadLimit work properly for CMS. > UseAdaptiveSizePolicy doesn't work and is disabled by default for > CMS, but UseGCOverheadLimit depends on logic in the > CMSAdaptiveSizePolicy class (which was removed in JDK9) to properly > collect major GC overhead. Your short paragraph simply sound to me that the GC overhead limit checking is misplaced in the AdaptiveSizePolicy class. Since the code is also basically very much geared to a two-generation heap setup, we should not put it into e.g. CollectorPolicy either. What do you think about factoring out this code into a single class to be reused by the generational collectors? It gets fed using some sample() methods (for young and full gc each; feel free to adjust the name) by the collectors, and provides the recent average costs (I saw that AdaptiveSizePolicy uses them) and that check_gc_overhead_limit() method? Then this functionality could also be used properly by CMS in all of its modes, i.e. whether using AdaptiveSizePolicy or not I believe. Note that above is based on some quick look at the code, there might be complications as usual :) But importing AdaptiveSizePolicy wholesale into G1 seems very ugly. Thanks, Thomas From thomas.schatzl at oracle.com Mon Oct 15 09:06:06 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 15 Oct 2018 11:06:06 +0200 Subject: RFR(XS) 8212074: Add method to peek the remaining tasks in task queues In-Reply-To: <4ada4955-27b6-aabb-cae2-5b626c9d4aca@redhat.com> References: <4ada4955-27b6-aabb-cae2-5b626c9d4aca@redhat.com> Message-ID: <0ac38c0f7b33749d9c3fbd18089ab5b232f1bf4e.camel@oracle.com> Hi Zhengyu, On Fri, 2018-10-12 at 09:28 -0400, Zhengyu Gu wrote: > Hi All, > > Shenandoah implemented an variant of termination protocol based on > [1], which requires to peek into task queues for remaining tasks, so > that it can wake just enough workers to steal tasks. > > The patch adds methods for this purpose, and allow subclasses of > ParallelTaskTerminator to access it members. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8212074 > Webrev: http://cr.openjdk.java.net/~zgu/8212074/webrev.00/ could you please move line 451 of taskqueue.hpp in the obviously following patch that adds the ShenandoahTaskTerminator to the Shenandoah specific code? It does not seem to fit here. Otherwise it looks good. Btw, feel free to take over JDK-8204947 - I just did not get the time due to other more important changes. Thanks, Thomas From shade at redhat.com Mon Oct 15 10:24:13 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 15 Oct 2018 12:24:13 +0200 Subject: RFR (XS) 8212177: Epsilon alignment adjustments can overflow max TLAB size Message-ID: Bug: https://bugs.openjdk.java.net/browse/JDK-8212177 This was spotted by Per in review last week. Fix: http://cr.openjdk.java.net/~shade/8212177/webrev.01/ Testing: Linux {x86_64, x86_32} gc/epsilon, jdk-submit (running) Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Mon Oct 15 10:59:06 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 15 Oct 2018 12:59:06 +0200 Subject: RFR (S): 8211388: Make OtherRegionsTable independent of the region it is for In-Reply-To: <13515ce5470f232d34dfb1d232624770fe4dff28.camel@oracle.com> References: <13515ce5470f232d34dfb1d232624770fe4dff28.camel@oracle.com> Message-ID: Hi all, ping for a second review... Thomas On Tue, 2018-10-02 at 17:58 +0200, Thomas Schatzl wrote: > Hi, > > the OtherRegionsTable is a set of a set of cards that comprises the > remembered set. > > It stores the HeapRegion it collects cards for to do filtering in the > FromCardCache. The set of cards should not have knowledge about what > region it is for, from an abstraction POV it should just be a dumb > container of cards. > > Move out the HeapRegion id from OtherRegionsTable (the "set of > cards") > into the enclosing HeapRegionRememberedSet (managing set of cards), > separating the FCC functionality from the raw data storage. > > This will make code that assigns a HeapRegionRememberedSet to > multiple > regions in the future a bit simpler too. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8211388 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8211388/webrev/ > Testing: > hs-tier1-3 > > Thanks, > Thomas > > From thomas.schatzl at oracle.com Mon Oct 15 10:59:02 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 15 Oct 2018 12:59:02 +0200 Subject: RFR (M): 8071913: Filter out entries to free/uncommitted regions during iteration In-Reply-To: <809796367d72a91d192bfdb6fdb05617a7506914.camel@oracle.com> References: <809796367d72a91d192bfdb6fdb05617a7506914.camel@oracle.com> Message-ID: <0ca1d5ef3f37d50a6e65a007db845a4ddf63ef23.camel@oracle.com> Hi all, ping for a second review. Thomas On Tue, 2018-10-02 at 13:28 +0200, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that makes sure that remembered > set entries to free or uncommitted regions are not touched during GC? > > In order to implement uncommitting regions at remark we need to make > sure that these regions and their corresponding side data structures > are not touched by GC any more. > > This change adds a simple filter in the remembered set iterator plus > some careful changes in various locations of the code to avoid just > that. > > There are alternatives to this changes that have been considered but > ultimately rejected: > > - drop remembered sets when freeing regions: this would require us > to > scan all remembered sets for such regions which is slow. > > - do the filtering directly in the iterator: while this has been > prototyped, and even gives a significant improvement in pause times > (~5%), the change would be much larger (1500 LOC). Since I am > currently > reworking the remembered sets completely, I found it a waste of > reviewer's time to let you do this review now, and shortly afterwards > throw away all that code. I will incorporate the improvements into > the > new code though. :) > > If the remembered set changes won't make it e.g. for 12, we might > consider taking these changes in at a later time. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8071913 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8071913/webrev/ > Testing: > hs-tier1-5,jdk-tier1-3, perf tested, no change in pause times or > (throughput) scores > > Quite a few gc tests do extensive commit/uncommit of regions with the > corresponding exercise of the code. > > Thanks, > Thomas > > From thomas.schatzl at oracle.com Mon Oct 15 10:59:12 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 15 Oct 2018 12:59:12 +0200 Subject: RFR (S): 6490394: G1: Allow heap shrinking / memory unmapping after reclaiming regions during Remark In-Reply-To: <10549c6a3761fe3a2ec76a1dc3b001fa708520f6.camel@oracle.com> References: <10549c6a3761fe3a2ec76a1dc3b001fa708520f6.camel@oracle.com> Message-ID: <0ce908bfa750d11e1c3fdd5b9bd5aa0b40c3405b.camel@oracle.com> Hi, ping for a second review.. Thomas On Tue, 2018-10-02 at 17:57 +0200, Thomas Schatzl wrote: > Hi all, > > allow me to provide a fix for the oldest g1-specific open issue on > the bug tracker (from 2006 :)): resize the heap (eg. uncommit) > according to existing sizing policies at remark time. > > It does exactly what the subject of the CR says. > > The feature is enabled by default - I did not see any reason to > disable > it as the functionality seems to actually be what would be expected > of > the concurrent cycle (as it mirrors the functionality of a full gc). > > Needs the change for JDK-8071913 currently out for review applied to > work (i.e. not crash after uncommit). > > The current work for "JDK-8204089: Timely Reducing Unused Committed > Memory" also depends on this change. > > CR: > https://bugs.openjdk.java.net/browse/JDK-6490394 > Webrev: > http://cr.openjdk.java.net/~tschatzl/6490394/webrev/ > Testing: > hs-tier1-5, jdk-tier1-3, perf testing ongoing but not expecting a > difference > > Thanks, > Thomas > > > From zgu at redhat.com Mon Oct 15 13:03:31 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 15 Oct 2018 09:03:31 -0400 Subject: RFR(XS) 8212074: Add method to peek the remaining tasks in task queues In-Reply-To: <0ac38c0f7b33749d9c3fbd18089ab5b232f1bf4e.camel@oracle.com> References: <4ada4955-27b6-aabb-cae2-5b626c9d4aca@redhat.com> <0ac38c0f7b33749d9c3fbd18089ab5b232f1bf4e.camel@oracle.com> Message-ID: Hi Thomas, Thanks for reviewing. On 10/15/2018 05:06 AM, Thomas Schatzl wrote: > Hi Zhengyu, > > On Fri, 2018-10-12 at 09:28 -0400, Zhengyu Gu wrote: >> Hi All, >> >> Shenandoah implemented an variant of termination protocol based on >> [1], which requires to peek into task queues for remaining tasks, so >> that it can wake just enough workers to steal tasks. >> >> The patch adds methods for this purpose, and allow subclasses of >> ParallelTaskTerminator to access it members. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8212074 >> Webrev: http://cr.openjdk.java.net/~zgu/8212074/webrev.00/ > > could you please move line 451 of taskqueue.hpp in the obviously > following patch that adds the ShenandoahTaskTerminator to the > Shenandoah specific code? It does not seem to fit here. Agree. Updated webrev: http://cr.openjdk.java.net/~zgu/8212074/webrev.01/ > > Otherwise it looks good. Btw, feel free to take over JDK-8204947 - I > just did not get the time due to other more important changes. Will do. Thanks, -Zhengyu > > Thanks, > Thomas > > From thomas.schatzl at oracle.com Mon Oct 15 13:05:20 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 15 Oct 2018 15:05:20 +0200 Subject: RFR(XS) 8212074: Add method to peek the remaining tasks in task queues In-Reply-To: References: <4ada4955-27b6-aabb-cae2-5b626c9d4aca@redhat.com> <0ac38c0f7b33749d9c3fbd18089ab5b232f1bf4e.camel@oracle.com> Message-ID: Hi, On Mon, 2018-10-15 at 09:03 -0400, Zhengyu Gu wrote: > Hi Thomas, > > Thanks for reviewing. > > On 10/15/2018 05:06 AM, Thomas Schatzl wrote: > > Hi Zhengyu, > > > > On Fri, 2018-10-12 at 09:28 -0400, Zhengyu Gu wrote: > > > Hi All, > > > > > > Shenandoah implemented an variant of termination protocol based > > > on > > > [1], which requires to peek into task queues for remaining tasks, > > > so > > > that it can wake just enough workers to steal tasks. > > > > > > The patch adds methods for this purpose, and allow subclasses of > > > ParallelTaskTerminator to access it members. > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8212074 > > > Webrev: http://cr.openjdk.java.net/~zgu/8212074/webrev.00/ > > > > could you please move line 451 of taskqueue.hpp in the obviously > > following patch that adds the ShenandoahTaskTerminator to the > > Shenandoah specific code? It does not seem to fit here. > > Agree. > > Updated webrev: http://cr.openjdk.java.net/~zgu/8212074/webrev.01/ Looks good. Thomas From rkennke at redhat.com Mon Oct 15 13:15:53 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 15 Oct 2018 15:15:53 +0200 Subject: RFR(XS): JDK-8212122: Allow ReferenceProcessor to always be MT processing In-Reply-To: References: Message-ID: <0349a726-6b74-09da-47dc-d3335762c5f2@redhat.com> Am 13.10.18 um 11:38 schrieb Roman Kennke: > >>> Currently, in ReferenceProcessor, the mt_processing flag is adjusted for >>> each phase depending on the number of workers: if it's 1 worker, then >>> mt_processing is disabled, for more than 1 worker, mt_processing is >>> enabled. Depending on the flag, ref-processing is either done through >>> AbstractRefProcTaskExecutor, or directly in VMThread. However, we have a >>> situation where we always want to execute via the executor, and never >>> directly. Instead we decide in the executor whether or not we want to >>> execute directly in the VMThread. >>> >>> A simple change allows a subclass of ReferenceProcessor to change this >>> behavior. >>> >>> In case you're interested why we need to do this: Shenandoah requires >>> (for each worker) a scope around reference processing. In current >>> scheme, we'd need to put one scope around >>> RP::process_discovered_reference() to cover the 1-threaded case, and one >>> scope inside each worker to cover those. However, we cannot handle this >>> sort of weirdly nested scoping. >>> >>> (If it was me, I'd rework this whole execution scheme to always call the >>> MT config, and let RP figure out and execute directly in VMThread if >>> it's only 1 worker, and not put this burden on the caller to set up both >>> for 1-threaded and N-threaded execution. This is just an ugly mess.) >> I agree with you. >> In addition to Kim's comment which covered ParallelGC case, >> 1) For CMS, single vs. MT processing should be decided before calling >> RP? because of CMS specific implementation. For more detail, please >> refer comments at https://bugs.openjdk.java.net/browse/JDK-6938732 >> 2) For G1, MT processing is set to true by default. But still caller has >> 2 paths as you described. I'm not sure whether single thread case is >> still needed there or not because RP is deciding to use single or >> multiple threads. Probably we can remove it unless an user really wants >> to force using single thread only. :) > > > The problem is the (rather new) ergonomics code in RP that scales number > of threads to spin up based on the amount of work to do. And if it ends > up being only 1 thread, then it goes down the single-threaded-path. > Otherwise I wouldn't actually need this change... (that might be another > approach to this problem: don't actually change _mt_processing flag at > all, and call the executor even with 1 thread. In Shenandoah, we do this > and it works fine, and we special-case the 1-thread-path by not spinning > up the workers and execute it in the calling thread (which is the VMThread). In other words, this is a fix that actually removes 'cruft', and seems less surprising API-wise: i.e. if caller asks for mt_processing, then don't change it, and leave it to caller to execute in VMThread if it wishes. Updated webrev: http://cr.openjdk.java.net/~rkennke/JDK-8212122/webrev.01/ An example how G1 can be made to strictly do the current behaviour (i.e. execute in VMThread when ergo_workers==1): http://cr.openjdk.java.net/~rkennke/JDK-8212122/webrev.01.g1/ ... altough I haven't tested it at all. It works well with Shenandoah, but I don't know if you'd actually do it/want it/ask me to do it ;-) What do you think? Which approach do you like best (except the huge parallel/CMS/WorkGang/ReferenceProcessor reworking)? Thanks, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Mon Oct 15 13:17:27 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 15 Oct 2018 15:17:27 +0200 Subject: RFR(XS) 8212074: Add method to peek the remaining tasks in task queues In-Reply-To: References: <4ada4955-27b6-aabb-cae2-5b626c9d4aca@redhat.com> <0ac38c0f7b33749d9c3fbd18089ab5b232f1bf4e.camel@oracle.com> Message-ID: <3f727413-1f78-6dc2-fbbc-31794f3797a6@redhat.com> Hi Zhengyu. >> Hi Zhengyu, >> >> On Fri, 2018-10-12 at 09:28 -0400, Zhengyu Gu wrote: >>> Hi All, >>> >>> Shenandoah implemented an variant of termination protocol based on >>> [1], which requires to peek into task queues for remaining tasks, so >>> that it can wake just enough workers to steal tasks. >>> >>> The patch adds methods for this purpose, and allow subclasses of >>> ParallelTaskTerminator to access it members. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8212074 >>> Webrev: http://cr.openjdk.java.net/~zgu/8212074/webrev.00/ >> >> ?? could you please move line 451 of taskqueue.hpp in the obviously >> following patch that adds the ShenandoahTaskTerminator to the >> Shenandoah specific code? It does not seem to fit here. > > Agree. > > Updated webrev: http://cr.openjdk.java.net/~zgu/8212074/webrev.01/ Looks good to me. Thanks, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Mon Oct 15 13:24:31 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 15 Oct 2018 09:24:31 -0400 Subject: RFR(XS) 8212074: Add method to peek the remaining tasks in task queues In-Reply-To: <3f727413-1f78-6dc2-fbbc-31794f3797a6@redhat.com> References: <4ada4955-27b6-aabb-cae2-5b626c9d4aca@redhat.com> <0ac38c0f7b33749d9c3fbd18089ab5b232f1bf4e.camel@oracle.com> <3f727413-1f78-6dc2-fbbc-31794f3797a6@redhat.com> Message-ID: Thanks for reviewing, Roman. -Zhengyu On 10/15/2018 09:17 AM, Roman Kennke wrote: > Hi Zhengyu. > >>> Hi Zhengyu, >>> >>> On Fri, 2018-10-12 at 09:28 -0400, Zhengyu Gu wrote: >>>> Hi All, >>>> >>>> Shenandoah implemented an variant of termination protocol based on >>>> [1], which requires to peek into task queues for remaining tasks, so >>>> that it can wake just enough workers to steal tasks. >>>> >>>> The patch adds methods for this purpose, and allow subclasses of >>>> ParallelTaskTerminator to access it members. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8212074 >>>> Webrev: http://cr.openjdk.java.net/~zgu/8212074/webrev.00/ >>> >>> ?? could you please move line 451 of taskqueue.hpp in the obviously >>> following patch that adds the ShenandoahTaskTerminator to the >>> Shenandoah specific code? It does not seem to fit here. >> >> Agree. >> >> Updated webrev: http://cr.openjdk.java.net/~zgu/8212074/webrev.01/ > > Looks good to me. > > Thanks, > Roman > > From shade at redhat.com Mon Oct 15 13:26:31 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 15 Oct 2018 15:26:31 +0200 Subject: RFR(XS) 8212074: Add method to peek the remaining tasks in task queues In-Reply-To: References: <4ada4955-27b6-aabb-cae2-5b626c9d4aca@redhat.com> <0ac38c0f7b33749d9c3fbd18089ab5b232f1bf4e.camel@oracle.com> Message-ID: <1cbae9a7-523d-4e66-7e9d-838d662c09b6@redhat.com> On 10/15/2018 03:03 PM, Zhengyu Gu wrote: > Updated webrev: http://cr.openjdk.java.net/~zgu/8212074/webrev.01/ Looks good to me. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Mon Oct 15 13:27:42 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 15 Oct 2018 15:27:42 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: <45395b5d-216c-eb40-1e03-fed6167b4b4f@oracle.com> References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <2aa7c91e-9b1e-3129-cd7a-d1fa50da422e@redhat.com> <07b75752-0430-68d0-a0b3-5bdf8a4b1d1d@redhat.com> <376d3148-6492-ada6-3514-fd0779e6786d@redhat.com> <9467830c-0983-fc12-7f3a-1446e5b85362@oracle.com> <9c425de5-387f-6d7d-6ee2-cf6160a0d46f@redhat.com> <13396609-82b6-be14-f1fe-cc168b8db95d@oracle.com> <45395b5d-216c-eb40-1e03-fed6167b4b4f@oracle.com> Message-ID: Hi Per, what do you think about putting this proposed change in jdk/jdk now, and work on the improvements later / on top of it? It does provide some reasonable consolidation I think, which seems worth as-is. Thanks, Roman > Hi Roman, > > On 10/11/2018 02:41 PM, Roman Kennke wrote: > [...] >> Those are the only functions where this matters. In-fact, the LAB >> reserve abstraction is the last one I have lined up. The remaining stuff >> that Shenandoah needs is a bunch of extra utility classes and C2 changes. > > Thanks for clarifying. That makes me much less worried. > > For C2, the strategy we had and the requirement we put on our selves > when integrating ZGC was basically this: > > We really didn't want to screw things up in C2. If things went south, we > wanted to just be able to disable building of ZGC and C2 would go back > to do _exactly_ what it did before we merged the ZGC patch. As you maybe > remember, we _almost_ got there. We had a very small set of places in C2 > where we did change/add code, which was _not_ disabled when disabling > ZGC. However, it was easy to prove that that code was safe/correct. This > strategy turned out to work well, and it forced us to think hard about > having abstractions in the right places, move as much code as possible > into the BarrierSet, etc. > > I'd recommend you do the same for Shenandoah, that way the pressure on > you (in case something breaks and results instability or a long bug > trail) will be reduced a lot. If shit hits the fan, you can always > disable building of Shenandoah until the problems are sorted out. > > [...] >>>>> Generally, having PLAB call Universe::heap()->tlab_alloc_reserve() >>>>> looks >>>>> wrong to me, since a PLAB is not a TLAB. If we really want to share >>>>> this >>>>> code I think we should try to find some other way. Maybe just give >>>>> tlab_alloc_reserve() a better name, but at the same time I'm not even >>>>> sure this belongs in CollectedHeap. >>>> >>>> We can change it to lab_alloc_reserve() instead. I think it should be >>>> shared between TLAB and PLAB because the concept is the same thing, and >>>> relies on the same properties: we need some reserve at the end in order >>>> to be able to fill it completely, and with Shenandoah this is GC >>>> specific. And since all the other code that deals with filling TLABs >>>> and >>>> PLABs resides in CollectedHeap, this naturally belongs there too, right >>>> next to CollectedHeap::fill_with_dummy_object(..) which is also used by >>>> both TLAB and PLAB. >>> >>> Maybe a stupid question, but why do you even need a Brooks pointer in >>> your filler objects? Is not having that maybe a solution here? >> >> Filler objects are objects. Laying them out differently would make >> parsing the heap impossible. > > Check. > > I thought some more about this. How about we rip out the filler-stuff > from CollectedHeap, and put it in a Fill utility class. You could think > of Fill as a cosine to the "class Copy" we have today. Something like: > > class Fill : public AllStatic { > private: > ? static int _min_size; > > public: > ? static void set_min_size(int min_size) { > ??? _min_size = min_size; > ? } > > ? static int min_size() { > ??? return _min_size; > ? } > > ? static int min_reserve_size() { > ??? return _min_size > MinObjAlignment ? _min_size : 0; > ? } > > ? static void with_objects(...); > ? static void with_object(...); > ? ... > }; > > Fill::_min_size is initialized to oopDesc::header_size() by default. A > collector that wants something different calls Fill::set_min_size(...) > during start up. TLAB/PLAB would call Fill::min_reserve_size(). > > In my mind, I think this would put things where they belong. And as a > bonus we would cut out some cruft from CollectedHeap. What do you think? > I started doing a patch for this to see if it seems like a good > approach. I hope to have something to show later today. > > cheers, > Per -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Mon Oct 15 13:28:22 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 15 Oct 2018 09:28:22 -0400 Subject: RFR(XS) 8212074: Add method to peek the remaining tasks in task queues In-Reply-To: <1cbae9a7-523d-4e66-7e9d-838d662c09b6@redhat.com> References: <4ada4955-27b6-aabb-cae2-5b626c9d4aca@redhat.com> <0ac38c0f7b33749d9c3fbd18089ab5b232f1bf4e.camel@oracle.com> <1cbae9a7-523d-4e66-7e9d-838d662c09b6@redhat.com> Message-ID: <5b6b4865-f5cb-282a-48f0-fe23d9b7625d@redhat.com> Thanks, Aleksey. -Zhengyu On 10/15/2018 09:26 AM, Aleksey Shipilev wrote: > On 10/15/2018 03:03 PM, Zhengyu Gu wrote: >> Updated webrev: http://cr.openjdk.java.net/~zgu/8212074/webrev.01/ > > Looks good to me. > > -Aleksey > From per.liden at oracle.com Mon Oct 15 14:41:41 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 15 Oct 2018 16:41:41 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <2aa7c91e-9b1e-3129-cd7a-d1fa50da422e@redhat.com> <07b75752-0430-68d0-a0b3-5bdf8a4b1d1d@redhat.com> <376d3148-6492-ada6-3514-fd0779e6786d@redhat.com> <9467830c-0983-fc12-7f3a-1446e5b85362@oracle.com> <9c425de5-387f-6d7d-6ee2-cf6160a0d46f@redhat.com> <13396609-82b6-be14-f1fe-cc168b8db95d@oracle.com> <45395b5d-216c-eb40-1e03-fed6167b4b4f@oracle.com> Message-ID: <7c637b25-7695-132b-551c-a8b06705c264@oracle.com> Hi Roman, On 10/15/2018 03:27 PM, Roman Kennke wrote: > Hi Per, > > what do you think about putting this proposed change in jdk/jdk now, and > work on the improvements later / on top of it? It does provide some > reasonable consolidation I think, which seems worth as-is. Sure. I still have at least one bug to sort out in my patch, and today I've been derailed by some higher priority stuff. So, go ahead and I'll follow up with my patch later. cheers, Per > > Thanks, > Roman > > >> Hi Roman, >> >> On 10/11/2018 02:41 PM, Roman Kennke wrote: >> [...] >>> Those are the only functions where this matters. In-fact, the LAB >>> reserve abstraction is the last one I have lined up. The remaining stuff >>> that Shenandoah needs is a bunch of extra utility classes and C2 changes. >> >> Thanks for clarifying. That makes me much less worried. >> >> For C2, the strategy we had and the requirement we put on our selves >> when integrating ZGC was basically this: >> >> We really didn't want to screw things up in C2. If things went south, we >> wanted to just be able to disable building of ZGC and C2 would go back >> to do _exactly_ what it did before we merged the ZGC patch. As you maybe >> remember, we _almost_ got there. We had a very small set of places in C2 >> where we did change/add code, which was _not_ disabled when disabling >> ZGC. However, it was easy to prove that that code was safe/correct. This >> strategy turned out to work well, and it forced us to think hard about >> having abstractions in the right places, move as much code as possible >> into the BarrierSet, etc. >> >> I'd recommend you do the same for Shenandoah, that way the pressure on >> you (in case something breaks and results instability or a long bug >> trail) will be reduced a lot. If shit hits the fan, you can always >> disable building of Shenandoah until the problems are sorted out. >> >> [...] >>>>>> Generally, having PLAB call Universe::heap()->tlab_alloc_reserve() >>>>>> looks >>>>>> wrong to me, since a PLAB is not a TLAB. If we really want to share >>>>>> this >>>>>> code I think we should try to find some other way. Maybe just give >>>>>> tlab_alloc_reserve() a better name, but at the same time I'm not even >>>>>> sure this belongs in CollectedHeap. >>>>> >>>>> We can change it to lab_alloc_reserve() instead. I think it should be >>>>> shared between TLAB and PLAB because the concept is the same thing, and >>>>> relies on the same properties: we need some reserve at the end in order >>>>> to be able to fill it completely, and with Shenandoah this is GC >>>>> specific. And since all the other code that deals with filling TLABs >>>>> and >>>>> PLABs resides in CollectedHeap, this naturally belongs there too, right >>>>> next to CollectedHeap::fill_with_dummy_object(..) which is also used by >>>>> both TLAB and PLAB. >>>> >>>> Maybe a stupid question, but why do you even need a Brooks pointer in >>>> your filler objects? Is not having that maybe a solution here? >>> >>> Filler objects are objects. Laying them out differently would make >>> parsing the heap impossible. >> >> Check. >> >> I thought some more about this. How about we rip out the filler-stuff >> from CollectedHeap, and put it in a Fill utility class. You could think >> of Fill as a cosine to the "class Copy" we have today. Something like: >> >> class Fill : public AllStatic { >> private: >> ? static int _min_size; >> >> public: >> ? static void set_min_size(int min_size) { >> ??? _min_size = min_size; >> ? } >> >> ? static int min_size() { >> ??? return _min_size; >> ? } >> >> ? static int min_reserve_size() { >> ??? return _min_size > MinObjAlignment ? _min_size : 0; >> ? } >> >> ? static void with_objects(...); >> ? static void with_object(...); >> ? ... >> }; >> >> Fill::_min_size is initialized to oopDesc::header_size() by default. A >> collector that wants something different calls Fill::set_min_size(...) >> during start up. TLAB/PLAB would call Fill::min_reserve_size(). >> >> In my mind, I think this would put things where they belong. And as a >> bonus we would cut out some cruft from CollectedHeap. What do you think? >> I started doing a patch for this to see if it seems like a good >> approach. I hope to have something to show later today. >> >> cheers, >> Per > From per.liden at oracle.com Mon Oct 15 14:42:44 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 15 Oct 2018 16:42:44 +0200 Subject: RFR: 8212184: Incorrect oop ref strength used for referents in FinalReference Message-ID: <9363d38d-eb3c-b06c-2c23-88d8f71fe21f@oracle.com> AccessBarrierSupport::resolve_unknown_oop_ref_strength() returns an incorrect oop ref strength for referents in FinalReference objects. It currently returns ON_WEAK_OOP_REF when it should return ON_STRONG_OOP_REF. This is not really an issue for any GC except ZGC when using the ZHeapIterator to walk the heap while resurrection is blocked. Bug: https://bugs.openjdk.java.net/browse/JDK-8212184 Webrev: http://cr.openjdk.java.net/~pliden/8212184/webrev.0 /Per From per.liden at oracle.com Mon Oct 15 14:44:44 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 15 Oct 2018 16:44:44 +0200 Subject: RFR: 8212181: ZGC: Fix incorrect root iteration in ZHeapIterator Message-ID: <4123e36b-fd4f-7ee3-e5b1-5e37674b3378@oracle.com> The ZHeapIterator should not make callbacks to the user supplied ObjectClosure while iterating over roots. Instead all roots should be pushed to the stack first, before any call to the user supplied ObjectClosure is made. This is needed since the user supplied ObjectClosure could otherwise modify the root set (e.g. the JVMTI TagMap) while we're iterating over it, which can lead to memory corruption. Bug: https://bugs.openjdk.java.net/browse/JDK-8212181 Webrev: http://cr.openjdk.java.net/~pliden/8212181/webrev.0 /Per From erik.osterlund at oracle.com Mon Oct 15 14:46:37 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 15 Oct 2018 16:46:37 +0200 Subject: RFR: 8212184: Incorrect oop ref strength used for referents in FinalReference In-Reply-To: <9363d38d-eb3c-b06c-2c23-88d8f71fe21f@oracle.com> References: <9363d38d-eb3c-b06c-2c23-88d8f71fe21f@oracle.com> Message-ID: <5BC4A84D.8040809@oracle.com> Hi Per, Looks good. Thanks, /Erik On 2018-10-15 16:42, Per Liden wrote: > AccessBarrierSupport::resolve_unknown_oop_ref_strength() returns an > incorrect oop ref strength for referents in FinalReference objects. It > currently returns ON_WEAK_OOP_REF when it should return > ON_STRONG_OOP_REF. This is not really an issue for any GC except ZGC > when using the ZHeapIterator to walk the heap while resurrection is > blocked. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8212184 > Webrev: http://cr.openjdk.java.net/~pliden/8212184/webrev.0 > > /Per From erik.osterlund at oracle.com Mon Oct 15 14:47:35 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 15 Oct 2018 16:47:35 +0200 Subject: RFR: 8212181: ZGC: Fix incorrect root iteration in ZHeapIterator In-Reply-To: <4123e36b-fd4f-7ee3-e5b1-5e37674b3378@oracle.com> References: <4123e36b-fd4f-7ee3-e5b1-5e37674b3378@oracle.com> Message-ID: <5BC4A886.9010809@oracle.com> Hi Per, Looks good. Thanks, /Erik On 2018-10-15 16:44, Per Liden wrote: > The ZHeapIterator should not make callbacks to the user supplied > ObjectClosure while iterating over roots. Instead all roots should be > pushed to the stack first, before any call to the user supplied > ObjectClosure is made. This is needed since the user supplied > ObjectClosure could otherwise modify the root set (e.g. the JVMTI > TagMap) while we're iterating over it, which can lead to memory > corruption. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8212181 > Webrev: http://cr.openjdk.java.net/~pliden/8212181/webrev.0 > > /Per From rkennke at redhat.com Mon Oct 15 14:47:43 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 15 Oct 2018 16:47:43 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: <7c637b25-7695-132b-551c-a8b06705c264@oracle.com> References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <2aa7c91e-9b1e-3129-cd7a-d1fa50da422e@redhat.com> <07b75752-0430-68d0-a0b3-5bdf8a4b1d1d@redhat.com> <376d3148-6492-ada6-3514-fd0779e6786d@redhat.com> <9467830c-0983-fc12-7f3a-1446e5b85362@oracle.com> <9c425de5-387f-6d7d-6ee2-cf6160a0d46f@redhat.com> <13396609-82b6-be14-f1fe-cc168b8db95d@oracle.com> <45395b5d-216c-eb40-1e03-fed6167b4b4f@oracle.com> <7c637b25-7695-132b-551c-a8b06705c264@oracle.com> Message-ID: <3c669729-c509-f3f3-7568-e572293336e9@redhat.com> Hi Per, Thanks! I take that as 'Reviewed' except that I believe we haven't been clear which one. The last one I posted was did also include changes for whitebox and jvmti and rename to 'cell_size': http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.03/ but I'm not sure we wanted that. And the one before was: http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.02/ I suggest to go with that for now. Ok? Roman > Hi Roman, > > On 10/15/2018 03:27 PM, Roman Kennke wrote: >> Hi Per, >> >> what do you think about putting this proposed change in jdk/jdk now, and >> work on the improvements later / on top of it? It does provide some >> reasonable consolidation I think, which seems worth as-is. > > Sure. I still have at least one bug to sort out in my patch, and today > I've been derailed by some higher priority stuff. So, go ahead and I'll > follow up with my patch later. > > cheers, > Per > >> >> Thanks, >> Roman >> >> >>> Hi Roman, >>> >>> On 10/11/2018 02:41 PM, Roman Kennke wrote: >>> [...] >>>> Those are the only functions where this matters. In-fact, the LAB >>>> reserve abstraction is the last one I have lined up. The remaining >>>> stuff >>>> that Shenandoah needs is a bunch of extra utility classes and C2 >>>> changes. >>> >>> Thanks for clarifying. That makes me much less worried. >>> >>> For C2, the strategy we had and the requirement we put on our selves >>> when integrating ZGC was basically this: >>> >>> We really didn't want to screw things up in C2. If things went south, we >>> wanted to just be able to disable building of ZGC and C2 would go back >>> to do _exactly_ what it did before we merged the ZGC patch. As you maybe >>> remember, we _almost_ got there. We had a very small set of places in C2 >>> where we did change/add code, which was _not_ disabled when disabling >>> ZGC. However, it was easy to prove that that code was safe/correct. This >>> strategy turned out to work well, and it forced us to think hard about >>> having abstractions in the right places, move as much code as possible >>> into the BarrierSet, etc. >>> >>> I'd recommend you do the same for Shenandoah, that way the pressure on >>> you (in case something breaks and results instability or a long bug >>> trail) will be reduced a lot. If shit hits the fan, you can always >>> disable building of Shenandoah until the problems are sorted out. >>> >>> [...] >>>>>>> Generally, having PLAB call Universe::heap()->tlab_alloc_reserve() >>>>>>> looks >>>>>>> wrong to me, since a PLAB is not a TLAB. If we really want to share >>>>>>> this >>>>>>> code I think we should try to find some other way. Maybe just give >>>>>>> tlab_alloc_reserve() a better name, but at the same time I'm not >>>>>>> even >>>>>>> sure this belongs in CollectedHeap. >>>>>> >>>>>> We can change it to lab_alloc_reserve() instead. I think it should be >>>>>> shared between TLAB and PLAB because the concept is the same >>>>>> thing, and >>>>>> relies on the same properties: we need some reserve at the end in >>>>>> order >>>>>> to be able to fill it completely, and with Shenandoah this is GC >>>>>> specific. And since all the other code that deals with filling TLABs >>>>>> and >>>>>> PLABs resides in CollectedHeap, this naturally belongs there too, >>>>>> right >>>>>> next to CollectedHeap::fill_with_dummy_object(..) which is also >>>>>> used by >>>>>> both TLAB and PLAB. >>>>> >>>>> Maybe a stupid question, but why do you even need a Brooks pointer in >>>>> your filler objects? Is not having that maybe a solution here? >>>> >>>> Filler objects are objects. Laying them out differently would make >>>> parsing the heap impossible. >>> >>> Check. >>> >>> I thought some more about this. How about we rip out the filler-stuff >>> from CollectedHeap, and put it in a Fill utility class. You could think >>> of Fill as a cosine to the "class Copy" we have today. Something like: >>> >>> class Fill : public AllStatic { >>> private: >>> ?? static int _min_size; >>> >>> public: >>> ?? static void set_min_size(int min_size) { >>> ???? _min_size = min_size; >>> ?? } >>> >>> ?? static int min_size() { >>> ???? return _min_size; >>> ?? } >>> >>> ?? static int min_reserve_size() { >>> ???? return _min_size > MinObjAlignment ? _min_size : 0; >>> ?? } >>> >>> ?? static void with_objects(...); >>> ?? static void with_object(...); >>> ?? ... >>> }; >>> >>> Fill::_min_size is initialized to oopDesc::header_size() by default. A >>> collector that wants something different calls Fill::set_min_size(...) >>> during start up. TLAB/PLAB would call Fill::min_reserve_size(). >>> >>> In my mind, I think this would put things where they belong. And as a >>> bonus we would cut out some cruft from CollectedHeap. What do you think? >>> I started doing a patch for this to see if it seems like a good >>> approach. I hope to have something to show later today. >>> >>> cheers, >>> Per >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Mon Oct 15 14:55:03 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 15 Oct 2018 16:55:03 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: <3c669729-c509-f3f3-7568-e572293336e9@redhat.com> References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <2aa7c91e-9b1e-3129-cd7a-d1fa50da422e@redhat.com> <07b75752-0430-68d0-a0b3-5bdf8a4b1d1d@redhat.com> <376d3148-6492-ada6-3514-fd0779e6786d@redhat.com> <9467830c-0983-fc12-7f3a-1446e5b85362@oracle.com> <9c425de5-387f-6d7d-6ee2-cf6160a0d46f@redhat.com> <13396609-82b6-be14-f1fe-cc168b8db95d@oracle.com> <45395b5d-216c-eb40-1e03-fed6167b4b4f@oracle.com> <7c637b25-7695-132b-551c-a8b06705c264@oracle.com> <3c669729-c509-f3f3-7568-e572293336e9@redhat.com> Message-ID: <894032be-e49f-a1f0-38b1-b8be0c49f87c@oracle.com> Hi, On 10/15/2018 04:47 PM, Roman Kennke wrote: > Hi Per, > > Thanks! I take that as 'Reviewed' except that I believe we haven't been > clear which one. The last one I posted was did also include changes for > whitebox and jvmti and rename to 'cell_size': > > http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.03/ > > but I'm not sure we wanted that. > > And the one before was: > http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.02/ > > I suggest to go with that for now. Ok? Yes, please go with webrev.02 for now and let's revisit the cell concept at a later time. cheers, Per > > Roman > > >> Hi Roman, >> >> On 10/15/2018 03:27 PM, Roman Kennke wrote: >>> Hi Per, >>> >>> what do you think about putting this proposed change in jdk/jdk now, and >>> work on the improvements later / on top of it? It does provide some >>> reasonable consolidation I think, which seems worth as-is. >> >> Sure. I still have at least one bug to sort out in my patch, and today >> I've been derailed by some higher priority stuff. So, go ahead and I'll >> follow up with my patch later. >> >> cheers, >> Per >> >>> >>> Thanks, >>> Roman >>> >>> >>>> Hi Roman, >>>> >>>> On 10/11/2018 02:41 PM, Roman Kennke wrote: >>>> [...] >>>>> Those are the only functions where this matters. In-fact, the LAB >>>>> reserve abstraction is the last one I have lined up. The remaining >>>>> stuff >>>>> that Shenandoah needs is a bunch of extra utility classes and C2 >>>>> changes. >>>> >>>> Thanks for clarifying. That makes me much less worried. >>>> >>>> For C2, the strategy we had and the requirement we put on our selves >>>> when integrating ZGC was basically this: >>>> >>>> We really didn't want to screw things up in C2. If things went south, we >>>> wanted to just be able to disable building of ZGC and C2 would go back >>>> to do _exactly_ what it did before we merged the ZGC patch. As you maybe >>>> remember, we _almost_ got there. We had a very small set of places in C2 >>>> where we did change/add code, which was _not_ disabled when disabling >>>> ZGC. However, it was easy to prove that that code was safe/correct. This >>>> strategy turned out to work well, and it forced us to think hard about >>>> having abstractions in the right places, move as much code as possible >>>> into the BarrierSet, etc. >>>> >>>> I'd recommend you do the same for Shenandoah, that way the pressure on >>>> you (in case something breaks and results instability or a long bug >>>> trail) will be reduced a lot. If shit hits the fan, you can always >>>> disable building of Shenandoah until the problems are sorted out. >>>> >>>> [...] >>>>>>>> Generally, having PLAB call Universe::heap()->tlab_alloc_reserve() >>>>>>>> looks >>>>>>>> wrong to me, since a PLAB is not a TLAB. If we really want to share >>>>>>>> this >>>>>>>> code I think we should try to find some other way. Maybe just give >>>>>>>> tlab_alloc_reserve() a better name, but at the same time I'm not >>>>>>>> even >>>>>>>> sure this belongs in CollectedHeap. >>>>>>> >>>>>>> We can change it to lab_alloc_reserve() instead. I think it should be >>>>>>> shared between TLAB and PLAB because the concept is the same >>>>>>> thing, and >>>>>>> relies on the same properties: we need some reserve at the end in >>>>>>> order >>>>>>> to be able to fill it completely, and with Shenandoah this is GC >>>>>>> specific. And since all the other code that deals with filling TLABs >>>>>>> and >>>>>>> PLABs resides in CollectedHeap, this naturally belongs there too, >>>>>>> right >>>>>>> next to CollectedHeap::fill_with_dummy_object(..) which is also >>>>>>> used by >>>>>>> both TLAB and PLAB. >>>>>> >>>>>> Maybe a stupid question, but why do you even need a Brooks pointer in >>>>>> your filler objects? Is not having that maybe a solution here? >>>>> >>>>> Filler objects are objects. Laying them out differently would make >>>>> parsing the heap impossible. >>>> >>>> Check. >>>> >>>> I thought some more about this. How about we rip out the filler-stuff >>>> from CollectedHeap, and put it in a Fill utility class. You could think >>>> of Fill as a cosine to the "class Copy" we have today. Something like: >>>> >>>> class Fill : public AllStatic { >>>> private: >>>> ?? static int _min_size; >>>> >>>> public: >>>> ?? static void set_min_size(int min_size) { >>>> ???? _min_size = min_size; >>>> ?? } >>>> >>>> ?? static int min_size() { >>>> ???? return _min_size; >>>> ?? } >>>> >>>> ?? static int min_reserve_size() { >>>> ???? return _min_size > MinObjAlignment ? _min_size : 0; >>>> ?? } >>>> >>>> ?? static void with_objects(...); >>>> ?? static void with_object(...); >>>> ?? ... >>>> }; >>>> >>>> Fill::_min_size is initialized to oopDesc::header_size() by default. A >>>> collector that wants something different calls Fill::set_min_size(...) >>>> during start up. TLAB/PLAB would call Fill::min_reserve_size(). >>>> >>>> In my mind, I think this would put things where they belong. And as a >>>> bonus we would cut out some cruft from CollectedHeap. What do you think? >>>> I started doing a patch for this to see if it seems like a good >>>> approach. I hope to have something to show later today. >>>> >>>> cheers, >>>> Per >>> > From rkennke at redhat.com Mon Oct 15 15:00:05 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 15 Oct 2018 17:00:05 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: <894032be-e49f-a1f0-38b1-b8be0c49f87c@oracle.com> References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <2aa7c91e-9b1e-3129-cd7a-d1fa50da422e@redhat.com> <07b75752-0430-68d0-a0b3-5bdf8a4b1d1d@redhat.com> <376d3148-6492-ada6-3514-fd0779e6786d@redhat.com> <9467830c-0983-fc12-7f3a-1446e5b85362@oracle.com> <9c425de5-387f-6d7d-6ee2-cf6160a0d46f@redhat.com> <13396609-82b6-be14-f1fe-cc168b8db95d@oracle.com> <45395b5d-216c-eb40-1e03-fed6167b4b4f@oracle.com> <7c637b25-7695-132b-551c-a8b06705c264@oracle.com> <3c669729-c509-f3f3-7568-e572293336e9@redhat.com> <894032be-e49f-a1f0-38b1-b8be0c49f87c@oracle.com> Message-ID: <4cf5908e-acee-15ca-b829-7820b9ac3eef@redhat.com> Thanks Per! Roman > Hi, > > On 10/15/2018 04:47 PM, Roman Kennke wrote: >> Hi Per, >> >> Thanks! I take that as 'Reviewed' except that I believe we haven't been >> clear which one. The last one I posted was did also include changes for >> whitebox and jvmti and rename to 'cell_size': >> >> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.03/ >> >> but I'm not sure we wanted that. >> >> And the one before was: >> http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.02/ >> >> I suggest to go with that for now. Ok? > > Yes, please go with webrev.02 for now and let's revisit the cell concept > at a later time. > > cheers, > Per > >> >> Roman >> >> >>> Hi Roman, >>> >>> On 10/15/2018 03:27 PM, Roman Kennke wrote: >>>> Hi Per, >>>> >>>> what do you think about putting this proposed change in jdk/jdk now, >>>> and >>>> work on the improvements later / on top of it? It does provide some >>>> reasonable consolidation I think, which seems worth as-is. >>> >>> Sure. I still have at least one bug to sort out in my patch, and today >>> I've been derailed by some higher priority stuff. So, go ahead and I'll >>> follow up with my patch later. >>> >>> cheers, >>> Per >>> >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi Roman, >>>>> >>>>> On 10/11/2018 02:41 PM, Roman Kennke wrote: >>>>> [...] >>>>>> Those are the only functions where this matters. In-fact, the LAB >>>>>> reserve abstraction is the last one I have lined up. The remaining >>>>>> stuff >>>>>> that Shenandoah needs is a bunch of extra utility classes and C2 >>>>>> changes. >>>>> >>>>> Thanks for clarifying. That makes me much less worried. >>>>> >>>>> For C2, the strategy we had and the requirement we put on our selves >>>>> when integrating ZGC was basically this: >>>>> >>>>> We really didn't want to screw things up in C2. If things went >>>>> south, we >>>>> wanted to just be able to disable building of ZGC and C2 would go back >>>>> to do _exactly_ what it did before we merged the ZGC patch. As you >>>>> maybe >>>>> remember, we _almost_ got there. We had a very small set of places >>>>> in C2 >>>>> where we did change/add code, which was _not_ disabled when disabling >>>>> ZGC. However, it was easy to prove that that code was safe/correct. >>>>> This >>>>> strategy turned out to work well, and it forced us to think hard about >>>>> having abstractions in the right places, move as much code as possible >>>>> into the BarrierSet, etc. >>>>> >>>>> I'd recommend you do the same for Shenandoah, that way the pressure on >>>>> you (in case something breaks and results instability or a long bug >>>>> trail) will be reduced a lot. If shit hits the fan, you can always >>>>> disable building of Shenandoah until the problems are sorted out. >>>>> >>>>> [...] >>>>>>>>> Generally, having PLAB call Universe::heap()->tlab_alloc_reserve() >>>>>>>>> looks >>>>>>>>> wrong to me, since a PLAB is not a TLAB. If we really want to >>>>>>>>> share >>>>>>>>> this >>>>>>>>> code I think we should try to find some other way. Maybe just give >>>>>>>>> tlab_alloc_reserve() a better name, but at the same time I'm not >>>>>>>>> even >>>>>>>>> sure this belongs in CollectedHeap. >>>>>>>> >>>>>>>> We can change it to lab_alloc_reserve() instead. I think it >>>>>>>> should be >>>>>>>> shared between TLAB and PLAB because the concept is the same >>>>>>>> thing, and >>>>>>>> relies on the same properties: we need some reserve at the end in >>>>>>>> order >>>>>>>> to be able to fill it completely, and with Shenandoah this is GC >>>>>>>> specific. And since all the other code that deals with filling >>>>>>>> TLABs >>>>>>>> and >>>>>>>> PLABs resides in CollectedHeap, this naturally belongs there too, >>>>>>>> right >>>>>>>> next to CollectedHeap::fill_with_dummy_object(..) which is also >>>>>>>> used by >>>>>>>> both TLAB and PLAB. >>>>>>> >>>>>>> Maybe a stupid question, but why do you even need a Brooks >>>>>>> pointer in >>>>>>> your filler objects? Is not having that maybe a solution here? >>>>>> >>>>>> Filler objects are objects. Laying them out differently would make >>>>>> parsing the heap impossible. >>>>> >>>>> Check. >>>>> >>>>> I thought some more about this. How about we rip out the filler-stuff >>>>> from CollectedHeap, and put it in a Fill utility class. You could >>>>> think >>>>> of Fill as a cosine to the "class Copy" we have today. Something like: >>>>> >>>>> class Fill : public AllStatic { >>>>> private: >>>>> ??? static int _min_size; >>>>> >>>>> public: >>>>> ??? static void set_min_size(int min_size) { >>>>> ????? _min_size = min_size; >>>>> ??? } >>>>> >>>>> ??? static int min_size() { >>>>> ????? return _min_size; >>>>> ??? } >>>>> >>>>> ??? static int min_reserve_size() { >>>>> ????? return _min_size > MinObjAlignment ? _min_size : 0; >>>>> ??? } >>>>> >>>>> ??? static void with_objects(...); >>>>> ??? static void with_object(...); >>>>> ??? ... >>>>> }; >>>>> >>>>> Fill::_min_size is initialized to oopDesc::header_size() by default. A >>>>> collector that wants something different calls Fill::set_min_size(...) >>>>> during start up. TLAB/PLAB would call Fill::min_reserve_size(). >>>>> >>>>> In my mind, I think this would put things where they belong. And as a >>>>> bonus we would cut out some cruft from CollectedHeap. What do you >>>>> think? >>>>> I started doing a patch for this to see if it seems like a good >>>>> approach. I hope to have something to show later today. >>>>> >>>>> cheers, >>>>> Per >>>> >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Mon Oct 15 15:01:56 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 15 Oct 2018 17:01:56 +0200 Subject: RFR: JDK-8211955: GC abstraction for LAB reserve In-Reply-To: <3c669729-c509-f3f3-7568-e572293336e9@redhat.com> References: <1770ac31-2ccc-dee7-b7df-1dbf302560d2@redhat.com> <80cbfe53-b633-cf42-57b9-97a9addd1bee@redhat.com> <2aa7c91e-9b1e-3129-cd7a-d1fa50da422e@redhat.com> <07b75752-0430-68d0-a0b3-5bdf8a4b1d1d@redhat.com> <376d3148-6492-ada6-3514-fd0779e6786d@redhat.com> <9467830c-0983-fc12-7f3a-1446e5b85362@oracle.com> <9c425de5-387f-6d7d-6ee2-cf6160a0d46f@redhat.com> <13396609-82b6-be14-f1fe-cc168b8db95d@oracle.com> <45395b5d-216c-eb40-1e03-fed6167b4b4f@oracle.com> <7c637b25-7695-132b-551c-a8b06705c264@oracle.com> <3c669729-c509-f3f3-7568-e572293336e9@redhat.com> Message-ID: On 10/15/2018 04:47 PM, Roman Kennke wrote: > And the one before was: > http://cr.openjdk.java.net/~rkennke/JDK-8211955/webrev.02/ This looks okay to me. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Mon Oct 15 16:10:14 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 15 Oct 2018 18:10:14 +0200 Subject: RFR (XXS): 8210492: PLAB object promotion events report object sizes in words Message-ID: <7530479baa7e750c8657c72b22c4255064b20429.camel@oracle.com> Hi all, can I have reviews for this trival fix that properly scales the values passed on to JFR by HeapWordSize? CR: https://bugs.openjdk.java.net/browse/JDK-8210492 Webrev: http://cr.openjdk.java.net/~tschatzl/8210492/webrev/ Testing: hs-tier1-3,jdk-tier1-3 Thanks, Thomas From erik.osterlund at oracle.com Mon Oct 15 17:05:59 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 15 Oct 2018 19:05:59 +0200 Subject: RFR: 8210064: ZGC: Introduce ZConcurrentRootsIterator for scanning a subset of strong IN_NATIVE roots concurrently In-Reply-To: <66825cea-9e7f-2736-1656-6bf7c8b9cf80@oracle.com> References: <28b4de13-e167-67b2-8dec-3948c1e8c8c8@oracle.com> <123ab70e-f1e3-e83b-a03c-671899386fee@oracle.com> <5B8E52BB.10700@oracle.com> <1044166e-7581-c3a7-ce84-eb363188a000@oracle.com> <5B8E5BC3.7090106@oracle.com> <66825cea-9e7f-2736-1656-6bf7c8b9cf80@oracle.com> Message-ID: <5ac1eb16-1f51-8bc6-d740-ed4ea524107b@oracle.com> Hi, This patch has been awaiting 3 patches to go in first due to unrelated test failures found in tier 6. Here is a rebase of these fixes. http://cr.openjdk.java.net/~eosterlund/8210064/webrev.03/ Compared to the last patch, I also 1) Introduce the STS joiner that is held while doing concurrent root iteration. The reason is that it is not safe to run this code through safepoints. 2) Treat the CLDs differently when marking vs heap iterating. Right now, I don't claim CLDs during heap iteration, because that will eventually break class unloading. Eventually, I will also need to walk different CLD sets during marking vs heap iteration (heap iteration will walk all CLDs, marking will only walk the strong ones), so the distinction is necessary. Thanks, /Erik On 2018-09-04 12:23, Per Liden wrote: > On 09/04/2018 12:17 PM, Erik ?sterlund wrote: >> Hi Per, >> >> On 2018-09-04 11:47, Per Liden wrote: >>> Hi Erik, >>> >>> On 09/04/2018 11:39 AM, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> Coleen pushed 8210155 before I pushed this. So now I need to wrap >>>> the CLDG iterations in a CLDG lock. >>>> I made a patch that adds the lock and relaxes the locking assert >>>> appropriately so that the thread invoking the worker gang can take >>>> the lock. This implies that the worker threads will not own the >>>> lock, but that is okay. >>>> >>>> Incremental: >>>> http://cr.openjdk.java.net/~eosterlund/8210064/webrev.01_02/ >>>> >>>> Full: >>>> http://cr.openjdk.java.net/~eosterlund/8210064/webrev.02/ >>> >>> Looks good. Just one small thing. It seems the old >>> assert_locked_or_safepoint() can now be expressed as >>> assert_locked_or_safepoint_weak() plus some extra conditions. >> >> Thanks for the review. Unfortunately it looks like I can't express >> the strong one with the weak one as it has early exit code, and I >> need to change the weak one to return a boolean to make that work. Oh >> well. > > Ah, you're right of course. Ignore that. > > /Per > >> >> Thanks, >> /Erik >> >>> /Per >>> >>>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2018-08-31 10:37, Erik ?sterlund wrote: >>>>> Hi Per, >>>>> >>>>> On 2018-08-31 09:57, Per Liden wrote: >>>>>> Hi Erik, >>>>>> >>>>>> On 08/30/2018 10:46 AM, Erik ?sterlund wrote: >>>>>>> Hi, >>>>>>> >>>>>>> We now have enough load barriers to support scanning of CLDs and >>>>>>> JNI handles concurrently. I propose to do that and move these >>>>>>> root sets out from ZRootsIterator, and hence the GC pause. They >>>>>>> will be scanned during concurrent marking (and heap iteration), >>>>>>> but no longer during relocation. >>>>>>> >>>>>>> I still perform ClassLoaderDataGraph::clear_claimed_marks() in >>>>>>> the pause because it seems cheap. But it can be moved out of the >>>>>>> pause when Coleen gets her new cool CLDG lock in. >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~eosterlund/8210064/webrev.00/ >>>>>> >>>>>> I have some minor requests. Instead of listing them all, I >>>>>> attached a patch which addresses those. >>>>>> >>>>>> The main thing is that I don't think ZDriver should know about >>>>>> "concurrent roots", just that it's doing "mark" or "mark >>>>>> continue", so I suggest we turn that into a "bool initial" >>>>>> argument to mark() instead of exposing a mark_concurrent_roots() >>>>>> function. >>>>> >>>>> Sure, that makes sense. >>>>> >>>>> Thanks for the review! >>>>> >>>>> /Erik >>>>> >>>>>> The other things are minor style adjustments. >>>>>> /Per >>>>>> >>>>>>> >>>>>>> Bug: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8210064 >>>>>>> >>>>>>> Tested through hs-tier1-3. >>>>>>> >>>>>>> Thanks, >>>>>>> /Erik >>>>> >>>> >> From kim.barrett at oracle.com Mon Oct 15 18:55:17 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 15 Oct 2018 14:55:17 -0400 Subject: RFR: 8212184: Incorrect oop ref strength used for referents in FinalReference In-Reply-To: <9363d38d-eb3c-b06c-2c23-88d8f71fe21f@oracle.com> References: <9363d38d-eb3c-b06c-2c23-88d8f71fe21f@oracle.com> Message-ID: <4052187A-9C1F-4BE3-8CE7-E10C06B2BEB9@oracle.com> > On Oct 15, 2018, at 10:42 AM, Per Liden wrote: > > AccessBarrierSupport::resolve_unknown_oop_ref_strength() returns an incorrect oop ref strength for referents in FinalReference objects. It currently returns ON_WEAK_OOP_REF when it should return ON_STRONG_OOP_REF. This is not really an issue for any GC except ZGC when using the ZHeapIterator to walk the heap while resurrection is blocked. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8212184 > Webrev: http://cr.openjdk.java.net/~pliden/8212184/webrev.0 > > /Per src/hotspot/share/gc/shared/accessBarrierSupport.cpp 32 if (!java_lang_ref_Reference::is_referent_field(base, offset) || 33 java_lang_ref_Reference::is_final(base)) { 34 ds |= ON_STRONG_OOP_REF; This doesn't seem right. Doesn't this give the wrong answer for G1? I'm not even sure "strong" is the right answer for ZGC in the described context. What am I missing? From jcbeyler at google.com Mon Oct 15 19:04:56 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 15 Oct 2018 12:04:56 -0700 Subject: RFR (S) 8212083: Handle remaining gc/lock native code and fix two strings In-Reply-To: References: Message-ID: Thanks Thomas! Here is the new webrev for the curious, which puts the includes in lexicographical order: Webrev: http://cr.openjdk.java.net/~jcbeyler/8212083/webrev.01/ And Bug is still here: https://bugs.openjdk.java.net/browse/JDK-8212083 I'll pass it through the submit repo and then, upon success I'll push it with you and Paul as reviewers. Thanks again, Jc On Mon, Oct 15, 2018 at 1:47 AM Thomas Schatzl wrote: > Hi, > > On Thu, 2018-10-11 at 15:23 -0700, JC Beyler wrote: > > Hi all, > > > > I've cleaned up the gc/lock tests to use the ExceptionCheckingJniEnv > > wrapper and now we have all JNI calls in that test group covered from > > a "checking exceptions and returns of JNI methods". > > > > Let me know what you think: > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8212083/webrev.00/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8212083 > > > > looks good to me as far as I understand the code. One nit: could you > fix the #include declarations: they should be ordered > lexicographically. > > Thanks, > Thomas > > > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Mon Oct 15 20:21:15 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 15 Oct 2018 22:21:15 +0200 Subject: RFR (S) 8212083: Handle remaining gc/lock native code and fix two strings In-Reply-To: References: Message-ID: Hi, On Mon, 2018-10-15 at 12:04 -0700, JC Beyler wrote: > Thanks Thomas! > > Here is the new webrev for the curious, which puts the includes in > lexicographical order: > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8212083/webrev.01/ > And Bug is still here: > https://bugs.openjdk.java.net/browse/JDK-8212083 > > I'll pass it through the submit repo and then, upon success I'll push > it with you and Paul as reviewers. looks good. Thomas From hohensee at amazon.com Mon Oct 15 20:29:01 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 15 Oct 2018 20:29:01 +0000 Subject: RFR (S) 8212083: Handle remaining gc/lock native code and fix two strings In-Reply-To: References: Message-ID: +1 Paul ?On 10/15/18, 4:23 PM, "hotspot-gc-dev on behalf of Thomas Schatzl" wrote: Hi, On Mon, 2018-10-15 at 12:04 -0700, JC Beyler wrote: > Thanks Thomas! > > Here is the new webrev for the curious, which puts the includes in > lexicographical order: > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8212083/webrev.01/ > And Bug is still here: > https://bugs.openjdk.java.net/browse/JDK-8212083 > > I'll pass it through the submit repo and then, upon success I'll push > it with you and Paul as reviewers. looks good. Thomas From jcbeyler at google.com Mon Oct 15 21:33:53 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 15 Oct 2018 14:33:53 -0700 Subject: RFR (XXS): 8210492: PLAB object promotion events report object sizes in words In-Reply-To: <7530479baa7e750c8657c72b22c4255064b20429.camel@oracle.com> References: <7530479baa7e750c8657c72b22c4255064b20429.camel@oracle.com> Message-ID: Hi Thomas, Not a reviewer but looks good to me :) Jc On Mon, Oct 15, 2018 at 9:11 AM Thomas Schatzl wrote: > Hi all, > > can I have reviews for this trival fix that properly scales the > values passed on to JFR by HeapWordSize? > > CR: > https://bugs.openjdk.java.net/browse/JDK-8210492 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8210492/webrev/ > Testing: > hs-tier1-3,jdk-tier1-3 > > Thanks, > Thomas > > > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Mon Oct 15 21:51:00 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 15 Oct 2018 17:51:00 -0400 Subject: RFR(XS): JDK-8212122: Allow ReferenceProcessor to always be MT processing In-Reply-To: <0349a726-6b74-09da-47dc-d3335762c5f2@redhat.com> References: <0349a726-6b74-09da-47dc-d3335762c5f2@redhat.com> Message-ID: > On Oct 15, 2018, at 9:15 AM, Roman Kennke wrote: >> The problem is the (rather new) ergonomics code in RP that scales number >> of threads to spin up based on the amount of work to do. And if it ends >> up being only 1 thread, then it goes down the single-threaded-path. >> Otherwise I wouldn't actually need this change... (that might be another >> approach to this problem: don't actually change _mt_processing flag at >> all, and call the executor even with 1 thread. In Shenandoah, we do this >> and it works fine, and we special-case the 1-thread-path by not spinning >> up the workers and execute it in the calling thread (which is the VMThread). > > In other words, this is a fix that actually removes 'cruft', and seems > less surprising API-wise: i.e. if caller asks for mt_processing, then > don't change it, and leave it to caller to execute in VMThread if it wishes. > > Updated webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8212122/webrev.01/ > > An example how G1 can be made to strictly do the current behaviour (i.e. > execute in VMThread when ergo_workers==1): > > http://cr.openjdk.java.net/~rkennke/JDK-8212122/webrev.01.g1/ > > ... altough I haven't tested it at all. It works well with Shenandoah, > but I don't know if you'd actually do it/want it/ask me to do it ;-) > > What do you think? Which approach do you like best (except the huge > parallel/CMS/WorkGang/ReferenceProcessor reworking)? > > Thanks, > Roman Sorry I haven't had time to give this much attention yet; I was planning to look at it today or tomorrow. But your latest comments (which I might not fully understand yet, so keep that in mind) are leading me to question this change. Putting it on callers of run_task to do the 1-thread optimization doesn't seem appropriate. I recently had a change that included that sort of thing, and Thomas convinced me that decision ought to be in run_task, rather than in callers of run_task. I think that hasn't happened yet because of a combination of nobody getting around to it and wondering whether there might be any uses where running the task directly from the VMThread might be a problem. Maybe the selection of whether to run the 1-thread case directly or via the executor should be a property of the executor? That seems to be kind of what you are proposing, except I think you are suggesting the executor's execute function be responsible for that. I think that didn't work for CMS and Parallel? Sangheon probably remembers better than exactly what the issues were there. The choice could instead be a predicate on the executor, called with the number of threads. From manc at google.com Mon Oct 15 22:07:11 2018 From: manc at google.com (Man Cao) Date: Mon, 15 Oct 2018 15:07:11 -0700 Subject: UseGCOverheadLimit for G1 In-Reply-To: <9a6fb903181b6f13ee91a7733761b89283246c43.camel@oracle.com> References: <9a6fb903181b6f13ee91a7733761b89283246c43.camel@oracle.com> Message-ID: Hi Thomas, Yes, I totally agree that refactoring code related to UseAdaptiveSizePolicy and UseGCOverheadLimit would be ideal for the long term. They should be independent of each other. I created: https://bugs.openjdk.java.net/browse/JDK-8212206 I wasn't sure if it is encouraged to do such refactoring to code used by ParallelGC. Good to know that it is encouraged! -Man On Mon, Oct 15, 2018 at 2:01 AM Thomas Schatzl wrote: > Hi Man, > > On Fri, 2018-10-12 at 10:47 -0700, Man Cao wrote: > > Sounds good. I'll take a stab at it. > > > > Does initiating the AdaptiveSizePolicy class for G1, or create a > > subclass G1AdaptiveSizePolicy sound like an acceptable high-level > > approach? > > I know it is a non-goal to make UseAdaptiveSizePolicy work for G1, > > but most logic for UseGCOverheadLimit is in the AdaptiveSizePolicy > > class. > > This seems the approach that reuse most existing code and does not > > affect ParallelGC. > > > > A side story is that we have a Google-local patch since JDK6 that > > makes UseGCOverheadLimit work properly for CMS. > > UseAdaptiveSizePolicy doesn't work and is disabled by default for > > CMS, but UseGCOverheadLimit depends on logic in the > > CMSAdaptiveSizePolicy class (which was removed in JDK9) to properly > > collect major GC overhead. > > Your short paragraph simply sound to me that the GC overhead limit > checking is misplaced in the AdaptiveSizePolicy class. > > Since the code is also basically very much geared to a two-generation > heap setup, we should not put it into e.g. CollectorPolicy either. > > What do you think about factoring out this code into a single class to > be reused by the generational collectors? It gets fed using some > sample() methods (for young and full gc each; feel free to adjust the > name) by the collectors, and provides the recent average costs (I saw > that AdaptiveSizePolicy uses them) and that check_gc_overhead_limit() > method? > > Then this functionality could also be used properly by CMS in all of > its modes, i.e. whether using AdaptiveSizePolicy or not I believe. > > Note that above is based on some quick look at the code, there might be > complications as usual :) But importing AdaptiveSizePolicy wholesale > into G1 seems very ugly. > > Thanks, > Thomas > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sangheon.kim at oracle.com Mon Oct 15 23:29:50 2018 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Mon, 15 Oct 2018 16:29:50 -0700 Subject: RFR(XS): JDK-8212122: Allow ReferenceProcessor to always be MT processing In-Reply-To: References: <0349a726-6b74-09da-47dc-d3335762c5f2@redhat.com> Message-ID: <568be465-dd88-4aec-5a68-f0f10b2e295f@oracle.com> > Maybe the selection of whether to run the 1-thread case directly or > via the executor should be a property of the executor? That seems to > be kind of what you are proposing, except I think you are suggesting > the executor's execute function be responsible for that. I think that > didn't work for CMS and Parallel? Sangheon probably remembers better > than exactly what the issues were there. 1. CMS has different caller side setting for single-multi thread cases. ( more details at https://bugs.openjdk.java.net/browse/JDK-6938732 ) 2. The reason for Parallel gc is a bit different. As it is not using WorkGang, changes are not trivial so it is not included at JDK-8043575. For this we have https://bugs.openjdk.java.net/browse/JDK-8204686. Thanks, Sangheon > The choice could instead be > a predicate on the executor, called with the number of threads. From sangheon.kim at oracle.com Mon Oct 15 23:33:48 2018 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Mon, 15 Oct 2018 16:33:48 -0700 Subject: RFR(XS): JDK-8212122: Allow ReferenceProcessor to always be MT processing In-Reply-To: <0349a726-6b74-09da-47dc-d3335762c5f2@redhat.com> References: <0349a726-6b74-09da-47dc-d3335762c5f2@redhat.com> Message-ID: <9f03a1d3-d598-461a-cd4d-2b8987eee2d4@oracle.com> Let me answer to your recent 2 emails together. On 10/15/18 6:15 AM, Roman Kennke wrote: > > Am 13.10.18 um 11:38 schrieb Roman Kennke: >>>> Currently, in ReferenceProcessor, the mt_processing flag is adjusted for >>>> each phase depending on the number of workers: if it's 1 worker, then >>>> mt_processing is disabled, for more than 1 worker, mt_processing is >>>> enabled. Depending on the flag, ref-processing is either done through >>>> AbstractRefProcTaskExecutor, or directly in VMThread. However, we have a >>>> situation where we always want to execute via the executor, and never >>>> directly. Instead we decide in the executor whether or not we want to >>>> execute directly in the VMThread. >>>> >>>> A simple change allows a subclass of ReferenceProcessor to change this >>>> behavior. >>>> >>>> In case you're interested why we need to do this: Shenandoah requires >>>> (for each worker) a scope around reference processing. In current >>>> scheme, we'd need to put one scope around >>>> RP::process_discovered_reference() to cover the 1-threaded case, and one >>>> scope inside each worker to cover those. However, we cannot handle this >>>> sort of weirdly nested scoping. >>>> >>>> (If it was me, I'd rework this whole execution scheme to always call the >>>> MT config, and let RP figure out and execute directly in VMThread if >>>> it's only 1 worker, and not put this burden on the caller to set up both >>>> for 1-threaded and N-threaded execution. This is just an ugly mess.) >>> I agree with you. >>> In addition to Kim's comment which covered ParallelGC case, >>> 1) For CMS, single vs. MT processing should be decided before calling >>> RP? because of CMS specific implementation. For more detail, please >>> refer comments at https://bugs.openjdk.java.net/browse/JDK-6938732 >>> 2) For G1, MT processing is set to true by default. But still caller has >>> 2 paths as you described. I'm not sure whether single thread case is >>> still needed there or not because RP is deciding to use single or >>> multiple threads. Probably we can remove it unless an user really wants >>> to force using single thread only. :) >> >> The problem is the (rather new) ergonomics code in RP that scales number >> of threads to spin up based on the amount of work to do. And if it ends >> up being only 1 thread, then it goes down the single-threaded-path. I was thinking why it is a problem.. And now I understand your statement is something like 'why not obey an provided executor' rather than single-thread stuff. Because current code is run in single thread which is VMThread (RP::_processing_is_mt = false). All existing GCs (when I worked on the code) have below setting, RP::_processing_is_mt = (ParallelGCThreads > 1) && ParallelRefProcEnabled which convinced me 1 worker case can be considered same as running on VMThread because ParallelGCThreads == 1 is not considered as mt processing. If you are suggesting that we should always use an executor if it is provided, it would be okay for this case. But I'm not sure whether it is the way we want to go. >> Otherwise I wouldn't actually need this change... (that might be another >> approach to this problem: don't actually change _mt_processing flag at >> all, and call the executor even with 1 thread. In Shenandoah, we do this >> and it works fine, and we special-case the 1-thread-path by not spinning >> up the workers and execute it in the calling thread (which is the VMThread). > In other words, this is a fix that actually removes 'cruft', and seems > less surprising API-wise: i.e. if caller asks for mt_processing, then > don't change it, and leave it to caller to execute in VMThread if it wishes. > > Updated webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8212122/webrev.01/ > > An example how G1 can be made to strictly do the current behaviour (i.e. > execute in VMThread when ergo_workers==1): > > http://cr.openjdk.java.net/~rkennke/JDK-8212122/webrev.01.g1/ > > ... altough I haven't tested it at all. It works well with Shenandoah, > but I don't know if you'd actually do it/want it/ask me to do it ;-) > > What do you think? Which approach do you like best (except the huge > parallel/CMS/WorkGang/ReferenceProcessor reworking)? As this could need more discussion, let me postpone answering it. :) Thanks, Sangheon > > Thanks, > Roman > From hohensee at amazon.com Mon Oct 15 23:44:49 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 15 Oct 2018 23:44:49 +0000 Subject: RFR (XXS): 8210492: PLAB object promotion events report object sizes in words In-Reply-To: References: <7530479baa7e750c8657c72b22c4255064b20429.camel@oracle.com> Message-ID: <51E17387-9F96-4725-A50B-C2BB28CA87ED@amazon.com> Looks good to me. Paul From: hotspot-gc-dev on behalf of JC Beyler Date: Monday, October 15, 2018 at 5:35 PM To: Thomas Schatzl Cc: "hotspot-gc-dev at openjdk.java.net" Subject: Re: RFR (XXS): 8210492: PLAB object promotion events report object sizes in words Hi Thomas, Not a reviewer but looks good to me :) Jc On Mon, Oct 15, 2018 at 9:11 AM Thomas Schatzl > wrote: Hi all, can I have reviews for this trival fix that properly scales the values passed on to JFR by HeapWordSize? CR: https://bugs.openjdk.java.net/browse/JDK-8210492 Webrev: http://cr.openjdk.java.net/~tschatzl/8210492/webrev/ Testing: hs-tier1-3,jdk-tier1-3 Thanks, Thomas -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladimir.kozlov at oracle.com Tue Oct 16 00:05:11 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 15 Oct 2018 17:05:11 -0700 Subject: RFC: How to deal with/abstract some offset-related Shenandoah changes? In-Reply-To: <27467623-88df-c77d-20a3-2e2e26a3ecee@redhat.com> References: <27467623-88df-c77d-20a3-2e2e26a3ecee@redhat.com> Message-ID: Hi Roman, I and others looked on this and agree with you that BrooksPointer::byte_offset() checks and other code you showed should be cleaned up. It may take time to come up with good suggestion. Please, wait. Regards, Vladimir On 10/12/18 1:57 AM, Roman Kennke wrote: > Hello, > > I'd like to get some opinions on how should we deal with some changes > that we need in Shenandoah, that really look a bit ugly, but that seem a > little out of place in a GC interface too. Let me show you the code: > > https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/asm/assembler.cpp.udiff.html > > There are 3 blocks there: > - The first deals with the aarch64 addressing bits. Only the lower > 48bits are used for addressing, and thus the upper 16 are masked away. > - the second block adjusts the offset by 8 bytes for Shenandoah when > computing the normalized heap-based-offset for narrow-oops. This is > needed to get combined decode+read-barrier instruction to work iirc. > - the last block allows implicit null-checks on Shenandoah fwd ptr loads > (now including the masking from block#1 above). > > I really am not sure how do deal with this. It does look like it would > require a two-way abstraction: > 1. CPU specific. i.e. move this method under src/cpu/$CPU and do the > right thing there (possibly duplicating for platforms that do common stuff) > 2. GC specific to deal with the -8 offset of Shenandoah > > However, I doubt that it would make the code any more easier to > understand/read/etc. And how would the GC abstraction look like? bool > BarrierSetAssembler::needs_explicit_null_check(..) ? And if GC says > 'false' then don't check any of the other stuff? > > I'm a bit at a loss here. Any suggestions are appreciated. > > > There is another place that deals with special offsets: > https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/ci/ciInstanceKlass.cpp.udiff.html > > (look at get_canonical_holder() there) > > Please let me know what you think. > > Roman > From kim.barrett at oracle.com Tue Oct 16 00:41:47 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 15 Oct 2018 20:41:47 -0400 Subject: RFR(XS): JDK-8212122: Allow ReferenceProcessor to always be MT processing In-Reply-To: <9f03a1d3-d598-461a-cd4d-2b8987eee2d4@oracle.com> References: <0349a726-6b74-09da-47dc-d3335762c5f2@redhat.com> <9f03a1d3-d598-461a-cd4d-2b8987eee2d4@oracle.com> Message-ID: > On Oct 15, 2018, at 7:33 PM, sangheon.kim at oracle.com wrote: > > Let me answer to your recent 2 emails together. > > On 10/15/18 6:15 AM, Roman Kennke wrote: >> >> Am 13.10.18 um 11:38 schrieb Roman Kennke: >>> >>> The problem is the (rather new) ergonomics code in RP that scales number >>> of threads to spin up based on the amount of work to do. And if it ends >>> up being only 1 thread, then it goes down the single-threaded-path. > I was thinking why it is a problem.. > And now I understand your statement is something like 'why not obey an provided executor' rather than single-thread stuff. Because current code is run in single thread which is VMThread (RP::_processing_is_mt = false). > > All existing GCs (when I worked on the code) have below setting, > RP::_processing_is_mt = (ParallelGCThreads > 1) && ParallelRefProcEnabled > which convinced me 1 worker case can be considered same as running on VMThread because ParallelGCThreads == 1 is not considered as mt processing. > > If you are suggesting that we should always use an executor if it is provided, it would be okay for this case. But I'm not sure whether it is the way we want to go. I think what Roman is suggesting is to always invoke the executor, and leave it to the executor to decide whether to run directly or not. I thought there was some problem with doing that (either for Parallel or CMS), but you (Sangheon) are more familiar with the difficulties encountered during JDK-8043575. I think the issue was that both Parallel and CMS required making the decision early (at the beginning) and couldn't change it on the fly for the various phases. Is that correct? That seems like it might make deferral to executor invocation problematic. From sangheon.kim at oracle.com Tue Oct 16 05:03:06 2018 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Mon, 15 Oct 2018 22:03:06 -0700 Subject: RFR(XS): JDK-8212122: Allow ReferenceProcessor to always be MT processing In-Reply-To: References: <0349a726-6b74-09da-47dc-d3335762c5f2@redhat.com> <9f03a1d3-d598-461a-cd4d-2b8987eee2d4@oracle.com> Message-ID: On 10/15/18 5:41 PM, Kim Barrett wrote: >> On Oct 15, 2018, at 7:33 PM, sangheon.kim at oracle.com wrote: >> >> Let me answer to your recent 2 emails together. >> >> On 10/15/18 6:15 AM, Roman Kennke wrote: >>> Am 13.10.18 um 11:38 schrieb Roman Kennke: >>>> The problem is the (rather new) ergonomics code in RP that scales number >>>> of threads to spin up based on the amount of work to do. And if it ends >>>> up being only 1 thread, then it goes down the single-threaded-path. >> I was thinking why it is a problem.. >> And now I understand your statement is something like 'why not obey an provided executor' rather than single-thread stuff. Because current code is run in single thread which is VMThread (RP::_processing_is_mt = false). >> >> All existing GCs (when I worked on the code) have below setting, >> RP::_processing_is_mt = (ParallelGCThreads > 1) && ParallelRefProcEnabled >> which convinced me 1 worker case can be considered same as running on VMThread because ParallelGCThreads == 1 is not considered as mt processing. >> >> If you are suggesting that we should always use an executor if it is provided, it would be okay for this case. But I'm not sure whether it is the way we want to go. > I think what Roman is suggesting is to always invoke the executor, and > leave it to the executor to decide whether to run directly or not. I > thought there was some problem with doing that (either for Parallel or > CMS), but you (Sangheon) are more familiar with the difficulties > encountered during JDK-8043575. I think the issue was that both > Parallel and CMS required making the decision early (at the beginning) > and couldn't change it on the fly for the various phases. Is that > correct? I answered in different email thread, but let me repeat here. It is hard for CMS. (https://bugs.openjdk.java.net/browse/JDK-6938732) Serial / parallel uses different mechanism to treat so it is hard to change it from? RP::process_discovered_references(). For Parallel GC, it is doable but it requires much work. And this is why we decided to exclude parallel gc support from JDK-8043575. Thanks, Sangheon > That seems like it might make deferral to executor invocation > problematic. > > > From rkennke at redhat.com Tue Oct 16 07:20:23 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 16 Oct 2018 09:20:23 +0200 Subject: RFC: How to deal with/abstract some offset-related Shenandoah changes? In-Reply-To: References: <27467623-88df-c77d-20a3-2e2e26a3ecee@redhat.com> Message-ID: <5413DFC4-170B-4C2F-93F4-EC963D8F2100@redhat.com> Hi Vladimir, thanks for the heads-up. Sure, I'll wait and try to come up with something reasonable myself in the meantime. Thanks! /Roman Am 16. Oktober 2018 02:05:11 MESZ schrieb Vladimir Kozlov : >Hi Roman, > >I and others looked on this and agree with you that >BrooksPointer::byte_offset() checks and other >code you showed should be cleaned up. It may take time to come up with >good suggestion. Please, wait. > >Regards, >Vladimir > >On 10/12/18 1:57 AM, Roman Kennke wrote: >> Hello, >> >> I'd like to get some opinions on how should we deal with some changes >> that we need in Shenandoah, that really look a bit ugly, but that >seem a >> little out of place in a GC interface too. Let me show you the code: >> >> >https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/asm/assembler.cpp.udiff.html >> >> There are 3 blocks there: >> - The first deals with the aarch64 addressing bits. Only the lower >> 48bits are used for addressing, and thus the upper 16 are masked >away. >> - the second block adjusts the offset by 8 bytes for Shenandoah when >> computing the normalized heap-based-offset for narrow-oops. This is >> needed to get combined decode+read-barrier instruction to work iirc. >> - the last block allows implicit null-checks on Shenandoah fwd ptr >loads >> (now including the masking from block#1 above). >> >> I really am not sure how do deal with this. It does look like it >would >> require a two-way abstraction: >> 1. CPU specific. i.e. move this method under src/cpu/$CPU and do the >> right thing there (possibly duplicating for platforms that do common >stuff) >> 2. GC specific to deal with the -8 offset of Shenandoah >> >> However, I doubt that it would make the code any more easier to >> understand/read/etc. And how would the GC abstraction look like? bool >> BarrierSetAssembler::needs_explicit_null_check(..) ? And if GC says >> 'false' then don't check any of the other stuff? >> >> I'm a bit at a loss here. Any suggestions are appreciated. >> >> >> There is another place that deals with special offsets: >> >https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/ci/ciInstanceKlass.cpp.udiff.html >> >> (look at get_canonical_holder() there) >> >> Please let me know what you think. >> >> Roman >> From rkennke at redhat.com Tue Oct 16 07:29:07 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 16 Oct 2018 09:29:07 +0200 Subject: RFR(XS): JDK-8212122: Allow ReferenceProcessor to always be MT processing In-Reply-To: References: <0349a726-6b74-09da-47dc-d3335762c5f2@redhat.com> Message-ID: <4056D06C-FA66-4673-BED5-97B31F60CEEE@redhat.com> Am 15. Oktober 2018 23:51:00 MESZ schrieb Kim Barrett : >> On Oct 15, 2018, at 9:15 AM, Roman Kennke wrote: >>> The problem is the (rather new) ergonomics code in RP that scales >number >>> of threads to spin up based on the amount of work to do. And if it >ends >>> up being only 1 thread, then it goes down the single-threaded-path. >>> Otherwise I wouldn't actually need this change... (that might be >another >>> approach to this problem: don't actually change _mt_processing flag >at >>> all, and call the executor even with 1 thread. In Shenandoah, we do >this >>> and it works fine, and we special-case the 1-thread-path by not >spinning >>> up the workers and execute it in the calling thread (which is the >VMThread). >> >> In other words, this is a fix that actually removes 'cruft', and >seems >> less surprising API-wise: i.e. if caller asks for mt_processing, then >> don't change it, and leave it to caller to execute in VMThread if it >wishes. >> >> Updated webrev: >> http://cr.openjdk.java.net/~rkennke/JDK-8212122/webrev.01/ >> >> An example how G1 can be made to strictly do the current behaviour >(i.e. >> execute in VMThread when ergo_workers==1): >> >> http://cr.openjdk.java.net/~rkennke/JDK-8212122/webrev.01.g1/ >> >> ... altough I haven't tested it at all. It works well with >Shenandoah, >> but I don't know if you'd actually do it/want it/ask me to do it ;-) >> >> What do you think? Which approach do you like best (except the huge >> parallel/CMS/WorkGang/ReferenceProcessor reworking)? >> >> Thanks, >> Roman > >Sorry I haven't had time to give this much attention yet; I was >planning to look at it today or tomorrow. But your latest comments >(which I might not fully understand yet, so keep that in mind) are >leading me to question this change. > >Putting it on callers of run_task to do the 1-thread optimization >doesn't seem appropriate. I recently had a change that included that >sort of thing, and Thomas convinced me that decision ought to be in >run_task, rather than in callers of run_task. I think that hasn't >happened yet because of a combination of nobody getting around to it >and wondering whether there might be any uses where running the task >directly from the VMThread might be a problem. Yeah I was initially thinking the same: put it in WorkGang directly. But as you said, this seems like a subtle change of behavior of which I'm not sure it'd break things. Plus it wouldn't work for GCs that don't use WorkGang at all. >Maybe the selection of whether to run the 1-thread case directly or >via the executor should be a property of the executor? That seems to >be kind of what you are proposing, except I think you are suggesting >the executor's execute function be responsible for that. I think that >didn't work for CMS and Parallel? Sangheon probably remembers better >than exactly what the issues were there. The choice could instead be >a predicate on the executor, called with the number of threads. True, it's even more reliable to decide that before diving into the executor and use the 'real' 1-threaded setup for 1-threaded execution. I'll come up with a patch for this later (I'm currently on the road..) Thanks, Roman From thomas.schatzl at oracle.com Tue Oct 16 09:28:44 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 16 Oct 2018 11:28:44 +0200 Subject: RFR (XXS): 8210492: PLAB object promotion events report object sizes in words In-Reply-To: <51E17387-9F96-4725-A50B-C2BB28CA87ED@amazon.com> References: <7530479baa7e750c8657c72b22c4255064b20429.camel@oracle.com> <51E17387-9F96-4725-A50B-C2BB28CA87ED@amazon.com> Message-ID: Hi JC, Paul, On Mon, 2018-10-15 at 23:44 +0000, Hohensee, Paul wrote: > Looks good to me. > > Paul > > From: hotspot-gc-dev on > behalf of JC Beyler > Date: Monday, October 15, 2018 at 5:35 PM > To: Thomas Schatzl > Cc: "hotspot-gc-dev at openjdk.java.net" < > hotspot-gc-dev at openjdk.java.net> > Subject: Re: RFR (XXS): 8210492: PLAB object promotion events report > object sizes in words > > Hi Thomas, > > Not a reviewer but looks good to me :) > Jc thanks for your reviews. Thomas From thomas.schatzl at oracle.com Tue Oct 16 09:32:13 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 16 Oct 2018 11:32:13 +0200 Subject: RFR (XS): 8211853: Avoid additional duplicate work when a reference in the task queue has already been evacuated In-Reply-To: References: Message-ID: <1968273d8823658fc9c59b009d396a69fe4ac560.camel@oracle.com> Hi, ping for some reviews... Thomas On Mon, 2018-10-08 at 16:58 +0200, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this tiny change that removes G1 doing some > duplicate work? > > Due to benign races we can sometimes get references that have already > been evacuated from the task queue. > > Current code in G1ParScanThreadState::do_oop_evac() needlessly > applies follow-up calculations (remembered set, humonguous liveness) > on this already evacuated reference - that all has been done the > first time around already. > > Just exit the method if we detect such a task. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8211853 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8211853/webrev/ > Testing: > local jtreg run of gc passes, some benchmarks > > Thanks, > Thomas > > From per.liden at oracle.com Tue Oct 16 09:52:53 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 16 Oct 2018 11:52:53 +0200 Subject: RFR: 8210064: ZGC: Introduce ZConcurrentRootsIterator for scanning a subset of strong IN_NATIVE roots concurrently In-Reply-To: <5ac1eb16-1f51-8bc6-d740-ed4ea524107b@oracle.com> References: <28b4de13-e167-67b2-8dec-3948c1e8c8c8@oracle.com> <123ab70e-f1e3-e83b-a03c-671899386fee@oracle.com> <5B8E52BB.10700@oracle.com> <1044166e-7581-c3a7-ce84-eb363188a000@oracle.com> <5B8E5BC3.7090106@oracle.com> <66825cea-9e7f-2736-1656-6bf7c8b9cf80@oracle.com> <5ac1eb16-1f51-8bc6-d740-ed4ea524107b@oracle.com> Message-ID: On 10/15/2018 07:05 PM, Erik ?sterlund wrote: > Hi, > > This patch has been awaiting 3 patches to go in first due to unrelated > test failures found in tier 6. > Here is a rebase of these fixes. > > http://cr.openjdk.java.net/~eosterlund/8210064/webrev.03/ Looks good. /Per > > Compared to the last patch, I also > > 1) Introduce the STS joiner that is held while doing concurrent root > iteration. The reason is that it is not safe to run this code through > safepoints. > 2) Treat the CLDs differently when marking vs heap iterating. Right now, > I don't claim CLDs during heap iteration, because that will eventually > break class unloading. Eventually, I will also need to walk different > CLD sets during marking vs heap iteration (heap iteration will walk all > CLDs, marking will only walk the strong ones), so the distinction is > necessary. > > Thanks, > /Erik > > On 2018-09-04 12:23, Per Liden wrote: >> On 09/04/2018 12:17 PM, Erik ?sterlund wrote: >>> Hi Per, >>> >>> On 2018-09-04 11:47, Per Liden wrote: >>>> Hi Erik, >>>> >>>> On 09/04/2018 11:39 AM, Erik ?sterlund wrote: >>>>> Hi, >>>>> >>>>> Coleen pushed 8210155 before I pushed this. So now I need to wrap >>>>> the CLDG iterations in a CLDG lock. >>>>> I made a patch that adds the lock and relaxes the locking assert >>>>> appropriately so that the thread invoking the worker gang can take >>>>> the lock. This implies that the worker threads will not own the >>>>> lock, but that is okay. >>>>> >>>>> Incremental: >>>>> http://cr.openjdk.java.net/~eosterlund/8210064/webrev.01_02/ >>>>> >>>>> Full: >>>>> http://cr.openjdk.java.net/~eosterlund/8210064/webrev.02/ >>>> >>>> Looks good. Just one small thing. It seems the old >>>> assert_locked_or_safepoint() can now be expressed as >>>> assert_locked_or_safepoint_weak() plus some extra conditions. >>> >>> Thanks for the review. Unfortunately it looks like I can't express >>> the strong one with the weak one as it has early exit code, and I >>> need to change the weak one to return a boolean to make that work. Oh >>> well. >> >> Ah, you're right of course. Ignore that. >> >> /Per >> >>> >>> Thanks, >>> /Erik >>> >>>> /Per >>>> >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>> On 2018-08-31 10:37, Erik ?sterlund wrote: >>>>>> Hi Per, >>>>>> >>>>>> On 2018-08-31 09:57, Per Liden wrote: >>>>>>> Hi Erik, >>>>>>> >>>>>>> On 08/30/2018 10:46 AM, Erik ?sterlund wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> We now have enough load barriers to support scanning of CLDs and >>>>>>>> JNI handles concurrently. I propose to do that and move these >>>>>>>> root sets out from ZRootsIterator, and hence the GC pause. They >>>>>>>> will be scanned during concurrent marking (and heap iteration), >>>>>>>> but no longer during relocation. >>>>>>>> >>>>>>>> I still perform ClassLoaderDataGraph::clear_claimed_marks() in >>>>>>>> the pause because it seems cheap. But it can be moved out of the >>>>>>>> pause when Coleen gets her new cool CLDG lock in. >>>>>>>> >>>>>>>> Webrev: >>>>>>>> http://cr.openjdk.java.net/~eosterlund/8210064/webrev.00/ >>>>>>> >>>>>>> I have some minor requests. Instead of listing them all, I >>>>>>> attached a patch which addresses those. >>>>>>> >>>>>>> The main thing is that I don't think ZDriver should know about >>>>>>> "concurrent roots", just that it's doing "mark" or "mark >>>>>>> continue", so I suggest we turn that into a "bool initial" >>>>>>> argument to mark() instead of exposing a mark_concurrent_roots() >>>>>>> function. >>>>>> >>>>>> Sure, that makes sense. >>>>>> >>>>>> Thanks for the review! >>>>>> >>>>>> /Erik >>>>>> >>>>>>> The other things are minor style adjustments. >>>>>>> /Per >>>>>>> >>>>>>>> >>>>>>>> Bug: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8210064 >>>>>>>> >>>>>>>> Tested through hs-tier1-3. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> /Erik >>>>>> >>>>> >>> > From erik.osterlund at oracle.com Tue Oct 16 09:55:35 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 16 Oct 2018 11:55:35 +0200 Subject: RFR: 8210064: ZGC: Introduce ZConcurrentRootsIterator for scanning a subset of strong IN_NATIVE roots concurrently In-Reply-To: References: <28b4de13-e167-67b2-8dec-3948c1e8c8c8@oracle.com> <123ab70e-f1e3-e83b-a03c-671899386fee@oracle.com> <5B8E52BB.10700@oracle.com> <1044166e-7581-c3a7-ce84-eb363188a000@oracle.com> <5B8E5BC3.7090106@oracle.com> <66825cea-9e7f-2736-1656-6bf7c8b9cf80@oracle.com> <5ac1eb16-1f51-8bc6-d740-ed4ea524107b@oracle.com> Message-ID: <5BC5B597.2010606@oracle.com> Hi Per, Thanks for the review! /Erik On 2018-10-16 11:52, Per Liden wrote: > On 10/15/2018 07:05 PM, Erik ?sterlund wrote: >> Hi, >> >> This patch has been awaiting 3 patches to go in first due to >> unrelated test failures found in tier 6. >> Here is a rebase of these fixes. >> >> http://cr.openjdk.java.net/~eosterlund/8210064/webrev.03/ > > Looks good. > > /Per > >> >> Compared to the last patch, I also >> >> 1) Introduce the STS joiner that is held while doing concurrent root >> iteration. The reason is that it is not safe to run this code through >> safepoints. >> 2) Treat the CLDs differently when marking vs heap iterating. Right >> now, I don't claim CLDs during heap iteration, because that will >> eventually break class unloading. Eventually, I will also need to >> walk different CLD sets during marking vs heap iteration (heap >> iteration will walk all CLDs, marking will only walk the strong >> ones), so the distinction is necessary. >> >> Thanks, >> /Erik >> >> On 2018-09-04 12:23, Per Liden wrote: >>> On 09/04/2018 12:17 PM, Erik ?sterlund wrote: >>>> Hi Per, >>>> >>>> On 2018-09-04 11:47, Per Liden wrote: >>>>> Hi Erik, >>>>> >>>>> On 09/04/2018 11:39 AM, Erik ?sterlund wrote: >>>>>> Hi, >>>>>> >>>>>> Coleen pushed 8210155 before I pushed this. So now I need to wrap >>>>>> the CLDG iterations in a CLDG lock. >>>>>> I made a patch that adds the lock and relaxes the locking assert >>>>>> appropriately so that the thread invoking the worker gang can >>>>>> take the lock. This implies that the worker threads will not own >>>>>> the lock, but that is okay. >>>>>> >>>>>> Incremental: >>>>>> http://cr.openjdk.java.net/~eosterlund/8210064/webrev.01_02/ >>>>>> >>>>>> Full: >>>>>> http://cr.openjdk.java.net/~eosterlund/8210064/webrev.02/ >>>>> >>>>> Looks good. Just one small thing. It seems the old >>>>> assert_locked_or_safepoint() can now be expressed as >>>>> assert_locked_or_safepoint_weak() plus some extra conditions. >>>> >>>> Thanks for the review. Unfortunately it looks like I can't express >>>> the strong one with the weak one as it has early exit code, and I >>>> need to change the weak one to return a boolean to make that work. >>>> Oh well. >>> >>> Ah, you're right of course. Ignore that. >>> >>> /Per >>> >>>> >>>> Thanks, >>>> /Erik >>>> >>>>> /Per >>>>> >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>>>> On 2018-08-31 10:37, Erik ?sterlund wrote: >>>>>>> Hi Per, >>>>>>> >>>>>>> On 2018-08-31 09:57, Per Liden wrote: >>>>>>>> Hi Erik, >>>>>>>> >>>>>>>> On 08/30/2018 10:46 AM, Erik ?sterlund wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> We now have enough load barriers to support scanning of CLDs >>>>>>>>> and JNI handles concurrently. I propose to do that and move >>>>>>>>> these root sets out from ZRootsIterator, and hence the GC >>>>>>>>> pause. They will be scanned during concurrent marking (and >>>>>>>>> heap iteration), but no longer during relocation. >>>>>>>>> >>>>>>>>> I still perform ClassLoaderDataGraph::clear_claimed_marks() in >>>>>>>>> the pause because it seems cheap. But it can be moved out of >>>>>>>>> the pause when Coleen gets her new cool CLDG lock in. >>>>>>>>> >>>>>>>>> Webrev: >>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8210064/webrev.00/ >>>>>>>> >>>>>>>> I have some minor requests. Instead of listing them all, I >>>>>>>> attached a patch which addresses those. >>>>>>>> >>>>>>>> The main thing is that I don't think ZDriver should know about >>>>>>>> "concurrent roots", just that it's doing "mark" or "mark >>>>>>>> continue", so I suggest we turn that into a "bool initial" >>>>>>>> argument to mark() instead of exposing a >>>>>>>> mark_concurrent_roots() function. >>>>>>> >>>>>>> Sure, that makes sense. >>>>>>> >>>>>>> Thanks for the review! >>>>>>> >>>>>>> /Erik >>>>>>> >>>>>>>> The other things are minor style adjustments. >>>>>>>> /Per >>>>>>>> >>>>>>>>> >>>>>>>>> Bug: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8210064 >>>>>>>>> >>>>>>>>> Tested through hs-tier1-3. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> /Erik >>>>>>> >>>>>> >>>> >> From rkennke at redhat.com Tue Oct 16 10:15:37 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 16 Oct 2018 12:15:37 +0200 Subject: RFR(XS): JDK-8212122: Allow ReferenceProcessor to always be MT processing In-Reply-To: References: <0349a726-6b74-09da-47dc-d3335762c5f2@redhat.com> Message-ID: <9d8caf90-f82c-cf7f-31ad-6f8a3d1f0b40@redhat.com> The more I think about it, the more messy this occurs to me. Again, to begin with, I believe we already have what we want: GCs can tell the RP if it wants mt_processing or not via the mt_processing flag. It used to be the case that RP always respected that. GCs set this to false if num-gc-treads==1 or if it didn't want mt_processing for other reasons (serial GC), otherwise to true, and RP would always use the executor if mt_processing==true, otherwise go the single-threaded path. With the change of using ergonomically determined num-workers, this has changed to dynamically use one or the other path, which is not good for GCs that actually always want mt_processing. I find this surprising behaviour. I would argue that when GC tells RP to to mt_processing, then yes, please always go the executor/mt-processing route. The executor is free to decide to not spin up workers for 1-threaded-execution as optimization. Other than that, I believe that 1-threaded and N-threaded execution should be equivalent. For this reason, I still think that simply removing this surprising adjustment of _mt_processing is the way to go, as proposed here: http://cr.openjdk.java.net/~rkennke/JDK-8212122/webrev.01/ If we really need to preserve the old (actually, rather new) behaviour to execute single-threaded using the specific single-threaded setup, we can do as proposed earlier: http://cr.openjdk.java.net/~rkennke/JDK-8212122/webrev.00/ and GC can override this by passing a suitable RP subclass. As much as I'd like to fix this mess, and care about CMS and ParallelGC, I currently don't have time for that (maybe later). What I need is a way to avoid the single-threaded-path altogether, because it breaks Shenandoah. Roman >> On Oct 15, 2018, at 9:15 AM, Roman Kennke wrote: >>> The problem is the (rather new) ergonomics code in RP that scales number >>> of threads to spin up based on the amount of work to do. And if it ends >>> up being only 1 thread, then it goes down the single-threaded-path. >>> Otherwise I wouldn't actually need this change... (that might be another >>> approach to this problem: don't actually change _mt_processing flag at >>> all, and call the executor even with 1 thread. In Shenandoah, we do this >>> and it works fine, and we special-case the 1-thread-path by not spinning >>> up the workers and execute it in the calling thread (which is the VMThread). >> >> In other words, this is a fix that actually removes 'cruft', and seems >> less surprising API-wise: i.e. if caller asks for mt_processing, then >> don't change it, and leave it to caller to execute in VMThread if it wishes. >> >> Updated webrev: >> http://cr.openjdk.java.net/~rkennke/JDK-8212122/webrev.01/ >> >> An example how G1 can be made to strictly do the current behaviour (i.e. >> execute in VMThread when ergo_workers==1): >> >> http://cr.openjdk.java.net/~rkennke/JDK-8212122/webrev.01.g1/ >> >> ... altough I haven't tested it at all. It works well with Shenandoah, >> but I don't know if you'd actually do it/want it/ask me to do it ;-) >> >> What do you think? Which approach do you like best (except the huge >> parallel/CMS/WorkGang/ReferenceProcessor reworking)? >> >> Thanks, >> Roman > > Sorry I haven't had time to give this much attention yet; I was > planning to look at it today or tomorrow. But your latest comments > (which I might not fully understand yet, so keep that in mind) are > leading me to question this change. > > Putting it on callers of run_task to do the 1-thread optimization > doesn't seem appropriate. I recently had a change that included that > sort of thing, and Thomas convinced me that decision ought to be in > run_task, rather than in callers of run_task. I think that hasn't > happened yet because of a combination of nobody getting around to it > and wondering whether there might be any uses where running the task > directly from the VMThread might be a problem. > > Maybe the selection of whether to run the 1-thread case directly or > via the executor should be a property of the executor? That seems to > be kind of what you are proposing, except I think you are suggesting > the executor's execute function be responsible for that. I think that > didn't work for CMS and Parallel? Sangheon probably remembers better > than exactly what the issues were there. The choice could instead be > a predicate on the executor, called with the number of threads. > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Tue Oct 16 11:45:56 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 16 Oct 2018 13:45:56 +0200 Subject: RFR(XS): JDK-8212122: Allow ReferenceProcessor to always be MT processing In-Reply-To: <9d8caf90-f82c-cf7f-31ad-6f8a3d1f0b40@redhat.com> References: <0349a726-6b74-09da-47dc-d3335762c5f2@redhat.com> <9d8caf90-f82c-cf7f-31ad-6f8a3d1f0b40@redhat.com> Message-ID: On Tue, 2018-10-16 at 12:15 +0200, Roman Kennke wrote: > The more I think about it, the more messy this occurs to me. > > Again, to begin with, I believe we already have what we want: GCs can > tell the RP if it wants mt_processing or not via the mt_processing > flag. > It used to be the case that RP always respected that. GCs set this to > false if num-gc-treads==1 or if it didn't want mt_processing for > other reasons (serial GC), otherwise to true, and RP would always use > the executor if mt_processing==true, otherwise go the single-threaded > path. > > With the change of using ergonomically determined num-workers, this > has changed to dynamically use one or the other path, which is not > good for GCs that actually always want mt_processing. I find this > surprising behaviour. I would argue that when GC tells RP to to I agree with that this may be surprising behavior. However you do not want all-MT for performance reasons - spinning up just one worker thread is significantly slower than just processing the few References inline. This particularly matters if you want very low-pause times. (This has been improved a bit with just not doing any work if the References are zero, which is a common case). So you somehow need to alternate between in-thread processing and real MT processing for maximum throughput, particularly in this case where you spin up and shut down threads a lot potentially in rapid succession. Now this can be fixed _partially_ with letting the executor deciding whether doing the work inline or not, with the design issue mentioned by Kim already ("isn't this the point of run_task()?"). The other issue is balancing the queues which is superfluous with only one thread - but we've had some very subtle issues with that. (Yeah, yeah, I do have a mostly done JDK-8202328 around, but....) > mt_processing, then then yes, please always go the executor/mt- > processing route. The executor is free to decide to not spin up > workers for 1-threaded-execution as optimization. Other than that, I Been there, done that, i.e. the programmer had been responsible for deciding whether to do the work inline or not. This had several issues (history lesson follows, just skip if not interested): - people forgot to do that in some places, leading to massive performance issues in situations the author hadn't thought of yet (like running on a multi-TB heap with thousands of HW threads). - people started optimizing for one or the other case, adding lots of code and so increasing maintenance costs a lot (e.g. some paths were basically never tested until the next crash, lots of duplicate and boilerplate code added etc) which made for lots of pain. - WorkGang itself does not give any kind of guarantees on which threads a particular AbstractGangTask is run in the first place, and users must not rely on that. Relying on that again and again just caused once- during-a-full-moon issues in the past; another example for a misguided assumption is that there is no guaranteed 1:1 relation between worker id passed to the AbstractGangTask and the thread's id. I.e. a single thread may execute an AbstractGangTask multiple times (with different worker_id of course), and some workgang threads may not execute any at all. (I assume that Shenandoah handles that case just fine?) Assuming otherwise is just a wrong assumption of what WorkGang provides to you and any code assuming so should simply be considered broken. These are the reasons why at some point (early JDK9 timeframe I think) we just removed the special casing and cleaned G1 AbstractGangTasks because we got weary of spending so much time on fixing these issues. Basically CMS' AbstractGangTasks are left to look over (which seems obviously broken for the RP case; but CMS'es removal keeps getting closer anyway). > believe that 1-threaded and N-threaded execution should be > equivalent. > > For this reason, I still think that simply removing this surprising > adjustment of _mt_processing is the way to go, as proposed here: > > http://cr.openjdk.java.net/~rkennke/JDK-8212122/webrev.01/ > > If we really need to preserve the old (actually, rather new) This behavior is fully in-line with the guarantees of WorkGang (assuming that ideally you would pass a WorkGang and AbstractGangTask to reference processing if for a moment we ignore the Executor framework which is an additional wrapper for Parallel GC). > behaviour to execute single-threaded using the specific single- > threaded setup, we can do as proposed earlier: > > http://cr.openjdk.java.net/~rkennke/JDK-8212122/webrev.00/ > > and GC can override this by passing a suitable RP subclass. > > As much as I'd like to fix this mess, and care about CMS and > ParallelGC, I currently don't have time for that (maybe later). What > I need is a way to avoid the single-threaded-path altogether, because > it breaks Shenandoah. >From what I have understood so far, Shenandoah code seems to have jumped into the same trap, i.e. seems to assume things about WorkGang that it should not (which is unfortunately probably not a quick fix either). If there is really need for getting this fixed asap, please preserve old behavior. The new behavior you propose may cause significant regressions (even if the Executor special-cases the 1-thread case) in reference processing as it seems to unconditionally enable queue balancing in the MT case - which is rather slow (and useless to actually attempt when there is only one thread to do reference processing). Thanks, Thomas From per.liden at oracle.com Tue Oct 16 11:46:41 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 16 Oct 2018 13:46:41 +0200 Subject: RFR: 8212181: ZGC: Fix incorrect root iteration in ZHeapIterator In-Reply-To: <5BC4A886.9010809@oracle.com> References: <4123e36b-fd4f-7ee3-e5b1-5e37674b3378@oracle.com> <5BC4A886.9010809@oracle.com> Message-ID: Thanks Erik! /Per On 10/15/2018 04:47 PM, Erik ?sterlund wrote: > Hi Per, > > Looks good. > > Thanks, > /Erik > > On 2018-10-15 16:44, Per Liden wrote: >> The ZHeapIterator should not make callbacks to the user supplied >> ObjectClosure while iterating over roots. Instead all roots should be >> pushed to the stack first, before any call to the user supplied >> ObjectClosure is made. This is needed since the user supplied >> ObjectClosure could otherwise modify the root set (e.g. the JVMTI >> TagMap) while we're iterating over it, which can lead to memory >> corruption. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8212181 >> Webrev: http://cr.openjdk.java.net/~pliden/8212181/webrev.0 >> >> /Per > From per.liden at oracle.com Tue Oct 16 11:55:51 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 16 Oct 2018 13:55:51 +0200 Subject: RFR: 8212184: Incorrect oop ref strength used for referents in FinalReference In-Reply-To: <4052187A-9C1F-4BE3-8CE7-E10C06B2BEB9@oracle.com> References: <9363d38d-eb3c-b06c-2c23-88d8f71fe21f@oracle.com> <4052187A-9C1F-4BE3-8CE7-E10C06B2BEB9@oracle.com> Message-ID: <890bb0a2-30af-3fa4-0c42-874b3d4376d4@oracle.com> Hi Kim, On 10/15/2018 08:55 PM, Kim Barrett wrote: >> On Oct 15, 2018, at 10:42 AM, Per Liden wrote: >> >> AccessBarrierSupport::resolve_unknown_oop_ref_strength() returns an incorrect oop ref strength for referents in FinalReference objects. It currently returns ON_WEAK_OOP_REF when it should return ON_STRONG_OOP_REF. This is not really an issue for any GC except ZGC when using the ZHeapIterator to walk the heap while resurrection is blocked. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8212184 >> Webrev: http://cr.openjdk.java.net/~pliden/8212184/webrev.0 >> >> /Per > > src/hotspot/share/gc/shared/accessBarrierSupport.cpp > 32 if (!java_lang_ref_Reference::is_referent_field(base, offset) || > 33 java_lang_ref_Reference::is_final(base)) { > 34 ds |= ON_STRONG_OOP_REF; > > This doesn't seem right. Doesn't this give the wrong answer for G1? > > I'm not even sure "strong" is the right answer for ZGC in the > described context. > > What am I missing? > There's no way for a mutator to get hold of the referent (except via Unsafe, which we've said we don't care about anyway). The only time we would randomly step on a referent in a Finalizer is when we're doing heap iteration in ZGC. An alternative, to perhaps make this more explicit, would be to have an ON_FINAL_OOP_REF, but it would end up doing the same thing as ON_STRONG_OOP_REF. /Per From per.liden at oracle.com Tue Oct 16 12:00:03 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 16 Oct 2018 14:00:03 +0200 Subject: RFR (XS) 8212177: Epsilon alignment adjustments can overflow max TLAB size In-Reply-To: References: Message-ID: <7375c540-7599-9592-abbc-f612e89d184c@oracle.com> Looks good to me! /Per On 10/15/2018 12:24 PM, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212177 > > This was spotted by Per in review last week. > > Fix: > http://cr.openjdk.java.net/~shade/8212177/webrev.01/ > > Testing: Linux {x86_64, x86_32} gc/epsilon, jdk-submit (running) > > Thanks, > -Aleksey > From thomas.schatzl at oracle.com Tue Oct 16 12:20:32 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 16 Oct 2018 14:20:32 +0200 Subject: RFR (XS) 8212177: Epsilon alignment adjustments can overflow max TLAB size In-Reply-To: References: Message-ID: <334e46c74a096b66f7b99e3d21d342da245a57fd.camel@oracle.com> Hi, On Mon, 2018-10-15 at 12:24 +0200, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212177 > > This was spotted by Per in review last week. > > Fix: > http://cr.openjdk.java.net/~shade/8212177/webrev.01/ > > Testing: Linux {x86_64, x86_32} gc/epsilon, jdk-submit (running) > looks good. Thomas From thomas.schatzl at oracle.com Tue Oct 16 13:59:41 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 16 Oct 2018 15:59:41 +0200 Subject: RFR (XS): 8211853: Avoid additional duplicate work when a reference in the task queue has already been evacuated In-Reply-To: References: Message-ID: <8f58feb1ec7da68af24d040a1fb126cc2397c842.camel@oracle.com> Hi all, On Mon, 2018-10-08 at 16:58 +0200, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this tiny change that removes G1 doing some > duplicate work? > > Due to benign races we can sometimes get references that have already > been evacuated from the task queue. > > Current code in G1ParScanThreadState::do_oop_evac() needlessly > applies follow-up calculations (remembered set, humonguous liveness) > on this already evacuated reference - that all has been done the > first time around already. StefanJ correctly pointed out that there must more than one reason to be allowed to do this. - we can skip the humongous liveness check because G1 never pushes oop* with references outside of the collection set onto the task queue. Humongous objects are always logically in old gen, so we should never see such a reference here. I added an additional assert to make that clear. - the second reason for being able to exit early if the reference in the oop* is in old gen is because then we know that that reference has already been processed (as we try to only push references into the collection set on the task queue), and the remembered set update etc already done. Note that in case of evacuation failure this check is not sufficient, i.e. the reference is still a reference into the collection set, so G1 still does the update_rs() again the next time we get here. However this is no problem either - naturally they are not references to humongous objects, and we do not care too much about performance in that rare case either (i.e. the additional update_rs() call is very quick anyway). > Just exit the method if we detect such a task. New webrev at http://cr.openjdk.java.net/~tschatzl/8211853/webrev.0_to_1 (diff) http://cr.openjdk.java.net/~tschatzl/8211853/webrev.1 (full) with the added assert. Passed hs-tier1-3 as expected. Thanks, Thomas From igor.ignatyev at oracle.com Tue Oct 16 16:06:06 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 16 Oct 2018 09:06:06 -0700 Subject: RFR(XS) : 8171097 : Convert TestReservedSpace_test to Gtest Message-ID: http://cr.openjdk.java.net/~iignatyev//8171097/webrev.00/index.html > 188 lines changed: 187 ins; 1 del; 0 mod; Hi all, could you please review this small (and hopefully trivial) patch which converts internal TestReservedSpace_test to gtest? since the old test is still used by WhiteBox::runMemoryUnitTests, the old code hasn't been removed. it will be removed later when all 4 tests used by runMemoryUnitTests are converted. webrev: http://cr.openjdk.java.net/~iignatyev//8171097/webrev.00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8171097 testing: - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants - build w/ precompiled-headers enabled and disabled PS the patch has been originally created by Kirill Zh, but hasn't been sent out for official review Thanks, -- Igor From shade at redhat.com Tue Oct 16 16:14:48 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 16 Oct 2018 18:14:48 +0200 Subject: RFR (XS) 8212177: Epsilon alignment adjustments can overflow max TLAB size In-Reply-To: <334e46c74a096b66f7b99e3d21d342da245a57fd.camel@oracle.com> References: <334e46c74a096b66f7b99e3d21d342da245a57fd.camel@oracle.com> Message-ID: <412e6c03-7da1-5ba1-a090-797d46d7d6d7@redhat.com> On 10/16/2018 02:20 PM, Thomas Schatzl wrote: > On Mon, 2018-10-15 at 12:24 +0200, Aleksey Shipilev wrote: >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8212177 >> >> This was spotted by Per in review last week. >> >> Fix: >> http://cr.openjdk.java.net/~shade/8212177/webrev.01/ >> >> Testing: Linux {x86_64, x86_32} gc/epsilon, jdk-submit (running) >> > > looks good. Thanks, pushed. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From jcbeyler at google.com Tue Oct 16 19:51:36 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 16 Oct 2018 12:51:36 -0700 Subject: RFR(XS) : 8171097 : Convert TestReservedSpace_test to Gtest In-Reply-To: References: Message-ID: Hi Igor, The change looks good to me, I compared old/new, the port looks good. For the reviewers, here is the original test code: http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1071 Thanks, Jc On Tue, Oct 16, 2018 at 9:06 AM Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8171097/webrev.00/index.html > > 188 lines changed: 187 ins; 1 del; 0 mod; > > Hi all, > > could you please review this small (and hopefully trivial) patch which > converts internal TestReservedSpace_test to gtest? > since the old test is still used by WhiteBox::runMemoryUnitTests, the old > code hasn't been removed. it will be removed later when all 4 tests used by > runMemoryUnitTests are converted. > > webrev: > http://cr.openjdk.java.net/~iignatyev//8171097/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8171097 > testing: > - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 > in product and fastdebug variants > - build w/ precompiled-headers enabled and disabled > > PS the patch has been originally created by Kirill Zh, but hasn't been > sent out for official review > > Thanks, > -- Igor -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From rkennke at redhat.com Tue Oct 16 21:07:21 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 16 Oct 2018 23:07:21 +0200 Subject: RFR(XS): JDK-8212122: Allow ReferenceProcessor to always be MT processing In-Reply-To: References: <0349a726-6b74-09da-47dc-d3335762c5f2@redhat.com> <9d8caf90-f82c-cf7f-31ad-6f8a3d1f0b40@redhat.com> Message-ID: <46a0dcb3-8e1d-0c93-4dfe-1e8404bbaa6b@redhat.com> Hi Thomas, >> The more I think about it, the more messy this occurs to me. >> >> Again, to begin with, I believe we already have what we want: GCs can >> tell the RP if it wants mt_processing or not via the mt_processing >> flag. >> It used to be the case that RP always respected that. GCs set this to >> false if num-gc-treads==1 or if it didn't want mt_processing for >> other reasons (serial GC), otherwise to true, and RP would always use >> the executor if mt_processing==true, otherwise go the single-threaded >> path. >> >> With the change of using ergonomically determined num-workers, this >> has changed to dynamically use one or the other path, which is not >> good for GCs that actually always want mt_processing. I find this >> surprising behaviour. I would argue that when GC tells RP to to > > I agree with that this may be surprising behavior. However you do not > want all-MT for performance reasons - spinning up just one worker > thread is significantly slower than just processing the few References > inline. This particularly matters if you want very low-pause times. Right. That's why we 'fixed' that by special-casing this in the executor. > (This has been improved a bit with just not doing any work if the > References are zero, which is a common case). > > So you somehow need to alternate between in-thread processing and real > MT processing for maximum throughput, particularly in this case where > you spin up and shut down threads a lot potentially in rapid > succession. > > Now this can be fixed _partially_ with letting the executor deciding > whether doing the work inline or not, with the design issue mentioned > by Kim already ("isn't this the point of run_task()?"). The other issue > is balancing the queues which is superfluous with only one thread - but > we've had some very subtle issues with that. Riiight. Very good point. > (Yeah, yeah, I do have a mostly done JDK-8202328 around, but....) > >> mt_processing, then then yes, please always go the executor/mt- >> processing route. The executor is free to decide to not spin up >> workers for 1-threaded-execution as optimization. Other than that, I > > Been there, done that, i.e. the programmer had been responsible for > deciding whether to do the work inline or not. This had several issues > (history lesson follows, just skip if not interested): > > - people forgot to do that in some places, leading to massive > performance issues in situations the author hadn't thought of yet (like > running on a multi-TB heap with thousands of HW threads). > > - people started optimizing for one or the other case, adding lots of > code and so increasing maintenance costs a lot (e.g. some paths were > basically never tested until the next crash, lots of duplicate and > boilerplate code added etc) which made for lots of pain. > > - WorkGang itself does not give any kind of guarantees on which threads > a particular AbstractGangTask is run in the first place, and users must > not rely on that. Relying on that again and again just caused once- > during-a-full-moon issues in the past; another example for a misguided > assumption is that there is no guaranteed 1:1 relation between worker > id passed to the AbstractGangTask and the thread's id. I.e. a single > thread may execute an AbstractGangTask multiple times (with different > worker_id of course), and some workgang threads may not execute any at > all. (I assume that Shenandoah handles that case just fine?) > Assuming otherwise is just a wrong assumption of what WorkGang provides > to you and any code assuming so should simply be considered broken. > > These are the reasons why at some point (early JDK9 timeframe I think) > we just removed the special casing and cleaned G1 AbstractGangTasks > because we got weary of spending so much time on fixing these issues. > Basically CMS' AbstractGangTasks are left to look over (which seems > obviously broken for the RP case; but CMS'es removal keeps getting > closer anyway). > >> believe that 1-threaded and N-threaded execution should be >> equivalent. >> >> For this reason, I still think that simply removing this surprising >> adjustment of _mt_processing is the way to go, as proposed here: >> >> http://cr.openjdk.java.net/~rkennke/JDK-8212122/webrev.01/ >> >> If we really need to preserve the old (actually, rather new) > > This behavior is fully in-line with the guarantees of WorkGang > (assuming that ideally you would pass a WorkGang and AbstractGangTask > to reference processing if for a moment we ignore the Executor > framework which is an additional wrapper for Parallel GC). > >> behaviour to execute single-threaded using the specific single- >> threaded setup, we can do as proposed earlier: >> >> http://cr.openjdk.java.net/~rkennke/JDK-8212122/webrev.00/ >> >> and GC can override this by passing a suitable RP subclass. >> >> As much as I'd like to fix this mess, and care about CMS and >> ParallelGC, I currently don't have time for that (maybe later). What >> I need is a way to avoid the single-threaded-path altogether, because >> it breaks Shenandoah. > > From what I have understood so far, Shenandoah code seems to have > jumped into the same trap, i.e. seems to assume things about WorkGang > that it should not (which is unfortunately probably not a quick fix > either). > > If there is really need for getting this fixed asap, please preserve > old behavior. > > The new behavior you propose may cause significant regressions (even if > the Executor special-cases the 1-thread case) in reference processing > as it seems to unconditionally enable queue balancing in the MT case - > which is rather slow (and useless to actually attempt when there is > only one thread to do reference processing). Thanks for the detailed explanations! I have reconsidered our problem and found another workaround, which does not require any RP changes and does use the separate 1-threaded and N-threaded setups for now. In the long run we'll probably figure out something significantly better, and until then we're good I guess. I'm withdrawing this RFR. Thanks, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed Oct 17 09:48:20 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 17 Oct 2018 11:48:20 +0200 Subject: RFR: JDK-8212186: JVMTI lacks a few GC barriers/hooks Message-ID: <1c409b76-cb66-c6fa-d77f-cca236cd6cc5@redhat.com> Erik recently pointed out that JvmtiTagHashmapEntry::equals() is lacking on obj-equals-barrier, and that JvmtiTagHashmap::hash() better does ::resolve() the object to a stable value before casting it to a raw address. This lead us to do some more JVMTI testing and we found that we have a little inconsistency with reporting object size too: the VMObjectAlloc callback now reports a different size that GetObjectSize(). This is also fixed by this proposed change: Bug: https://bugs.openjdk.java.net/browse/JDK-8212186 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8212186/webrev.00/ Testing: hotspot/tier1, nsk/jvmti with Shenandoah Ok? Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From leo.korinth at oracle.com Wed Oct 17 15:24:54 2018 From: leo.korinth at oracle.com (Leo Korinth) Date: Wed, 17 Oct 2018 17:24:54 +0200 Subject: RFR: 8212595: Remove unused size_helper() in oop_oop_iterate* in instanceKlass.inline.hpp Message-ID: <4a254b3d-7d19-2025-5fd5-b30ce9abbf8b@oracle.com> Hi! This is a small clean-up. I am removing calls to size_helper() in instanceKlass.inline.hpp. The size is never used, and the corresponding methods outside InstanceKlass already returns void instead of int. Fast disassembly inspection of the oop_oop_iterate_reverse version reveals that the compiler output remains the same. That is, the compiler already understood that the code was unused. No performance improvement thus. The code becomes a bit simpler and corresponds better to the methods in the other *Klass files. Enhancement: https://bugs.openjdk.java.net/browse/JDK-8212595 Webrev: http://cr.openjdk.java.net/~lkorinth/8212595/00/ Testing: - mach5 tier1-3 Thanks, Leo From shade at redhat.com Wed Oct 17 15:27:49 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 17 Oct 2018 17:27:49 +0200 Subject: RFR: 8212595: Remove unused size_helper() in oop_oop_iterate* in instanceKlass.inline.hpp In-Reply-To: <4a254b3d-7d19-2025-5fd5-b30ce9abbf8b@oracle.com> References: <4a254b3d-7d19-2025-5fd5-b30ce9abbf8b@oracle.com> Message-ID: On 10/17/2018 05:24 PM, Leo Korinth wrote: > Webrev: > http://cr.openjdk.java.net/~lkorinth/8212595/00/ This looks good to me. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From erik.osterlund at oracle.com Wed Oct 17 16:36:34 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Wed, 17 Oct 2018 18:36:34 +0200 Subject: RFR: JDK-8212186: JVMTI lacks a few GC barriers/hooks In-Reply-To: <1c409b76-cb66-c6fa-d77f-cca236cd6cc5@redhat.com> References: <1c409b76-cb66-c6fa-d77f-cca236cd6cc5@redhat.com> Message-ID: <5E1355DC-4F5C-4C23-B02E-C41111E42D24@oracle.com> Hi Roman, Looks fantastic. BTW, out of curiosity, how do you do the heap walk - scanning or tracing? Thanks, /Erik > On 17 Oct 2018, at 11:48, Roman Kennke wrote: > > Erik recently pointed out that JvmtiTagHashmapEntry::equals() is > lacking on obj-equals-barrier, and that JvmtiTagHashmap::hash() better > does ::resolve() the object to a stable value before casting it to a raw > address. > > This lead us to do some more JVMTI testing and we found that we have a > little inconsistency with reporting object size too: the VMObjectAlloc > callback now reports a different size that GetObjectSize(). This is also > fixed by this proposed change: > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212186 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8212186/webrev.00/ > > Testing: hotspot/tier1, nsk/jvmti with Shenandoah > > Ok? > > Roman > From rkennke at redhat.com Wed Oct 17 17:10:57 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 17 Oct 2018 19:10:57 +0200 Subject: RFR: JDK-8212186: JVMTI lacks a few GC barriers/hooks In-Reply-To: <5E1355DC-4F5C-4C23-B02E-C41111E42D24@oracle.com> References: <1c409b76-cb66-c6fa-d77f-cca236cd6cc5@redhat.com> <5E1355DC-4F5C-4C23-B02E-C41111E42D24@oracle.com> Message-ID: <9d3e24ee-21fd-de57-3089-969553edfbf9@redhat.com> > Hi Roman, > > Looks fantastic. Thanks! > BTW, out of curiosity, how do you do the heap walk - scanning or tracing? Tracing. Scanning would require to carry around complete marking info (to be able to avoid holes of dead objects with recycled Klass*), which we don't. Roman > Thanks, > /Erik > >> On 17 Oct 2018, at 11:48, Roman Kennke wrote: >> >> Erik recently pointed out that JvmtiTagHashmapEntry::equals() is >> lacking on obj-equals-barrier, and that JvmtiTagHashmap::hash() better >> does ::resolve() the object to a stable value before casting it to a raw >> address. >> >> This lead us to do some more JVMTI testing and we found that we have a >> little inconsistency with reporting object size too: the VMObjectAlloc >> callback now reports a different size that GetObjectSize(). This is also >> fixed by this proposed change: >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8212186 >> Webrev: >> http://cr.openjdk.java.net/~rkennke/JDK-8212186/webrev.00/ >> >> Testing: hotspot/tier1, nsk/jvmti with Shenandoah >> >> Ok? >> >> Roman >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Wed Oct 17 17:19:00 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 17 Oct 2018 19:19:00 +0200 Subject: RFR: JDK-8212186: JVMTI lacks a few GC barriers/hooks In-Reply-To: <1c409b76-cb66-c6fa-d77f-cca236cd6cc5@redhat.com> References: <1c409b76-cb66-c6fa-d77f-cca236cd6cc5@redhat.com> Message-ID: <479e1961-9906-b380-59a7-cd7ed7d115f5@redhat.com> On 10/17/2018 11:48 AM, Roman Kennke wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212186 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8212186/webrev.00/ Looks good to me too. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From igor.ignatyev at oracle.com Wed Oct 17 17:54:51 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 17 Oct 2018 10:54:51 -0700 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest Message-ID: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > 141 lines changed: 140 ins; 1 del; 0 mod; Hi all, could you please review this small (and hopefully trivial) patch which converts internal TestVirtualSpace_test to gtest? again, the old test is still used by WhiteBox::runMemoryUnitTests, the old code hasn't been removed. it will be removed later when all 4 tests used by runMemoryUnitTests are converted. for the sake of reviewer, http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 is the old version. webrev: http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 testing: - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants - build w/ precompiled-headers enabled and disabled Thanks, -- Igor From igor.ignatyev at oracle.com Wed Oct 17 17:55:05 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 17 Oct 2018 10:55:05 -0700 Subject: RFR(XS) : 8171097 : Convert TestReservedSpace_test to Gtest In-Reply-To: References: Message-ID: Hi JC, thanks for your review! Cheers, -- Igor > On Oct 16, 2018, at 12:51 PM, JC Beyler wrote: > > Hi Igor, > > The change looks good to me, I compared old/new, the port looks good. > > For the reviewers, here is the original test code: > http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1071 > > Thanks, > Jc > > On Tue, Oct 16, 2018 at 9:06 AM Igor Ignatyev > wrote: > http://cr.openjdk.java.net/~iignatyev//8171097/webrev.00/index.html > > 188 lines changed: 187 ins; 1 del; 0 mod; > > Hi all, > > could you please review this small (and hopefully trivial) patch which converts internal TestReservedSpace_test to gtest? > since the old test is still used by WhiteBox::runMemoryUnitTests, the old code hasn't been removed. it will be removed later when all 4 tests used by runMemoryUnitTests are converted. > > webrev: http://cr.openjdk.java.net/~iignatyev//8171097/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8171097 > testing: > - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants > - build w/ precompiled-headers enabled and disabled > > PS the patch has been originally created by Kirill Zh, but hasn't been sent out for official review > > Thanks, > -- Igor > > > -- > > Thanks, > Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.stuefe at gmail.com Wed Oct 17 18:16:55 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 17 Oct 2018 20:16:55 +0200 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest In-Reply-To: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> References: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> Message-ID: Hi Igor, test_virtual_space_actual_committed_space() and actual_committed_space_one_large_page(): Won't we leak the reserved space now when we run into an ASSERT and then continue with the next gtests? Cheers, Thomas On Wed, Oct 17, 2018 at 7:55 PM Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > > 141 lines changed: 140 ins; 1 del; 0 mod; > > Hi all, > > could you please review this small (and hopefully trivial) patch which converts internal TestVirtualSpace_test to gtest? > again, the old test is still used by WhiteBox::runMemoryUnitTests, the old code hasn't been removed. it will be removed later when all 4 tests used by runMemoryUnitTests are converted. > > for the sake of reviewer, http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 is the old version. > > webrev: http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 > testing: > - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants > - build w/ precompiled-headers enabled and disabled > > Thanks, > -- Igor From igor.ignatyev at oracle.com Wed Oct 17 18:29:11 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 17 Oct 2018 11:29:11 -0700 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest In-Reply-To: References: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> Message-ID: <8C78BE56-2C11-4A8B-B961-FBA777AA6404@oracle.com> Hi Thomas, 1st of all, thanks for looking at it, appreciate that. it seems we can. there are two ways I can fix that either replace ASSERT by EXCEPT or introduce a text fixture w/ SetUp and TearDown. which do you prefer? -- Igor > On Oct 17, 2018, at 11:16 AM, Thomas St?fe wrote: > > Hi Igor, > > test_virtual_space_actual_committed_space() and > actual_committed_space_one_large_page(): > > Won't we leak the reserved space now when we run into an ASSERT and > then continue with the next gtests? > > Cheers, Thomas > > On Wed, Oct 17, 2018 at 7:55 PM Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html >>> 141 lines changed: 140 ins; 1 del; 0 mod; >> >> Hi all, >> >> could you please review this small (and hopefully trivial) patch which converts internal TestVirtualSpace_test to gtest? >> again, the old test is still used by WhiteBox::runMemoryUnitTests, the old code hasn't been removed. it will be removed later when all 4 tests used by runMemoryUnitTests are converted. >> >> for the sake of reviewer, http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 is the old version. >> >> webrev: http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html >> JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 >> testing: >> - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants >> - build w/ precompiled-headers enabled and disabled >> >> Thanks, >> -- Igor From stefan.johansson at oracle.com Wed Oct 17 18:51:27 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 17 Oct 2018 20:51:27 +0200 Subject: RFR (XS): 8211853: Avoid additional duplicate work when a reference in the task queue has already been evacuated In-Reply-To: <8f58feb1ec7da68af24d040a1fb126cc2397c842.camel@oracle.com> References: <8f58feb1ec7da68af24d040a1fb126cc2397c842.camel@oracle.com> Message-ID: Looks great, Stefan On 2018-10-16 15:59, Thomas Schatzl wrote: > Hi all, > > On Mon, 2018-10-08 at 16:58 +0200, Thomas Schatzl wrote: >> Hi all, >> >> can I have reviews for this tiny change that removes G1 doing some >> duplicate work? >> >> Due to benign races we can sometimes get references that have already >> been evacuated from the task queue. >> >> Current code in G1ParScanThreadState::do_oop_evac() needlessly >> applies follow-up calculations (remembered set, humonguous liveness) >> on this already evacuated reference - that all has been done the >> first time around already. > StefanJ correctly pointed out that there must more than one reason to > be allowed to do this. > > - we can skip the humongous liveness check because G1 never pushes oop* > with references outside of the collection set onto the task queue. > Humongous objects are always logically in old gen, so we should never > see such a reference here. > I added an additional assert to make that clear. > > - the second reason for being able to exit early if the reference in > the oop* is in old gen is because then we know that that reference has > already been processed (as we try to only push references into the > collection set on the task queue), and the remembered set update etc > already done. > > Note that in case of evacuation failure this check is not sufficient, > i.e. the reference is still a reference into the collection set, so G1 > still does the update_rs() again the next time we get here. However > this is no problem either - naturally they are not references to > humongous objects, and we do not care too much about performance in > that rare case either (i.e. the additional update_rs() call is very > quick anyway). > >> Just exit the method if we detect such a task. > New webrev at > http://cr.openjdk.java.net/~tschatzl/8211853/webrev.0_to_1 (diff) > http://cr.openjdk.java.net/~tschatzl/8211853/webrev.1 (full) > with the added assert. > > Passed hs-tier1-3 as expected. > > Thanks, > Thomas > > From thomas.stuefe at gmail.com Wed Oct 17 18:53:37 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 17 Oct 2018 20:53:37 +0200 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest In-Reply-To: <8C78BE56-2C11-4A8B-B961-FBA777AA6404@oracle.com> References: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> <8C78BE56-2C11-4A8B-B961-FBA777AA6404@oracle.com> Message-ID: On Wed, Oct 17, 2018 at 8:29 PM Igor Ignatyev wrote: > > Hi Thomas, > > 1st of all, thanks for looking at it, appreciate that. > > it seems we can. there are two ways I can fix that either replace ASSERT by EXCEPT or introduce a text fixture w/ SetUp and TearDown. which do you prefer? > > -- Igor Slight preference for SetUp/TearDown (EXCEPT would quit the testing altogether, wont it?). Lazy me would have just used an RAII object like this: struct Cleaner { ReservedSpace* rs; ~Cleaner() { rs.release(); } }; But I leave it up to you. ... Thomas > > > On Oct 17, 2018, at 11:16 AM, Thomas St?fe wrote: > > > > Hi Igor, > > > > test_virtual_space_actual_committed_space() and > > actual_committed_space_one_large_page(): > > > > Won't we leak the reserved space now when we run into an ASSERT and > > then continue with the next gtests? > > > > Cheers, Thomas > > > > On Wed, Oct 17, 2018 at 7:55 PM Igor Ignatyev wrote: > >> > >> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > >>> 141 lines changed: 140 ins; 1 del; 0 mod; > >> > >> Hi all, > >> > >> could you please review this small (and hopefully trivial) patch which converts internal TestVirtualSpace_test to gtest? > >> again, the old test is still used by WhiteBox::runMemoryUnitTests, the old code hasn't been removed. it will be removed later when all 4 tests used by runMemoryUnitTests are converted. > >> > >> for the sake of reviewer, http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 is the old version. > >> > >> webrev: http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > >> JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 > >> testing: > >> - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants > >> - build w/ precompiled-headers enabled and disabled > >> > >> Thanks, > >> -- Igor > From jcbeyler at google.com Wed Oct 17 18:58:54 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 17 Oct 2018 11:58:54 -0700 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest In-Reply-To: References: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> <8C78BE56-2C11-4A8B-B961-FBA777AA6404@oracle.com> Message-ID: Hi Igor, Looks good to me as well, I agree with Thomas' comments and also prefer RAII, especially since SetUp/TearDown here is going to be weird since you don't always create the object the same way. If you make a good RAII object, then both tests could use the same kind of code instead of one using a helper method and another creating the ReservedSpace directly too. Thanks! Jc On Wed, Oct 17, 2018 at 11:54 AM Thomas St?fe wrote: > On Wed, Oct 17, 2018 at 8:29 PM Igor Ignatyev > wrote: > > > > Hi Thomas, > > > > 1st of all, thanks for looking at it, appreciate that. > > > > it seems we can. there are two ways I can fix that either replace ASSERT > by EXCEPT or introduce a text fixture w/ SetUp and TearDown. which do you > prefer? > > > > -- Igor > > Slight preference for SetUp/TearDown (EXCEPT would quit the testing > altogether, wont it?). > > Lazy me would have just used an RAII object like this: > > struct Cleaner { > ReservedSpace* rs; > ~Cleaner() { rs.release(); } > }; > > But I leave it up to you. > > ... Thomas > > > > > > On Oct 17, 2018, at 11:16 AM, Thomas St?fe > wrote: > > > > > > Hi Igor, > > > > > > test_virtual_space_actual_committed_space() and > > > actual_committed_space_one_large_page(): > > > > > > Won't we leak the reserved space now when we run into an ASSERT and > > > then continue with the next gtests? > > > > > > Cheers, Thomas > > > > > > On Wed, Oct 17, 2018 at 7:55 PM Igor Ignatyev < > igor.ignatyev at oracle.com> wrote: > > >> > > >> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > > >>> 141 lines changed: 140 ins; 1 del; 0 mod; > > >> > > >> Hi all, > > >> > > >> could you please review this small (and hopefully trivial) patch > which converts internal TestVirtualSpace_test to gtest? > > >> again, the old test is still used by WhiteBox::runMemoryUnitTests, > the old code hasn't been removed. it will be removed later when all 4 tests > used by runMemoryUnitTests are converted. > > >> > > >> for the sake of reviewer, > http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 > is the old version. > > >> > > >> webrev: > http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > > >> JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 > > >> testing: > > >> - converted tests on linux-x64, windows-x64, macosx-x64, > solaris-sparcv9 in product and fastdebug variants > > >> - build w/ precompiled-headers enabled and disabled > > >> > > >> Thanks, > > >> -- Igor > > > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.ignatyev at oracle.com Wed Oct 17 19:00:43 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 17 Oct 2018 12:00:43 -0700 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest In-Reply-To: References: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> <8C78BE56-2C11-4A8B-B961-FBA777AA6404@oracle.com> Message-ID: <452FE76E-15DD-4B37-87F4-BC966DD9FB05@oracle.com> EXPECT would just mark a test as failed and allow the method to continue, so rs.release() will be called. test fixture will basically do the same as an RAII object, but it will be a bit more obvious as test preparation and cleaning steps will be separated from test actions, so I'll go w/ setUp/tearDown. -- Igor > On Oct 17, 2018, at 11:53 AM, Thomas St?fe wrote: > > On Wed, Oct 17, 2018 at 8:29 PM Igor Ignatyev wrote: >> >> Hi Thomas, >> >> 1st of all, thanks for looking at it, appreciate that. >> >> it seems we can. there are two ways I can fix that either replace ASSERT by EXCEPT or introduce a text fixture w/ SetUp and TearDown. which do you prefer? >> >> -- Igor > > Slight preference for SetUp/TearDown (EXCEPT would quit the testing > altogether, wont it?). > > Lazy me would have just used an RAII object like this: > > struct Cleaner { > ReservedSpace* rs; > ~Cleaner() { rs.release(); } > }; > > But I leave it up to you. > > ... Thomas > >> >>> On Oct 17, 2018, at 11:16 AM, Thomas St?fe wrote: >>> >>> Hi Igor, >>> >>> test_virtual_space_actual_committed_space() and >>> actual_committed_space_one_large_page(): >>> >>> Won't we leak the reserved space now when we run into an ASSERT and >>> then continue with the next gtests? >>> >>> Cheers, Thomas >>> >>> On Wed, Oct 17, 2018 at 7:55 PM Igor Ignatyev wrote: >>>> >>>> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html >>>>> 141 lines changed: 140 ins; 1 del; 0 mod; >>>> >>>> Hi all, >>>> >>>> could you please review this small (and hopefully trivial) patch which converts internal TestVirtualSpace_test to gtest? >>>> again, the old test is still used by WhiteBox::runMemoryUnitTests, the old code hasn't been removed. it will be removed later when all 4 tests used by runMemoryUnitTests are converted. >>>> >>>> for the sake of reviewer, http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 is the old version. >>>> >>>> webrev: http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 >>>> testing: >>>> - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants >>>> - build w/ precompiled-headers enabled and disabled >>>> >>>> Thanks, >>>> -- Igor >> From thomas.stuefe at gmail.com Wed Oct 17 19:04:07 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 17 Oct 2018 21:04:07 +0200 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest In-Reply-To: <452FE76E-15DD-4B37-87F4-BC966DD9FB05@oracle.com> References: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> <8C78BE56-2C11-4A8B-B961-FBA777AA6404@oracle.com> <452FE76E-15DD-4B37-87F4-BC966DD9FB05@oracle.com> Message-ID: On Wed 17. Oct 2018 at 21:00, Igor Ignatyev wrote: > EXPECT would just mark a test as failed and allow the method to continue, > so rs.release() will be called. Oh, okay. Thanks for clarifying. > > test fixture will basically do the same as an RAII object, but it will be > a bit more obvious as test preparation and cleaning steps will be separated > from test actions, so I'll go w/ setUp/tearDown. Okay fine by me. ..thomas > > -- Igor > > > On Oct 17, 2018, at 11:53 AM, Thomas St?fe > wrote: > > > > On Wed, Oct 17, 2018 at 8:29 PM Igor Ignatyev > wrote: > >> > >> Hi Thomas, > >> > >> 1st of all, thanks for looking at it, appreciate that. > >> > >> it seems we can. there are two ways I can fix that either replace > ASSERT by EXCEPT or introduce a text fixture w/ SetUp and TearDown. which > do you prefer? > >> > >> -- Igor > > > > Slight preference for SetUp/TearDown (EXCEPT would quit the testing > > altogether, wont it?). > > > > Lazy me would have just used an RAII object like this: > > > > struct Cleaner { > > ReservedSpace* rs; > > ~Cleaner() { rs.release(); } > > }; > > > > But I leave it up to you. > > > > ... Thomas > > > >> > >>> On Oct 17, 2018, at 11:16 AM, Thomas St?fe > wrote: > >>> > >>> Hi Igor, > >>> > >>> test_virtual_space_actual_committed_space() and > >>> actual_committed_space_one_large_page(): > >>> > >>> Won't we leak the reserved space now when we run into an ASSERT and > >>> then continue with the next gtests? > >>> > >>> Cheers, Thomas > >>> > >>> On Wed, Oct 17, 2018 at 7:55 PM Igor Ignatyev < > igor.ignatyev at oracle.com> wrote: > >>>> > >>>> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > >>>>> 141 lines changed: 140 ins; 1 del; 0 mod; > >>>> > >>>> Hi all, > >>>> > >>>> could you please review this small (and hopefully trivial) patch > which converts internal TestVirtualSpace_test to gtest? > >>>> again, the old test is still used by WhiteBox::runMemoryUnitTests, > the old code hasn't been removed. it will be removed later when all 4 tests > used by runMemoryUnitTests are converted. > >>>> > >>>> for the sake of reviewer, > http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 > is the old version. > >>>> > >>>> webrev: > http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 > >>>> testing: > >>>> - converted tests on linux-x64, windows-x64, macosx-x64, > solaris-sparcv9 in product and fastdebug variants > >>>> - build w/ precompiled-headers enabled and disabled > >>>> > >>>> Thanks, > >>>> -- Igor > >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From per.liden at oracle.com Wed Oct 17 20:13:50 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 17 Oct 2018 22:13:50 +0200 Subject: RFR: 8212595: Remove unused size_helper() in oop_oop_iterate* in instanceKlass.inline.hpp In-Reply-To: <4a254b3d-7d19-2025-5fd5-b30ce9abbf8b@oracle.com> References: <4a254b3d-7d19-2025-5fd5-b30ce9abbf8b@oracle.com> Message-ID: <0e27c0ca-539a-ad70-b153-589ac4c336f8@oracle.com> Looks good. /Per On 10/17/2018 05:24 PM, Leo Korinth wrote: > Hi! > > This is a small clean-up. > > I am removing calls to size_helper() in instanceKlass.inline.hpp. The > size is never used, and the corresponding methods outside InstanceKlass > already returns void instead of int. > > Fast disassembly inspection of the oop_oop_iterate_reverse version > reveals that the compiler output remains the same. That is, the compiler > already understood that the code was unused. No performance improvement > thus. The code becomes a bit simpler and corresponds better to the > methods in the other *Klass files. > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8212595 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/8212595/00/ > > Testing: > - mach5 tier1-3 > > Thanks, Leo From hohensee at amazon.com Wed Oct 17 20:14:58 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Wed, 17 Oct 2018 20:14:58 +0000 Subject: Backport 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: <7565CED8-5174-4CB2-8AC1-F2544BD3B316@amazon.com> References: <7565CED8-5174-4CB2-8AC1-F2544BD3B316@amazon.com> Message-ID: <22380FF1-A213-464A-9F5B-8A7D6815909E@amazon.com> Ping. Reviews please, before the cutoff? Thanks, Paul From: hotspot-gc-dev on behalf of "Hohensee, Paul" Date: Friday, October 12, 2018 at 10:14 AM To: JC Beyler Cc: "serviceability-dev at openjdk.java.net" , "jdk8u-dev at openjdk.java.net" , "hotspot-gc-dev at openjdk.java.net" Subject: Re: Backport 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results Thanks for the review, new webrev at http://cr.openjdk.java.net/~phh/8195115/webrev.06/. The different requires were an artifact of me trying to get TestOldGenCollectionUsage.java to run. Good catch. Jiangli changed the heap size to 14m to get it to work with CDS, see https://bugs.openjdk.java.net/browse/JDK-8210193. Paul From: JC Beyler Date: Thursday, October 11, 2018 at 10:37 PM To: "Hohensee, Paul" Cc: "hotspot-gc-dev at openjdk.java.net" , "serviceability-dev at openjdk.java.net" , "jdk8u-dev at openjdk.java.net" Subject: Re: RFR: Backport 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results Hi Paul, The biggest thing I saw in this RFR was that the flags for the test: http://cr.openjdk.java.net/~phh/8195115/webrev.05/test/gc/g1/mixedgc/TestOldGenCollectionUsage.java.html were changed it seems: - the @requires are different for the backport (you accept null for JDK8 for GC and also removed the @requires vm.opt.MaxGCPauseMillis == "null") - the @run flags are different (-Xms/Xmx are 14m for the backport; they were 12 originally; there is a comment below in the backport saying this requires normally 12m though you ask for 14 in the @run) What are the reasons for these differences? Apart from that, the backport seemed ok but I'm not that well versed in the GC changes :) Jc On Thu, Oct 11, 2018 at 5:04 PM Hohensee, Paul > wrote: Please review a backport to jdk8u. Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.05/ JDK11 patch: http://hg.openjdk.java.net/jdk/jdk/rev/5d3c5af82654 The backport is slightly different from the JDK11 patch due to G1 refactoring, hence my request for new review. I?ll ask for jdk8u approval once the backport is reviewed. I backported two jtreg tests from JDK11, which pass. Also, all the hotspot gc jtreg tests pass as well as they do for jdk8u-dev. There was a CSR involved, https://bugs.openjdk.java.net/browse/JDK-8196719. Does that have to be re-approved for jdk8u as well, and if so, what?s the process? Thanks, Paul -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From hohensee at amazon.com Wed Oct 17 20:17:25 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Wed, 17 Oct 2018 20:17:25 +0000 Subject: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions In-Reply-To: References: <8C241BB9-2089-491B-A27E-57C4F43FBA9E@amazon.com> Message-ID: Ping. From: serviceability-dev on behalf of "Hohensee, Paul" Date: Thursday, October 11, 2018 at 6:46 PM To: "hotspot-gc-dev at openjdk.java.net" , "serviceability-dev at openjdk.java.net" Subject: Re: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions Any takers? :) From: serviceability-dev on behalf of "Hohensee, Paul" Date: Monday, October 8, 2018 at 7:50 PM To: "hotspot-gc-dev at openjdk.java.net" , "serviceability-dev at openjdk.java.net" Subject: RFR: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions Bug: https://bugs.openjdk.java.net/browse/JDK-8196989 CSR: https://bugs.openjdk.java.net/browse/JDK-8196991 Webrev: http://cr.openjdk.java.net/~phh/8196989/webrev.02/ As requested, I split the jstat counter update off from the MXBean update. This is the MXBean update. The jstat counter RFE is https://bugs.openjdk.java.net/browse/JDK-8210965 and its CSR is https://bugs.openjdk.java.net/browse/JDK-8210966. The MXBean CSR is in draft state, I?d greatly appreciate review and sign-off. It?s been suggested that we add another pool to represent the free region set, but doing so would be incompatible with existing MXBean use invariants for all GCs. These are: 1. The sum of the pools? MemoryUsage.max properties is the total reserved heap size. 2. The sum of the pools? MemoryUsage.committed properties is the total committed size. 3. The sum of the pools? MemoryUsage.used properties is the total size of the memory containing objects, live and dead-and-yet-to-be-collected, as the case might be, plus intentional gaps between them. 4. The total free space is (sum of the max properties ? sum of the used properties). 5. The total uncommitted space is (sum of the max properties ? sum of the committed properties). 6. The total committed free space is (2) ? (3). To keep invariants 1, 2 and 3, the free region pool?s ?max? property should be ?undefined? (i.e., -1). The intuitive, to me, ?used? property value would be the total free space, but that would violate invariant 4 above. Defining the ?committed? property as the total committed free space would violate invariants 2 and 6. The patch passes the submit repo, hotspot tier1, and, separately, the serviceability, jfr, and gc jtreg tests. I?m uncertain how to construct a test that checks for valid MXBean content: the existing tests don?t. Any such test will be fragile due to possible future Hotspot changes that affect the values, and to run-to-run variability. I?ve done by-hand comparisons between the old and new MXBean content using the SwingSet2 demo, including using App CDS, and the numbers look reasonable. The guts of the change are in G1MonitoringSupport::recalculate_sizes, initialize_serviceability, memory_managers, memory_pools, and G1MonitoringScope. I also defined TraceConcMemoryManagerStats to track the concurrent cycle in a way analogous to TraceCMSMemoryManagerStats. The changes to the includes in g1FullGCOopClosures.inline.hpp and g1HeapVerifier.cpp are to satisfy compiler complaints. I changed the 3rd argument to the G1MonitoringScope constructor to a mixed_gc flag, and use accessor methods instead of direct field accesses when accessor methods exist. I believe I?ve minimized the latter. I updated the copyright date to 2018 in memoryService.hpp because I neglected to do so in my previous G1 MXBean patch. Thanks, Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Wed Oct 17 20:46:30 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 17 Oct 2018 13:46:30 -0700 Subject: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions In-Reply-To: References: <8C241BB9-2089-491B-A27E-57C4F43FBA9E@amazon.com> Message-ID: Hi Paul, I looked at this but it took time for me to "digest" it and I haven't entirely gone through the real GC changes :) My few remarks on the webrev itself are: - http://cr.openjdk.java.net/~phh/8196989/webrev.02/src/hotspot/share/services/memoryService.hpp.udiff.html - There is no need to change the copyright, right? - http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/hotspot/jtreg/gc/TestMemoryMXBeansAndPoolsPresence.java.udiff.html - the break seems to be on the wrong line, no? + } break; - http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/jdk/java/lang/management/MemoryMXBean/LowMemoryTest.java.udiff.html and http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/jdk/java/lang/management/MemoryMXBean/MemoryManagement.java.udiff.html + // In G1, humongous objects are tracked in the old space only in + // legacy monitoring mode. In default mode, G1 tracks humongous + // objects in the humongous space, which latter also supports a + // usage threshold. Since we're allocating humongous objects in + // this test, in default mode the old space doesn't change. For + // this test, we use the old space in legacy mode (it's called + // "G1 Old Gen" and the humongous space in default mode. If we + // used "G1 Old Space" in default mode, notification would never + // happen. -> latter seems ot be the wrong word or something is missing in that sentence -> the parenthesis is never closed (it's called.... is missing a ) somewhere Thanks, Jc On Wed, Oct 17, 2018 at 1:18 PM Hohensee, Paul wrote: > Ping. > > > > *From: *serviceability-dev > on behalf of "Hohensee, Paul" > *Date: *Thursday, October 11, 2018 at 6:46 PM > *To: *"hotspot-gc-dev at openjdk.java.net" , > "serviceability-dev at openjdk.java.net" > > *Subject: *Re: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector > MXBean definitions > > > > Any takers? :) > > > > *From: *serviceability-dev > on behalf of "Hohensee, Paul" > *Date: *Monday, October 8, 2018 at 7:50 PM > *To: *"hotspot-gc-dev at openjdk.java.net" , > "serviceability-dev at openjdk.java.net" > > *Subject: *RFR: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector > MXBean definitions > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8196989 > > CSR: https://bugs.openjdk.java.net/browse/JDK-8196991 > > Webrev: http://cr.openjdk.java.net/~phh/8196989/webrev.02/ > > > > As requested, I split the jstat counter update off from the MXBean update. > This is the MXBean update. The jstat counter RFE is > https://bugs.openjdk.java.net/browse/JDK-8210965 and its CSR is > https://bugs.openjdk.java.net/browse/JDK-8210966. > > > > The MXBean CSR is in draft state, I?d greatly appreciate review and > sign-off. > > > > It?s been suggested that we add another pool to represent the free region > set, but doing so would be incompatible with existing MXBean use invariants > for all GCs. These are: > > > > 1. The sum of the pools? MemoryUsage.max properties is the total > reserved heap size. > 2. The sum of the pools? MemoryUsage.committed properties is the total > committed size. > 3. The sum of the pools? MemoryUsage.used properties is the total size > of the memory containing objects, live and dead-and-yet-to-be-collected, as > the case might be, plus intentional gaps between them. > 4. The total free space is (sum of the max properties ? sum of the > used properties). > 5. The total uncommitted space is (sum of the max properties ? sum of > the committed properties). > 6. The total committed free space is (2) ? (3). > > > > To keep invariants 1, 2 and 3, the free region pool?s ?max? property > should be ?undefined? (i.e., -1). The intuitive, to me, ?used? property > value would be the total free space, but that would violate invariant 4 > above. Defining the ?committed? property as the total committed free space > would violate invariants 2 and 6. > > > > The patch passes the submit repo, hotspot tier1, and, separately, the > serviceability, jfr, and gc jtreg tests. I?m uncertain how to construct a > test that checks for valid MXBean content: the existing tests don?t. Any > such test will be fragile due to possible future Hotspot changes that > affect the values, and to run-to-run variability. I?ve done by-hand > comparisons between the old and new MXBean content using the SwingSet2 > demo, including using App CDS, and the numbers look reasonable. > > > > The guts of the change are in G1MonitoringSupport::recalculate_sizes, > initialize_serviceability, memory_managers, memory_pools, and > G1MonitoringScope. I also defined TraceConcMemoryManagerStats to track the > concurrent cycle in a way analogous to TraceCMSMemoryManagerStats. The > changes to the includes in g1FullGCOopClosures.inline.hpp and > g1HeapVerifier.cpp are to satisfy compiler complaints. I changed the 3rd > argument to the G1MonitoringScope constructor to a mixed_gc flag, and use > accessor methods instead of direct field accesses when accessor methods > exist. I believe I?ve minimized the latter. I updated the copyright date to > 2018 in memoryService.hpp because I neglected to do so in my previous G1 > MXBean patch. > > > > Thanks, > > > > Paul > > > > > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.ignatyev at oracle.com Wed Oct 17 21:45:38 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 17 Oct 2018 14:45:38 -0700 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest In-Reply-To: References: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> <8C78BE56-2C11-4A8B-B961-FBA777AA6404@oracle.com> <452FE76E-15DD-4B37-87F4-BC966DD9FB05@oracle.com> Message-ID: Hi Thomas, JC I tried to use a test fixture and the code became very clumsy, so I went w/ a simple RAII. http://cr.openjdk.java.net/~iignatyev//8177709/webrev.0-1/index.html I've also noticed that we have same problem in TestReservedSpace_test tests -- http://cr.openjdk.java.net/~iignatyev//8171097/webrev.0-1/index.html Thanks, -- Igor > On Oct 17, 2018, at 12:04 PM, Thomas St?fe wrote: > > > > On Wed 17. Oct 2018 at 21:00, Igor Ignatyev > wrote: > EXPECT would just mark a test as failed and allow the method to continue, so rs.release() will be called. > > Oh, okay. Thanks for clarifying. > > > > test fixture will basically do the same as an RAII object, but it will be a bit more obvious as test preparation and cleaning steps will be separated from test actions, so I'll go w/ setUp/tearDown. > > Okay fine by me. > > ..thomas > > > > -- Igor > > > On Oct 17, 2018, at 11:53 AM, Thomas St?fe > wrote: > > > > On Wed, Oct 17, 2018 at 8:29 PM Igor Ignatyev > wrote: > >> > >> Hi Thomas, > >> > >> 1st of all, thanks for looking at it, appreciate that. > >> > >> it seems we can. there are two ways I can fix that either replace ASSERT by EXCEPT or introduce a text fixture w/ SetUp and TearDown. which do you prefer? > >> > >> -- Igor > > > > Slight preference for SetUp/TearDown (EXCEPT would quit the testing > > altogether, wont it?). > > > > Lazy me would have just used an RAII object like this: > > > > struct Cleaner { > > ReservedSpace* rs; > > ~Cleaner() { rs.release(); } > > }; > > > > But I leave it up to you. > > > > ... Thomas > > > >> > >>> On Oct 17, 2018, at 11:16 AM, Thomas St?fe > wrote: > >>> > >>> Hi Igor, > >>> > >>> test_virtual_space_actual_committed_space() and > >>> actual_committed_space_one_large_page(): > >>> > >>> Won't we leak the reserved space now when we run into an ASSERT and > >>> then continue with the next gtests? > >>> > >>> Cheers, Thomas > >>> > >>> On Wed, Oct 17, 2018 at 7:55 PM Igor Ignatyev > wrote: > >>>> > >>>> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > >>>>> 141 lines changed: 140 ins; 1 del; 0 mod; > >>>> > >>>> Hi all, > >>>> > >>>> could you please review this small (and hopefully trivial) patch which converts internal TestVirtualSpace_test to gtest? > >>>> again, the old test is still used by WhiteBox::runMemoryUnitTests, the old code hasn't been removed. it will be removed later when all 4 tests used by runMemoryUnitTests are converted. > >>>> > >>>> for the sake of reviewer, http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 is the old version. > >>>> > >>>> webrev: http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html > >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 > >>>> testing: > >>>> - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants > >>>> - build w/ precompiled-headers enabled and disabled > >>>> > >>>> Thanks, > >>>> -- Igor > >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Wed Oct 17 22:15:20 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 17 Oct 2018 15:15:20 -0700 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest In-Reply-To: References: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> <8C78BE56-2C11-4A8B-B961-FBA777AA6404@oracle.com> <452FE76E-15DD-4B37-87F4-BC966DD9FB05@oracle.com> Message-ID: Hi Igor, Both look good to me now :) Thanks! Jc On Wed, Oct 17, 2018 at 2:45 PM Igor Ignatyev wrote: > Hi Thomas, JC > > I tried to use a test fixture and the code became very clumsy, so I went > w/ a simple RAII. > http://cr.openjdk.java.net/~iignatyev//8177709/webrev.0-1/index.html > > I've also noticed that we have same problem in TestReservedSpace_test > tests -- > http://cr.openjdk.java.net/~iignatyev//8171097/webrev.0-1/index.html > > Thanks, > -- Igor > > On Oct 17, 2018, at 12:04 PM, Thomas St?fe > wrote: > > > > On Wed 17. Oct 2018 at 21:00, Igor Ignatyev > wrote: > >> EXPECT would just mark a test as failed and allow the method to continue, >> so rs.release() will be called. > > > Oh, okay. Thanks for clarifying. > > >> >> test fixture will basically do the same as an RAII object, but it will be >> a bit more obvious as test preparation and cleaning steps will be separated >> from test actions, so I'll go w/ setUp/tearDown. > > > Okay fine by me. > > ..thomas > > >> >> -- Igor >> >> > On Oct 17, 2018, at 11:53 AM, Thomas St?fe >> wrote: >> > >> > On Wed, Oct 17, 2018 at 8:29 PM Igor Ignatyev >> wrote: >> >> >> >> Hi Thomas, >> >> >> >> 1st of all, thanks for looking at it, appreciate that. >> >> >> >> it seems we can. there are two ways I can fix that either replace >> ASSERT by EXCEPT or introduce a text fixture w/ SetUp and TearDown. which >> do you prefer? >> >> >> >> -- Igor >> > >> > Slight preference for SetUp/TearDown (EXCEPT would quit the testing >> > altogether, wont it?). >> > >> > Lazy me would have just used an RAII object like this: >> > >> > struct Cleaner { >> > ReservedSpace* rs; >> > ~Cleaner() { rs.release(); } >> > }; >> > >> > But I leave it up to you. >> > >> > ... Thomas >> > >> >> >> >>> On Oct 17, 2018, at 11:16 AM, Thomas St?fe >> wrote: >> >>> >> >>> Hi Igor, >> >>> >> >>> test_virtual_space_actual_committed_space() and >> >>> actual_committed_space_one_large_page(): >> >>> >> >>> Won't we leak the reserved space now when we run into an ASSERT and >> >>> then continue with the next gtests? >> >>> >> >>> Cheers, Thomas >> >>> >> >>> On Wed, Oct 17, 2018 at 7:55 PM Igor Ignatyev < >> igor.ignatyev at oracle.com> wrote: >> >>>> >> >>>> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html >> >>>>> 141 lines changed: 140 ins; 1 del; 0 mod; >> >>>> >> >>>> Hi all, >> >>>> >> >>>> could you please review this small (and hopefully trivial) patch >> which converts internal TestVirtualSpace_test to gtest? >> >>>> again, the old test is still used by WhiteBox::runMemoryUnitTests, >> the old code hasn't been removed. it will be removed later when all 4 tests >> used by runMemoryUnitTests are converted. >> >>>> >> >>>> for the sake of reviewer, >> http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 >> is the old version. >> >>>> >> >>>> webrev: >> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html >> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 >> >>>> testing: >> >>>> - converted tests on linux-x64, windows-x64, macosx-x64, >> solaris-sparcv9 in product and fastdebug variants >> >>>> - build w/ precompiled-headers enabled and disabled >> >>>> >> >>>> Thanks, >> >>>> -- Igor >> >> >> >> > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Thu Oct 18 02:21:35 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 Oct 2018 12:21:35 +1000 Subject: RFR: JDK-8212186: JVMTI lacks a few GC barriers/hooks In-Reply-To: <1c409b76-cb66-c6fa-d77f-cca236cd6cc5@redhat.com> References: <1c409b76-cb66-c6fa-d77f-cca236cd6cc5@redhat.com> Message-ID: <5979ecdc-90ad-4226-fc76-e5d5032cfc3e@oracle.com> Hi Roman, Erik, On 17/10/2018 7:48 PM, Roman Kennke wrote: > Erik recently pointed out that JvmtiTagHashmapEntry::equals() is > lacking on obj-equals-barrier, and that JvmtiTagHashmap::hash() better > does ::resolve() the object to a stable value before casting it to a raw > address. Can the resolve be incorporated into the cast_from_oop? It's just not obvious to me how one realizes when the resolve is necessary. ?? > This lead us to do some more JVMTI testing and we found that we have a > little inconsistency with reporting object size too: the VMObjectAlloc > callback now reports a different size that GetObjectSize(). This is also > fixed by this proposed change: So when is it valid to use obj->size() and when not? Thanks, David ----- > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212186 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8212186/webrev.00/ > > Testing: hotspot/tier1, nsk/jvmti with Shenandoah > > Ok? > > Roman > From thomas.stuefe at gmail.com Thu Oct 18 05:39:53 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 18 Oct 2018 07:39:53 +0200 Subject: RFR(XXS) : 8177709 : Convert TestVirtualSpace_test to GTest In-Reply-To: References: <19DDC8C6-0254-4300-8EA3-86D7E2A47488@oracle.com> <8C78BE56-2C11-4A8B-B961-FBA777AA6404@oracle.com> <452FE76E-15DD-4B37-87F4-BC966DD9FB05@oracle.com> Message-ID: Looks both good Igor. +1 for the const pointers :) ..Thomas On Wed, Oct 17, 2018 at 11:45 PM Igor Ignatyev wrote: > > Hi Thomas, JC > > I tried to use a test fixture and the code became very clumsy, so I went w/ a simple RAII. > http://cr.openjdk.java.net/~iignatyev//8177709/webrev.0-1/index.html > > I've also noticed that we have same problem in TestReservedSpace_test tests -- http://cr.openjdk.java.net/~iignatyev//8171097/webrev.0-1/index.html > > Thanks, > -- Igor > > On Oct 17, 2018, at 12:04 PM, Thomas St?fe wrote: > > > > On Wed 17. Oct 2018 at 21:00, Igor Ignatyev wrote: >> >> EXPECT would just mark a test as failed and allow the method to continue, so rs.release() will be called. > > > Oh, okay. Thanks for clarifying. > >> >> >> test fixture will basically do the same as an RAII object, but it will be a bit more obvious as test preparation and cleaning steps will be separated from test actions, so I'll go w/ setUp/tearDown. > > > Okay fine by me. > > ..thomas > >> >> >> -- Igor >> >> > On Oct 17, 2018, at 11:53 AM, Thomas St?fe wrote: >> > >> > On Wed, Oct 17, 2018 at 8:29 PM Igor Ignatyev wrote: >> >> >> >> Hi Thomas, >> >> >> >> 1st of all, thanks for looking at it, appreciate that. >> >> >> >> it seems we can. there are two ways I can fix that either replace ASSERT by EXCEPT or introduce a text fixture w/ SetUp and TearDown. which do you prefer? >> >> >> >> -- Igor >> > >> > Slight preference for SetUp/TearDown (EXCEPT would quit the testing >> > altogether, wont it?). >> > >> > Lazy me would have just used an RAII object like this: >> > >> > struct Cleaner { >> > ReservedSpace* rs; >> > ~Cleaner() { rs.release(); } >> > }; >> > >> > But I leave it up to you. >> > >> > ... Thomas >> > >> >> >> >>> On Oct 17, 2018, at 11:16 AM, Thomas St?fe wrote: >> >>> >> >>> Hi Igor, >> >>> >> >>> test_virtual_space_actual_committed_space() and >> >>> actual_committed_space_one_large_page(): >> >>> >> >>> Won't we leak the reserved space now when we run into an ASSERT and >> >>> then continue with the next gtests? >> >>> >> >>> Cheers, Thomas >> >>> >> >>> On Wed, Oct 17, 2018 at 7:55 PM Igor Ignatyev wrote: >> >>>> >> >>>> http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html >> >>>>> 141 lines changed: 140 ins; 1 del; 0 mod; >> >>>> >> >>>> Hi all, >> >>>> >> >>>> could you please review this small (and hopefully trivial) patch which converts internal TestVirtualSpace_test to gtest? >> >>>> again, the old test is still used by WhiteBox::runMemoryUnitTests, the old code hasn't been removed. it will be removed later when all 4 tests used by runMemoryUnitTests are converted. >> >>>> >> >>>> for the sake of reviewer, http://hg.openjdk.java.net/jdk/jdk/file/2e928420389d/src/hotspot/share/memory/virtualspace.cpp#l1261 is the old version. >> >>>> >> >>>> webrev: http://cr.openjdk.java.net/~iignatyev//8177709/webrev.00/index.html >> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177709 >> >>>> testing: >> >>>> - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants >> >>>> - build w/ precompiled-headers enabled and disabled >> >>>> >> >>>> Thanks, >> >>>> -- Igor >> >> >> > From per.liden at oracle.com Thu Oct 18 07:13:44 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 18 Oct 2018 09:13:44 +0200 Subject: RFR: JDK-8212186: JVMTI lacks a few GC barriers/hooks In-Reply-To: <9d3e24ee-21fd-de57-3089-969553edfbf9@redhat.com> References: <1c409b76-cb66-c6fa-d77f-cca236cd6cc5@redhat.com> <5E1355DC-4F5C-4C23-B02E-C41111E42D24@oracle.com> <9d3e24ee-21fd-de57-3089-969553edfbf9@redhat.com> Message-ID: <0a42b88f-c3aa-81c8-dac9-ceafc881c5fa@oracle.com> On 10/17/2018 07:10 PM, Roman Kennke wrote: > >> Hi Roman, >> >> Looks fantastic. > > Thanks! > >> BTW, out of curiosity, how do you do the heap walk - scanning or tracing? > > Tracing. Scanning would require to carry around complete marking info > (to be able to avoid holes of dead objects with recycled Klass*), which > we don't. So that makes me wonder why you needed JDK-8211955 (GC abstraction for LAB reserve) and why you insert filler objects at all? Am I missing something? cheers, Per > > Roman > >> Thanks, >> /Erik >> >>> On 17 Oct 2018, at 11:48, Roman Kennke wrote: >>> >>> Erik recently pointed out that JvmtiTagHashmapEntry::equals() is >>> lacking on obj-equals-barrier, and that JvmtiTagHashmap::hash() better >>> does ::resolve() the object to a stable value before casting it to a raw >>> address. >>> >>> This lead us to do some more JVMTI testing and we found that we have a >>> little inconsistency with reporting object size too: the VMObjectAlloc >>> callback now reports a different size that GetObjectSize(). This is also >>> fixed by this proposed change: >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8212186 >>> Webrev: >>> http://cr.openjdk.java.net/~rkennke/JDK-8212186/webrev.00/ >>> >>> Testing: hotspot/tier1, nsk/jvmti with Shenandoah >>> >>> Ok? >>> >>> Roman >>> > From shade at redhat.com Thu Oct 18 07:24:13 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 18 Oct 2018 09:24:13 +0200 Subject: RFR: JDK-8212186: JVMTI lacks a few GC barriers/hooks In-Reply-To: <0a42b88f-c3aa-81c8-dac9-ceafc881c5fa@oracle.com> References: <1c409b76-cb66-c6fa-d77f-cca236cd6cc5@redhat.com> <5E1355DC-4F5C-4C23-B02E-C41111E42D24@oracle.com> <9d3e24ee-21fd-de57-3089-969553edfbf9@redhat.com> <0a42b88f-c3aa-81c8-dac9-ceafc881c5fa@oracle.com> Message-ID: <34419a02-23ed-9554-b691-3645b0669e71@redhat.com> On 10/18/2018 09:13 AM, Per Liden wrote: >> Tracing. Scanning would require to carry around complete marking info >> (to be able to avoid holes of dead objects with recycled Klass*), which >> we don't. > > So that makes me wonder why you needed JDK-8211955 (GC abstraction for LAB reserve) and why you > insert filler objects at all? Am I missing something? I think Roman meant that Shenandoah does tracing for _external_ heap walk, e.g. for heap dumps. The internal heap walks, e.g. for evac/update-refs, are still pretty much scanning the self-parsable heap, which requires us doing fillers right. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Thu Oct 18 07:38:40 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 18 Oct 2018 09:38:40 +0200 Subject: RFR: JDK-8212186: JVMTI lacks a few GC barriers/hooks In-Reply-To: <34419a02-23ed-9554-b691-3645b0669e71@redhat.com> References: <1c409b76-cb66-c6fa-d77f-cca236cd6cc5@redhat.com> <5E1355DC-4F5C-4C23-B02E-C41111E42D24@oracle.com> <9d3e24ee-21fd-de57-3089-969553edfbf9@redhat.com> <0a42b88f-c3aa-81c8-dac9-ceafc881c5fa@oracle.com> <34419a02-23ed-9554-b691-3645b0669e71@redhat.com> Message-ID: On 10/18/2018 09:24 AM, Aleksey Shipilev wrote: > On 10/18/2018 09:13 AM, Per Liden wrote: >>> Tracing. Scanning would require to carry around complete marking info >>> (to be able to avoid holes of dead objects with recycled Klass*), which >>> we don't. >> >> So that makes me wonder why you needed JDK-8211955 (GC abstraction for LAB reserve) and why you >> insert filler objects at all? Am I missing something? > > I think Roman meant that Shenandoah does tracing for _external_ heap walk, e.g. for heap dumps. The > internal heap walks, e.g. for evac/update-refs, are still pretty much scanning the self-parsable > heap, which requires us doing fillers right. Don't you have the problem of potentially stepping on an object with a stale klass pointer in that case too? cheers, Per From shade at redhat.com Thu Oct 18 07:46:26 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 18 Oct 2018 09:46:26 +0200 Subject: RFR: JDK-8212186: JVMTI lacks a few GC barriers/hooks In-Reply-To: References: <1c409b76-cb66-c6fa-d77f-cca236cd6cc5@redhat.com> <5E1355DC-4F5C-4C23-B02E-C41111E42D24@oracle.com> <9d3e24ee-21fd-de57-3089-969553edfbf9@redhat.com> <0a42b88f-c3aa-81c8-dac9-ceafc881c5fa@oracle.com> <34419a02-23ed-9554-b691-3645b0669e71@redhat.com> Message-ID: <1f67d40d-873d-fd54-4d3d-98e199c35ac9@redhat.com> On 10/18/2018 09:38 AM, Per Liden wrote: > On 10/18/2018 09:24 AM, Aleksey Shipilev wrote: >> On 10/18/2018 09:13 AM, Per Liden wrote: >>>> Tracing. Scanning would require to carry around complete marking info >>>> (to be able to avoid holes of dead objects with recycled Klass*), which >>>> we don't. >>> >>> So that makes me wonder why you needed JDK-8211955 (GC abstraction for LAB reserve) and why you >>> insert filler objects at all? Am I missing something? >> >> I think Roman meant that Shenandoah does tracing for _external_ heap walk, e.g. for heap dumps. The >> internal heap walks, e.g. for evac/update-refs, are still pretty much scanning the self-parsable >> heap, which requires us doing fillers right. > > Don't you have the problem of potentially stepping on an object with a stale klass pointer in that > case too? No, we do not. Objects below TAMS would be walked using marking bitmap data, which would bypass dead objects, and object above TAMS would be walked as self-parsable heap, but those objects (and their klasses) are guaranteed to be alive. Shenandoah does tracing for external heap walks, on the off chance external walk is requested mid-marking. There are other ways to do this, e.g. wait for marking to complete before doing external walk, which gets us back to heap parsability :) -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Thu Oct 18 08:13:06 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 18 Oct 2018 10:13:06 +0200 Subject: RFR: JDK-8212186: JVMTI lacks a few GC barriers/hooks In-Reply-To: <1f67d40d-873d-fd54-4d3d-98e199c35ac9@redhat.com> References: <1c409b76-cb66-c6fa-d77f-cca236cd6cc5@redhat.com> <5E1355DC-4F5C-4C23-B02E-C41111E42D24@oracle.com> <9d3e24ee-21fd-de57-3089-969553edfbf9@redhat.com> <0a42b88f-c3aa-81c8-dac9-ceafc881c5fa@oracle.com> <34419a02-23ed-9554-b691-3645b0669e71@redhat.com> <1f67d40d-873d-fd54-4d3d-98e199c35ac9@redhat.com> Message-ID: <01550086-0451-07f4-ff44-313fca9c5f79@oracle.com> On 10/18/2018 09:46 AM, Aleksey Shipilev wrote: > On 10/18/2018 09:38 AM, Per Liden wrote: >> On 10/18/2018 09:24 AM, Aleksey Shipilev wrote: >>> On 10/18/2018 09:13 AM, Per Liden wrote: >>>>> Tracing. Scanning would require to carry around complete marking info >>>>> (to be able to avoid holes of dead objects with recycled Klass*), which >>>>> we don't. >>>> >>>> So that makes me wonder why you needed JDK-8211955 (GC abstraction for LAB reserve) and why you >>>> insert filler objects at all? Am I missing something? >>> >>> I think Roman meant that Shenandoah does tracing for _external_ heap walk, e.g. for heap dumps. The >>> internal heap walks, e.g. for evac/update-refs, are still pretty much scanning the self-parsable >>> heap, which requires us doing fillers right. >> >> Don't you have the problem of potentially stepping on an object with a stale klass pointer in that >> case too? > > No, we do not. Objects below TAMS would be walked using marking bitmap data, which would bypass dead > objects, and object above TAMS would be walked as self-parsable heap, but those objects (and their > klasses) are guaranteed to be alive. Ah, sounds good. > > Shenandoah does tracing for external heap walks, on the off chance external walk is requested > mid-marking. There are other ways to do this, e.g. wait for marking to complete before doing > external walk, which gets us back to heap parsability :) We initially did that in ZGC, but found that forcing the GC to be in a specific state before you can do certain operations just wasn't very attractive (more complex code, potentially stalled operations, strange error paths, etc). So, I think you made the right choice ;) cheers, Per From shade at redhat.com Thu Oct 18 08:32:32 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 18 Oct 2018 10:32:32 +0200 Subject: RFR: JDK-8212186: JVMTI lacks a few GC barriers/hooks In-Reply-To: <01550086-0451-07f4-ff44-313fca9c5f79@oracle.com> References: <1c409b76-cb66-c6fa-d77f-cca236cd6cc5@redhat.com> <5E1355DC-4F5C-4C23-B02E-C41111E42D24@oracle.com> <9d3e24ee-21fd-de57-3089-969553edfbf9@redhat.com> <0a42b88f-c3aa-81c8-dac9-ceafc881c5fa@oracle.com> <34419a02-23ed-9554-b691-3645b0669e71@redhat.com> <1f67d40d-873d-fd54-4d3d-98e199c35ac9@redhat.com> <01550086-0451-07f4-ff44-313fca9c5f79@oracle.com> Message-ID: <07147477-ec1d-9165-a016-de562b9a4b07@redhat.com> On 10/18/2018 10:13 AM, Per Liden wrote: > On 10/18/2018 09:46 AM, Aleksey Shipilev wrote: >> Shenandoah does tracing for external heap walks, on the off chance external walk is requested >> mid-marking. There are other ways to do this, e.g. wait for marking to complete before doing >> external walk, which gets us back to heap parsability :) > > We initially did that in ZGC, but found that forcing the GC to be in a specific state before you can > do certain operations just wasn't very attractive (more complex code, potentially stalled > operations, strange error paths, etc). So, I think you made the right choice ;) Roman would be happy to hear that. *I* was insisting to drop the additional tracing code and just wait for marking to be over. :) -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From leo.korinth at oracle.com Thu Oct 18 09:09:27 2018 From: leo.korinth at oracle.com (Leo Korinth) Date: Thu, 18 Oct 2018 11:09:27 +0200 Subject: RFR: 8212595: Remove unused size_helper() in oop_oop_iterate* in instanceKlass.inline.hpp In-Reply-To: References: <4a254b3d-7d19-2025-5fd5-b30ce9abbf8b@oracle.com> Message-ID: <9ce0f786-58e7-cc39-a343-6274b0b5155f@oracle.com> Hi Aleksey. Thank you for the review. /Leo On 17/10/2018 17:27, Aleksey Shipilev wrote: > On 10/17/2018 05:24 PM, Leo Korinth wrote: >> Webrev: >> http://cr.openjdk.java.net/~lkorinth/8212595/00/ > > This looks good to me. > > -Aleksey > From leo.korinth at oracle.com Thu Oct 18 09:10:26 2018 From: leo.korinth at oracle.com (Leo Korinth) Date: Thu, 18 Oct 2018 11:10:26 +0200 Subject: RFR: 8212595: Remove unused size_helper() in oop_oop_iterate* in instanceKlass.inline.hpp In-Reply-To: <0e27c0ca-539a-ad70-b153-589ac4c336f8@oracle.com> References: <4a254b3d-7d19-2025-5fd5-b30ce9abbf8b@oracle.com> <0e27c0ca-539a-ad70-b153-589ac4c336f8@oracle.com> Message-ID: <296c2bea-d0e0-ae29-c887-ba158a621574@oracle.com> Hi Per. Thank you for the review. /Leo On 17/10/2018 22:13, Per Liden wrote: > Looks good. > > /Per > > On 10/17/2018 05:24 PM, Leo Korinth wrote: >> Hi! >> >> This is a small clean-up. >> >> I am removing calls to size_helper() in instanceKlass.inline.hpp. The >> size is never used, and the corresponding methods outside >> InstanceKlass already returns void instead of int. >> >> Fast disassembly inspection of the oop_oop_iterate_reverse version >> reveals that the compiler output remains the same. That is, the >> compiler already understood that the code was unused. No performance >> improvement thus. The code becomes a bit simpler and corresponds >> better to the methods in the other *Klass files. >> >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8212595 >> >> Webrev: >> http://cr.openjdk.java.net/~lkorinth/8212595/00/ >> >> Testing: >> - mach5 tier1-3 >> >> Thanks, Leo From per.liden at oracle.com Thu Oct 18 12:52:58 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 18 Oct 2018 14:52:58 +0200 Subject: RFR: 8212656: Clean up heap filler insertion Message-ID: <1d6ca575-7362-db59-7adc-49c70062d503@oracle.com> While reviewing JDK-8211955, I suggested that the filler insertion code in CollectedHeap should be broken out into a Fill class (to match the somewhat related Copy class we already have). The filler insertion code doesn't have to live in the already crowded CollectedHeap class, as it's basically just a bunch of utility functions (just like the functions in the Copy class). With this patch CollectedHeap calls Fill::initialize() in it's constructor. This will set up the default behavior. Collectors with special needs (G1, ZGC and Shenandoah) can then call one or more of the Fill::set_*() functions to adjust filling behavior. Bug: https://bugs.openjdk.java.net/browse/JDK-8212656 Webrev: http://cr.openjdk.java.net/~pliden/8212656/webrev.0 Testing: Passed tier{3,5,6} on all Oracle supported platforms, running additional tiers right now. /Per From rkennke at redhat.com Thu Oct 18 13:57:57 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 18 Oct 2018 15:57:57 +0200 Subject: RFR: 8212656: Clean up heap filler insertion In-Reply-To: <1d6ca575-7362-db59-7adc-49c70062d503@oracle.com> References: <1d6ca575-7362-db59-7adc-49c70062d503@oracle.com> Message-ID: Hi Per, I'll try to weave that into Shenandoah and see how that goes :-) One thing that I don't see how it could work is how we could do extra initialization for the fwd ptr? We can currently do it by overriding fill_with_dummy_object(..) like this: void ShenandoahHeap::fill_with_dummy_object(HeapWord* start, HeapWord* end, bool zap) { HeapWord* obj = post_allocation_setup(start); // Bumps start by 1 word CollectedHeap::fill_with_object(obj, end); } This seems to have gone with your patch. Roman > While reviewing JDK-8211955, I suggested that the filler insertion code > in CollectedHeap should be broken out into a Fill class (to match the > somewhat related Copy class we already have). The filler insertion code > doesn't have to live in the already crowded CollectedHeap class, as it's > basically just a bunch of utility functions (just like the functions in > the Copy class). > > With this patch CollectedHeap calls Fill::initialize() in it's > constructor. This will set up the default behavior. Collectors with > special needs (G1, ZGC and Shenandoah) can then call one or more of the > Fill::set_*() functions to adjust filling behavior. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8212656 > Webrev: http://cr.openjdk.java.net/~pliden/8212656/webrev.0 > > Testing: Passed tier{3,5,6} on all Oracle supported platforms, running > additional tiers right now. > > /Per -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Thu Oct 18 14:45:06 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 18 Oct 2018 16:45:06 +0200 Subject: RFR: 8212656: Clean up heap filler insertion In-Reply-To: References: <1d6ca575-7362-db59-7adc-49c70062d503@oracle.com> Message-ID: Hi Roman, On 10/18/2018 03:57 PM, Roman Kennke wrote: > Hi Per, > > I'll try to weave that into Shenandoah and see how that goes :-) > > One thing that I don't see how it could work is how we could do extra > initialization for the fwd ptr? We can currently do it by overriding > fill_with_dummy_object(..) like this: > > void ShenandoahHeap::fill_with_dummy_object(HeapWord* start, HeapWord* > end, bool zap) { > HeapWord* obj = post_allocation_setup(start); // Bumps start by 1 word > CollectedHeap::fill_with_object(obj, end); > } > > This seems to have gone with your patch. Ok, so if we do something like this: --- a/src/hotspot/share/gc/shared/fill.cpp +++ b/src/hotspot/share/gc/shared/fill.cpp @@ -76,14 +76,14 @@ void Fill::fill_with_object(HeapWord* start) { const size_t size = object_header_size(); - ObjAllocator allocator(SystemDictionary::Object_klass(), size); - allocator.initialize(start); + ObjAllocator allocator(SystemDictionary::Object_klass(), size - _extra_header_size); + allocator.initialize(start + _extra_header_size); } void Fill::fill_with_array(HeapWord* start, size_t size) { - const size_t length = array_length(size); - ObjArrayAllocator allocator(Universe::intArrayKlassObj(), size, (int)length, false /* do_zero */); - allocator.initialize(start); + const size_t length = array_length(size - _extra_header_size); + ObjArrayAllocator allocator(Universe::intArrayKlassObj(), size - _extra_header_size, (int)length, false /* do_zero */); + allocator.initialize(start + _extra_header_size); if (ZapFillerObjects) { HeapWord* const payload_start = start + array_header_size(); Then the object will be initialized in the correct position. You will not have the forwarding pointer initialized (we only made space for it to keep the layout intact), but I suspect that should not be needed for this purpose. The only reason this object exists is to make the heap parsable, so you should only be reading the klass pointer anyway, right? cheers, Per > > Roman > >> While reviewing JDK-8211955, I suggested that the filler insertion code >> in CollectedHeap should be broken out into a Fill class (to match the >> somewhat related Copy class we already have). The filler insertion code >> doesn't have to live in the already crowded CollectedHeap class, as it's >> basically just a bunch of utility functions (just like the functions in >> the Copy class). >> >> With this patch CollectedHeap calls Fill::initialize() in it's >> constructor. This will set up the default behavior. Collectors with >> special needs (G1, ZGC and Shenandoah) can then call one or more of the >> Fill::set_*() functions to adjust filling behavior. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8212656 >> Webrev: http://cr.openjdk.java.net/~pliden/8212656/webrev.0 >> >> Testing: Passed tier{3,5,6} on all Oracle supported platforms, running >> additional tiers right now. >> >> /Per > From rkennke at redhat.com Thu Oct 18 14:55:24 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 18 Oct 2018 16:55:24 +0200 Subject: RFR: 8212656: Clean up heap filler insertion In-Reply-To: References: <1d6ca575-7362-db59-7adc-49c70062d503@oracle.com> Message-ID: <16e85ae1-bac2-0ed7-c9ad-75dab26e3bf8@redhat.com> Hi Per, > Hi Roman, > > On 10/18/2018 03:57 PM, Roman Kennke wrote: >> Hi Per, >> >> I'll try to weave that into Shenandoah and see how that goes :-) >> >> One thing that I don't see how it could work is how we could do extra >> initialization for the fwd ptr? We can currently do it by overriding >> fill_with_dummy_object(..) like this: >> >> void ShenandoahHeap::fill_with_dummy_object(HeapWord* start, HeapWord* >> end, bool zap) { >> ?? HeapWord* obj = post_allocation_setup(start); // Bumps start by 1 word >> ?? CollectedHeap::fill_with_object(obj, end); >> } >> >> This seems to have gone with your patch. > > Ok, so if we do something like this: > > --- a/src/hotspot/share/gc/shared/fill.cpp > +++ b/src/hotspot/share/gc/shared/fill.cpp > @@ -76,14 +76,14 @@ > > ?void Fill::fill_with_object(HeapWord* start) { > ?? const size_t size = object_header_size(); > -? ObjAllocator allocator(SystemDictionary::Object_klass(), size); > -? allocator.initialize(start); > +? ObjAllocator allocator(SystemDictionary::Object_klass(), size - > _extra_header_size); > +? allocator.initialize(start + _extra_header_size); > ?} > > ?void Fill::fill_with_array(HeapWord* start, size_t size) { > -? const size_t length = array_length(size); > -? ObjArrayAllocator allocator(Universe::intArrayKlassObj(), size, > (int)length, false /* do_zero */); > -? allocator.initialize(start); > +? const size_t length = array_length(size - _extra_header_size); > +? ObjArrayAllocator allocator(Universe::intArrayKlassObj(), size - > _extra_header_size, (int)length, false /* do_zero */); > +? allocator.initialize(start + _extra_header_size); > > ?? if (ZapFillerObjects) { > ???? HeapWord* const payload_start = start + array_header_size(); > > Then the object will be initialized in the correct position. You will > not have the forwarding pointer initialized (we only made space for it > to keep the layout intact), but I suspect that should not be needed for > this purpose. The only reason this object exists is to make the heap > parsable, so you should only be reading the klass pointer anyway, right? Or, instead of hardwiring to ObjAllocator and ArrayAllocator, call CH::obj_allocate() and CH::array_allocate() which will be overridden by our own stuff, and should do the right thing? I'll try this approach in a bit. Thanks, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Thu Oct 18 15:44:11 2018 From: per.liden at oracle.com (=?utf-8?Q?Per_Lid=C3=A9n?=) Date: Thu, 18 Oct 2018 17:44:11 +0200 Subject: RFR: 8212656: Clean up heap filler insertion In-Reply-To: <16e85ae1-bac2-0ed7-c9ad-75dab26e3bf8@redhat.com> References: <1d6ca575-7362-db59-7adc-49c70062d503@oracle.com> <16e85ae1-bac2-0ed7-c9ad-75dab26e3bf8@redhat.com> Message-ID: > On 18 Oct 2018, at 16:55, Roman Kennke wrote: > > Hi Per, > >> Hi Roman, >> >>> On 10/18/2018 03:57 PM, Roman Kennke wrote: >>> Hi Per, >>> >>> I'll try to weave that into Shenandoah and see how that goes :-) >>> >>> One thing that I don't see how it could work is how we could do extra >>> initialization for the fwd ptr? We can currently do it by overriding >>> fill_with_dummy_object(..) like this: >>> >>> void ShenandoahHeap::fill_with_dummy_object(HeapWord* start, HeapWord* >>> end, bool zap) { >>> HeapWord* obj = post_allocation_setup(start); // Bumps start by 1 word >>> CollectedHeap::fill_with_object(obj, end); >>> } >>> >>> This seems to have gone with your patch. >> >> Ok, so if we do something like this: >> >> --- a/src/hotspot/share/gc/shared/fill.cpp >> +++ b/src/hotspot/share/gc/shared/fill.cpp >> @@ -76,14 +76,14 @@ >> >> void Fill::fill_with_object(HeapWord* start) { >> const size_t size = object_header_size(); >> - ObjAllocator allocator(SystemDictionary::Object_klass(), size); >> - allocator.initialize(start); >> + ObjAllocator allocator(SystemDictionary::Object_klass(), size - >> _extra_header_size); >> + allocator.initialize(start + _extra_header_size); >> } >> >> void Fill::fill_with_array(HeapWord* start, size_t size) { >> - const size_t length = array_length(size); >> - ObjArrayAllocator allocator(Universe::intArrayKlassObj(), size, >> (int)length, false /* do_zero */); >> - allocator.initialize(start); >> + const size_t length = array_length(size - _extra_header_size); >> + ObjArrayAllocator allocator(Universe::intArrayKlassObj(), size - >> _extra_header_size, (int)length, false /* do_zero */); >> + allocator.initialize(start + _extra_header_size); >> >> if (ZapFillerObjects) { >> HeapWord* const payload_start = start + array_header_size(); >> >> Then the object will be initialized in the correct position. You will >> not have the forwarding pointer initialized (we only made space for it >> to keep the layout intact), but I suspect that should not be needed for >> this purpose. The only reason this object exists is to make the heap >> parsable, so you should only be reading the klass pointer anyway, right? > > Or, instead of hardwiring to ObjAllocator and ArrayAllocator, call > CH::obj_allocate() and CH::array_allocate() which will be overridden by > our own stuff, and should do the right thing? I'll try this approach in > a bit. I?m afraid that will call allocator.allocate() rather than just allocator.initialize(). /Per > > Thanks, > Roman > From rkennke at redhat.com Thu Oct 18 17:34:32 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 18 Oct 2018 19:34:32 +0200 Subject: RFR: 8212656: Clean up heap filler insertion In-Reply-To: References: <1d6ca575-7362-db59-7adc-49c70062d503@oracle.com> Message-ID: <774d232a-5a33-327a-6122-11314ce7df6f@redhat.com> Hi Per, > Hi Roman, > > On 10/18/2018 03:57 PM, Roman Kennke wrote: >> Hi Per, >> >> I'll try to weave that into Shenandoah and see how that goes :-) >> >> One thing that I don't see how it could work is how we could do extra >> initialization for the fwd ptr? We can currently do it by overriding >> fill_with_dummy_object(..) like this: >> >> void ShenandoahHeap::fill_with_dummy_object(HeapWord* start, HeapWord* >> end, bool zap) { >> ?? HeapWord* obj = post_allocation_setup(start); // Bumps start by 1 word >> ?? CollectedHeap::fill_with_object(obj, end); >> } >> >> This seems to have gone with your patch. > > Ok, so if we do something like this: > > --- a/src/hotspot/share/gc/shared/fill.cpp > +++ b/src/hotspot/share/gc/shared/fill.cpp > @@ -76,14 +76,14 @@ > > ?void Fill::fill_with_object(HeapWord* start) { > ?? const size_t size = object_header_size(); > -? ObjAllocator allocator(SystemDictionary::Object_klass(), size); > -? allocator.initialize(start); > +? ObjAllocator allocator(SystemDictionary::Object_klass(), size - > _extra_header_size); > +? allocator.initialize(start + _extra_header_size); > ?} > > ?void Fill::fill_with_array(HeapWord* start, size_t size) { > -? const size_t length = array_length(size); > -? ObjArrayAllocator allocator(Universe::intArrayKlassObj(), size, > (int)length, false /* do_zero */); > -? allocator.initialize(start); > +? const size_t length = array_length(size - _extra_header_size); > +? ObjArrayAllocator allocator(Universe::intArrayKlassObj(), size - > _extra_header_size, (int)length, false /* do_zero */); > +? allocator.initialize(start + _extra_header_size); > > ?? if (ZapFillerObjects) { > ???? HeapWord* const payload_start = start + array_header_size(); > > Then the object will be initialized in the correct position. You will > not have the forwarding pointer initialized (we only made space for it > to keep the layout intact), but I suspect that should not be needed for > this purpose. The only reason this object exists is to make the heap > parsable, so you should only be reading the klass pointer anyway, right? > With the above changes, I can almost get it to work. I still need: diff --git a/src/hotspot/share/gc/shared/fill.cpp b/src/hotspot/share/gc/shared/fill.cpp --- a/src/hotspot/share/gc/shared/fill.cpp +++ b/src/hotspot/share/gc/shared/fill.cpp @@ -47,7 +47,7 @@ } size_t Fill::array_length(size_t size) { - return (size - array_header_size()) * (HeapWordSize / sizeof(jint)); + return (size - typeArrayOopDesc::header_size(T_INT)) * (HeapWordSize / sizeof(jint)); } void Fill::initialize() { Otherwise the arraylength will be inconsistent. You are right, very strictly speaking the fwd ptr field doesn't matter there. However, it would still be nice to have a way to initialize that. Our verifier would otherwise go crazy if it encounters a bad fwd ptr. :-) Maybe it would be better to design the whole filler stuff with some virtual methods, pretty much as it is now in CollectedHeap? This whole business of _extra_words is already quite Shenandoah specific. We had a similar concept earlier in our Shenandoah repo, where we'd have a CH::extra_oop_words() method that would be used in allocation paths, filler paths, object size paths, etc. But then I preferred to not have this rather specific notion of extra words, plus the implicit idea that the extra words *precede* the actual object in heap layout. This is all implicit Shenandoah stuff. The way we have it now, this is completely opaque in the interfaces and implementation. Can the Fill code be designed like that instead? Roman > cheers, > Per > >> >> Roman >> >>> While reviewing JDK-8211955, I suggested that the filler insertion code >>> in CollectedHeap should be broken out into a Fill class (to match the >>> somewhat related Copy class we already have). The filler insertion code >>> doesn't have to live in the already crowded CollectedHeap class, as it's >>> basically just a bunch of utility functions (just like the functions in >>> the Copy class). >>> >>> With this patch CollectedHeap calls Fill::initialize() in it's >>> constructor. This will set up the default behavior. Collectors with >>> special needs (G1, ZGC and Shenandoah) can then call one or more of the >>> Fill::set_*() functions to adjust filling behavior. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8212656 >>> Webrev: http://cr.openjdk.java.net/~pliden/8212656/webrev.0 >>> >>> Testing: Passed tier{3,5,6} on all Oracle supported platforms, running >>> additional tiers right now. >>> >>> /Per >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From hohensee at amazon.com Thu Oct 18 22:18:35 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Thu, 18 Oct 2018 22:18:35 +0000 Subject: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions In-Reply-To: References: <8C241BB9-2089-491B-A27E-57C4F43FBA9E@amazon.com> Message-ID: <773340F4-C9A2-4904-A634-1CDE95253E88@amazon.com> Thanks for your review, JC. New webrev: http://cr.openjdk.java.net/~phh/8196989/webrev.03/ I updated the copyright date in memoryService.hpp because I forgot to do so in the patch for https://bugs.openjdk.java.net/browse/JDK-8195115. Thomas asked me to fix in it a separate CR, so I?ve reverted it. Ditto the #include fixes in g1FullGCOopClosures.inline.hpp and g1HeapVerifier.cpp. At one point during development, clang complained about the latter, but no longer does. The ?break? on the same line as the ?}? was in the original version, but I?ve moved it. :) The comment is indeed a bit opaque. I changed it to: // Only check heap pools that support a usage threshold. // This is typically only the old generation space // since the other spaces are expected to get filled up. if (p.getType() == MemoryType.HEAP && p.isUsageThresholdSupported()) { // In all collectors except G1, only the old generation supports a // usage threshold. The G1 legacy mode "G1 Old Gen" also does. In // G1 default mode, both the old space ("G1 Old Space": it's not // really a generation in the non-G1 collector sense) and the // humongous space ("G1 Humongous Space"), support a usage threshold. // So, the following condition is true for all non-G1 old generations, // for the G1 legacy old gen, and for the G1 default humongous space. // It is not true for the G1 default old gen. // // We're allocating humongous objects in this test, so the G1 default // mode "G1 Old Space" occupancy doesn't change, because humongous // objects are allocated in the "G1 Humongous Space". If we allowed // the G1 default mode "G1 Old Space", notification would never // happen because no objects are allocated there. if (!p.getName().equals("G1 Old Space")) { Finally, the G1MonitoringScope constructor now does a better job of selecting a memory manager. Paul From: JC Beyler Date: Wednesday, October 17, 2018 at 4:47 PM To: "Hohensee, Paul" Cc: "hotspot-gc-dev at openjdk.java.net" , "serviceability-dev at openjdk.java.net" Subject: Re: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions Hi Paul, I looked at this but it took time for me to "digest" it and I haven't entirely gone through the real GC changes :) My few remarks on the webrev itself are: - http://cr.openjdk.java.net/~phh/8196989/webrev.02/src/hotspot/share/services/memoryService.hpp.udiff.html - There is no need to change the copyright, right? - http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/hotspot/jtreg/gc/TestMemoryMXBeansAndPoolsPresence.java.udiff.html - the break seems to be on the wrong line, no? + } break; - http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/jdk/java/lang/management/MemoryMXBean/LowMemoryTest.java.udiff.html and http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/jdk/java/lang/management/MemoryMXBean/MemoryManagement.java.udiff.html + // In G1, humongous objects are tracked in the old space only in + // legacy monitoring mode. In default mode, G1 tracks humongous + // objects in the humongous space, which latter also supports a + // usage threshold. Since we're allocating humongous objects in + // this test, in default mode the old space doesn't change. For + // this test, we use the old space in legacy mode (it's called + // "G1 Old Gen" and the humongous space in default mode. If we + // used "G1 Old Space" in default mode, notification would never + // happen. -> latter seems ot be the wrong word or something is missing in that sentence -> the parenthesis is never closed (it's called.... is missing a ) somewhere Thanks, Jc On Wed, Oct 17, 2018 at 1:18 PM Hohensee, Paul > wrote: Ping. From: serviceability-dev > on behalf of "Hohensee, Paul" > Date: Thursday, October 11, 2018 at 6:46 PM To: "hotspot-gc-dev at openjdk.java.net" >, "serviceability-dev at openjdk.java.net" > Subject: Re: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions Any takers? :) From: serviceability-dev > on behalf of "Hohensee, Paul" > Date: Monday, October 8, 2018 at 7:50 PM To: "hotspot-gc-dev at openjdk.java.net" >, "serviceability-dev at openjdk.java.net" > Subject: RFR: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions Bug: https://bugs.openjdk.java.net/browse/JDK-8196989 CSR: https://bugs.openjdk.java.net/browse/JDK-8196991 Webrev: http://cr.openjdk.java.net/~phh/8196989/webrev.02/ As requested, I split the jstat counter update off from the MXBean update. This is the MXBean update. The jstat counter RFE is https://bugs.openjdk.java.net/browse/JDK-8210965 and its CSR is https://bugs.openjdk.java.net/browse/JDK-8210966. The MXBean CSR is in draft state, I?d greatly appreciate review and sign-off. It?s been suggested that we add another pool to represent the free region set, but doing so would be incompatible with existing MXBean use invariants for all GCs. These are: 1. The sum of the pools? MemoryUsage.max properties is the total reserved heap size. 2. The sum of the pools? MemoryUsage.committed properties is the total committed size. 3. The sum of the pools? MemoryUsage.used properties is the total size of the memory containing objects, live and dead-and-yet-to-be-collected, as the case might be, plus intentional gaps between them. 4. The total free space is (sum of the max properties ? sum of the used properties). 5. The total uncommitted space is (sum of the max properties ? sum of the committed properties). 6. The total committed free space is (2) ? (3). To keep invariants 1, 2 and 3, the free region pool?s ?max? property should be ?undefined? (i.e., -1). The intuitive, to me, ?used? property value would be the total free space, but that would violate invariant 4 above. Defining the ?committed? property as the total committed free space would violate invariants 2 and 6. The patch passes the submit repo, hotspot tier1, and, separately, the serviceability, jfr, and gc jtreg tests. I?m uncertain how to construct a test that checks for valid MXBean content: the existing tests don?t. Any such test will be fragile due to possible future Hotspot changes that affect the values, and to run-to-run variability. I?ve done by-hand comparisons between the old and new MXBean content using the SwingSet2 demo, including using App CDS, and the numbers look reasonable. The guts of the change are in G1MonitoringSupport::recalculate_sizes, initialize_serviceability, memory_managers, memory_pools, and G1MonitoringScope. I also defined TraceConcMemoryManagerStats to track the concurrent cycle in a way analogous to TraceCMSMemoryManagerStats. The changes to the includes in g1FullGCOopClosures.inline.hpp and g1HeapVerifier.cpp are to satisfy compiler complaints. I changed the 3rd argument to the G1MonitoringScope constructor to a mixed_gc flag, and use accessor methods instead of direct field accesses when accessor methods exist. I believe I?ve minimized the latter. I updated the copyright date to 2018 in memoryService.hpp because I neglected to do so in my previous G1 MXBean patch. Thanks, Paul -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Thu Oct 18 23:09:45 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 18 Oct 2018 19:09:45 -0400 Subject: RFR (XS): 8211853: Avoid additional duplicate work when a reference in the task queue has already been evacuated In-Reply-To: <8f58feb1ec7da68af24d040a1fb126cc2397c842.camel@oracle.com> References: <8f58feb1ec7da68af24d040a1fb126cc2397c842.camel@oracle.com> Message-ID: > On Oct 16, 2018, at 9:59 AM, Thomas Schatzl wrote: > > New webrev at > http://cr.openjdk.java.net/~tschatzl/8211853/webrev.0_to_1 (diff) > http://cr.openjdk.java.net/~tschatzl/8211853/webrev.1 (full) > with the added assert. > > Passed hs-tier1-3 as expected. > > Thanks, > Thomas Looks good. From jcbeyler at google.com Fri Oct 19 02:45:59 2018 From: jcbeyler at google.com (JC Beyler) Date: Thu, 18 Oct 2018 19:45:59 -0700 Subject: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions In-Reply-To: <773340F4-C9A2-4904-A634-1CDE95253E88@amazon.com> References: <8C241BB9-2089-491B-A27E-57C4F43FBA9E@amazon.com> <773340F4-C9A2-4904-A634-1CDE95253E88@amazon.com> Message-ID: Hi Paul, Looks much better to me. The other question I have is about the legacy mode. I understand why, from a tool's perspective, having a legacy mode is practical. By doing it this way, we are ensuring we don't break any tools (or at least they can use a flag to be "unbroken") and give time to migrate. This also provides an easier means to backport this fix to older JDKs because now the legacy mode can be used to not break anything and yet provide a means to migrate to a more sane vision of G1 collector definitions. Should the flag perhaps be automatically put in deprecation and then we can mark it as obsolete for JDK13? That would give a limited time for a flag but again I'm not sure this is really done? Or is the plan to keep the flag for a given number of versions, try out these new pools and ensure they provide what we need? Thanks! Jc On Thu, Oct 18, 2018 at 3:18 PM Hohensee, Paul wrote: > Thanks for your review, JC. New webrev: > http://cr.openjdk.java.net/~phh/8196989/webrev.03/ > > > > I updated the copyright date in memoryService.hpp because I forgot to do > so in the patch for https://bugs.openjdk.java.net/browse/JDK-8195115. > Thomas asked me to fix in it a separate CR, so I?ve reverted it. Ditto the > #include fixes in g1FullGCOopClosures.inline.hpp and g1HeapVerifier.cpp. At > one point during development, clang complained about the latter, but no > longer does. > > > > The ?break? on the same line as the ?}? was in the original version, but > I?ve moved it. :) > > > > The comment is indeed a bit opaque. I changed it to: > > > > // Only check heap pools that support a usage threshold. > > // This is typically only the old generation space > > // since the other spaces are expected to get filled up. > > if (p.getType() == MemoryType.HEAP && > > p.isUsageThresholdSupported()) { > > // In all collectors except G1, only the old generation > supports a > > // usage threshold. The G1 legacy mode "G1 Old Gen" also > does. In > > // G1 default mode, both the old space ("G1 Old Space": > it's not > > // really a generation in the non-G1 collector sense) and > the > > // humongous space ("G1 Humongous Space"), support a usage > threshold. > > // So, the following condition is true for all non-G1 old > generations, > > // for the G1 legacy old gen, and for the G1 default > humongous space. > > // It is not true for the G1 default old gen. > > // > > // We're allocating humongous objects in this test, so the > G1 default > > // mode "G1 Old Space" occupancy doesn't change, because > humongous > > // objects are allocated in the "G1 Humongous Space". If > we allowed > > // the G1 default mode "G1 Old Space", notification would > never > > // happen because no objects are allocated there. > > if (!p.getName().equals("G1 Old Space")) { > > > > Finally, the G1MonitoringScope constructor now does a better job of > selecting a memory manager. > > > > Paul > > > > *From: *JC Beyler > *Date: *Wednesday, October 17, 2018 at 4:47 PM > *To: *"Hohensee, Paul" > *Cc: *"hotspot-gc-dev at openjdk.java.net" , > "serviceability-dev at openjdk.java.net" > > *Subject: *Re: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector > MXBean definitions > > > > Hi Paul, > > > > I looked at this but it took time for me to "digest" it and I haven't > entirely gone through the real GC changes :) > > > > My few remarks on the webrev itself are: > > - > http://cr.openjdk.java.net/~phh/8196989/webrev.02/src/hotspot/share/services/memoryService.hpp.udiff.html > > - There is no need to change the copyright, right? > > > > - > http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/hotspot/jtreg/gc/TestMemoryMXBeansAndPoolsPresence.java.udiff.html > > - the break seems to be on the wrong line, no? > > > > + } break; > > > > > > - > http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/jdk/java/lang/management/MemoryMXBean/LowMemoryTest.java.udiff.html > > and > > > http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/jdk/java/lang/management/MemoryMXBean/MemoryManagement.java.udiff.html > > > > + // In G1, humongous objects are tracked in the > old space only in > > + // legacy monitoring mode. In default mode, G1 > tracks humongous > > + // objects in the humongous space, which latter > also supports a > > + // usage threshold. Since we're allocating > humongous objects in > > + // this test, in default mode the old space > doesn't change. For > > + // this test, we use the old space in legacy mode > (it's called > > + // "G1 Old Gen" and the humongous space in > default mode. If we > > + // used "G1 Old Space" in default mode, > notification would never > > + // happen. > > > > -> latter seems ot be the wrong word or something is missing in that > sentence > > -> the parenthesis is never closed (it's called.... is missing a ) > somewhere > > > > Thanks, > > Jc > > > > On Wed, Oct 17, 2018 at 1:18 PM Hohensee, Paul > wrote: > > Ping. > > > > *From: *serviceability-dev > on behalf of "Hohensee, Paul" > *Date: *Thursday, October 11, 2018 at 6:46 PM > *To: *"hotspot-gc-dev at openjdk.java.net" , > "serviceability-dev at openjdk.java.net" > > *Subject: *Re: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector > MXBean definitions > > > > Any takers? :) > > > > *From: *serviceability-dev > on behalf of "Hohensee, Paul" > *Date: *Monday, October 8, 2018 at 7:50 PM > *To: *"hotspot-gc-dev at openjdk.java.net" , > "serviceability-dev at openjdk.java.net" > > *Subject: *RFR: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector > MXBean definitions > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8196989 > > CSR: https://bugs.openjdk.java.net/browse/JDK-8196991 > > Webrev: http://cr.openjdk.java.net/~phh/8196989/webrev.02/ > > > > As requested, I split the jstat counter update off from the MXBean update. > This is the MXBean update. The jstat counter RFE is > https://bugs.openjdk.java.net/browse/JDK-8210965 and its CSR is > https://bugs.openjdk.java.net/browse/JDK-8210966. > > > > The MXBean CSR is in draft state, I?d greatly appreciate review and > sign-off. > > > > It?s been suggested that we add another pool to represent the free region > set, but doing so would be incompatible with existing MXBean use invariants > for all GCs. These are: > > > > 1. The sum of the pools? MemoryUsage.max properties is the total > reserved heap size. > 2. The sum of the pools? MemoryUsage.committed properties is the total > committed size. > 3. The sum of the pools? MemoryUsage.used properties is the total size > of the memory containing objects, live and dead-and-yet-to-be-collected, as > the case might be, plus intentional gaps between them. > 4. The total free space is (sum of the max properties ? sum of the > used properties). > 5. The total uncommitted space is (sum of the max properties ? sum of > the committed properties). > 6. The total committed free space is (2) ? (3). > > > > To keep invariants 1, 2 and 3, the free region pool?s ?max? property > should be ?undefined? (i.e., -1). The intuitive, to me, ?used? property > value would be the total free space, but that would violate invariant 4 > above. Defining the ?committed? property as the total committed free space > would violate invariants 2 and 6. > > > > The patch passes the submit repo, hotspot tier1, and, separately, the > serviceability, jfr, and gc jtreg tests. I?m uncertain how to construct a > test that checks for valid MXBean content: the existing tests don?t. Any > such test will be fragile due to possible future Hotspot changes that > affect the values, and to run-to-run variability. I?ve done by-hand > comparisons between the old and new MXBean content using the SwingSet2 > demo, including using App CDS, and the numbers look reasonable. > > > > The guts of the change are in G1MonitoringSupport::recalculate_sizes, > initialize_serviceability, memory_managers, memory_pools, and > G1MonitoringScope. I also defined TraceConcMemoryManagerStats to track the > concurrent cycle in a way analogous to TraceCMSMemoryManagerStats. The > changes to the includes in g1FullGCOopClosures.inline.hpp and > g1HeapVerifier.cpp are to satisfy compiler complaints. I changed the 3rd > argument to the G1MonitoringScope constructor to a mixed_gc flag, and use > accessor methods instead of direct field accesses when accessor methods > exist. I believe I?ve minimized the latter. I updated the copyright date to > 2018 in memoryService.hpp because I neglected to do so in my previous G1 > MXBean patch. > > > > Thanks, > > > > Paul > > > > > > > > > -- > > > > Thanks, > > Jc > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Fri Oct 19 13:23:27 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 19 Oct 2018 15:23:27 +0200 Subject: RFR: Backport 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: <9D007BB7-F6FE-4A24-81BC-6629B68DEF70@amazon.com> References: <9D007BB7-F6FE-4A24-81BC-6629B68DEF70@amazon.com> Message-ID: Hi Paul, On Fri, 2018-10-12 at 00:03 +0000, Hohensee, Paul wrote: > Please review a backport to jdk8u. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.05/ > JDK11 patch: http://hg.openjdk.java.net/jdk/jdk/rev/5d3c5af82654 > > The backport is slightly different from the JDK11 patch due to G1 > refactoring, hence my request for new review. I?ll ask for jdk8u > approval once the backport is reviewed. > I backported two jtreg tests from JDK11, which pass. Also, all the > hotspot gc jtreg tests pass as well as they do for jdk8u-dev. I think the backport is good. Others need to decide whether this change is worth backporting. > There was a CSR involved, > https://bugs.openjdk.java.net/browse/JDK-8196719. Does that have to > be re-approved for jdk8u as well, and if so, what?s the process? CC'ed Joe Darcy as I am not sure either. Thanks, Thomas From hohensee at amazon.com Fri Oct 19 16:05:09 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 19 Oct 2018 16:05:09 +0000 Subject: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions In-Reply-To: References: <8C241BB9-2089-491B-A27E-57C4F43FBA9E@amazon.com> <773340F4-C9A2-4904-A634-1CDE95253E88@amazon.com> Message-ID: <6C5B0F49-6829-4793-BF1A-AD310ABC1B2A@amazon.com> If we put the flag into deprecation, I?d like to keep it for a year so people have time to change their monitoring code (one release to change their code, and another to run with their new code), which would be two releases. I don?t know how the deprecation process works either. Note that if/when this gets backported to jdk8u and/or jdk11u, there?s no mechanism there to obsolete a flag. Discovered an issue with the jdk/java/lang/management/MemoryMXBean/CollectionUsageThreshold.java test, new new webrev at http://cr.openjdk.java.net/~phh/8196989/webrev.04/ Paul From: JC Beyler Date: Thursday, October 18, 2018 at 10:47 PM To: "Hohensee, Paul" Cc: "hotspot-gc-dev at openjdk.java.net" , "serviceability-dev at openjdk.java.net" Subject: Re: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions Hi Paul, Looks much better to me. The other question I have is about the legacy mode. I understand why, from a tool's perspective, having a legacy mode is practical. By doing it this way, we are ensuring we don't break any tools (or at least they can use a flag to be "unbroken") and give time to migrate. This also provides an easier means to backport this fix to older JDKs because now the legacy mode can be used to not break anything and yet provide a means to migrate to a more sane vision of G1 collector definitions. Should the flag perhaps be automatically put in deprecation and then we can mark it as obsolete for JDK13? That would give a limited time for a flag but again I'm not sure this is really done? Or is the plan to keep the flag for a given number of versions, try out these new pools and ensure they provide what we need? Thanks! Jc On Thu, Oct 18, 2018 at 3:18 PM Hohensee, Paul > wrote: Thanks for your review, JC. New webrev: http://cr.openjdk.java.net/~phh/8196989/webrev.03/ I updated the copyright date in memoryService.hpp because I forgot to do so in the patch for https://bugs.openjdk.java.net/browse/JDK-8195115. Thomas asked me to fix in it a separate CR, so I?ve reverted it. Ditto the #include fixes in g1FullGCOopClosures.inline.hpp and g1HeapVerifier.cpp. At one point during development, clang complained about the latter, but no longer does. The ?break? on the same line as the ?}? was in the original version, but I?ve moved it. :) The comment is indeed a bit opaque. I changed it to: // Only check heap pools that support a usage threshold. // This is typically only the old generation space // since the other spaces are expected to get filled up. if (p.getType() == MemoryType.HEAP && p.isUsageThresholdSupported()) { // In all collectors except G1, only the old generation supports a // usage threshold. The G1 legacy mode "G1 Old Gen" also does. In // G1 default mode, both the old space ("G1 Old Space": it's not // really a generation in the non-G1 collector sense) and the // humongous space ("G1 Humongous Space"), support a usage threshold. // So, the following condition is true for all non-G1 old generations, // for the G1 legacy old gen, and for the G1 default humongous space. // It is not true for the G1 default old gen. // // We're allocating humongous objects in this test, so the G1 default // mode "G1 Old Space" occupancy doesn't change, because humongous // objects are allocated in the "G1 Humongous Space". If we allowed // the G1 default mode "G1 Old Space", notification would never // happen because no objects are allocated there. if (!p.getName().equals("G1 Old Space")) { Finally, the G1MonitoringScope constructor now does a better job of selecting a memory manager. Paul From: JC Beyler > Date: Wednesday, October 17, 2018 at 4:47 PM To: "Hohensee, Paul" > Cc: "hotspot-gc-dev at openjdk.java.net" >, "serviceability-dev at openjdk.java.net" > Subject: Re: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions Hi Paul, I looked at this but it took time for me to "digest" it and I haven't entirely gone through the real GC changes :) My few remarks on the webrev itself are: - http://cr.openjdk.java.net/~phh/8196989/webrev.02/src/hotspot/share/services/memoryService.hpp.udiff.html - There is no need to change the copyright, right? - http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/hotspot/jtreg/gc/TestMemoryMXBeansAndPoolsPresence.java.udiff.html - the break seems to be on the wrong line, no? + } break; - http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/jdk/java/lang/management/MemoryMXBean/LowMemoryTest.java.udiff.html and http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/jdk/java/lang/management/MemoryMXBean/MemoryManagement.java.udiff.html + // In G1, humongous objects are tracked in the old space only in + // legacy monitoring mode. In default mode, G1 tracks humongous + // objects in the humongous space, which latter also supports a + // usage threshold. Since we're allocating humongous objects in + // this test, in default mode the old space doesn't change. For + // this test, we use the old space in legacy mode (it's called + // "G1 Old Gen" and the humongous space in default mode. If we + // used "G1 Old Space" in default mode, notification would never + // happen. -> latter seems ot be the wrong word or something is missing in that sentence -> the parenthesis is never closed (it's called.... is missing a ) somewhere Thanks, Jc On Wed, Oct 17, 2018 at 1:18 PM Hohensee, Paul > wrote: Ping. From: serviceability-dev > on behalf of "Hohensee, Paul" > Date: Thursday, October 11, 2018 at 6:46 PM To: "hotspot-gc-dev at openjdk.java.net" >, "serviceability-dev at openjdk.java.net" > Subject: Re: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions Any takers? :) From: serviceability-dev > on behalf of "Hohensee, Paul" > Date: Monday, October 8, 2018 at 7:50 PM To: "hotspot-gc-dev at openjdk.java.net" >, "serviceability-dev at openjdk.java.net" > Subject: RFR: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions Bug: https://bugs.openjdk.java.net/browse/JDK-8196989 CSR: https://bugs.openjdk.java.net/browse/JDK-8196991 Webrev: http://cr.openjdk.java.net/~phh/8196989/webrev.02/ As requested, I split the jstat counter update off from the MXBean update. This is the MXBean update. The jstat counter RFE is https://bugs.openjdk.java.net/browse/JDK-8210965 and its CSR is https://bugs.openjdk.java.net/browse/JDK-8210966. The MXBean CSR is in draft state, I?d greatly appreciate review and sign-off. It?s been suggested that we add another pool to represent the free region set, but doing so would be incompatible with existing MXBean use invariants for all GCs. These are: 1. The sum of the pools? MemoryUsage.max properties is the total reserved heap size. 2. The sum of the pools? MemoryUsage.committed properties is the total committed size. 3. The sum of the pools? MemoryUsage.used properties is the total size of the memory containing objects, live and dead-and-yet-to-be-collected, as the case might be, plus intentional gaps between them. 4. The total free space is (sum of the max properties ? sum of the used properties). 5. The total uncommitted space is (sum of the max properties ? sum of the committed properties). 6. The total committed free space is (2) ? (3). To keep invariants 1, 2 and 3, the free region pool?s ?max? property should be ?undefined? (i.e., -1). The intuitive, to me, ?used? property value would be the total free space, but that would violate invariant 4 above. Defining the ?committed? property as the total committed free space would violate invariants 2 and 6. The patch passes the submit repo, hotspot tier1, and, separately, the serviceability, jfr, and gc jtreg tests. I?m uncertain how to construct a test that checks for valid MXBean content: the existing tests don?t. Any such test will be fragile due to possible future Hotspot changes that affect the values, and to run-to-run variability. I?ve done by-hand comparisons between the old and new MXBean content using the SwingSet2 demo, including using App CDS, and the numbers look reasonable. The guts of the change are in G1MonitoringSupport::recalculate_sizes, initialize_serviceability, memory_managers, memory_pools, and G1MonitoringScope. I also defined TraceConcMemoryManagerStats to track the concurrent cycle in a way analogous to TraceCMSMemoryManagerStats. The changes to the includes in g1FullGCOopClosures.inline.hpp and g1HeapVerifier.cpp are to satisfy compiler complaints. I changed the 3rd argument to the G1MonitoringScope constructor to a mixed_gc flag, and use accessor methods instead of direct field accesses when accessor methods exist. I believe I?ve minimized the latter. I updated the copyright date to 2018 in memoryService.hpp because I neglected to do so in my previous G1 MXBean patch. Thanks, Paul -- Thanks, Jc -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Fri Oct 19 21:37:38 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 19 Oct 2018 23:37:38 +0200 Subject: RFR (XS): 8211853: Avoid additional duplicate work when a reference in the task queue has already been evacuated In-Reply-To: References: <8f58feb1ec7da68af24d040a1fb126cc2397c842.camel@oracle.com> Message-ID: Hi Kim, On Thu, 2018-10-18 at 19:09 -0400, Kim Barrett wrote: > > On Oct 16, 2018, at 9:59 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > New webrev at > > http://cr.openjdk.java.net/~tschatzl/8211853/webrev.0_to_1 (diff) > > http://cr.openjdk.java.net/~tschatzl/8211853/webrev.1 (full) > > with the added assert. > > > > Passed hs-tier1-3 as expected. > > > > Thanks, > > Thomas > > Looks good. > thanks for your review. Thomas From dthomson at google.com Fri Oct 19 22:04:49 2018 From: dthomson at google.com (Derek Thomson) Date: Fri, 19 Oct 2018 15:04:49 -0700 Subject: RFR (XS) 8209802: JFR failed to build when certain garbage collectors are disabled Message-ID: Hi all, I saw this bug and thought I could take a stab at it. I tweaked the #ifs to make it work for -g1gc, then decided to keep going and make sure all valid GC exclusions work. It's worth noting that you can't say '-serialgc' without also using '-cmsgc', as CMS relies on serial. Could someone review it please? Webrev: http://cr.openjdk.java.net/~jcbeyler/8209802/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8209802 Thanks, Derek -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Sun Oct 21 22:07:57 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 22 Oct 2018 08:07:57 +1000 Subject: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions In-Reply-To: <6C5B0F49-6829-4793-BF1A-AD310ABC1B2A@amazon.com> References: <8C241BB9-2089-491B-A27E-57C4F43FBA9E@amazon.com> <773340F4-C9A2-4904-A634-1CDE95253E88@amazon.com> <6C5B0F49-6829-4793-BF1A-AD310ABC1B2A@amazon.com> Message-ID: <56a2527c-e882-7868-1d3c-ff85aed15ea5@oracle.com> Hi Paul, On 20/10/2018 2:05 AM, Hohensee, Paul wrote: > If we put the flag into deprecation, I?d like to keep it for a year so > people have time to change their monitoring code (one release to change > their code, and another to run with their new code), which would be two > releases. I don?t know how the deprecation process works either. Note > that if/when this gets backported to jdk8u and/or jdk11u, there?s no > mechanism there to obsolete a flag. First the new flag has to be added to the CSR to get approval to be added. It would be quite strange to add a new flag and deprecate it at the same time - not completely out of the question given its legacy compatibility nature, but still odd. So I'd suggest not initially deprecating it but rather file a new bug and CSR to deprecate in 13, obsolete in 14 and expire in 15. That gives you 12 and 13 where the flag is active - though you'll only get a deprecation warning in 13. The obsoletion in 14 will require new bug, but not CSR. The expiration is normally handled en-masse when we bump the JDK release version. For 8u the deprecation process is more manual. You need to add an explicit check and warning in arguments.cpp, then when you're ready to obsolete it add it to the obsolete flag table and remove the deprecation warning. Cheers, David ----- > Discovered an issue with the > jdk/java/lang/management/MemoryMXBean/CollectionUsageThreshold.java > test, new new webrev at > > http://cr.openjdk.java.net/~phh/8196989/webrev.04/ > > > Paul > > *From: *JC Beyler > *Date: *Thursday, October 18, 2018 at 10:47 PM > *To: *"Hohensee, Paul" > *Cc: *"hotspot-gc-dev at openjdk.java.net" > , "serviceability-dev at openjdk.java.net" > > *Subject: *Re: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector > MXBean definitions > > Hi Paul, > > Looks much better to me. The other question I have is about the legacy > mode. I understand why, from a tool's perspective, having a legacy mode > is practical. By doing it this way, we are ensuring we don't break any > tools (or at least they can use a flag to be "unbroken") and give time > to migrate. This also provides an easier means to backport this fix to > older JDKs because now the legacy mode can be used to not break anything > and yet provide a means to migrate to a more sane vision of G1 collector > definitions. > > Should the flag perhaps be automatically put in deprecation and then we > can mark it as obsolete for JDK13? That would give a limited time for a > flag but again I'm not sure this is really done? > > Or is the plan to keep the flag for a given number of versions, try out > these new pools and ensure they provide what we need? > > Thanks! > > Jc > > On Thu, Oct 18, 2018 at 3:18 PM Hohensee, Paul > wrote: > > Thanks for your review, JC.? New webrev: > http://cr.openjdk.java.net/~phh/8196989/webrev.03/ > > > I updated the copyright date in memoryService.hpp because I forgot > to do so in the patch for > https://bugs.openjdk.java.net/browse/JDK-8195115. Thomas asked me to > fix in it a separate CR, so I?ve reverted it. Ditto the #include > fixes in g1FullGCOopClosures.inline.hpp and g1HeapVerifier.cpp. At > one point during development, clang complained about the latter, but > no longer does. > > The ?break? on the same line as the ?}? was in the original version, > but I?ve moved it. :) > > The comment is indeed a bit opaque. I changed it to: > > ??????? // Only check heap pools that support a usage threshold. > > ??? ????// This is typically only the old generation space > > ??????? // since the other spaces are expected to get filled up. > > ??????? if (p.getType() == MemoryType.HEAP && > > ???????????p.isUsageThresholdSupported()) { > > ???????????????// In all collectors except G1, only the old > generation supports a > > ? ??????????????// usage threshold. The G1 legacy mode "G1 Old Gen" > also does. In > > ? ??????????????// G1 default mode, both the old space ("G1 Old > Space": it's not > > ??? ????????????// really a generation in the non-G1 collector > sense) and the > > ???? ???????????// humongous space ("G1 Humongous Space"), support > a usage threshold. > > ????? ??????????// So, the following condition is true for all > non-G1 old generations, > > ? ??????????????// for the G1 legacy old gen, and for the G1 > default humongous space. > > ???????????????// It is not true for the G1 default old gen. > > ? ??????????????// > > ? ??????????????// We're allocating humongous objects in this test, > so the G1 default > > ?? ?????????????// mode "G1 Old Space" occupancy doesn't change, > because humongous > > ??? ????????????// objects are allocated in the "G1 Humongous > Space". If we allowed > > ???????? ???????// the G1 default mode "G1 Old Space", notification > would never > > ??? ????????????// happen because no objects are allocated there. > > ???????????????if (!p.getName().equals("G1 Old Space")) { > > Finally, the G1MonitoringScope constructor now does a better job of > selecting a memory manager. > > Paul > > *From: *JC Beyler > > *Date: *Wednesday, October 17, 2018 at 4:47 PM > *To: *"Hohensee, Paul" > > *Cc: *"hotspot-gc-dev at openjdk.java.net > " > >, > "serviceability-dev at openjdk.java.net > " > > > *Subject: *Re: 8196989: Revamp G1 JMX MemoryPool and > GarbageCollector MXBean definitions > > Hi Paul, > > I looked at this but it took time for me to "digest" it and I > haven't entirely gone through the real GC changes :) > > My few remarks on the webrev itself are: > > ? ?- > http://cr.openjdk.java.net/~phh/8196989/webrev.02/src/hotspot/share/services/memoryService.hpp.udiff.html > > > ? ? ? - There is no need to change the copyright, right? > > ? - > http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/hotspot/jtreg/gc/TestMemoryMXBeansAndPoolsPresence.java.udiff.html > > > ? ? ?- the break seems to be on the wrong line, no? > > +? ? ? ? ? ? ? ? }? ? ? ? ? ? ? ? break; > > ? ? - > http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/jdk/java/lang/management/MemoryMXBean/LowMemoryTest.java.udiff.html > > > ? ? and > > http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/jdk/java/lang/management/MemoryMXBean/MemoryManagement.java.udiff.html > > > +? ? ? ? ? ? ? ? ? ? ? ? // In G1, humongous objects are tracked in > the old space only in > > +? ? ? ? ? ? ? ? ? ? ? ? // legacy monitoring mode. In default mode, > G1 tracks humongous > > +? ? ? ? ? ? ? ? ? ? ? ? // objects in the humongous space, which > latter also supports a > > +? ? ? ? ? ? ? ? ? ? ? ? // usage threshold. Since we're allocating > humongous objects in > > +? ? ? ? ? ? ? ? ? ? ? ? // this test, in default mode the old space > doesn't change. For > > +? ? ? ? ? ? ? ? ? ? ? ? // this test, we use the old space in > legacy mode (it's called > > +? ? ? ? ? ? ? ? ? ? ? ? // "G1 Old Gen" and the humongous space in > default mode. If we > > +? ? ? ? ? ? ? ? ? ? ? ? // used "G1 Old Space" in default mode, > notification would never > > +? ? ? ? ? ? ? ? ? ? ? ? // happen. > > -> latter seems ot be the wrong word or something is missing in that > sentence > > -> the parenthesis is never closed (it's called.... is missing a ) > somewhere > > Thanks, > > Jc > > On Wed, Oct 17, 2018 at 1:18 PM Hohensee, Paul > wrote: > > Ping. > > *From: *serviceability-dev > > on behalf > of "Hohensee, Paul" > > *Date: *Thursday, October 11, 2018 at 6:46 PM > *To: *"hotspot-gc-dev at openjdk.java.net > " > >, > "serviceability-dev at openjdk.java.net > " > > > *Subject: *Re: 8196989: Revamp G1 JMX MemoryPool and > GarbageCollector MXBean definitions > > Any takers? :) > > *From: *serviceability-dev > > on behalf > of "Hohensee, Paul" > > *Date: *Monday, October 8, 2018 at 7:50 PM > *To: *"hotspot-gc-dev at openjdk.java.net > " > >, > "serviceability-dev at openjdk.java.net > " > > > *Subject: *RFR: 8196989: Revamp G1 JMX MemoryPool and > GarbageCollector MXBean definitions > > Bug: https://bugs.openjdk.java.net/browse/JDK-8196989 > > CSR: https://bugs.openjdk.java.net/browse/JDK-8196991 > > Webrev: http://cr.openjdk.java.net/~phh/8196989/webrev.02/ > > > As requested, I split the jstat counter update off from the > MXBean update. This is the MXBean update. The jstat counter RFE > is https://bugs.openjdk.java.net/browse/JDK-8210965 and its CSR > is https://bugs.openjdk.java.net/browse/JDK-8210966. > > The MXBean CSR is in draft state, I?d greatly appreciate review > and sign-off. > > It?s been suggested that we add another pool to represent the > free region set, but doing so would be incompatible with > existing MXBean use invariants for all GCs. These are: > > 1. The sum of the pools? MemoryUsage.max properties is the > total reserved heap size. > 2. The sum of the pools? MemoryUsage.committed properties is > the total committed size. > 3. The sum of the pools? MemoryUsage.used properties is the > total size of the memory containing objects, live and > dead-and-yet-to-be-collected, as the case might be, plus > intentional gaps between them. > 4. The total free space is (sum of the max properties ? sum of > the used properties). > 5. The total uncommitted space is (sum of the max properties ? > sum of the committed properties). > 6. The total committed free space is (2) ? (3). > > To keep invariants 1, 2 and 3, the free region pool?s ?max? > property should be ?undefined? (i.e., -1). The intuitive, to me, > ?used? property value would be the total free space, but that > would violate invariant 4 above. Defining the ?committed? > property as the total committed free space would violate > invariants 2 and 6. > > The patch passes the submit repo, hotspot tier1, and, > separately, the serviceability, jfr, and gc jtreg tests. I?m > uncertain how to construct a test that checks for valid MXBean > content: the existing tests don?t. Any such test will be fragile > due to possible future Hotspot changes that affect the values, > and to run-to-run variability. I?ve done by-hand comparisons > between the old and new MXBean content using the SwingSet2 demo, > including using App CDS, and the numbers look reasonable. > > The guts of the change are in > G1MonitoringSupport::recalculate_sizes, > initialize_serviceability, memory_managers, memory_pools, and > G1MonitoringScope. I also defined TraceConcMemoryManagerStats to > track the concurrent cycle in a way analogous to > TraceCMSMemoryManagerStats. The changes to the includes in > g1FullGCOopClosures.inline.hpp and g1HeapVerifier.cpp are to > satisfy compiler complaints. I changed the 3^rd argument to the > G1MonitoringScope constructor to a mixed_gc flag, and use > accessor methods instead of direct field accesses when accessor > methods exist. I believe I?ve minimized the latter. I updated > the copyright date to 2018 in memoryService.hpp because I > neglected to do so in my previous G1 MXBean patch. > > Thanks, > > Paul > > > -- > > Thanks, > > Jc > > > -- > > Thanks, > > Jc > From thomas.schatzl at oracle.com Mon Oct 22 09:48:22 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 22 Oct 2018 11:48:22 +0200 Subject: RFR (XS): 8211853: Avoid additional duplicate work when a reference in the task queue has already been evacuated In-Reply-To: References: <8f58feb1ec7da68af24d040a1fb126cc2397c842.camel@oracle.com> Message-ID: <4afdee77518932b8b5f94de8147b6152a70f53c5.camel@oracle.com> Hi Kim, Stefan, On Thu, 2018-10-18 at 19:09 -0400, Kim Barrett wrote: > > On Oct 16, 2018, at 9:59 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > New webrev at > > http://cr.openjdk.java.net/~tschatzl/8211853/webrev.0_to_1 (diff) > > http://cr.openjdk.java.net/~tschatzl/8211853/webrev.1 (full) > > with the added assert. > > > > Passed hs-tier1-3 as expected. > > > > Thanks, > > Thomas > > Looks good. thanks for your reviews. Thomas From thomas.schatzl at oracle.com Mon Oct 22 13:02:13 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 22 Oct 2018 15:02:13 +0200 Subject: RFR (XS): 8212753: Improve oopDesc::forward_to_atomic Message-ID: <137c34c66a1a2f18b203da248e741663b52534b1.camel@oracle.com> Hi all, can I have reviews for this micro-optimization that de-clutters oopDesc::forward_to_atomic used in CMS and G1? Reasons for this change: - the mark is reloaded in the method although all collectors could pass in their current copy (the mark is a volatile member, so it can't be optimized away) - it checks whether the mark is already set (i.e. already forwarded) before doing the CAS, and after doing the CAS. The former is waste of time because forwarding undo-statistics show that the number of failures (i.e. an object has already been forwarded) is extremely rate. - the check after the CAS whether the mark is set is superfluous, because after the CAS, independent of whether it was successful or not, we know that the mark must have been forwarded. There is no particular performance change. Just removes needlessly executed code. (Just before somebody asks: the two asserts in the changed forward_to_atomic() method are conditionalized on CMS because it will CAS in a special value into the mark word that do not fit the rest of the condition to "reserve" it for old gen promotion). CR: https://bugs.openjdk.java.net/browse/JDK-8212753 Webrev: http://cr.openjdk.java.net/~tschatzl/8212753/webrev/ Testing: hs-tier1-3 Thanks, Thomas From martin.doerr at sap.com Mon Oct 22 14:00:05 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 22 Oct 2018 14:00:05 +0000 Subject: RFR (XS): 8212753: Improve oopDesc::forward_to_atomic In-Reply-To: <137c34c66a1a2f18b203da248e741663b52534b1.camel@oracle.com> References: <137c34c66a1a2f18b203da248e741663b52534b1.camel@oracle.com> Message-ID: <9f5c1c44be904d0ba213ff975635d641@sap.com> Hi Thomas, this looks like a nice cleanup. Thanks for improving. Your changes look correct. Best regards, Martin -----Original Message----- From: hotspot-gc-dev On Behalf Of Thomas Schatzl Sent: Montag, 22. Oktober 2018 15:02 To: hotspot-gc-dev at openjdk.java.net Subject: RFR (XS): 8212753: Improve oopDesc::forward_to_atomic Hi all, can I have reviews for this micro-optimization that de-clutters oopDesc::forward_to_atomic used in CMS and G1? Reasons for this change: - the mark is reloaded in the method although all collectors could pass in their current copy (the mark is a volatile member, so it can't be optimized away) - it checks whether the mark is already set (i.e. already forwarded) before doing the CAS, and after doing the CAS. The former is waste of time because forwarding undo-statistics show that the number of failures (i.e. an object has already been forwarded) is extremely rate. - the check after the CAS whether the mark is set is superfluous, because after the CAS, independent of whether it was successful or not, we know that the mark must have been forwarded. There is no particular performance change. Just removes needlessly executed code. (Just before somebody asks: the two asserts in the changed forward_to_atomic() method are conditionalized on CMS because it will CAS in a special value into the mark word that do not fit the rest of the condition to "reserve" it for old gen promotion). CR: https://bugs.openjdk.java.net/browse/JDK-8212753 Webrev: http://cr.openjdk.java.net/~tschatzl/8212753/webrev/ Testing: hs-tier1-3 Thanks, Thomas From thomas.schatzl at oracle.com Mon Oct 22 15:34:56 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 22 Oct 2018 17:34:56 +0200 Subject: RFR (XS): 8212753: Improve oopDesc::forward_to_atomic In-Reply-To: <9f5c1c44be904d0ba213ff975635d641@sap.com> References: <137c34c66a1a2f18b203da248e741663b52534b1.camel@oracle.com> <9f5c1c44be904d0ba213ff975635d641@sap.com> Message-ID: <1b8238aea73cb4f0abf94e4c86ecd15291d92974.camel@oracle.com> Hi Martin, On Mon, 2018-10-22 at 14:00 +0000, Doerr, Martin wrote: > Hi Thomas, > > this looks like a nice cleanup. Thanks for improving. Your changes > look correct. thanks for your review Thomas From igor.ignatyev at oracle.com Mon Oct 22 18:36:17 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 22 Oct 2018 11:36:17 -0700 Subject: RFR(XS) : 8177708 : Convert TestReserveMemorySpecial_test to Gtest Message-ID: http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html > 331 lines changed: 330 ins; 1 del; 0 mod; Hi all, could you please review this small and trivial patch which converts TestReserveMemorySpecial_test tests to Gtest? the old tests[1,2] haven't been removed as they are used by WhiteBox::runMemoryUnitTests they will be removed when the rest tests are converted. webrev: http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8177708 [1] http://hg.openjdk.java.net/jdk/jdk/file/tip/src/hotspot/os/linux/os_linux.cpp#l5969 [2] http://hg.openjdk.java.net/jdk/jdk/file/tip/src/hotspot/os/windows/os_windows.cpp#l5564 Thanks, -- Igor From kim.barrett at oracle.com Tue Oct 23 01:49:23 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 22 Oct 2018 21:49:23 -0400 Subject: RFR (XS): 8212753: Improve oopDesc::forward_to_atomic In-Reply-To: <137c34c66a1a2f18b203da248e741663b52534b1.camel@oracle.com> References: <137c34c66a1a2f18b203da248e741663b52534b1.camel@oracle.com> Message-ID: <8F27E1FA-93E3-4631-AD33-64308EE2C58E@oracle.com> > On Oct 22, 2018, at 9:02 AM, Thomas Schatzl wrote: > > CR: > https://bugs.openjdk.java.net/browse/JDK-8212753 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8212753/webrev/ > Testing: > hs-tier1-3 > > Thanks, > Thomas There?s an additional change to the G1 code here: src/hotspot/share/gc/g1/g1ParScanThreadState.cpp 313 } else if (!obj->is_typeArray()) { which seems unrelated to the described change and unmentioned in the RFR. Is this change intentional here? Is this a measurable optimization? For the CMS-conditionalized asserts in forward_to_atomic, please add a comment that CMS may CAS in a special pseudo-oop value. From thomas.schatzl at oracle.com Tue Oct 23 11:03:42 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 23 Oct 2018 13:03:42 +0200 Subject: RFR (XS): 8212753: Improve oopDesc::forward_to_atomic In-Reply-To: <8F27E1FA-93E3-4631-AD33-64308EE2C58E@oracle.com> References: <137c34c66a1a2f18b203da248e741663b52534b1.camel@oracle.com> <8F27E1FA-93E3-4631-AD33-64308EE2C58E@oracle.com> Message-ID: <5a80a7420221534ae4f05f62355d88a246f13fa7.camel@oracle.com> Hi, thanks for your review. On Mon, 2018-10-22 at 21:49 -0400, Kim Barrett wrote: > > On Oct 22, 2018, at 9:02 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8212753 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8212753/webrev/ > > Testing: > > hs-tier1-3 > > > > Thanks, > > Thomas > > There?s an additional change to the G1 code here: > src/hotspot/share/gc/g1/g1ParScanThreadState.cpp > 313 } else if (!obj->is_typeArray()) { > > which seems unrelated to the described change and unmentioned in the > RFR. > Is this change intentional here? No. > Is this a measurable optimization? On parallel GC and jbb2005 (and probably some specjvm2008 applications as I believe this is the class of applications parallel has been optimized for), it seems to decrease pause time measurably slightly (did some ad-hoc testing right now). For G1 the situation is more fuzzy, as its pause time readings are much more noisy. The change for G1 is very old now, lying around for maybe years, so it slipped in when cutting up some recent improvements.Sorry, wasn't intentional. > For the CMS-conditionalized asserts in forward_to_atomic, please add > a comment that CMS may CAS in a special pseudo-oop value. Done. http://cr.openjdk.java.net/~tschatzl/8212753/webrev.0_to_1 (diff) http://cr.openjdk.java.net/~tschatzl/8212753/webrev.1/ (full) Thanks, Thomas From thomas.schatzl at oracle.com Tue Oct 23 11:43:17 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 23 Oct 2018 13:43:17 +0200 Subject: RFR (XS) 8209802: JFR failed to build when certain garbage collectors are disabled In-Reply-To: References: Message-ID: Hi Derek, On Fri, 2018-10-19 at 15:04 -0700, Derek Thomson wrote: > Hi all, > > I saw this bug and thought I could take a stab at it. I tweaked the > #ifs to make it work for -g1gc, then decided to keep going and make > sure all valid GC exclusions work. > > It's worth noting that you can't say '-serialgc' without also using > '-cmsgc', as CMS relies on serial. > > Could someone review it please? > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8209802/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8209802 - wouldn't it be more useful to tweak the makefiles so that setting -serialgc implies -cmsgc, and conversely, +cmsgc implies +serialgc? Or, for me as fine, if conflicting options are set, return with an error. Because it looks like with this change, with -serialgc +cmsgc you will get a non-functioning CMS anyway. I.e. generate a nonsensical situation. This removes a lot of #ifdef SERIALGC I think. I am actually not sure it is actually possible to compile (and succesfully run any of the "old" gcs) without serial gc at the moment. - the friend declaration in heapRegion.hpp can be removed instead of ifdef'ed. G1 does not use these methods. HeapRegion is still a CompactibleSpace/Space, but it actually should not any more. (And applying the Serial gc full gc to G1 will crash the VM) Anyway, for this change, just remove the friend declaration completely. - in jfr.cpp, I would prefer the ifdef's completely removed the TRACE_REQUEST_FUNC(G1HeapRegionInformation), if possible. Otherwise, I think it is better to remove starting from the body of this method. - in jfrType.cpp the question is if compiling without g1 could remove the entire declaration of G1HeapRegionTypeConstant. Cc'ing hotspot-jfr-dev at openjdk.java.net as the jfr devs might have an idea how to accomplish that. Thanks, Thomas From thomas.schatzl at oracle.com Tue Oct 23 13:37:43 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 23 Oct 2018 15:37:43 +0200 Subject: RFR: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions In-Reply-To: <8C241BB9-2089-491B-A27E-57C4F43FBA9E@amazon.com> References: <8C241BB9-2089-491B-A27E-57C4F43FBA9E@amazon.com> Message-ID: <208c3313a6ce24a97d121abf718b0b9dbf6ab023.camel@oracle.com> Hi Paul, sorry for the long wait. On Mon, 2018-10-08 at 23:49 +0000, Hohensee, Paul wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8196989 > CSR: https://bugs.openjdk.java.net/browse/JDK-8196991 > Webrev: http://cr.openjdk.java.net/~phh/8196989/webrev.02/ Note that I reviewed the latest change at http://cr.openjdk.java.net/~phh/8196989/webrev.04/ Currently pushing this through our testing. At that point the thread itself has moved to a question of a particular detail, with too many indentations to the email text I wanted to actually answer, so I answered to the original email. :) > > As requested, I split the jstat counter update off from the MXBean > update. This is the MXBean update. The jstat counter RFE is > https://bugs.openjdk.java.net/browse/JDK-8210965 and its CSR is > https://bugs.openjdk.java.net/browse/JDK-8210966. > > The MXBean CSR is in draft state, I?d greatly appreciate review and > sign-off. I will look at it. As David already mentioned, it also needs to mention the new flag regardless of whether introducing it as deprecated or not. > It?s been suggested that we add another pool to represent the free > region set, but doing so would be incompatible with existing MXBean > use invariants for all GCs. These are: > > 1) The sum of the pools? MemoryUsage.max properties is the total > reserved heap size. > 2) The sum of the pools? MemoryUsage.committed properties is the > total committed size. > 3) The sum of the pools? MemoryUsage.used properties is the total > size of the memory containing objects, live and dead-and-yet-to-be- > collected, as the case might be, plus intentional gaps between them. > 4) The total free space is (sum of the max properties ? sum of the > used properties). > 5) The total uncommitted space is (sum of the max properties ? sum of > the committed properties). > 6) The total committed free space is (2) ? (3). > I think we discussed this before, but none of these so-called invariants are specified anywhere and artifacts of the particular heap layout and MXBean implementations for Serial GC/Parallel GC and CMS. My understanding is that there are implementations out there that assume just that - but those can just use the "legacy" mxbeans. Currently, the MXBean values for G1 have been hacked to conform to this scheme for "compatibility reasons" (e.g. eden regions max is -1; or just look at the sorry code bending over backwards in recalculate_sizes()). I would prefer to not carry on with this, particularly with the new collectors (ZGC and Shenandoah) in the future in mind, with a potential "young gen" going to have the same issues (not sure about Shenandoah going to have something like this, but I assume so for various reasons). This change would be a good opportunity to do things correctly and return values as the VM sees them (like proper max values for all MXBeans), and avoid another round of having a "UseLegacyMemoryMXBeans2" switch later. Additionally most of these invariants do not even hold for the "old" collectors if you look closely. One simple case is a GC occuring between readings of the MemoryMXBean (with the known workaround retry until you get consisten values instead of calling the right method), and the issue related to JDK-8207200 (which *exactly* did rely on these "invariants", and was broken because of that). I also want to point out that the jstat counters already return heap size as max capacity (which probably most of the MemoryMXBeans would also return if fixed). So overall I honestly do not like the scope of this change, i.e. this half-fixing of G1 MemoryMXBeans for all these reasons (while the change itself is good). Changing the values returned by MemoryMXBeans would not decrease the amount of information or quality you get, but rather improve both, and make sure that future generational enhancements of collectors won't have to bend backwards for broken code that assumes non-"invariants" again. > To keep invariants 1, 2 and 3, the free region pool?s ?max? property > should be ?undefined? (i.e., -1). The intuitive, to me, ?used? > property value would be the total free space, but that would violate > invariant 4 above. Defining the ?committed? property as the total > committed free space would violate invariants 2 and 6. My intuitive take on the values of the Free region space would be: - used = 0, there are never objects in there, it's the *free* space after all. - committed = number of regions committed to Free regions - max = current max heap size - other committed size, as this would then also encompass the so-called "Not Available" Free regions in G1. It would probably be too complicated and not relevant to the users to have an extra MXBean for those. > The patch passes the submit repo, hotspot tier1, and, separately, the > serviceability, jfr, and gc jtreg tests. I?m uncertain how to > construct a test that checks for valid MXBean content: the existing > tests don?t. Any such test will be fragile due to possible future > Hotspot changes that affect the values, and to run-to-run > variability. I?ve done by-hand comparisons between the old and new > MXBean content using the SwingSet2 demo, including using App CDS, and > the numbers look reasonable. > > The guts of the change are in G1MonitoringSupport::recalculate_sizes, > initialize_serviceability, memory_managers, memory_pools, and > G1MonitoringScope. I also defined TraceConcMemoryManagerStats to > track the concurrent cycle in a way analogous to > TraceCMSMemoryManagerStats. The changes to the includes in > g1FullGCOopClosures.inline.hpp and g1HeapVerifier.cpp are to satisfy > compiler complaints. I changed the 3rd argument to the > G1MonitoringScope constructor to a mixed_gc flag, and use accessor > methods instead of direct field accesses when accessor methods exist. > I believe I?ve minimized the latter. I updated the copyright date to > 2018 in memoryService.hpp because I neglected to do so in my previous > G1 MXBean patch. - I am not sure about the reasons for the change in G1CollectedHeap::post_initialize(). Did you experience some problems, or is this some cleanup? - the comment in g1Collected.hpp:1224 is misplaced, if you touch it, please remove. The "surv rate groups" moved to G1Policy long ago. - the two "Record start/end, but take no time" comments in g1ConcurrentMark.cpp seem to not be very useful. The call below tells you that exactly that happens, but not why, adding no information. I would remove them. - the comments at the top of g1MemoryPool.hpp and g1MemoryPool.hpp:66-68 and 95-98 are nice but seem to mostly duplicate the information in the comment in g1MonitoringSupport.hpp. Maybe they could be merged a bit to avoid having too much duplicate information around (and refer to it in g1MemoryPool). - in the G1MonitoringSupport constructor, please avoid specifying multiple member variable initializations in the same line using additional formatting via spaces (I'm fine with additional newlines if you want to group them). That is somewhat uncommon to read. - g1MonitoringSupport.cpp:288: please remove the unused variable old_regions_count. - the comment in g1MonitoringSupport.hpp:277 needs to be made more concise (thanks for fixing up lots of comments in this change!) by adding something like: "Values between different returned MemoryUsage may not be consistent with each other." - memoryManager.cpp:214/248: please add braces around the return statement. Rest of the changes seems good. Thanks, Thomas From manc at google.com Tue Oct 23 21:37:20 2018 From: manc at google.com (Man Cao) Date: Tue, 23 Oct 2018 14:37:20 -0700 Subject: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions In-Reply-To: <56a2527c-e882-7868-1d3c-ff85aed15ea5@oracle.com> References: <8C241BB9-2089-491B-A27E-57C4F43FBA9E@amazon.com> <773340F4-C9A2-4904-A634-1CDE95253E88@amazon.com> <6C5B0F49-6829-4793-BF1A-AD310ABC1B2A@amazon.com> <56a2527c-e882-7868-1d3c-ff85aed15ea5@oracle.com> Message-ID: Hi Paul, I looked at the code in detail, and didn't find any major problem. A few small issues below. I'm not a Reviewer, though. ----------------------------File Separator---------------------------------- In http://cr.openjdk.java.net/~phh/8196989/webrev.04/src/hotspot/share/gc/g1/g1MonitoringSupport.hpp.udiff.html -// - young_gen_used = current young region capacity +// - young_gen_committed = current young region capacity // - survivor_used = survivor_capacity // - eden_used = young_gen_used - survivor_used "young_gen_used" is still referenced several times in comment, although there is no function or field in code to compute young_gen_used. Maybe the following is more accurate? // * Occupancy // // - young_gen_committed = current young region capacity // - survivor_used = survivor_capacity // - eden_used = sum of eden regions allocated // In legacy mode: // - old_used = overall_used - survivor_used - eden_used // Otherwise: // - humongous_used = sum of humongous regions allocated // - archive_used = sum of archive regions allocated // - old_used = overall_used - survivor_used - eden_used - // humongous_used - archive_used ----------------------------File Separator---------------------------------- In http://cr.openjdk.java.net/~phh/8196989/webrev.04/src/hotspot/share/gc/g1/g1CollectedHeap.cpp.udiff.html void G1CollectedHeap::post_initialize() { + // Necessary to satisfy locking discipline assertions. + MutexLockerEx x(Heap_lock); + CollectedHeap::post_initialize(); ref_processing_init(); } I couldn't immediately see which assertion requires the Heap_lock. Is the lock required by the call to G1MonitoringSupport::update_sizes() in G1MonitoringSupport::initialize_serviceability()? Other collectors (Parallel, CMS, etc.) do not seem to hold the Heap_lock for post_initialize() and initialize_serviceability(), either. Should this locking statement be put at a more general place, e.g. universe_post_init() in universe.cpp; or if it is G1-specific, at a place closer to where it is required in G1? ----------------------------File Separator---------------------------------- In http://cr.openjdk.java.net/~phh/8196989/webrev.04/src/hotspot/share/services/memoryManager.cpp.udiff.html void GCMemoryManager::gc_begin(bool recordGCBeginTime, bool recordPreGCUsage, bool recordAccumulatedGCTime) { - assert(_last_gc_stat != NULL && _current_gc_stat != NULL, "Just checking"); + // Inactive memory managers (concurrent in G1 legacy mode) will not be initialized. + if (_last_gc_stat == NULL && _current_gc_stat == NULL) return; + void GCMemoryManager::gc_end(bool recordPostGCUsage, bool recordAccumulatedGCTime, bool recordGCEndTime, bool countCollection, GCCause::Cause cause, bool allMemoryPoolsAffected) { + if (_last_gc_stat == NULL && _current_gc_stat == NULL) return; + Because they are only for handling the special case for g1mm()->conc_memory_manager(), it is probably better not to change memoryManager.cpp, but let TraceConcMemoryManagerStats handle them. I was considering if we can make the calls to gc_begin() and gc_end() conditional on G1UseLegacyMonitoring, but C++ does not allow complete overriding of the base class's destructor ~TraceMemoryManagerStats(), which calls GCMemoryManager::gc_end(). One option is to keep the code as-is, but I recommend adding assertions that the two branches can only be taken when G1UseLegacyMonitoring && this == g1mm()->conc_memory_manager(). The other option is to implement TraceConcMemoryManagerStats independent of TraceMemoryManagerStats, so it can have a destructor that conditionally calls GCMemoryManager::gc_end(). ----------------------------File Separator---------------------------------- In http://cr.openjdk.java.net/~phh/8196989/webrev.04/test/hotspot/jtreg/gc/g1/mixedgc/TestOldGenCollectionUsage.java.udiff.html - if (newCollectionCount <= collectionCount) { + if (useLegacyMonitoring) { + if (newCollectionCount <= youngCollectionCount) { throw new RuntimeException("No new collection"); } - if (newCollectionTime <= collectionTime) { - throw new RuntimeException("Collector has not run some more"); + } else { + if (newCollectionCount <= 0) { + throw new RuntimeException("Mixed collection count <= 0"); + } + if (newCollectionTime <= 0) { + throw new RuntimeException("Mixed collector did not run"); + } } It seems under the useLegacyMonitoring==true branch, the check for "newCollectionTime <= collectionTime" was removed. In addition, I think this test add a check that youngCollector and mixedCollector point to the same instance if useLegacyMonitoring==true. ----------------------------File Separator---------------------------------- In http://cr.openjdk.java.net/~phh/8196989/webrev.04/test/jdk/jdk/jfr/event/gc/stacktrace/AllocationStackTrace.java.udiff.html private static GarbageCollectorMXBean garbageCollectorMXBean(String name) throws Exception { MBeanServer server = ManagementFactory.getPlatformMBeanServer(); - GarbageCollectorMXBean bean = ManagementFactory.newPlatformMXBeanProxy( - server, "java.lang:type=GarbageCollector,name=" + name, GarbageCollectorMXBean.class); + GarbageCollectorMXBean bean; + try { + bean = ManagementFactory.newPlatformMXBeanProxy(server, + "java.lang:type=GarbageCollector,name=" + name, + GarbageCollectorMXBean.class); + } catch (IllegalArgumentException e) { + bean = null; + } + return bean; + } + + private static GarbageCollectorMXBean g1YoungGarbageCollectorMXBean() throws Exception { + GarbageCollectorMXBean bean = garbageCollectorMXBean("G1 Young Generation"); + if (bean == null) { + bean = garbageCollectorMXBean("G1 Young"); + } + return bean; + } + + private static GarbageCollectorMXBean g1FullGarbageCollectorMXBean() throws Exception { + GarbageCollectorMXBean bean = garbageCollectorMXBean("G1 Old Generation"); + if (bean == null) { + bean = garbageCollectorMXBean("G1 Full"); + } return bean; } It is probably better to add the LegacyMonitoring checker class to this file and use LegacyMonitoring.use() to determine the appropriate name of the MXBeans. Catching IllegalArgumentException and retrying with a different name is like using exception for control flow. Thanks, Man On Sun, Oct 21, 2018 at 3:08 PM David Holmes wrote: > > Hi Paul, > > On 20/10/2018 2:05 AM, Hohensee, Paul wrote: > > If we put the flag into deprecation, I?d like to keep it for a year so > > people have time to change their monitoring code (one release to change > > their code, and another to run with their new code), which would be two > > releases. I don?t know how the deprecation process works either. Note > > that if/when this gets backported to jdk8u and/or jdk11u, there?s no > > mechanism there to obsolete a flag. > > First the new flag has to be added to the CSR to get approval to be > added. It would be quite strange to add a new flag and deprecate it at > the same time - not completely out of the question given its legacy > compatibility nature, but still odd. So I'd suggest not initially > deprecating it but rather file a new bug and CSR to deprecate in 13, > obsolete in 14 and expire in 15. That gives you 12 and 13 where the flag > is active - though you'll only get a deprecation warning in 13. The > obsoletion in 14 will require new bug, but not CSR. The expiration is > normally handled en-masse when we bump the JDK release version. > > For 8u the deprecation process is more manual. You need to add an > explicit check and warning in arguments.cpp, then when you're ready to > obsolete it add it to the obsolete flag table and remove the deprecation > warning. > > Cheers, > David > ----- > > > > Discovered an issue with the > > jdk/java/lang/management/MemoryMXBean/CollectionUsageThreshold.java > > test, new new webrev at > > > > http://cr.openjdk.java.net/~phh/8196989/webrev.04/ > > > > > > Paul > > > > *From: *JC Beyler > > *Date: *Thursday, October 18, 2018 at 10:47 PM > > *To: *"Hohensee, Paul" > > *Cc: *"hotspot-gc-dev at openjdk.java.net" > > , "serviceability-dev at openjdk.java.net" > > > > *Subject: *Re: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector > > MXBean definitions > > > > Hi Paul, > > > > Looks much better to me. The other question I have is about the legacy > > mode. I understand why, from a tool's perspective, having a legacy mode > > is practical. By doing it this way, we are ensuring we don't break any > > tools (or at least they can use a flag to be "unbroken") and give time > > to migrate. This also provides an easier means to backport this fix to > > older JDKs because now the legacy mode can be used to not break anything > > and yet provide a means to migrate to a more sane vision of G1 collector > > definitions. > > > > Should the flag perhaps be automatically put in deprecation and then we > > can mark it as obsolete for JDK13? That would give a limited time for a > > flag but again I'm not sure this is really done? > > > > Or is the plan to keep the flag for a given number of versions, try out > > these new pools and ensure they provide what we need? > > > > Thanks! > > > > Jc > > > > On Thu, Oct 18, 2018 at 3:18 PM Hohensee, Paul > > wrote: > > > > Thanks for your review, JC. New webrev: > > http://cr.openjdk.java.net/~phh/8196989/webrev.03/ > > > > > > I updated the copyright date in memoryService.hpp because I forgot > > to do so in the patch for > > https://bugs.openjdk.java.net/browse/JDK-8195115. Thomas asked me to > > fix in it a separate CR, so I?ve reverted it. Ditto the #include > > fixes in g1FullGCOopClosures.inline.hpp and g1HeapVerifier.cpp. At > > one point during development, clang complained about the latter, but > > no longer does. > > > > The ?break? on the same line as the ?}? was in the original version, > > but I?ve moved it. :) > > > > The comment is indeed a bit opaque. I changed it to: > > > > // Only check heap pools that support a usage threshold. > > > > // This is typically only the old generation space > > > > // since the other spaces are expected to get filled up. > > > > if (p.getType() == MemoryType.HEAP && > > > > p.isUsageThresholdSupported()) { > > > > // In all collectors except G1, only the old > > generation supports a > > > > // usage threshold. The G1 legacy mode "G1 Old Gen" > > also does. In > > > > // G1 default mode, both the old space ("G1 Old > > Space": it's not > > > > // really a generation in the non-G1 collector > > sense) and the > > > > // humongous space ("G1 Humongous Space"), support > > a usage threshold. > > > > // So, the following condition is true for all > > non-G1 old generations, > > > > // for the G1 legacy old gen, and for the G1 > > default humongous space. > > > > // It is not true for the G1 default old gen. > > > > // > > > > // We're allocating humongous objects in this test, > > so the G1 default > > > > // mode "G1 Old Space" occupancy doesn't change, > > because humongous > > > > // objects are allocated in the "G1 Humongous > > Space". If we allowed > > > > // the G1 default mode "G1 Old Space", notification > > would never > > > > // happen because no objects are allocated there. > > > > if (!p.getName().equals("G1 Old Space")) { > > > > Finally, the G1MonitoringScope constructor now does a better job of > > selecting a memory manager. > > > > Paul > > > > *From: *JC Beyler > > > *Date: *Wednesday, October 17, 2018 at 4:47 PM > > *To: *"Hohensee, Paul" > > > > *Cc: *"hotspot-gc-dev at openjdk.java.net > > " > > > >, > > "serviceability-dev at openjdk.java.net > > " > > > > > > *Subject: *Re: 8196989: Revamp G1 JMX MemoryPool and > > GarbageCollector MXBean definitions > > > > Hi Paul, > > > > I looked at this but it took time for me to "digest" it and I > > haven't entirely gone through the real GC changes :) > > > > My few remarks on the webrev itself are: > > > > - > > http://cr.openjdk.java.net/~phh/8196989/webrev.02/src/hotspot/share/services/memoryService.hpp.udiff.html > > > > > > - There is no need to change the copyright, right? > > > > - > > http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/hotspot/jtreg/gc/TestMemoryMXBeansAndPoolsPresence.java.udiff.html > > > > > > - the break seems to be on the wrong line, no? > > > > + } break; > > > > - > > http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/jdk/java/lang/management/MemoryMXBean/LowMemoryTest.java.udiff.html > > > > > > and > > > > http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/jdk/java/lang/management/MemoryMXBean/MemoryManagement.java.udiff.html > > > > > > + // In G1, humongous objects are tracked in > > the old space only in > > > > + // legacy monitoring mode. In default mode, > > G1 tracks humongous > > > > + // objects in the humongous space, which > > latter also supports a > > > > + // usage threshold. Since we're allocating > > humongous objects in > > > > + // this test, in default mode the old space > > doesn't change. For > > > > + // this test, we use the old space in > > legacy mode (it's called > > > > + // "G1 Old Gen" and the humongous space in > > default mode. If we > > > > + // used "G1 Old Space" in default mode, > > notification would never > > > > + // happen. > > > > -> latter seems ot be the wrong word or something is missing in that > > sentence > > > > -> the parenthesis is never closed (it's called.... is missing a ) > > somewhere > > > > Thanks, > > > > Jc > > > > On Wed, Oct 17, 2018 at 1:18 PM Hohensee, Paul > > wrote: > > > > Ping. > > > > *From: *serviceability-dev > > > > on behalf > > of "Hohensee, Paul" > > > > *Date: *Thursday, October 11, 2018 at 6:46 PM > > *To: *"hotspot-gc-dev at openjdk.java.net > > " > > > >, > > "serviceability-dev at openjdk.java.net > > " > > > > > > *Subject: *Re: 8196989: Revamp G1 JMX MemoryPool and > > GarbageCollector MXBean definitions > > > > Any takers? :) > > > > *From: *serviceability-dev > > > > on behalf > > of "Hohensee, Paul" > > > > *Date: *Monday, October 8, 2018 at 7:50 PM > > *To: *"hotspot-gc-dev at openjdk.java.net > > " > > > >, > > "serviceability-dev at openjdk.java.net > > " > > > > > > *Subject: *RFR: 8196989: Revamp G1 JMX MemoryPool and > > GarbageCollector MXBean definitions > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8196989 > > > > CSR: https://bugs.openjdk.java.net/browse/JDK-8196991 > > > > Webrev: http://cr.openjdk.java.net/~phh/8196989/webrev.02/ > > > > > > As requested, I split the jstat counter update off from the > > MXBean update. This is the MXBean update. The jstat counter RFE > > is https://bugs.openjdk.java.net/browse/JDK-8210965 and its CSR > > is https://bugs.openjdk.java.net/browse/JDK-8210966. > > > > The MXBean CSR is in draft state, I?d greatly appreciate review > > and sign-off. > > > > It?s been suggested that we add another pool to represent the > > free region set, but doing so would be incompatible with > > existing MXBean use invariants for all GCs. These are: > > > > 1. The sum of the pools? MemoryUsage.max properties is the > > total reserved heap size. > > 2. The sum of the pools? MemoryUsage.committed properties is > > the total committed size. > > 3. The sum of the pools? MemoryUsage.used properties is the > > total size of the memory containing objects, live and > > dead-and-yet-to-be-collected, as the case might be, plus > > intentional gaps between them. > > 4. The total free space is (sum of the max properties ? sum of > > the used properties). > > 5. The total uncommitted space is (sum of the max properties ? > > sum of the committed properties). > > 6. The total committed free space is (2) ? (3). > > > > To keep invariants 1, 2 and 3, the free region pool?s ?max? > > property should be ?undefined? (i.e., -1). The intuitive, to me, > > ?used? property value would be the total free space, but that > > would violate invariant 4 above. Defining the ?committed? > > property as the total committed free space would violate > > invariants 2 and 6. > > > > The patch passes the submit repo, hotspot tier1, and, > > separately, the serviceability, jfr, and gc jtreg tests. I?m > > uncertain how to construct a test that checks for valid MXBean > > content: the existing tests don?t. Any such test will be fragile > > due to possible future Hotspot changes that affect the values, > > and to run-to-run variability. I?ve done by-hand comparisons > > between the old and new MXBean content using the SwingSet2 demo, > > including using App CDS, and the numbers look reasonable. > > > > The guts of the change are in > > G1MonitoringSupport::recalculate_sizes, > > initialize_serviceability, memory_managers, memory_pools, and > > G1MonitoringScope. I also defined TraceConcMemoryManagerStats to > > track the concurrent cycle in a way analogous to > > TraceCMSMemoryManagerStats. The changes to the includes in > > g1FullGCOopClosures.inline.hpp and g1HeapVerifier.cpp are to > > satisfy compiler complaints. I changed the 3^rd argument to the > > G1MonitoringScope constructor to a mixed_gc flag, and use > > accessor methods instead of direct field accesses when accessor > > methods exist. I believe I?ve minimized the latter. I updated > > the copyright date to 2018 in memoryService.hpp because I > > neglected to do so in my previous G1 MXBean patch. > > > > Thanks, > > > > Paul > > > > > > -- > > > > Thanks, > > > > Jc > > > > > > -- > > > > Thanks, > > > > Jc > > From sangheon.kim at oracle.com Tue Oct 23 21:53:34 2018 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Tue, 23 Oct 2018 14:53:34 -0700 Subject: RFR(M): 8211425: Allocation of old generation of java heap on alternate memory devices - G1 GC In-Reply-To: References: Message-ID: <6660face-b5a7-3523-7a66-4e0b0e417149@oracle.com> Hi Kishor, Sorry for replying late. On 10/3/18 7:08 PM, Kharbas, Kishor wrote: > > Hi all, > > Requesting review of the patch for allocating old generation of g1 gc > on alternate memory devices such nv-dimm. > > The design of this implementation is explained on the bug page - > https://bugs.openjdk.java.net/browse/JDK-8211425 > > Please follow the parent issue here : > https://bugs.openjdk.java.net/browse/JDK-8202286 > . > > Webrev: http://cr.openjdk.java.net/~kkharbas/8211425/webrev.00 > > General comments. 1. Please file CSR for newly added options. 2. Please run jtreg tests with and without your patch. 3. Did you test with AppCds as well? You can skip style comments. :) ======================================================== src/hotspot/share/gc/g1/g1CollectedHeap.cpp 1489?? _is_hetero_heap(false), - _is_hetero_heap(AllocateOldGenAt != NULL) at initialization list. 1644???? max_byte_size *= 2; - It would be better to explain why multiple by 2? 1668???? G1RegionToSpaceMapper::create_heap_mapper(g1_rs, 1669????????????????????????????????????????? g1_rs.size(), 1670????????????????????????????????????????? page_size, 1671 HeapRegion::GrainBytes, 1672????????????????????????????????????????? 1, 1673????????????????????????????????????????? mtJavaHeap); - If we change 'create_heap_mapper' to return NULL in case of failing to creating mapping file etc.., we can exit with error code. And add NULL check around line 1674. 1704?? if (is_hetero_heap()) { 1705???? _hrm = new HeapRegionManagerForHeteroHeap((uint)((max_byte_size / 2) / HeapRegion::GrainBytes /*heap size as num of regions*/)); 1706?? } 1707?? else { 1708???? _hrm = new HeapRegionManager(); 1709?? } - If we want to minimize adding the check for hetero heap, these codes can be moved line 1644. ======================================================== src/hotspot/share/gc/g1/g1FullCollector.cpp ?169?? if (_heap->is_hetero_heap()) { ?170???? static_cast (_heap->_hrm)->prepare_for_full_collection_start(); ?186???? static_cast (_heap->_hrm)->prepare_for_full_collection_end(); ?187?? } - Stype. hrm() instead of _hrm? ======================================================== src/hotspot/share/gc/g1/g1Policy.cpp ?222?? // Resize dram regions set if called after full collection end. ?223?? if(_g1h->is_hetero_heap() && (Thread::current()->is_VM_thread() || Heap_lock->owned_by_self())) { ?224???? static_cast (_g1h->hrm())->resize_dram_regions(_young_list_target_length, _g1h->workers()); ?225?? } - Instead of checking vmthread or heap lock, couldn't move somewhere else? e.g. G1Policy::update_young_list_max_and_target_length() would be good fit if your concern is not being called from rset sampling thread. ======================================================== src/hotspot/share/gc/g1/g1RegionToSpaceMapper.cpp ?206?? /* Toggle HeteroHeap ?207?? // We map first part of size Xmx to DRAM. ?208?? ReservedSpace rs_dram = rs.first_part(MaxHeapSize); ?209?? // Second half of reserved memory is mapped to NV-DIMM. ?210?? ReservedSpace rs_nvdimm = rs.last_part(MaxHeapSize);*/ ?236 /* Toggle HeteroHeap ?237?? _start_index_of_nvdimm = (uint)(rs_dram.size() / alloc_granularity); ?238?? _start_index_of_dram = 0; */ ?246?? /* Toggle HeteroHeap ?247?? uint num_nvdimm = end_idx >= _start_index_of_nvdimm ? MIN2((end_idx - _start_index_of_nvdimm + 1), (uint)num_regions) : 0; ?248?? uint num_dram = (uint)num_regions - num_nvdimm;*/ ?257???? /* Toggle HeteroHeap ?258????? _dram_mapper->commit_regions(start_idx, num_dram, pretouch_gang); */ ?266?? /* Toggle HeterHeap ?267?? uint num_nvdimm = end_idx >= _start_index_of_nvdimm ? MIN2((end_idx - _start_index_of_nvdimm + 1), (uint)num_regions) : 0; ?268?? uint num_dram = (uint)num_regions - num_nvdimm;*/ - There are some commentted out blocks, please remove them. ?173 void G1RegionToHeteroSpaceMapper::map_nvdimm_space(ReservedSpace rs) { - Your previous change of adding AllocateHeapAt created nv file inside of ReservedHeapSpace but this-AllocateOldGenAt create inside of mapper. Couldn't old gen also create its file near there? I feel like creating here seems un-natural. - This is just an idea. Instead of adding AllocateOldGenAt, couldn't re-use AllocateHeapAt and add type option? e.g. 'all' or 'old' or 'off'. ?177???? vm_exit_during_initialization( - As the caller 'jint G1CollectedHeap::initialize()' is returning its status, I would prefer to return its status. In addition, it would be better not remapping nvdimm in ctor of G1RegionToHeteroSpaceMapper. i.e. Split initialization part(including map_nvdimm_space(), os::attempt_reserve_memory_at() etc) into other method. And call the newly added method after creating G1RegionToHeteroSpaceMapper. ?178?????? err_msg("Could not create file for Old generation at location %s", AllocateOldGenAt)); ?179?? } - Include "utilities/formatBuffer.hpp" to avoid compile error in Solaris because of 'err_msg()'. ?182?? //char* ret = os::replace_existing_mapping_with_file_mapping(rs.base(), rs.size(), _backing_fd); - Please remove this line. ======================================================== src/hotspot/share/gc/g1/g1RegionToSpaceMapper.hpp ?114?? uint num_committed_dram(); ?115?? uint num_committed_nvdimm(); - Add 'const'? ======================================================== src/hotspot/share/gc/g1/heapRegionManager.hpp ? 74?? protected: ? 75?? G1HeapRegionTable _regions; ? 76?? G1RegionToSpaceMapper* _heap_mapper; ? 77?? private: ? 78?? G1RegionToSpaceMapper* _prev_bitmap_mapper; ? 79?? G1RegionToSpaceMapper* _next_bitmap_mapper; ? 80?? G1RegionToSpaceMapper* _bot_mapper; ? 81?? G1RegionToSpaceMapper* _cardtable_mapper; ? 82?? G1RegionToSpaceMapper* _card_counts_mapper; ? 83 ? 84?? protected: ? 85?? FreeRegionList _free_list; ? 86?? private: ? 87 ? 88?? // Each bit in this bitmap indicates that the corresponding region is available ? 89?? // for allocation. ? 90?? CHeapBitMap _available_map; ? 91 ? 92??? // The number of regions committed in the heap. ? 93?? uint _num_committed; ? 94 ? 95?? // Internal only. The highest heap region +1 we allocated a HeapRegion instance for. ? 96?? uint _allocated_heapregions_length; ? 97 ? 98?? HeapWord* heap_bottom() const { return _regions.bottom_address_mapped(); } ? 99?? HeapWord* heap_end() const {return _regions.end_address_mapped(); } ?100 ?101?? protected: ?102?? void make_regions_available(uint index, uint num_regions = 1, WorkGang* pretouch_gang = NULL); ?103?? void uncommit_regions(uint index, size_t num_regions = 1); ?104?? private: - How about regrouping instead of adding bunch of access specifiers? ?128 #else ?129?? // Returns whether the given region is available for allocation. ?130 protected: ?131?? bool is_available(uint region) const; ?132 #endif - This seems weird. Why is_available(uint) const and 'protected:' have a new condition of !ASSERT? ?208?? // Return maximum number of regions that heap can expand to. ?209?? virtual uint max_expandable_length() const { return (uint)_regions.length(); } - It would be better to explain the difference with max_capacity(). ======================================================== src/hotspot/share/gc/g1/heapRegionType.hpp ? 79???? PreMatureOldTag?????? = PreMatureOldMask, ? 80???? // Archive regions are regions with immutable content (i.e. not reclaimed, and - Please add a new line between 79 and 80. ======================================================== src/hotspot/share/runtime/globals.hpp 2612?? experimental(uintx, G1YoungExpansionBufferPerc, 10,?????????????????????? \ 2613??????????????? "When heterogenous heap is enabled by AllocateOldGenAt "???? \ 2614??????????????? "option, after every GC, yg gen is re-sized which involves " \ 2615??????????????? "system calls to commit/uncommit memory. To reduce these "?? \ 2616??????????????? "calls, we keep a buffer of extra regions to absorb small "? \ 2617??????????????? "changes in yg gen length. This flag takes the buffer "????? \ 2618??????????????? "size as an percentage of young gen length") - Add 'range(0, 100)' - yg gen -> young gen - This G1 specific option should be moved to g1_globals.hpp. ======================================================== src/hotspot/share/gc/g1/heapRegionManagerForHeteroHeap.cpp ? 34 // expand_by() is called to grow the heap. We grow into nvdimm now. ? 35 // Dram regions are committed later as needed during mutator region allocation or ? 36 // when young list target length is determined after gc cycle. ? 37 uint HeapRegionManagerForHeteroHeap::expand_by(uint num_regions, WorkGang* pretouch_workers) { ? 38?? uint num_expanded = expand_nvdimm(MIN2(num_regions, max_expandable_length() - total_regions_committed()), pretouch_workers); ? 39?? assert(total_regions_committed() <= max_expandable_length(), "must be"); ? 40?? return num_expanded; ? 41 } - I guess the only reason of not adjusting dram here is that we don't have information of 'is_old'? Looks a bit weired but I don't have any suggestion. (see more below at allocate_free_region) ? 50?? num_regions = MIN2(num_regions, max_expandable_length() - total_regions_committed()); - Style. I would prefer to have a new variable instead of reusing. ? 68 #ifdef ASSERT ? 69????? uint total_committed_before = total_regions_committed(); ? 70 #endif ? 71???? uint can_be_made_available = shrink_nvdimm(to_be_made_available); - There is one more space at line 69. ?225?? while (cur >= start_idx && (is_available(cur) && at(cur)->is_empty())) { - Redundant parenthesis set before is_available(cur). ?239 HeapRegion* HeapRegionManagerForHeteroHeap::allocate_free_region(bool is_old) { - If is_old=true from G1CollectedHeap::new_region(), your patch allows to expand 2 times for nvdimm. One from this method(new behavior), 1 from G1CollectedHeap::expand(). - If is_old=false in above case, there's no big difference w.r.t expanding POV. But the location of expanding is different. For now, I don't see other than mentioned difference, but would like to hear others' opinions. ======================================================== src/hotspot/share/gc/g1/heapRegionManagerForHeteroHeap.hpp ? 40?? uint _max_regions; - Add 'const'? ?111?? // Override. Expand in nv-dimm. - Style comment. I would prefer adding 'virtual' instead of override comment. There are more if you agree to change. > Testing : Passed tier1_gc and tier1_runtime jtret tests. > > ???????????????? I added extra options > "-XX:+UnlockExperimentalVMOptions -XX:AllocateOldGenAt=/home/Kishor" > to each test. There are some tests which I had to exclude since they > won't work with this flag. Example: tests in ConcMarkSweepGC which > does not support this flag, tests requiring CompressedOops to be > enabled, etc. > As mentioned at the beginning, please test without the new option as well. Thanks, Sangheon > Thanks > > Kishor > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Wed Oct 24 09:57:37 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 24 Oct 2018 11:57:37 +0200 Subject: RFR: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions In-Reply-To: <208c3313a6ce24a97d121abf718b0b9dbf6ab023.camel@oracle.com> References: <8C241BB9-2089-491B-A27E-57C4F43FBA9E@amazon.com> <208c3313a6ce24a97d121abf718b0b9dbf6ab023.camel@oracle.com> Message-ID: <51c03ea06f29f1f22235c1a9ebb95e81820bb180.camel@oracle.com> Hi again, On Tue, 2018-10-23 at 15:37 +0200, Thomas Schatzl wrote: > Hi Paul, > > sorry for the long wait. > > On Mon, 2018-10-08 at 23:49 +0000, Hohensee, Paul wrote: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8196989 > > CSR: https://bugs.openjdk.java.net/browse/JDK-8196991 > > Webrev: http://cr.openjdk.java.net/~phh/8196989/webrev.02/ > > Note that I reviewed the latest change at > > http://cr.openjdk.java.net/~phh/8196989/webrev.04/ > > Currently pushing this through our testing. There are several failures that I think can be directly attributed to this change: jdk tests (run jtreg from test/jdk) com/sun/management/GarbageCollectorMXBean/GarbageCollectionNotification Test.java: "notifications have not been sent for java.lang:name=G1 Mixed, type=GarbageCollector" com/sun/management/GarbageCollectorMXBean/GarbageCollectionNotification ContentTest.java: fails with a NullPointerException I can reproduce these two locally when running without any additional options. They seem to only occur with G1. hotspot tests (run jtreg from test/hotspot) java/lang/management/MemoryMXBean/MemoryTest.java: "Number of heap pools = 3, but expected 5" Note that the test may be called with other collectors than G1 from outside, just not with ZGC as per the @requires tag, i.e. with -vmoption:-XX:+UseParallelGC. Thanks, Thomas From thomas.schatzl at oracle.com Wed Oct 24 10:39:21 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 24 Oct 2018 12:39:21 +0200 Subject: RFR: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions In-Reply-To: <51c03ea06f29f1f22235c1a9ebb95e81820bb180.camel@oracle.com> References: <8C241BB9-2089-491B-A27E-57C4F43FBA9E@amazon.com> <208c3313a6ce24a97d121abf718b0b9dbf6ab023.camel@oracle.com> <51c03ea06f29f1f22235c1a9ebb95e81820bb180.camel@oracle.com> Message-ID: Hi, On Wed, 2018-10-24 at 11:57 +0200, Thomas Schatzl wrote: > Hi again, > > On Tue, 2018-10-23 at 15:37 +0200, Thomas Schatzl wrote: > > Hi Paul, > > > > sorry for the long wait. > > > > On Mon, 2018-10-08 at 23:49 +0000, Hohensee, Paul wrote: > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8196989 > > > CSR: https://bugs.openjdk.java.net/browse/JDK-8196991 > > > Webrev: http://cr.openjdk.java.net/~phh/8196989/webrev.02/ > > > > Note that I reviewed the latest change at > > > > http://cr.openjdk.java.net/~phh/8196989/webrev.04/ > > > > Currently pushing this through our testing. > > There are several failures that I think can be directly attributed to > this change: > > jdk tests (run jtreg from test/jdk) > > com/sun/management/GarbageCollectorMXBean/GarbageCollectionNotificati > onTest.java: "notifications have not been sent for java.lang:name=G1 > Mixed, type=GarbageCollector" > com/sun/management/GarbageCollectorMXBean/GarbageCollectionNotificati > onContentTest.java: fails with a NullPointerException > > I can reproduce these two locally when running without any additional > options. They seem to only occur with G1. > > hotspot tests (run jtreg from test/hotspot) > > java/lang/management/MemoryMXBean/MemoryTest.java: "Number of heap > pools = 3, but expected 5" > > Note that the test may be called with other collectors than G1 from > outside, just not with ZGC as per the @requires tag, i.e. with > -vmoption:-XX:+UseParallelGC. Also vmTestbase/nsk/monitoring/MemoryNotificationInfo/from/from001/TestDescr iption.java from the same location starts to time out with the changes. Thanks, Thomas From thomas.schatzl at oracle.com Wed Oct 24 11:36:48 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 24 Oct 2018 13:36:48 +0200 Subject: RFR (S): 8212911: Unify and micro-optimize handling of non-in-collection set references in oop closures Message-ID: <2cad32398c23732a6a91b430bcafaf9ea2a7c611.camel@oracle.com> Hi all, can I have reviews for this small change that does some minor cleanup and micro-optimization of our main g1 stw/concurrent gc closures? - at the moment G1 randomly either adds a reference directly to the remembered set or puts it into the DCQS. I propose to, at least for closures called during stw to always put the reference into the DCQS and let the concurrent phase (or the next GC phase) deal with that as the current remembered set implementation is pretty bad. We may or may not get to fixing this for 12, but for now it seems to be better to do so. (No significant benchmark differences except for one 3% improvement of specjvm2008-xml, but overall it seems to make no difference) - enqueuing references always checks whether the from-region is in young to skip them. For that purpose, the closures get passed the from- region. Now the problem with that is that actually we do not need the from-region, but the result that the from-region is a young region (then we do not need to push that reference onto the DCQS), so the code calculates that result, i.e. from.is_young(), for every such reference. However at the caller of oop_iterate(), we often already know whether the from region is young (from other calculations), or specifically determine the from-region just for the iteration. The idea here is to pass from->is_young() directly, so we do not need to recalculate it for every reference. Now for the G1ScanEvacuatedObjClosure one could imagine to specialize the closure itself based on the from-region (e.g. having both a G1ScanEvacuatedObjFromYoungClosure and G1ScanEvacuatedObjFromOldClosure), however that results in code duplication at the caller (at least without splitting the task queue) the need for exactly that distinction there as well. Further, measurements did not yield a difference, so I kept the code a little more compact for now (there is an RFE out for this). We may re-evalute the latter as soon as Scan RS time variance has been improved a bit (for which this change helps a bit, and I have some changes in the pipeline. Actually this change has been broken out from these). - unify on using HeapRegion::is_in_same_region() instead of comparing HeapRegion* to avoid the table lookup to get the to-HeapRegion* as the memory load is presumably slower than some arithmetic on two local values. But the main driver for that change was that it allows just passing the is_young() result directly. CR: https://bugs.openjdk.java.net/browse/JDK-8212911 Webrev: http://cr.openjdk.java.net/~tschatzl/8212911/webrev/ Testing: hs-tier1-3, several hs-tier1-5 runs with other changes, benchmark suite etc. Thanks, Thomas From kim.barrett at oracle.com Wed Oct 24 13:01:03 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 24 Oct 2018 09:01:03 -0400 Subject: RFR (XS): 8212753: Improve oopDesc::forward_to_atomic In-Reply-To: <5a80a7420221534ae4f05f62355d88a246f13fa7.camel@oracle.com> References: <137c34c66a1a2f18b203da248e741663b52534b1.camel@oracle.com> <8F27E1FA-93E3-4631-AD33-64308EE2C58E@oracle.com> <5a80a7420221534ae4f05f62355d88a246f13fa7.camel@oracle.com> Message-ID: > On Oct 23, 2018, at 7:03 AM, Thomas Schatzl wrote: > > http://cr.openjdk.java.net/~tschatzl/8212753/webrev.0_to_1 (diff) > http://cr.openjdk.java.net/~tschatzl/8212753/webrev.1/ (full) > > Thanks, > Thomas Looks good. From thomas.schatzl at oracle.com Wed Oct 24 13:49:01 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 24 Oct 2018 15:49:01 +0200 Subject: RFR (XS): 8212766: TestPromotionEventWithG1.java failed due to "RuntimeException: PLAB size is smaller than object size." Message-ID: Hi, can I have reviews for this change that fixes a bug introduced in "JDK-8210492 PLAB object promotion events report object sizes in words"? In JDK-8210492, for some JFR event, the object size has been changed to be passed in bytes. However the JFR event gets passed another size, the PLAB size, which has been forgotten to be fixed accordingly. So it happens that PLAB size (in words) is compared to object size (in bytes) which fails if PLAB size happens accidentally to be smaller than object size. This change fixes that. CR: https://bugs.openjdk.java.net/browse/JDK-8212766 Webrev: http://cr.openjdk.java.net/~tschatzl/8212766/ Testing: Test fails with -XX:MinTLABSize=576 -XX:TLABSize=576 without the patch, passes otherwise. Thanks, Thomas From shade at redhat.com Wed Oct 24 13:55:21 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 24 Oct 2018 15:55:21 +0200 Subject: RFR (XS): 8212766: TestPromotionEventWithG1.java failed due to "RuntimeException: PLAB size is smaller than object size." In-Reply-To: References: Message-ID: On 10/24/2018 03:49 PM, Thomas Schatzl wrote: > CR: > https://bugs.openjdk.java.net/browse/JDK-8212766 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8212766/ The actual code change is commented out? 206 alloc_buf->word_sz() /* * HeapWordSize */); It looks okay with that code uncommented :) -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Wed Oct 24 13:57:37 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 24 Oct 2018 15:57:37 +0200 Subject: RFR (XS): 8212766: TestPromotionEventWithG1.java failed due to "RuntimeException: PLAB size is smaller than object size." In-Reply-To: References: Message-ID: <7f5b6ab9e078a69e6e1be61ea8d34878e07209b8.camel@oracle.com> Hi, On Wed, 2018-10-24 at 15:55 +0200, Aleksey Shipilev wrote: > On 10/24/2018 03:49 PM, Thomas Schatzl wrote: > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8212766 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8212766/webrev > > The actual code change is commented out? > > 206 alloc_buf->word_sz() /* * HeapWordSize */); > > It looks okay with that code uncommented :) fixed. Last minute testing just to make sure... :( Thanks for the review. Thomas From thomas.schatzl at oracle.com Wed Oct 24 14:00:50 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 24 Oct 2018 16:00:50 +0200 Subject: RFR (XS): 8212753: Improve oopDesc::forward_to_atomic In-Reply-To: References: <137c34c66a1a2f18b203da248e741663b52534b1.camel@oracle.com> <8F27E1FA-93E3-4631-AD33-64308EE2C58E@oracle.com> <5a80a7420221534ae4f05f62355d88a246f13fa7.camel@oracle.com> Message-ID: Hi Kim, On Wed, 2018-10-24 at 09:01 -0400, Kim Barrett wrote: > > On Oct 23, 2018, at 7:03 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > http://cr.openjdk.java.net/~tschatzl/8212753/webrev.0_to_1 (diff) > > http://cr.openjdk.java.net/~tschatzl/8212753/webrev.1/ (full) > > > > Thanks, > > Thomas > > Looks good. thanks for your review. Thomas From stefan.johansson at oracle.com Thu Oct 25 08:04:11 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 25 Oct 2018 10:04:11 +0200 Subject: RFR (XS): 8212766: TestPromotionEventWithG1.java failed due to "RuntimeException: PLAB size is smaller than object size." In-Reply-To: <7f5b6ab9e078a69e6e1be61ea8d34878e07209b8.camel@oracle.com> References: <7f5b6ab9e078a69e6e1be61ea8d34878e07209b8.camel@oracle.com> Message-ID: <5d4f2b66-6a90-1830-1377-7f3579c2e0e6@oracle.com> On 2018-10-24 15:57, Thomas Schatzl wrote: > Hi, > > On Wed, 2018-10-24 at 15:55 +0200, Aleksey Shipilev wrote: >> On 10/24/2018 03:49 PM, Thomas Schatzl wrote: >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8212766 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8212766/webrev >> >> The actual code change is commented out? >> >> 206 alloc_buf->word_sz() /* * HeapWordSize */); >> >> It looks okay with that code uncommented :) > > fixed. Last minute testing just to make sure... :( > Looks good! StefanJ > Thanks for the review. > > Thomas > From thomas.schatzl at oracle.com Thu Oct 25 09:42:35 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 25 Oct 2018 11:42:35 +0200 Subject: RFR (XS): 8212766: TestPromotionEventWithG1.java failed due to "RuntimeException: PLAB size is smaller than object size." In-Reply-To: <5d4f2b66-6a90-1830-1377-7f3579c2e0e6@oracle.com> References: <7f5b6ab9e078a69e6e1be61ea8d34878e07209b8.camel@oracle.com> <5d4f2b66-6a90-1830-1377-7f3579c2e0e6@oracle.com> Message-ID: <525db9b1795efde075eb221b9e077f0ab769bc5d.camel@oracle.com> Hi Aleksey, Stefan, On Thu, 2018-10-25 at 10:04 +0200, Stefan Johansson wrote: > > On 2018-10-24 15:57, Thomas Schatzl wrote: > > Hi, > > > > On Wed, 2018-10-24 at 15:55 +0200, Aleksey Shipilev wrote: > > > On 10/24/2018 03:49 PM, Thomas Schatzl wrote: > > > > CR: > > > > https://bugs.openjdk.java.net/browse/JDK-8212766 > > > > Webrev: > > > > http://cr.openjdk.java.net/~tschatzl/8212766/webrev > > > > > > The actual code change is commented out? > > > > > > 206 alloc_buf->word_sz() /* * HeapWordSize */); > > > > > > It looks okay with that code uncommented :) > > > > fixed. Last minute testing just to make sure... :( > > > > Looks good! > StefanJ Thanks for the reviews. Thomas From thomas.schatzl at oracle.com Thu Oct 25 09:45:25 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 25 Oct 2018 11:45:25 +0200 Subject: RFR (S): 8212911: Unify and micro-optimize handling of non-in-collection set references in oop closures In-Reply-To: <2cad32398c23732a6a91b430bcafaf9ea2a7c611.camel@oracle.com> References: <2cad32398c23732a6a91b430bcafaf9ea2a7c611.camel@oracle.com> Message-ID: <646a59f02646ff542659ed3f6fb16dc9df1462e9.camel@oracle.com> Hi all, On Wed, 2018-10-24 at 13:36 +0200, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this small change that does some minor > cleanup and micro-optimization of our main g1 stw/concurrent gc > closures? > > [...] > CR: > https://bugs.openjdk.java.net/browse/JDK-8212911 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8212911/webrev/ > Testing: > hs-tier1-3, several hs-tier1-5 runs with other changes, benchmark > suite etc. Stefan pointed out that _from_in_young is only used in G1ScanEvacuatedObjClosure any more, so it could be moved from G1ScanClosureBase. Also we brainstormed a bit about the name of _from_in_young, and agreed on changing this to _scanning_in_young. New webrevs: http://cr.openjdk.java.net/~tschatzl/8212911/webrev.1 (full) http://cr.openjdk.java.net/~tschatzl/8212911/webrev.0_to_1 (diff) Thanks, Thomas From stefan.johansson at oracle.com Thu Oct 25 09:44:54 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 25 Oct 2018 11:44:54 +0200 Subject: RFR (S): 8212911: Unify and micro-optimize handling of non-in-collection set references in oop closures In-Reply-To: <646a59f02646ff542659ed3f6fb16dc9df1462e9.camel@oracle.com> References: <2cad32398c23732a6a91b430bcafaf9ea2a7c611.camel@oracle.com> <646a59f02646ff542659ed3f6fb16dc9df1462e9.camel@oracle.com> Message-ID: <54091a90-a889-6e76-f913-39fbee690156@oracle.com> On 2018-10-25 11:45, Thomas Schatzl wrote: > Hi all, > > On Wed, 2018-10-24 at 13:36 +0200, Thomas Schatzl wrote: >> Hi all, >> >> can I have reviews for this small change that does some minor >> cleanup and micro-optimization of our main g1 stw/concurrent gc >> closures? >> >> [...] >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8212911 >> Webrev: >> http://cr.openjdk.java.net/~tschatzl/8212911/webrev/ >> Testing: >> hs-tier1-3, several hs-tier1-5 runs with other changes, benchmark >> suite etc. > > Stefan pointed out that _from_in_young is only used in > G1ScanEvacuatedObjClosure any more, so it could be moved from > G1ScanClosureBase. > > Also we brainstormed a bit about the name of _from_in_young, and agreed > on changing this to _scanning_in_young. > > New webrevs: > http://cr.openjdk.java.net/~tschatzl/8212911/webrev.1 (full) > http://cr.openjdk.java.net/~tschatzl/8212911/webrev.0_to_1 (diff) > Look great! =) StefanJ > Thanks, > Thomas > > From erik.osterlund at oracle.com Thu Oct 25 13:18:02 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 25 Oct 2018 15:18:02 +0200 Subject: RFR: 8212921: Relax code cache oop verification in ZGC Message-ID: <5BD1C28A.8050804@oracle.com> Hi, With concurrent class unloading, the normal state of affairs for ZGC is that nmethods have messed up oops in all GC safepoints, and fix them up during concurrent phases. Therefore, Universe::verify needs to relax the assumption that oops are okay for ZGC. Note that during Universe::verify, the oops in the nmethods were redundantly verified. Once by walking the oops in the code cache, and once when calling nmethod::verify(), which did the same verification again. I removed the latter. Webrev: http://cr.openjdk.java.net/~eosterlund/8212921/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8212921 Thanks, /Erik From erik.osterlund at oracle.com Thu Oct 25 13:44:34 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 25 Oct 2018 15:44:34 +0200 Subject: RFR: 8212748: ZGC: Add reentrant locking functionality Message-ID: <5BD1C8C2.5060103@oracle.com> Hi, ZGC needs a per-nmethod lock to be used for concurrent IC cleaning, protecting misaligned oops from concurrently being patched by nmethod entry barriers, and read using CompiledMethod::is_unloading(), with interactions crossing JavaThreads and non-Java threads. This patch adds that utility. Webrev: http://cr.openjdk.java.net/~eosterlund/8212748/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8212748 Thanks, /Erik From per.liden at oracle.com Thu Oct 25 15:27:23 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 25 Oct 2018 17:27:23 +0200 Subject: RFR: 8212748: ZGC: Add reentrant locking functionality In-Reply-To: <5BD1C8C2.5060103@oracle.com> References: <5BD1C8C2.5060103@oracle.com> Message-ID: Hi Erik, On 2018-10-25 15:44, Erik ?sterlund wrote: > Hi, > > ZGC needs a per-nmethod lock to be used for concurrent IC cleaning, > protecting misaligned oops from concurrently being patched by nmethod > entry barriers, and read using CompiledMethod::is_unloading(), with > interactions crossing JavaThreads and non-Java threads. This patch adds > that utility. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8212748/webrev.00/ I'd like to suggest that ZReentrantLock doens't inherit from ZLock, but that it instead has a ZLock. And that ZLocker is adjusted to take a T* instead of a ZLock*. Also, I'm not sure I see the need for the reentrant_lock() function. Shouldn't we just have a lock counter and let lock/unlock do the right thing? cheers, Per > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212748 > > Thanks, > /Erik From thomas.schatzl at oracle.com Thu Oct 25 15:35:20 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 25 Oct 2018 17:35:20 +0200 Subject: RFR (T): 8212974: Update RS Skipped cards uses wrong enum to register to phase Message-ID: Hi all, can I have a review for this trivial change that fixes a benign typo? During initialization of the phase time records the code uses the wrong enum to register the "Skipped Cards" to the "Update RS" logging structure. I.e. _gc_par_phases[UpdateRS]- >link_thread_work_items(_update_rs_skipped_cards, ScanRSSkippedCards); which should be _gc_par_phases[UpdateRS]- >link_thread_work_items(_update_rs_skipped_cards, UpdateRSSkippedCards); Since the value of ScanRSSkippedCards equals UpdateRSSkippedCards there is no harm. CR: https://bugs.openjdk.java.net/browse/JDK-8212974 Webrev: http://cr.openjdk.java.net/~tschatzl/8212974/webrev/ Testing: local compilation; there is no way to create a test for that for above reasons. Thanks, Thomas From erik.osterlund at oracle.com Thu Oct 25 16:20:06 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 25 Oct 2018 18:20:06 +0200 Subject: RFR: 8212748: ZGC: Add reentrant locking functionality In-Reply-To: References: <5BD1C8C2.5060103@oracle.com> Message-ID: <2c455f90-d5e1-f4fd-8d89-7ab25cd38f43@oracle.com> Hi Per, Thanks for having a look at this. On 2018-10-25 17:27, Per Liden wrote: > Hi Erik, > > On 2018-10-25 15:44, Erik ?sterlund wrote: >> Hi, >> >> ZGC needs a per-nmethod lock to be used for concurrent IC cleaning, >> protecting misaligned oops from concurrently being patched by nmethod >> entry barriers, and read using CompiledMethod::is_unloading(), with >> interactions crossing JavaThreads and non-Java threads. This patch >> adds that utility. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8212748/webrev.00/ > > I'd like to suggest that ZReentrantLock doens't inherit from ZLock, but > that it instead has a ZLock. And that ZLocker is adjusted to take a T* > instead of a ZLock*. Wouldn't it be annoying that you can't use a plain ZLocker then, but instead would have to use ZLocker? I'm not sure I see the advantage of that approach, only a usability disadvantage. I'm not opposed either though if you prefer it to be done in that way. > Also, I'm not sure I see the need for the reentrant_lock() function. > Shouldn't we just have a lock counter and let lock/unlock do the right > thing? That is a possibility indeed. However, there are contexts where the lock is not (and must not be) taken in a reentrant way, and it is an important assumption that it is in fact not reentrant, and I want it to not work if that assumption is wrong. And I wanted that to be the default and be the behaviour when using the ZLocker. Plus given that motivation, I also don't know if I want the extra counter state that seems unnecessary for the contexts where reentrancy is handled. Again, I'm not opposed to what you are suggesting, just trying to weigh the pros and cons. What do you think? Thanks /Erik > cheers, > Per > >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8212748 >> >> Thanks, >> /Erik From kim.barrett at oracle.com Fri Oct 26 00:00:15 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 25 Oct 2018 20:00:15 -0400 Subject: RFR: 8212748: ZGC: Add reentrant locking functionality In-Reply-To: <2c455f90-d5e1-f4fd-8d89-7ab25cd38f43@oracle.com> References: <5BD1C8C2.5060103@oracle.com> <2c455f90-d5e1-f4fd-8d89-7ab25cd38f43@oracle.com> Message-ID: <08A81EFE-B2B3-46B6-BE93-010E82DB69F4@oracle.com> > On Oct 25, 2018, at 12:20 PM, Erik ?sterlund wrote: > > Hi Per, > > Thanks for having a look at this. > > On 2018-10-25 17:27, Per Liden wrote: >> Hi Erik, >> On 2018-10-25 15:44, Erik ?sterlund wrote: >>> Hi, >>> >>> ZGC needs a per-nmethod lock to be used for concurrent IC cleaning, protecting misaligned oops from concurrently being patched by nmethod entry barriers, and read using CompiledMethod::is_unloading(), with interactions crossing JavaThreads and non-Java threads. This patch adds that utility. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8212748/webrev.00/ >> I'd like to suggest that ZReentrantLock doens't inherit from ZLock, but that it instead has a ZLock. And that ZLocker is adjusted to take a T* instead of a ZLock*. > > Wouldn't it be annoying that you can't use a plain ZLocker then, but instead would have to use ZLocker? I'm not sure I see the advantage of that approach, only a usability disadvantage. I'm not opposed either though if you prefer it to be done in that way. For what its worth, that?s exactly how the similar C++11 facility works, e.g. one says std::lock_guard lk(m); or std::lock_guard lk(rm); From stefan.johansson at oracle.com Fri Oct 26 14:32:02 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Fri, 26 Oct 2018 16:32:02 +0200 Subject: RFR(M): 8211425: Allocation of old generation of java heap on alternate memory devices - G1 GC In-Reply-To: References: Message-ID: <2ce910ad-69d3-01dd-c984-d1c169a88004@oracle.com> Hi Kishor, I'll start with a few general observations and then there are some specific code comments below. I think the general design looks promising, not having to set a fixed limit for what can end up on NV-dimm is great. Having a separate HeapRegionManager looks like a good abstraction and when adding larger features like this that is really important. I also agree with Sangheon that you should do more testing, both with and without the featured turned on. I also recommend you to build with pre-compiled headers disabled, to find places where includes have been forgotten. To configure such build add --disable-precompiled-headers to your configure call. On 2018-10-04 04:08, Kharbas, Kishor wrote: > Hi all, > > Requesting review of the patch for allocating old generation of g1 gc on > alternate memory devices such nv-dimm. > > The design of this implementation is explained on the bug page - > https://bugs.openjdk.java.net/browse/JDK-8211425 > > Please follow the parent issue here : > https://bugs.openjdk.java.net/browse/JDK-8202286. > > Webrev: http://cr.openjdk.java.net/~kkharbas/8211425/webrev.00 > src/hotspot/share/gc/g1/g1Allocator.inline.hpp ---------------------------------------------- 100 size_t length = static_cast (Universe::heap())->max_reserved_capacity(); You can avoid the cast by using G1CollectedHeap::heap() instead of Universe::heap(). --- src/hotspot/share/gc/g1/g1CollectedHeap.cpp ------------------------------------------- 1642 if (AllocateOldGenAt != NULL) { 1643 _is_hetero_heap = true; 1644 max_byte_size *= 2; 1645 } I would like this to be abstracted out of G1CollectedHeap, not entirely sure how but I'm thinking something like adding a G1HeteroCollectorPolicy which answers correctly on how much memory needs to be reserved and how big the heap actually is. 1668 G1RegionToSpaceMapper::create_heap_mapper(g1_rs, 1669 g1_rs.size(), 1670 page_size, 1671 HeapRegion::GrainBytes, 1672 1, 1673 mtJavaHeap); This function could then also make use of the new policy, something like: create_heap_mapper(g1_rs, _collector_policy, page_size); 1704 if (is_hetero_heap()) { 1705 _hrm = new HeapRegionManagerForHeteroHeap((uint)((max_byte_size / 2) / HeapRegion::GrainBytes /*heap size as num of regions*/)); 1706 } 1707 else { 1708 _hrm = new HeapRegionManager(); 1709 } This code could then become something like: _hrm = HeapRegionManager::create_manager(_collector_policy) 3925 if(g1h->is_hetero_heap()) { 3926 if(!r->is_old()) { .... 3929 r->set_premature_old(); 3930 } 3931 } else { 3932 r->set_old(); 3933 } So if I understand this correctly, premature_old is used when we get evac failures and we use it to force these regions to be included in the next Mixed collection. I'm not sure this is needed, in fact I think one of the cool things with nv-dimm is that we can avoid evac failures. G1 normally needs to stop handing out regions to promote to when there are no regions left, but when using nv-dimm the old regions come from another pool and we should be able to avoid this case. --- src/hotspot/share/gc/g1/g1FullCollector.cpp ------------------------------------------- 169 if (_heap->is_hetero_heap()) { 170 static_cast (_heap->_hrm)->prepare_for_full_collection_start(); 171 } Move this to: G1CollectedHeap::prepare_heap_for_full_collection() 185 if (_heap->is_hetero_heap()) { 186 static_cast (_heap->_hrm)->prepare_for_full_collection_end(); 187 } Move this to: G1CollectedHeap::prepare_heap_for_mutators() And if you make the prepare-functions virtual and not doing anything on HeapRegionManager we can avoid the checks. --- src/hotspot/share/gc/g1/g1Policy.cpp ------------------------------------ 223 if(_g1h->is_hetero_heap() && (Thread::current()->is_VM_thread() || Heap_lock->owned_by_self())) { 224 static_cast (_g1h->hrm())->resize_dram_regions(_young_list_target_length, _g1h->workers()); 225 } Feels like this code should be part of the prepare_for_full_collection_end() above, but the _young_list_target_length isn't updated until right after prepare_heap_for_mutators() so you might need to restructure the code a bit more to make it fit. --- src/hotspot/share/gc/g1/g1RegionToSpaceMapper.cpp ------------------------------------------------- Had to add these two includes to make it compile. #include "runtime/java.hpp" #include "utilities/formatBuffer.hpp" Please also clean out all code commented out. --- src/hotspot/share/gc/g1/heapRegionManager.hpp --------------------------------------------- I agree with Sangheon, please group the members that should be protected and remember to update the init-list for the constructor. Also agree that the #else on #ifdef ASSERT should be removed. --- src/hotspot/share/gc/g1/heapRegionSet.cpp ----------------------------------------- 240 bool started = false; 241 while (cur != NULL && cur->hrm_index() <= end) { 242 if (!started && cur->hrm_index() >= start) { 243 started = true; 244 } 245 if(started) { 246 num++; 247 } 248 cur = cur->next(); 249 } To me this looks more complex than it is because of the multiple conditions, what do you think about: while (cur != NULL) { uint index = cur->hrm_index(); if (index > end) { break; } else if (index >= start) { num++; } cur = cur->next(); } --- src/hotspot/share/runtime/arguments.cpp --------------------------------------- 2072 if(!FLAG_IS_DEFAULT(AllocateHeapAt) && !FLAG_IS_DEFAULT(AllocateOldGenAt)) { 2073 jio_fprintf(defaultStream::error_stream(), 2074 "AllocateHeapAt and AllocateOldGenAt cannot be used together.\n"); 2075 status = false; 2076 } 2077 2078 if (!FLAG_IS_DEFAULT(AllocateOldGenAt) && (UseSerialGC || UseConcMarkSweepGC || UseEpsilonGC || UseZGC)) { 2079 jio_fprintf(defaultStream::error_stream(), 2080 "AllocateOldGenAt not supported for selected GC.\n"); 2081 status = false; 2082 } This code would fit much better in GCArguments. There is currently no method handling this case but please add check_args_consistency(). You can look at CompilerConfig::check_args_consistency for inspiration. --- src/hotspot/share/runtime/globals.hpp ------------------------------------- 2612 experimental(uintx, G1YoungExpansionBufferPerc, 10, Move to g1_globals.hpp and call it G1YoungExpansionBufferPercent. --- src/hotspot/share/gc/g1/heapRegionManagerForHeteroHeap.*pp ---------------------------------------------------------- Haven't reviewed this code in detail yet but will do in a later review. One thought I have already is about the name, what do you think about: HeterogeneousHeapRegionManager --- Thanks, Stefan > > Testing : Passed tier1_gc and tier1_runtime jtret tests. > > ???????????????? I added extra options > "-XX:+UnlockExperimentalVMOptions -XX:AllocateOldGenAt=/home/Kishor" to > each test. There are some tests which I had to exclude since they won't > work with this flag. Example: tests in ConcMarkSweepGC which does not > support this flag, tests requiring CompressedOops to be enabled, etc. > > Thanks > > Kishor > From kim.barrett at oracle.com Fri Oct 26 17:37:12 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 26 Oct 2018 13:37:12 -0400 Subject: RFR (T): 8212974: Update RS Skipped cards uses wrong enum to register to phase In-Reply-To: References: Message-ID: > On Oct 25, 2018, at 11:35 AM, Thomas Schatzl wrote: > > Hi all, > > can I have a review for this trivial change that fixes a benign typo? > > During initialization of the phase time records the code uses the wrong > enum to register the "Skipped Cards" to the "Update RS" logging > structure. > > I.e. > _gc_par_phases[UpdateRS]- >> link_thread_work_items(_update_rs_skipped_cards, ScanRSSkippedCards); > > which should be > > _gc_par_phases[UpdateRS]- >> link_thread_work_items(_update_rs_skipped_cards, > UpdateRSSkippedCards); > > Since the value of ScanRSSkippedCards equals UpdateRSSkippedCards there > is no harm. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8212974 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8212974/webrev/ > Testing: > local compilation; there is no way to create a test for that for above > reasons. > > Thanks, > Thomas Looks good, and trivial. From igor.ignatyev at oracle.com Fri Oct 26 23:23:32 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 26 Oct 2018 16:23:32 -0700 Subject: RFR(S) : 8157728 : Covert GCTimer_test to GTest In-Reply-To: <32170658-7540-402A-ABB7-93C25E1D0A0F@oracle.com> References: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> <32170658-7540-402A-ABB7-93C25E1D0A0F@oracle.com> Message-ID: <9805F73F-2631-455A-8C44-3A2A2E4A7C51@oracle.com> ping... still looking for a Reviewer -- Igor > On Oct 10, 2018, at 11:33 AM, Igor Ignatyev wrote: > > Hi David, > > thanks for spotting the typo, I've fixed the commit message. > still looking for a Reviewer though. > > Cheers, > -- Igor > >> On Oct 9, 2018, at 6:49 PM, David Holmes wrote: >> >> Hi Igor, >> >> Not a review - I fixed the typo in the bug synopsis: Covert -> Convert :) >> >> Please ensure you commit with corrected synopsis. >> >> Thanks, >> David >> >> On 10/10/2018 3:23 AM, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html >>>> 450 lines changed: 238 ins; 211 del; 1 mod; >>> Hi all, >>> could you please review this small (and hopefully trivial) patch which converts internal GCTimer_test to gtest? >>> webrev: http://cr.openjdk.java.net/~iignatyev//8157728/webrev.00/index.html >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8157728 >>> testing: >>> - converted tests on linux-x64, windows-x64, macosx-x64, solaris-sparcv9 in product and fastdebug variants >>> - build w/ precompiled-headers enabled and disabled >>> PS the patch has been originally created by Kirill Zh, but hasn't been sent out for official review >>> Thanks, >>> -- Igor >>> > From igor.ignatyev at oracle.com Fri Oct 26 23:24:10 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 26 Oct 2018 16:24:10 -0700 Subject: RFR(XS) : 8177708 : Convert TestReserveMemorySpecial_test to Gtest In-Reply-To: References: Message-ID: <25A5310B-02AB-4CC5-A3B9-49AAD772B79E@oracle.com> ping -- Igor > On Oct 22, 2018, at 11:36 AM, Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html >> 331 lines changed: 330 ins; 1 del; 0 mod; > Hi all, > > could you please review this small and trivial patch which converts TestReserveMemorySpecial_test tests to Gtest? the old tests[1,2] haven't been removed as they are used by WhiteBox::runMemoryUnitTests they will be removed when the rest tests are converted. > > webrev: http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8177708 > [1] http://hg.openjdk.java.net/jdk/jdk/file/tip/src/hotspot/os/linux/os_linux.cpp#l5969 > [2] http://hg.openjdk.java.net/jdk/jdk/file/tip/src/hotspot/os/windows/os_windows.cpp#l5564 > > Thanks, > -- Igor From thomas.schatzl at oracle.com Mon Oct 29 07:52:45 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 29 Oct 2018 08:52:45 +0100 Subject: RFR (T): 8212974: Update RS Skipped cards uses wrong enum to register to phase In-Reply-To: References: Message-ID: <890e02807d489e53e424c7341ba7d0f22019d7c4.camel@oracle.com> Hi Kim, On Fri, 2018-10-26 at 13:37 -0400, Kim Barrett wrote: > > On Oct 25, 2018, at 11:35 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > Hi all, > > > > can I have a review for this trivial change that fixes a benign > > typo? > > > > [...] > > Looks good, and trivial. > thanks for your review. Thomas From thomas.schatzl at oracle.com Mon Oct 29 08:58:49 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 29 Oct 2018 09:58:49 +0100 Subject: RFR(S) : 8157728 : Covert GCTimer_test to GTest In-Reply-To: <32170658-7540-402A-ABB7-93C25E1D0A0F@oracle.com> References: <32DDAF88-5F67-4CD0-AC0C-3CB9F9A13869@oracle.com> <32170658-7540-402A-ABB7-93C25E1D0A0F@oracle.com> Message-ID: Hi Igor, On Wed, 2018-10-10 at 11:33 -0700, Igor Ignatyev wrote: > Hi David, > > thanks for spotting the typo, I've fixed the commit message. > still looking for a Reviewer though. looks good. Thomas From sangheon.kim at oracle.com Mon Oct 29 22:05:03 2018 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Mon, 29 Oct 2018 15:05:03 -0700 Subject: RFR (S): 8211388: Make OtherRegionsTable independent of the region it is for In-Reply-To: References: <13515ce5470f232d34dfb1d232624770fe4dff28.camel@oracle.com> Message-ID: <0a94cc4a-255d-061e-73f5-8b9b6bff208a@oracle.com> Hi Thomas, On 10/15/18 3:59 AM, Thomas Schatzl wrote: > Hi all, > > ping for a second review... > > Thomas > > On Tue, 2018-10-02 at 17:58 +0200, Thomas Schatzl wrote: >> Hi, >> >> the OtherRegionsTable is a set of a set of cards that comprises the >> remembered set. >> >> It stores the HeapRegion it collects cards for to do filtering in the >> FromCardCache. The set of cards should not have knowledge about what >> region it is for, from an abstraction POV it should just be a dumb >> container of cards. >> >> Move out the HeapRegion id from OtherRegionsTable (the "set of >> cards") >> into the enclosing HeapRegionRememberedSet (managing set of cards), >> separating the FCC functionality from the raw data storage. >> >> This will make code that assigns a HeapRegionRememberedSet to >> multiple >> regions in the future a bit simpler too. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8211388 >> Webrev: >> http://cr.openjdk.java.net/~tschatzl/8211388/webrev/ Looks good. But please modify below line before pushing it. I don't need extra webrev for this. ------------------------------------------- src/hotspot/share/gc/g1/heapRegionRemSet.hpp 126 // Create a new remembered set for the given heap region. The given mutex should 127 // be used to ensure consistency. - You removed heap region parameter, so some part of above comment should be modified. :) Thanks, Sangheon >> Testing: >> hs-tier1-3 >> >> Thanks, >> Thomas >> >> > From sangheon.kim at oracle.com Mon Oct 29 22:18:29 2018 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Mon, 29 Oct 2018 15:18:29 -0700 Subject: RFR (S): 6490394: G1: Allow heap shrinking / memory unmapping after reclaiming regions during Remark In-Reply-To: <10549c6a3761fe3a2ec76a1dc3b001fa708520f6.camel@oracle.com> References: <10549c6a3761fe3a2ec76a1dc3b001fa708520f6.camel@oracle.com> Message-ID: <1bf1332f-2cab-398a-0b0d-852595e2a365@oracle.com> Hi Thomas, On 10/2/18 8:57 AM, Thomas Schatzl wrote: > Hi all, > > allow me to provide a fix for the oldest g1-specific open issue on > the bug tracker (from 2006 :)): resize the heap (eg. uncommit) > according to existing sizing policies at remark time. > > It does exactly what the subject of the CR says. > > The feature is enabled by default - I did not see any reason to disable > it as the functionality seems to actually be what would be expected of > the concurrent cycle (as it mirrors the functionality of a full gc). > > Needs the change for JDK-8071913 currently out for review applied to > work (i.e. not crash after uncommit). > > The current work for "JDK-8204089: Timely Reducing Unused Committed > Memory" also depends on this change. > > CR: > https://bugs.openjdk.java.net/browse/JDK-6490394 > Webrev: > http://cr.openjdk.java.net/~tschatzl/6490394/webrev/ Looks good. Just minor nits so okay going with current webrev. And I don't need extra webrev if you accept it. ----------------------------------------- src/hotspot/share/gc/g1/g1CollectedHeap.cpp 4399 assert(r->rem_set()->is_empty(), "Remembered sets should be empty."); 4404 assert(r->rem_set()->is_empty(), "Remembered sets should be empty."); - Can we differentiate newly added 2 same assert? This can be 'Remembered sets should be empty, if the region is empty'. ----------------------------------------- test/hotspot/jtreg/gc/g1/humongousObjects/objectGraphTest/TestObjectGraphAfterGC.java - Copyright update Thanks, Sangheon > Testing: > hs-tier1-5, jdk-tier1-3, perf testing ongoing but not expecting a > difference > > Thanks, > Thomas > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Tue Oct 30 00:54:48 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 29 Oct 2018 20:54:48 -0400 Subject: RFR (S): 8212911: Unify and micro-optimize handling of non-in-collection set references in oop closures In-Reply-To: <646a59f02646ff542659ed3f6fb16dc9df1462e9.camel@oracle.com> References: <2cad32398c23732a6a91b430bcafaf9ea2a7c611.camel@oracle.com> <646a59f02646ff542659ed3f6fb16dc9df1462e9.camel@oracle.com> Message-ID: <2AC26D1C-7D02-4FE9-B955-CBB15829F63A@oracle.com> > On Oct 25, 2018, at 5:45 AM, Thomas Schatzl wrote: > > Hi all, > > On Wed, 2018-10-24 at 13:36 +0200, Thomas Schatzl wrote: >> Hi all, >> >> can I have reviews for this small change that does some minor >> cleanup and micro-optimization of our main g1 stw/concurrent gc >> closures? >> >> [...] >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8212911 >> Webrev: >> http://cr.openjdk.java.net/~tschatzl/8212911/webrev/ >> Testing: >> hs-tier1-3, several hs-tier1-5 runs with other changes, benchmark >> suite etc. > > Stefan pointed out that _from_in_young is only used in > G1ScanEvacuatedObjClosure any more, so it could be moved from > G1ScanClosureBase. > > Also we brainstormed a bit about the name of _from_in_young, and agreed > on changing this to _scanning_in_young. > > New webrevs: > http://cr.openjdk.java.net/~tschatzl/8212911/webrev.1 (full) > http://cr.openjdk.java.net/~tschatzl/8212911/webrev.0_to_1 (diff) > > Thanks, > Thomas ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1OopClosures.inline.hpp 178 } else { 179 if (HeapRegion::is_in_same_region(p, obj)) { 180 return; 181 } I think this would be easier to read if it were written as 178 } else if (!HeapRegion::is_in_same_region(p, obj)) { There are similar snippets at line 86 and line 200. I don?t need a new review if you want to make these changes. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1OopClosures.hpp 91 G1ScanClosureBase(g1h, par_scan_state), _scanning_in_young(false) { } I kind of dislike the initialization of _scanning_in_young to false. But that seems consistent with previous usage of _from. For catching bugs, a kind of tri-state and an RAII-style setter (from "unset" to true or false then back) might be better. If you like that idea but don't want to do it right now, feel free to file an RFE. (The reset back to "unset" could be DEBUG_ONLY, along with all of the relevant checking being in asserts.) ------------------------------------------------------------------------------ Looks good, up to the above optional to address issues. From kim.barrett at oracle.com Tue Oct 30 02:31:00 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 29 Oct 2018 22:31:00 -0400 Subject: RFR (M): 8071913: Filter out entries to free/uncommitted regions during iteration In-Reply-To: <809796367d72a91d192bfdb6fdb05617a7506914.camel@oracle.com> References: <809796367d72a91d192bfdb6fdb05617a7506914.camel@oracle.com> Message-ID: <970FEE82-AE60-462E-A2C1-16052E3DAA30@oracle.com> > On Oct 2, 2018, at 7:28 AM, Thomas Schatzl wrote: > > [?] > > CR: > https://bugs.openjdk.java.net/browse/JDK-8071913 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8071913/webrev/ > Testing: > hs-tier1-5,jdk-tier1-3, perf tested, no change in pause times or > (throughput) scores > > Quite a few gc tests do extensive commit/uncommit of regions with the > corresponding exercise of the code. > > Thanks, > Thomas Looks good. A few minor comments, for which I don't need a new webrev. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1CollectedHeap.hpp 1123 inline HeapRegion* region_at_or_null(uint index) const; Needs comment updating. Presumably this too requires the index to be valid, and only returns NULL if the region is unmapped. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1CollectedHeap.inline.hpp 95 uint const region_idx = addr_to_region(addr); 96 return _hrm.is_available(region_idx) ? region_at(region_idx) : NULL; Shouldn't this just be _hrm.at_or_null(addr_to_region(addr)); ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1CollectedHeap.cpp 2672 if (!g1h->is_in_closed_subset(ct->addr_for(card_ptr))) { 2673 continue; 2674 } I prefer the original code, without this change. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1CollectedHeap.cpp [deleted code] 2679 assert(hrrs.n_yielded() == r->rem_set()->occupied(), 2680 "Remembered set hash maps out of sync, cur: " SIZE_FORMAT " entries, next: " SIZE_FORMAT " entries", 2681 hrrs.n_yielded(), r->rem_set()->occupied()); I'm not sure how this code deletion relates to the other changes in this webrev. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/heapRegionManager.hpp 126 public: Moving that makes for easier reviewing, which is good. But I think it might be better to move the declaration of is_available to some better part of the public functions, maybe near at_or_null? ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1RemSet.hpp src/hotspot/share/gc/g1/heapRegion.hpp src/hotspot/share/gc/g1/sparsePRT.hpp src/hotspot/share/gc/g1/sparsePRT.cpp [no changes] ------------------------------------------------------------------------------ From thomas.schatzl at oracle.com Tue Oct 30 09:39:52 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 30 Oct 2018 10:39:52 +0100 Subject: RFR (M): 8071913: Filter out entries to free/uncommitted regions during iteration In-Reply-To: <970FEE82-AE60-462E-A2C1-16052E3DAA30@oracle.com> References: <809796367d72a91d192bfdb6fdb05617a7506914.camel@oracle.com> <970FEE82-AE60-462E-A2C1-16052E3DAA30@oracle.com> Message-ID: Hi Kim, thanks for your review. On Mon, 2018-10-29 at 22:31 -0400, Kim Barrett wrote: > > On Oct 2, 2018, at 7:28 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > [?] > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8071913 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8071913/webrev/ > > Testing: > > hs-tier1-5,jdk-tier1-3, perf tested, no change in pause times or > > (throughput) scores > > > > Quite a few gc tests do extensive commit/uncommit of regions with > > the corresponding exercise of the code. > > > > Thanks, > > Thomas > > Looks good. > just for reference, here is what I intend to push later after some testing: http://cr.openjdk.java.net/~tschatzl/8071913/webrev.0_to_1/ (diff) http://cr.openjdk.java.net/~tschatzl/8071913/webrev.1/ (full) > A few minor comments, for which I don't need a new webrev. > > [...] > > src/hotspot/share/gc/g1/g1CollectedHeap.cpp > 2672 if (!g1h->is_in_closed_subset(ct->addr_for(card_ptr))) > { > 2673 continue; > 2674 } > > I prefer the original code, without this change. > Changed back. > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1CollectedHeap.cpp > [deleted code] > 2679 assert(hrrs.n_yielded() == r->rem_set()->occupied(), > 2680 "Remembered set hash maps out of sync, cur: " > SIZE_FORMAT " entries, next: " SIZE_FORMAT " entries", > 2681 hrrs.n_yielded(), r->rem_set()->occupied()); > > I'm not sure how this code deletion relates to the other changes in > this webrev. > This assert is wrong, just by the way we select humongous candidates it has not been triggered yet, and has been removed in different places before. Since dumping the remsets of humongous candidates into the DCQ will be changed significantly in the future, and this is no pressing issue, I reinstated the code for now. Thanks again, Thomas From thomas.schatzl at oracle.com Tue Oct 30 09:42:51 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 30 Oct 2018 10:42:51 +0100 Subject: RFR (S): 6490394: G1: Allow heap shrinking / memory unmapping after reclaiming regions during Remark In-Reply-To: <1bf1332f-2cab-398a-0b0d-852595e2a365@oracle.com> References: <10549c6a3761fe3a2ec76a1dc3b001fa708520f6.camel@oracle.com> <1bf1332f-2cab-398a-0b0d-852595e2a365@oracle.com> Message-ID: <6cd675c4d2e75191c445d03528b29fb762e086b1.camel@oracle.com> Hi Sanghon, thanks for your review. On Mon, 2018-10-29 at 15:18 -0700, sangheon.kim at oracle.com wrote: > Hi Thomas, > > On 10/2/18 8:57 AM, Thomas Schatzl wrote: > > Hi all, > > > > allow me to provide a fix for the oldest g1-specific open issue > > on the bug tracker (from 2006 :)): resize the heap (eg. uncommit) > > according to existing sizing policies at remark time. > > > > It does exactly what the subject of the CR says. > > > > [...] > > The current work for "JDK-8204089: Timely Reducing Unused Committed > > Memory" also depends on this change. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-6490394 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/6490394/webrev/ > Looks good. > Just minor nits so okay going with current webrev. And I don't need > extra webrev if you accept it. I changed the code as suggested. For reference (no need to re-review it), the webrevs: http://cr.openjdk.java.net/~tschatzl/6490394/webrev.0_to_1/ (diff) http://cr.openjdk.java.net/~tschatzl/6490394/webrev.1/ (full) Thanks again, Thomas From thomas.schatzl at oracle.com Tue Oct 30 09:45:07 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 30 Oct 2018 10:45:07 +0100 Subject: RFR (S): 8211388: Make OtherRegionsTable independent of the region it is for In-Reply-To: <0a94cc4a-255d-061e-73f5-8b9b6bff208a@oracle.com> References: <13515ce5470f232d34dfb1d232624770fe4dff28.camel@oracle.com> <0a94cc4a-255d-061e-73f5-8b9b6bff208a@oracle.com> Message-ID: <47912a6e769ea86e98a6c9b19fd4e4a89a2f9460.camel@oracle.com> Hi Sangheon, On Mon, 2018-10-29 at 15:05 -0700, sangheon.kim at oracle.com wrote: > Hi Thomas, > > On 10/15/18 3:59 AM, Thomas Schatzl wrote: > > Hi all, > > > > ping for a second review... > > > > Thomas > > > > On Tue, 2018-10-02 at 17:58 +0200, Thomas Schatzl wrote: > > > Hi, > > > > > > the OtherRegionsTable is a set of a set of cards that > > > comprises the remembered set. > > > [...] > > > CR: > > > https://bugs.openjdk.java.net/browse/JDK-8211388 > > > Webrev: > > > http://cr.openjdk.java.net/~tschatzl/8211388/webrev/ > > Looks good. > But please modify below line before pushing it. I don't need extra > webrev for this. > > ------------------------------------------- > src/hotspot/share/gc/g1/heapRegionRemSet.hpp > > 126 // Create a new remembered set for the given heap region. The > given mutex should > 127 // be used to ensure consistency. > > - You removed heap region parameter, so some part of above comment > should be modified. :) :) For reference, the new webrevs: http://cr.openjdk.java.net/~tschatzl/8211388/webrev.0_to_1 (diff) http://cr.openjdk.java.net/~tschatzl/8211388/webrev.1 (full) Thanks, Thomas From thomas.schatzl at oracle.com Tue Oct 30 10:34:59 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 30 Oct 2018 11:34:59 +0100 Subject: RFR (S): 8212911: Unify and micro-optimize handling of non-in-collection set references in oop closures In-Reply-To: <2AC26D1C-7D02-4FE9-B955-CBB15829F63A@oracle.com> References: <2cad32398c23732a6a91b430bcafaf9ea2a7c611.camel@oracle.com> <646a59f02646ff542659ed3f6fb16dc9df1462e9.camel@oracle.com> <2AC26D1C-7D02-4FE9-B955-CBB15829F63A@oracle.com> Message-ID: Hi Kim, On Mon, 2018-10-29 at 20:54 -0400, Kim Barrett wrote: > > On Oct 25, 2018, at 5:45 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > Hi all, > > > > On Wed, 2018-10-24 at 13:36 +0200, Thomas Schatzl wrote: > > > Hi all, > > > > > > can I have reviews for this small change that does some minor > > > cleanup and micro-optimization of our main g1 stw/concurrent gc > > > closures? > > > > > > [...] > > > CR: > > > https://bugs.openjdk.java.net/browse/JDK-8212911 > > > Webrev: > > > http://cr.openjdk.java.net/~tschatzl/8212911/webrev/ > > > Testing: > > > hs-tier1-3, several hs-tier1-5 runs with other changes, benchmark > > > suite etc. > > > > Stefan pointed out that _from_in_young is only used in > > G1ScanEvacuatedObjClosure any more, so it could be moved from > > G1ScanClosureBase. > > > > Also we brainstormed a bit about the name of _from_in_young, and > > agreed on changing this to _scanning_in_young. > > > > New webrevs: > > http://cr.openjdk.java.net/~tschatzl/8212911/webrev.1 (full) > > http://cr.openjdk.java.net/~tschatzl/8212911/webrev.0_to_1 (diff) > > > > Thanks, > > Thomas > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1OopClosures.inline.hpp > 178 } else { > 179 if (HeapRegion::is_in_same_region(p, obj)) { > 180 return; > 181 } > > I think this would be easier to read if it were written as > > 178 } else if (!HeapRegion::is_in_same_region(p, obj)) { > > There are similar snippets at line 86 and line 200. I don?t need > a new review if you want to make these changes. Fixed. > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1OopClosures.hpp > 91 G1ScanClosureBase(g1h, par_scan_state), > _scanning_in_young(false) { } > > I kind of dislike the initialization of _scanning_in_young to false. "False" is the safe option, potentially enqueuing a reference that is not needed to enqueue (if you forget). > But that seems consistent with previous usage of _from. For catching > bugs, a kind of tri-state and an RAII-style setter (from "unset" to > true or false then back) might be better. If you like that idea but > don't want to do it right now, feel free to file an RFE. (The reset > back to "unset" could be DEBUG_ONLY, along with all of the relevant > checking being in asserts.) I filed JDK-8213142 for that, with an RFR coming soon. Thanks, Thomas From thomas.schatzl at oracle.com Tue Oct 30 10:37:01 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 30 Oct 2018 11:37:01 +0100 Subject: RFR (S): 8212911: Unify and micro-optimize handling of non-in-collection set references in oop closures In-Reply-To: References: <2cad32398c23732a6a91b430bcafaf9ea2a7c611.camel@oracle.com> <646a59f02646ff542659ed3f6fb16dc9df1462e9.camel@oracle.com> <2AC26D1C-7D02-4FE9-B955-CBB15829F63A@oracle.com> Message-ID: Hi, ... aaaand I forgot to add the webrevs in the last email for reference: http://cr.openjdk.java.net/~tschatzl/8212911/webrev.1_to_2 (diff) http://cr.openjdk.java.net/~tschatzl/8212911/webrev.2 (full) Thanks, Thomas On Tue, 2018-10-30 at 11:34 +0100, Thomas Schatzl wrote: > Hi Kim, > > On Mon, 2018-10-29 at 20:54 -0400, Kim Barrett wrote: > > > On Oct 25, 2018, at 5:45 AM, Thomas Schatzl < > > > thomas.schatzl at oracle.com> wrote: > > > > > > Hi all, > > > > > > On Wed, 2018-10-24 at 13:36 +0200, Thomas Schatzl wrote: > > > > Hi all, > > > > > > > > can I have reviews for this small change that does some minor > > > > cleanup and micro-optimization of our main g1 stw/concurrent gc > > > > closures? > > > > > > > > [...] > > > > CR: > > > > https://bugs.openjdk.java.net/browse/JDK-8212911 > > > > Webrev: > > > > http://cr.openjdk.java.net/~tschatzl/8212911/webrev/ > > > > Testing: > > > > hs-tier1-3, several hs-tier1-5 runs with other changes, > > > > benchmark > > > > suite etc. > > > > > > Stefan pointed out that _from_in_young is only used in > > > G1ScanEvacuatedObjClosure any more, so it could be moved from > > > G1ScanClosureBase. > > > > > > Also we brainstormed a bit about the name of _from_in_young, and > > > agreed on changing this to _scanning_in_young. > > > > > > New webrevs: > > > http://cr.openjdk.java.net/~tschatzl/8212911/webrev.1 (full) > > > http://cr.openjdk.java.net/~tschatzl/8212911/webrev.0_to_1 (diff) > > > > > > Thanks, > > > Thomas > > > > ------------------------------------------------------------------- > > ----------- > > src/hotspot/share/gc/g1/g1OopClosures.inline.hpp > > 178 } else { > > 179 if (HeapRegion::is_in_same_region(p, obj)) { > > 180 return; > > 181 } > > > > I think this would be easier to read if it were written as > > > > 178 } else if (!HeapRegion::is_in_same_region(p, obj)) { > > > > There are similar snippets at line 86 and line 200. I don?t need > > a new review if you want to make these changes. > > Fixed. > > > ------------------------------------------------------------------- > > ----------- > > src/hotspot/share/gc/g1/g1OopClosures.hpp > > 91 G1ScanClosureBase(g1h, par_scan_state), > > _scanning_in_young(false) { } > > > > I kind of dislike the initialization of _scanning_in_young to > > false. > > "False" is the safe option, potentially enqueuing a reference that is > not needed to enqueue (if you forget). > > > But that seems consistent with previous usage of _from. For > > catching > > bugs, a kind of tri-state and an RAII-style setter (from "unset" to > > true or false then back) might be better. If you like that idea > > but > > don't want to do it right now, feel free to file an RFE. (The > > reset > > back to "unset" could be DEBUG_ONLY, along with all of the relevant > > checking being in asserts.) > > I filed JDK-8213142 for that, with an RFR coming soon. > > Thanks, > Thomas > > From thomas.schatzl at oracle.com Tue Oct 30 10:42:02 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 30 Oct 2018 11:42:02 +0100 Subject: RFR (S): 8213142: Use RAII to set the scanning source in G1ScanEvacuatedObjClosure Message-ID: <9d637ef4e7462bdf36015600033f299016a6f158.camel@oracle.com> Hi all, in the review for JDK-8212911 Kim suggested to use RAII to set the _scanning_in_young field of the G1ScanEvacuatedObjClosure class. This also allows making sure that the field is actually initialized when used. This is the implementation for this idea. CR: https://bugs.openjdk.java.net/browse/JDK-8213142 Webrev: http://cr.openjdk.java.net/~tschatzl/8213142/webrev/ Testing: local gc jtreg run Thanks, Thomas From sangheon.kim at oracle.com Tue Oct 30 13:55:21 2018 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Tue, 30 Oct 2018 06:55:21 -0700 Subject: RFR (S): 8211388: Make OtherRegionsTable independent of the region it is for In-Reply-To: <47912a6e769ea86e98a6c9b19fd4e4a89a2f9460.camel@oracle.com> References: <13515ce5470f232d34dfb1d232624770fe4dff28.camel@oracle.com> <0a94cc4a-255d-061e-73f5-8b9b6bff208a@oracle.com> <47912a6e769ea86e98a6c9b19fd4e4a89a2f9460.camel@oracle.com> Message-ID: Hi Thomas, On 10/30/18 2:45 AM, Thomas Schatzl wrote: > Hi Sangheon, > > On Mon, 2018-10-29 at 15:05 -0700, sangheon.kim at oracle.com wrote: >> Hi Thomas, >> >> On 10/15/18 3:59 AM, Thomas Schatzl wrote: >>> Hi all, >>> >>> ping for a second review... >>> >>> Thomas >>> >>> On Tue, 2018-10-02 at 17:58 +0200, Thomas Schatzl wrote: >>>> Hi, >>>> >>>> the OtherRegionsTable is a set of a set of cards that >>>> comprises the remembered set. >>>> [...] >>>> CR: >>>> https://bugs.openjdk.java.net/browse/JDK-8211388 >>>> Webrev: >>>> http://cr.openjdk.java.net/~tschatzl/8211388/webrev/ >> Looks good. >> But please modify below line before pushing it. I don't need extra >> webrev for this. >> >> ------------------------------------------- >> src/hotspot/share/gc/g1/heapRegionRemSet.hpp >> >> 126 // Create a new remembered set for the given heap region. The >> given mutex should >> 127 // be used to ensure consistency. >> >> - You removed heap region parameter, so some part of above comment >> should be modified. :) > :) > > For reference, the new webrevs: > http://cr.openjdk.java.net/~tschatzl/8211388/webrev.0_to_1 (diff) > http://cr.openjdk.java.net/~tschatzl/8211388/webrev.1 (full) webrev.1 looks good to me. Thanks for the webrev. Sangheon > > Thanks, > Thomas > > From sangheon.kim at oracle.com Tue Oct 30 13:56:53 2018 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Tue, 30 Oct 2018 06:56:53 -0700 Subject: RFR (S): 6490394: G1: Allow heap shrinking / memory unmapping after reclaiming regions during Remark In-Reply-To: <6cd675c4d2e75191c445d03528b29fb762e086b1.camel@oracle.com> References: <10549c6a3761fe3a2ec76a1dc3b001fa708520f6.camel@oracle.com> <1bf1332f-2cab-398a-0b0d-852595e2a365@oracle.com> <6cd675c4d2e75191c445d03528b29fb762e086b1.camel@oracle.com> Message-ID: <98fe27b3-ee2f-ec8f-dee5-923b9b341d26@oracle.com> Hi Thomas, On 10/30/18 2:42 AM, Thomas Schatzl wrote: > Hi Sanghon, > > thanks for your review. > > On Mon, 2018-10-29 at 15:18 -0700, sangheon.kim at oracle.com wrote: >> Hi Thomas, >> >> On 10/2/18 8:57 AM, Thomas Schatzl wrote: >>> Hi all, >>> >>> allow me to provide a fix for the oldest g1-specific open issue >>> on the bug tracker (from 2006 :)): resize the heap (eg. uncommit) >>> according to existing sizing policies at remark time. >>> >>> It does exactly what the subject of the CR says. >>> >>> [...] >>> The current work for "JDK-8204089: Timely Reducing Unused Committed >>> Memory" also depends on this change. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-6490394 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/6490394/webrev/ >> Looks good. >> Just minor nits so okay going with current webrev. And I don't need >> extra webrev if you accept it. > I changed the code as suggested. For reference (no need to re-review > it), the webrevs: > > http://cr.openjdk.java.net/~tschatzl/6490394/webrev.0_to_1/ (diff) > http://cr.openjdk.java.net/~tschatzl/6490394/webrev.1/ (full) Looks good to me. Thanks, Sangheon > > Thanks again, > Thomas > > From kim.barrett at oracle.com Tue Oct 30 14:49:16 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 30 Oct 2018 10:49:16 -0400 Subject: RFR (S): 8212911: Unify and micro-optimize handling of non-in-collection set references in oop closures In-Reply-To: References: <2cad32398c23732a6a91b430bcafaf9ea2a7c611.camel@oracle.com> <646a59f02646ff542659ed3f6fb16dc9df1462e9.camel@oracle.com> <2AC26D1C-7D02-4FE9-B955-CBB15829F63A@oracle.com> Message-ID: <6183933F-7619-433C-91A7-862F2E632F5A@oracle.com> > On Oct 30, 2018, at 6:37 AM, Thomas Schatzl wrote: > > Hi, > > ... aaaand I forgot to add the webrevs in the last email for > reference: > > http://cr.openjdk.java.net/~tschatzl/8212911/webrev.1_to_2 (diff) > http://cr.openjdk.java.net/~tschatzl/8212911/webrev.2 (full) And for the record, looks good. From sangheon.kim at oracle.com Tue Oct 30 21:39:35 2018 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Tue, 30 Oct 2018 14:39:35 -0700 Subject: RFR (S): 8213142: Use RAII to set the scanning source in G1ScanEvacuatedObjClosure In-Reply-To: <9d637ef4e7462bdf36015600033f299016a6f158.camel@oracle.com> References: <9d637ef4e7462bdf36015600033f299016a6f158.camel@oracle.com> Message-ID: <756c5e2c-754f-e362-2c64-141e5f361d26@oracle.com> Hi Thomas, On 10/30/18 3:42 AM, Thomas Schatzl wrote: > Hi all, > > in the review for JDK-8212911 Kim suggested to use RAII to set the > _scanning_in_young field of the G1ScanEvacuatedObjClosure class. This > also allows making sure that the field is actually initialized when > used. > > This is the implementation for this idea. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8213142 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8213142/webrev/ Looks good to me. Thanks, Sangheon > Testing: > local gc jtreg run > > Thanks, > Thomas > > From kim.barrett at oracle.com Wed Oct 31 06:00:49 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 31 Oct 2018 02:00:49 -0400 Subject: RFR (S): 8213142: Use RAII to set the scanning source in G1ScanEvacuatedObjClosure In-Reply-To: <9d637ef4e7462bdf36015600033f299016a6f158.camel@oracle.com> References: <9d637ef4e7462bdf36015600033f299016a6f158.camel@oracle.com> Message-ID: > On Oct 30, 2018, at 6:42 AM, Thomas Schatzl wrote: > > Hi all, > > in the review for JDK-8212911 Kim suggested to use RAII to set the > _scanning_in_young field of the G1ScanEvacuatedObjClosure class. This > also allows making sure that the field is actually initialized when > used. > > This is the implementation for this idea. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8213142 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8213142/webrev/ > Testing: > local gc jtreg run > > Thanks, > Thomas Looks good. From per.liden at oracle.com Wed Oct 31 14:10:32 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 31 Oct 2018 15:10:32 +0100 Subject: RFR: 8212921: Relax code cache oop verification in ZGC In-Reply-To: <5BD1C28A.8050804@oracle.com> References: <5BD1C28A.8050804@oracle.com> Message-ID: <452a3f9a-a257-a939-0b55-d02e471f98b9@oracle.com> On 10/25/2018 03:18 PM, Erik ?sterlund wrote: > Hi, > > With concurrent class unloading, the normal state of affairs for ZGC is > that nmethods have messed up oops in all GC safepoints, and fix them up > during concurrent phases. Therefore, Universe::verify needs to relax the > assumption that oops are okay for ZGC. > > Note that during Universe::verify, the oops in the nmethods were > redundantly verified. Once by walking the oops in the code cache, and > once when calling nmethod::verify(), which did the same verification > again. I removed the latter. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8212921/webrev.00/ I discussed this with Erik off-line. Instead of adjusting the verification code to do what ZGC needs, we could instead move ZGC's verification to after resurrection is unblocked. At this time, all oops we find will be good and no special nmethod verification will be needed. Here's a webrev to adjust to that model: http://cr.openjdk.java.net/~pliden/8212921/webrev.0 I also noticed that we could remove some boilerplate code by fusing ZVerifyRootOopClosure and ZVerifyHeapOopClosure into a single ZVerifyOopClosure, which I did. I still agree that the current double verification of nmethod oops seems completely unnecessary, but it's harmless so that can be handled by a separate RFE. cheers, Per > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212921 > > Thanks, > /Erik From erik.osterlund at oracle.com Wed Oct 31 14:19:15 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 31 Oct 2018 15:19:15 +0100 Subject: RFR: 8212921: Relax code cache oop verification in ZGC In-Reply-To: <452a3f9a-a257-a939-0b55-d02e471f98b9@oracle.com> References: <5BD1C28A.8050804@oracle.com> <452a3f9a-a257-a939-0b55-d02e471f98b9@oracle.com> Message-ID: <5BD9B9E3.9010207@oracle.com> Hi Per, I prefer your approach - it allows more verification which is good. Looks good. I'll reassign this to you. Thanks, /Erik On 2018-10-31 15:10, Per Liden wrote: > On 10/25/2018 03:18 PM, Erik ?sterlund wrote: >> Hi, >> >> With concurrent class unloading, the normal state of affairs for ZGC >> is that nmethods have messed up oops in all GC safepoints, and fix >> them up during concurrent phases. Therefore, Universe::verify needs >> to relax the assumption that oops are okay for ZGC. >> >> Note that during Universe::verify, the oops in the nmethods were >> redundantly verified. Once by walking the oops in the code cache, and >> once when calling nmethod::verify(), which did the same verification >> again. I removed the latter. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8212921/webrev.00/ > > I discussed this with Erik off-line. Instead of adjusting the > verification code to do what ZGC needs, we could instead move ZGC's > verification to after resurrection is unblocked. At this time, all > oops we find will be good and no special nmethod verification will be > needed. > > Here's a webrev to adjust to that model: > > http://cr.openjdk.java.net/~pliden/8212921/webrev.0 > > I also noticed that we could remove some boilerplate code by fusing > ZVerifyRootOopClosure and ZVerifyHeapOopClosure into a single > ZVerifyOopClosure, which I did. > > I still agree that the current double verification of nmethod oops > seems completely unnecessary, but it's harmless so that can be handled > by a separate RFE. > > cheers, > Per > >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8212921 >> >> Thanks, >> /Erik From per.liden at oracle.com Wed Oct 31 15:35:59 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 31 Oct 2018 16:35:59 +0100 Subject: RFR: 8212184: Incorrect oop ref strength used for referents in FinalReference In-Reply-To: <890bb0a2-30af-3fa4-0c42-874b3d4376d4@oracle.com> References: <9363d38d-eb3c-b06c-2c23-88d8f71fe21f@oracle.com> <4052187A-9C1F-4BE3-8CE7-E10C06B2BEB9@oracle.com> <890bb0a2-30af-3fa4-0c42-874b3d4376d4@oracle.com> Message-ID: <972367f4-2ee4-2357-25ed-b327b419bd7a@oracle.com> Hi, After some off-line discussions with Kim and Erik about whether this should be STRONG or PHANTOM it was agreed that it technically doesn't matter. So it comes down to preference. Kim (who preferred PHANTOM) agreed to go with STRONG (which Erik and I preferred) as long there was a comment explaining why. So I updated the patch accordingly: http://cr.openjdk.java.net/~pliden/8212184/webrev.1 cheers, Per On 10/16/2018 01:55 PM, Per Liden wrote: > Hi Kim, > > On 10/15/2018 08:55 PM, Kim Barrett wrote: >>> On Oct 15, 2018, at 10:42 AM, Per Liden wrote: >>> >>> AccessBarrierSupport::resolve_unknown_oop_ref_strength() returns an >>> incorrect oop ref strength for referents in FinalReference objects. >>> It currently returns ON_WEAK_OOP_REF when it should return >>> ON_STRONG_OOP_REF. This is not really an issue for any GC except ZGC >>> when using the ZHeapIterator to walk the heap while resurrection is >>> blocked. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8212184 >>> Webrev: http://cr.openjdk.java.net/~pliden/8212184/webrev.0 >>> >>> /Per >> >> src/hotspot/share/gc/shared/accessBarrierSupport.cpp >> ?? 32?? if (!java_lang_ref_Reference::is_referent_field(base, offset) || >> ?? 33?????? java_lang_ref_Reference::is_final(base)) { >> ?? 34???? ds |= ON_STRONG_OOP_REF; >> >> This doesn't seem right.? Doesn't this give the wrong answer for G1? >> >> I'm not even sure "strong" is the right answer for ZGC in the >> described context. >> >> What am I missing? >> > > There's no way for a mutator to get hold of the referent (except via > Unsafe, which we've said we don't care about anyway). The only time we > would randomly step on a referent in a Finalizer is when we're doing > heap iteration in ZGC. > > An alternative, to perhaps make this more explicit, would be to have an > ON_FINAL_OOP_REF, but it would end up doing the same thing as > ON_STRONG_OOP_REF. > > /Per From erik.osterlund at oracle.com Wed Oct 31 15:39:03 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 31 Oct 2018 16:39:03 +0100 Subject: RFR: 8212184: Incorrect oop ref strength used for referents in FinalReference In-Reply-To: <972367f4-2ee4-2357-25ed-b327b419bd7a@oracle.com> References: <9363d38d-eb3c-b06c-2c23-88d8f71fe21f@oracle.com> <4052187A-9C1F-4BE3-8CE7-E10C06B2BEB9@oracle.com> <890bb0a2-30af-3fa4-0c42-874b3d4376d4@oracle.com> <972367f4-2ee4-2357-25ed-b327b419bd7a@oracle.com> Message-ID: <5BD9CC97.1000905@oracle.com> Hi Per, Looks good. Thanks, /Erik On 2018-10-31 16:35, Per Liden wrote: > Hi, > > After some off-line discussions with Kim and Erik about whether this > should be STRONG or PHANTOM it was agreed that it technically doesn't > matter. So it comes down to preference. Kim (who preferred PHANTOM) > agreed to go with STRONG (which Erik and I preferred) as long there > was a comment explaining why. So I updated the patch accordingly: > > http://cr.openjdk.java.net/~pliden/8212184/webrev.1 > > cheers, > Per > > > On 10/16/2018 01:55 PM, Per Liden wrote: >> Hi Kim, >> >> On 10/15/2018 08:55 PM, Kim Barrett wrote: >>>> On Oct 15, 2018, at 10:42 AM, Per Liden wrote: >>>> >>>> AccessBarrierSupport::resolve_unknown_oop_ref_strength() returns an >>>> incorrect oop ref strength for referents in FinalReference objects. >>>> It currently returns ON_WEAK_OOP_REF when it should return >>>> ON_STRONG_OOP_REF. This is not really an issue for any GC except >>>> ZGC when using the ZHeapIterator to walk the heap while >>>> resurrection is blocked. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8212184 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8212184/webrev.0 >>>> >>>> /Per >>> >>> src/hotspot/share/gc/shared/accessBarrierSupport.cpp >>> 32 if (!java_lang_ref_Reference::is_referent_field(base, >>> offset) || >>> 33 java_lang_ref_Reference::is_final(base)) { >>> 34 ds |= ON_STRONG_OOP_REF; >>> >>> This doesn't seem right. Doesn't this give the wrong answer for G1? >>> >>> I'm not even sure "strong" is the right answer for ZGC in the >>> described context. >>> >>> What am I missing? >>> >> >> There's no way for a mutator to get hold of the referent (except via >> Unsafe, which we've said we don't care about anyway). The only time >> we would randomly step on a referent in a Finalizer is when we're >> doing heap iteration in ZGC. >> >> An alternative, to perhaps make this more explicit, would be to have >> an ON_FINAL_OOP_REF, but it would end up doing the same thing as >> ON_STRONG_OOP_REF. >> >> /Per From per.liden at oracle.com Wed Oct 31 15:47:44 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 31 Oct 2018 16:47:44 +0100 Subject: RFR: 8212921: Relax code cache oop verification in ZGC In-Reply-To: <5BD9B9E3.9010207@oracle.com> References: <5BD1C28A.8050804@oracle.com> <452a3f9a-a257-a939-0b55-d02e471f98b9@oracle.com> <5BD9B9E3.9010207@oracle.com> Message-ID: Ok, thanks for reviewing, Erik. /Per On 10/31/2018 03:19 PM, Erik ?sterlund wrote: > Hi Per, > > I prefer your approach - it allows more verification which is good. > Looks good. I'll reassign this to you. > > Thanks, > /Erik > > On 2018-10-31 15:10, Per Liden wrote: >> On 10/25/2018 03:18 PM, Erik ?sterlund wrote: >>> Hi, >>> >>> With concurrent class unloading, the normal state of affairs for ZGC >>> is that nmethods have messed up oops in all GC safepoints, and fix >>> them up during concurrent phases. Therefore, Universe::verify needs >>> to relax the assumption that oops are okay for ZGC. >>> >>> Note that during Universe::verify, the oops in the nmethods were >>> redundantly verified. Once by walking the oops in the code cache, and >>> once when calling nmethod::verify(), which did the same verification >>> again. I removed the latter. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8212921/webrev.00/ >> >> I discussed this with Erik off-line. Instead of adjusting the >> verification code to do what ZGC needs, we could instead move ZGC's >> verification to after resurrection is unblocked. At this time, all >> oops we find will be good and no special nmethod verification will be >> needed. >> >> Here's a webrev to adjust to that model: >> >> http://cr.openjdk.java.net/~pliden/8212921/webrev.0 >> >> I also noticed that we could remove some boilerplate code by fusing >> ZVerifyRootOopClosure and ZVerifyHeapOopClosure into a single >> ZVerifyOopClosure, which I did. >> >> I still agree that the current double verification of nmethod oops >> seems completely unnecessary, but it's harmless so that can be handled >> by a separate RFE. >> >> cheers, >> Per >> >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8212921 >>> >>> Thanks, >>> /Erik > From kim.barrett at oracle.com Wed Oct 31 16:50:33 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 31 Oct 2018 12:50:33 -0400 Subject: RFR: 8212184: Incorrect oop ref strength used for referents in FinalReference In-Reply-To: <972367f4-2ee4-2357-25ed-b327b419bd7a@oracle.com> References: <9363d38d-eb3c-b06c-2c23-88d8f71fe21f@oracle.com> <4052187A-9C1F-4BE3-8CE7-E10C06B2BEB9@oracle.com> <890bb0a2-30af-3fa4-0c42-874b3d4376d4@oracle.com> <972367f4-2ee4-2357-25ed-b327b419bd7a@oracle.com> Message-ID: <587898D9-0B5C-442F-9E58-A73FB8A8DB51@oracle.com> > On Oct 31, 2018, at 11:35 AM, Per Liden wrote: > > Hi, > > After some off-line discussions with Kim and Erik about whether this should be STRONG or PHANTOM it was agreed that it technically doesn't matter. So it comes down to preference. Kim (who preferred PHANTOM) agreed to go with STRONG (which Erik and I preferred) as long there was a comment explaining why. So I updated the patch accordingly: > > http://cr.openjdk.java.net/~pliden/8212184/webrev.1 Looks good. > > cheers, > Per > > > On 10/16/2018 01:55 PM, Per Liden wrote: >> Hi Kim, >> On 10/15/2018 08:55 PM, Kim Barrett wrote: >>>> On Oct 15, 2018, at 10:42 AM, Per Liden wrote: >>>> >>>> AccessBarrierSupport::resolve_unknown_oop_ref_strength() returns an incorrect oop ref strength for referents in FinalReference objects. It currently returns ON_WEAK_OOP_REF when it should return ON_STRONG_OOP_REF. This is not really an issue for any GC except ZGC when using the ZHeapIterator to walk the heap while resurrection is blocked. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8212184 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8212184/webrev.0 >>>> >>>> /Per >>> >>> src/hotspot/share/gc/shared/accessBarrierSupport.cpp >>> 32 if (!java_lang_ref_Reference::is_referent_field(base, offset) || >>> 33 java_lang_ref_Reference::is_final(base)) { >>> 34 ds |= ON_STRONG_OOP_REF; >>> >>> This doesn't seem right. Doesn't this give the wrong answer for G1? >>> >>> I'm not even sure "strong" is the right answer for ZGC in the >>> described context. >>> >>> What am I missing? >>> >> There's no way for a mutator to get hold of the referent (except via Unsafe, which we've said we don't care about anyway). The only time we would randomly step on a referent in a Finalizer is when we're doing heap iteration in ZGC. >> An alternative, to perhaps make this more explicit, would be to have an ON_FINAL_OOP_REF, but it would end up doing the same thing as ON_STRONG_OOP_REF. >> /Per From rkennke at redhat.com Wed Oct 31 17:27:15 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 31 Oct 2018 18:27:15 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() Message-ID: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> GCs (like Shenandoah) may emit additional loads or stores. In the case of Shenandoah, we emit forward pointer loads from offset -8. Those accesses don't match the pattern currently checked in Assembler::needs_explicit_null_check() which means that explicit null checks would be emitted for forwarding pointer loads. But we don't want that because we know that the base object cannot be NULL (everything else would be an error). I'm proposing to put an abstraction in needs_explicit_null_check(). I am basically trying to solve the mess that we have in Shenandoah in this place: https://builds.shipilev.net/patch-openjdk-shenandoah-jdk-only-shared/latest/src/hotspot/share/asm/assembler.cpp.sdiff.html After experimenting back-and-forth with a bunch of approaches, it seems to be most useful and least intrusive to simply put the whole method under GC's control by moving its body into BarrierSetAssembler: http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.00/src/hotspot/share/gc/shared/barrierSetAssembler.cpp.html The corresponding Shenandoah implementation for x86 would then look like this: http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.shenandoah/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp.sdiff.html As an additional bonus, this approach provides not only a GC-abstraction but also implicitely a cpu-abstraction, which helps to avoid the #ifdef mess that we made in that method. Bug: https://bugs.openjdk.java.net/browse/JDK-8213199 Full webrev: http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.00/ Testing: passes hotspot/jtreg:tier3 on x86/fastdebug Can I please get reviews on this? Also, any suggestions for improvement are very welcome! Thanks, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From kim.barrett at oracle.com Wed Oct 31 19:27:22 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 31 Oct 2018 15:27:22 -0400 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> Message-ID: <8496DA21-3DAD-4C20-BFCF-BB2EB1400AC9@oracle.com> > On Oct 31, 2018, at 1:27 PM, Roman Kennke wrote: > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8213199 > Full webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.00/ > > Testing: passes hotspot/jtreg:tier3 on x86/fastdebug > > Can I please get reviews on this? Also, any suggestions for improvement > are very welcome! > > Thanks, > Roman I think BarrierSetAssembler::needs_explicit_null_check() should be const. Other than that, looks good. From erik.osterlund at oracle.com Wed Oct 31 19:39:38 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Wed, 31 Oct 2018 20:39:38 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> Message-ID: <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> Hi Roman, Pretty sure this will break zero. Thanks, /Erik > On 31 Oct 2018, at 18:27, Roman Kennke wrote: > > GCs (like Shenandoah) may emit additional loads or stores. In the case > of Shenandoah, we emit forward pointer loads from offset -8. Those > accesses don't match the pattern currently checked in > Assembler::needs_explicit_null_check() which means that explicit null > checks would be emitted for forwarding pointer loads. But we don't want > that because we know that the base object cannot be NULL (everything > else would be an error). > > I'm proposing to put an abstraction in needs_explicit_null_check(). I am > basically trying to solve the mess that we have in Shenandoah in this place: > > https://builds.shipilev.net/patch-openjdk-shenandoah-jdk-only-shared/latest/src/hotspot/share/asm/assembler.cpp.sdiff.html > > After experimenting back-and-forth with a bunch of approaches, it seems > to be most useful and least intrusive to simply put the whole method > under GC's control by moving its body into BarrierSetAssembler: > > http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.00/src/hotspot/share/gc/shared/barrierSetAssembler.cpp.html > > The corresponding Shenandoah implementation for x86 would then look like > this: > > http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.shenandoah/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp.sdiff.html > > As an additional bonus, this approach provides not only a GC-abstraction > but also implicitely a cpu-abstraction, which helps to avoid the #ifdef > mess that we made in that method. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8213199 > Full webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.00/ > > Testing: passes hotspot/jtreg:tier3 on x86/fastdebug > > Can I please get reviews on this? Also, any suggestions for improvement > are very welcome! > > Thanks, > Roman > From manc at google.com Wed Oct 31 20:51:46 2018 From: manc at google.com (Man Cao) Date: Wed, 31 Oct 2018 13:51:46 -0700 Subject: RFR (XS): 8213113: Dead code related to UseAdaptiveSizePolicy in ParNewGeneration Message-ID: Hi all, Could anyone review this small clean up? I discovered it while working on refactoring code in adaptiveSizePolicy.hpp/cpp. Webrev: https://cr.openjdk.java.net/~manc/8213113/webrev.00/ RFE: https://bugs.openjdk.java.net/browse/JDK-8213113 Tested on submit repo with the help from JC (CCed). Thanks, Man -------------- next part -------------- An HTML attachment was scrubbed... URL: From rkennke at redhat.com Wed Oct 31 22:14:58 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 31 Oct 2018 23:14:58 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> Message-ID: <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> Hi Erik, right. Fixed this, and what what Kim mentioned plus a missing include: Incremental: http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01.diff/ Full: http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01/ Ok now? Roman > Hi Roman, > > Pretty sure this will break zero. > > Thanks, > /Erik > >> On 31 Oct 2018, at 18:27, Roman Kennke wrote: >> >> GCs (like Shenandoah) may emit additional loads or stores. In the case >> of Shenandoah, we emit forward pointer loads from offset -8. Those >> accesses don't match the pattern currently checked in >> Assembler::needs_explicit_null_check() which means that explicit null >> checks would be emitted for forwarding pointer loads. But we don't want >> that because we know that the base object cannot be NULL (everything >> else would be an error). >> >> I'm proposing to put an abstraction in needs_explicit_null_check(). I am >> basically trying to solve the mess that we have in Shenandoah in this place: >> >> https://builds.shipilev.net/patch-openjdk-shenandoah-jdk-only-shared/latest/src/hotspot/share/asm/assembler.cpp.sdiff.html >> >> After experimenting back-and-forth with a bunch of approaches, it seems >> to be most useful and least intrusive to simply put the whole method >> under GC's control by moving its body into BarrierSetAssembler: >> >> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.00/src/hotspot/share/gc/shared/barrierSetAssembler.cpp.html >> >> The corresponding Shenandoah implementation for x86 would then look like >> this: >> >> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.shenandoah/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp.sdiff.html >> >> As an additional bonus, this approach provides not only a GC-abstraction >> but also implicitely a cpu-abstraction, which helps to avoid the #ifdef >> mess that we made in that method. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8213199 >> Full webrev: >> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.00/ >> >> Testing: passes hotspot/jtreg:tier3 on x86/fastdebug >> >> Can I please get reviews on this? Also, any suggestions for improvement >> are very welcome! >> >> Thanks, >> Roman >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From kim.barrett at oracle.com Wed Oct 31 22:27:40 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 31 Oct 2018 18:27:40 -0400 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> Message-ID: <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> > On Oct 31, 2018, at 6:14 PM, Roman Kennke wrote: > > Hi Erik, > > right. Fixed this, and what what Kim mentioned plus a missing include: > > Incremental: > http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01.diff/ > Full: > http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01/ > > Ok now? Looks good.