From shade at redhat.com Mon Jul 1 08:29:41 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 1 Jul 2019 10:29:41 +0200 Subject: RFR(T) 8226957: Shenandoah: Remove obsoleted "ShenandoahStoreCheck" option In-Reply-To: References: Message-ID: <8bb9b0f1-ef2e-262b-e88f-c8c9cf4e752f@redhat.com> On 6/28/19 11:13 PM, Zhengyu Gu wrote: > The option is obsoleted. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8226957 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8226957/webrev.00/ All right, fine. It was used before for the cooperative checks in the generated code, that was removed long time ago. After that, we used to keep this flag to implement the aliased-heap verifier on top of it, but it does not seem to happen any time soon. So the flag can go. -- Thanks, -Aleksey From rkennke at redhat.com Mon Jul 1 14:51:03 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 1 Jul 2019 16:51:03 +0200 Subject: RFR: 8226822: GC interface for C1 runtime calls Message-ID: <19eb6e9c-5d5b-da71-339c-2cff003ca83e@redhat.com> Currently, it is not possible for GC to emit runtime calls in C1 without touching shared code, because it needs to be known in c1_Runtime.hpp/cpp. This patch adds a little hook to allow the GC to make runtime calls known. This is a prerequisite for: https://bugs.openjdk.java.net/browse/JDK-8226695 Bug: https://bugs.openjdk.java.net/browse/JDK-8226822 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8226822/webrev/ Can I please get reviews? Thanks, Roman From rkennke at redhat.com Mon Jul 1 15:28:37 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 1 Jul 2019 17:28:37 +0200 Subject: RFR: 8226823: Adjust BarrierSetC2 for C2 runtime calls Message-ID: In JDK-8226695 we want to emit runtime calls in C2 ideal graph that return an oop. This requires some adjustments in the GC interface for escape analysis in order to work. Specifically, we need: - Access to a bunch of stuff in ConnectionGraph that is currently private. Unfortunately we can't just befriend BarrierSetC2 and transitively its subclasses. - Apply the GC hook to EA earlier such that it can also handle calls. Bug: https://bugs.openjdk.java.net/browse/JDK-8226823 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8226823/webrev.00/ Can I please get reviews? Thanks, Roman From kim.barrett at oracle.com Mon Jul 1 22:31:08 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 1 Jul 2019 18:31:08 -0400 Subject: RFR (M): 8224741: Optimize the scan area during the Scan Heap Roots phase In-Reply-To: <7b434e2bc4dbe649a3795689073e1b9909abbdd0.camel@oracle.com> References: <7b434e2bc4dbe649a3795689073e1b9909abbdd0.camel@oracle.com> Message-ID: <7613C669-75F3-47A2-9071-EEAA46F1D129@oracle.com> > On May 24, 2019, at 11:31 AM, Thomas Schatzl wrote: > > Hi all, > > can I have reviews for this small change based on JDK-8213108 (out > for review right now) that optimizes the scanning of blocks within a > chunk (see comment of g1RemSet.cpp introduced in JDK-8213108) a bit? > > [?] > The only reason this is a separate CR is because I thought that the > change in JDK-8213108 is large enough already as is, and easier to > understand without. Thank you for that :) > CR: > https://bugs.openjdk.java.net/browse/JDK-8224741 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8224741/webrev/ > Testing: > hs-tier1-5, local jtreg gc and gc/g1 testing, perf testing > > Thanks, > Thomas ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/heapRegion.hpp 296 // Returns the address after the last actually scanned. This doesn't mention the possible NULL result when the iteration couldn't be done (only in the concurrent case). Of course, the previous comment didn't describe the return value at all. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1RemSet.cpp 923 assert(scanned_to >= mr.end(), "Scanned to " PTR_FORMAT " less than range " PTR_FORMAT, p2i(scanned_to), p2i(mr.end())); I think this assertion isn't correct. Suppose we have a humongous non-objarray object that fits in a single region. And suppose the range of dirty cards covers the entire object, and beyond (because the object's size is not a multiple of card size). The start of the scanned region includes (is) the start of the object, and the end of the scan region is beyond the end of the object. The scan of the region will hit the clause in do_oops_on_card_in_humongous that (with this change) returns obj end, which is less than mr.end(), tripping the assertion. But see next comment. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1RemSet.cpp 296 return (HeapWord*)obj + size; Maybe return the greator of end of object and end of region? Similarly to the dead case, there can't be other objects in this region, and we've scanned the entire object. Doing so would also address the assert issue above. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1RemSet.cpp 260 HeapWord* HeapRegion::do_oops_on_card_in_humongous(MemRegion mr, ... 281 if (!g1h->is_obj_dead(obj, sr)) { ... 298 } 299 // The object is dead. There can be no other object in this region, so return 300 // the end of that region. 301 return end(); The structure here has become somewhat confusing with the return value change. It used to be that we always returned true at the end, once we reached the is-dead check. But now we return different values in every clause. I think it would be easier to read if the control structure were flattened out to something like if (g1h->is_obj_dead(obj, sr)) { ... return ...; } else if (obj->is_objArray() || (sr->bottom() < mr.start())) { ... return ...; } else { ... return ...; } ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1RemSet.cpp 349 bool is_imprecise = false; I think the variable name is backward, and should be is_precise. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1RemSet.cpp 367 return is_imprecise ? mr.end() : cur; s/mr.end()/end/ ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/heapRegion.hpp 648 // Iterate over the objects overlapping part of a card, applying cl [pre-existing] Before JDK-8213108, oops_on_card_seq_iterate_careful and its helpers were applied to a MemRegion that covered no more than a single card. 8213108 changed that, now possibly applying the iteration to a MemRegion intersecting multiple cards. But comments for the iterate functions haven't been updated. ------------------------------------------------------------------------------ From shade at redhat.com Tue Jul 2 08:10:20 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 2 Jul 2019 10:10:20 +0200 Subject: RFR 8215523: jstat reports incorrect values for OU for CMS GC In-Reply-To: <4bb04022-9460-3956-cf86-447bafab0bf4@oracle.com> References: <64c5a88a-a39d-fedb-7738-998bca61b6a7@oracle.com> <4bb04022-9460-3956-cf86-447bafab0bf4@oracle.com> Message-ID: <3744fdba-2755-34fb-f0d5-76543a1faa68@redhat.com> Hi, On 6/21/19 10:30 PM, Poonam Parhar wrote: > On 6/21/19 12:21 PM, Poonam Parhar wrote: >> Bug 8215523 : jstat reports incorrect values for >> OU for CMS GC This bug is non-public, was it really meant to be? -- Thanks, -Aleksey From rkennke at redhat.com Tue Jul 2 09:32:25 2019 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 2 Jul 2019 11:32:25 +0200 Subject: RFR: JDK-8226757: Shenandoah: Make Traversal a separate mode In-Reply-To: <6d1681df-3ed9-4f25-08a4-5e2be80eb9c8@redhat.com> References: <6d1681df-3ed9-4f25-08a4-5e2be80eb9c8@redhat.com> Message-ID: There was a small mistake in one of the tests. Updated webrev: http://cr.openjdk.java.net/~rkennke/JDK-8226757/webrev.02/ Roman > Hi Aleksey, > > New webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8226757/webrev.01/ > > I addressed all issues you mentioned. I introduced a new ShenandoahMode > and 3 subclasses for traversal, normal and passive. I also had to adjust > all tests which was sometimes a bit hairy. > > Sorry, I messed up to make an incremental webrev. Only full one here. > > Better now? > > Roman > > >> Brief look: >> >> *) I wonder if it makes sense to rework it more thoroughly: >> modes: {passive, normal, traversal} >> heuristics: {aggressive, compact, adaptive, static} >> >> ...so that modes select barriers, and heuristics only selects when to start the cycle. >> >> *) should_start_normal_gc() is now simply should_start_gc()? >> >> *) I guess this can be done in "default_cause", near "default_mode": >> >> 156 cause = mode == concurrent_traversal ? GCCause::_shenandoah_traversal_gc : >> GCCause::_shenandoah_concurrent_gc; >> >> *) Looks to me, there would be no message printed if we supply wrong mode? >> >> *) I don't think it is fair to call it scavenge. Description also implies "normal" is not >> concurrent? I'd say "normal (mark, then evac, then update)" and "traversal (mark+evac+update)". >> >> 75 experimental(ccstr, ShenandoahGCMode, "normal", \ >> 76 "The GC mode to use in Shenandoah GC. Possible values" \ >> 77 " *) normal - normal GC (mark-evac-update)" \ >> 78 " *) traversal - Traversal GC (concurent scavange)") \ >> >> *) TestWrongBarrierDisable.java: indenting, L58: >> >> 54 shouldFailAll("-XX:ShenandoahGCHeuristics=adaptive", concurrent); >> 55 shouldFailAll("-XX:ShenandoahGCHeuristics=static", concurrent); >> 56 shouldFailAll("-XX:ShenandoahGCHeuristics=compact", concurrent); >> 57 shouldFailAll("-XX:ShenandoahGCHeuristics=aggressive", concurrent); >> 58 shouldFailAll("-XX:ShenandoahGCMode=traversal", traversal); >> 59 shouldPassAll("-XX:ShenandoahGCHeuristics=passive", concurrent); >> 60 shouldPassAll("-XX:ShenandoahGCHeuristics=passive", traversal); >> > From thomas.schatzl at oracle.com Tue Jul 2 10:34:06 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 02 Jul 2019 12:34:06 +0200 Subject: RFR(S): 8223575: add subspace transitions to gc+metaspace=info log lines In-Reply-To: References: Message-ID: <356397d601e11b52ac73f2520e5a3a618b9cc609.camel@oracle.com> Hi, On Wed, 2019-06-05 at 13:25 -0400, Tony Printezis wrote: > Thanks Thomas, still need to get clarification re: adding Twitter > copyright notice to the new files. I?ll report back when I have an > answer. > Some minor nits: - historically method that print log messages are prefixed with "print_" and not with "log_" like "log_metaspace_change" (which originally has also been called "print_metaspace_change"). There does not seem to be lots of reason to change this given that this method appears a few times in the vicinity of "print_" methods. For the sake of uniformity I would prefer if that would be kept; I am open to changing all of these at once (later). Just pointing out. - just a thought: the units of the output for metaspace logging are hardcoded with "K" (and previously were). It might be useful to adjust the unit as needed like other similar output. That's probably one for another CR though. - it would be nice to have a test case for this, just checking the general format of the output for with/without coops. [... snipped long text...] > > > > -- > > > > > > > > I am not sure if this needs a CSR and/or a Release Note since > > > > we change the format of the Metaspace entgc log entries and > > > > that may confuse people. If it does, I can review the CSR for > > > > you and hopefully it goes fast. I wait for the second reviewers > > > > opinion on this (preferably someone from Oracles GC group). > > > > > > I have no idea if a CSR is needed. Was there a CSR for the > > > -Xlog:safepoint* changes? I?ll assume one is not needed unless > > > I?m told otherwise. ;-) Log messages are not part of the end user interface and can be changed as needed (but should not randomly). Potentially this change is worth mentioning in a release note though if you want to get word out. Hopefully I did not overlook some important question in the very long email thread so far. Thanks, Thomas From thomas.schatzl at oracle.com Tue Jul 2 10:36:47 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 02 Jul 2019 12:36:47 +0200 Subject: RFR 8215523: jstat reports incorrect values for OU for CMS GC In-Reply-To: <3744fdba-2755-34fb-f0d5-76543a1faa68@redhat.com> References: <64c5a88a-a39d-fedb-7738-998bca61b6a7@oracle.com> <4bb04022-9460-3956-cf86-447bafab0bf4@oracle.com> <3744fdba-2755-34fb-f0d5-76543a1faa68@redhat.com> Message-ID: <4e7eee4a06416761e1ee63be678c71f1426c9adb.camel@oracle.com> Hi, On Tue, 2019-07-02 at 10:10 +0200, Aleksey Shipilev wrote: > Hi, > > On 6/21/19 10:30 PM, Poonam Parhar wrote: > > On 6/21/19 12:21 PM, Poonam Parhar wrote: > > > Bug 8215523 : > > > jstat reports incorrect values for > > > OU for CMS GC > > This bug is non-public, was it really meant to be? > there does not seem to be anything confidential in the public areas of the bug. Maybe Poonam can open it after looking at it again, and eventually open it (and add a token "Description" ;) ). Thanks, Thomas From thomas.schatzl at oracle.com Tue Jul 2 10:40:14 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 02 Jul 2019 12:40:14 +0200 Subject: RFR (M): 8224741: Optimize the scan area during the Scan Heap Roots phase In-Reply-To: <7613C669-75F3-47A2-9071-EEAA46F1D129@oracle.com> References: <7b434e2bc4dbe649a3795689073e1b9909abbdd0.camel@oracle.com> <7613C669-75F3-47A2-9071-EEAA46F1D129@oracle.com> Message-ID: <0fcd10da4ffa77651b464ab528baf2d611a0adc4.camel@oracle.com> Hi Kim, thanks for your review. On Mon, 2019-07-01 at 18:31 -0400, Kim Barrett wrote: > > On May 24, 2019, at 11:31 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > Hi all, > > > > can I have reviews for this small change based on JDK-8213108 (out > > for review right now) that optimizes the scanning of blocks within > > a chunk (see comment of g1RemSet.cpp introduced in JDK-8213108) a > > bit? > > > > [?] > > The only reason this is a separate CR is because I thought that the > > change in JDK-8213108 is large enough already as is, and easier to > > understand without. > > Thank you for that :) > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8224741 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8224741/webrev/ > > Testing: > > hs-tier1-5, local jtreg gc and gc/g1 testing, perf testing > > > > Thanks, > > Thomas > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/heapRegion.hpp > 296 // Returns the address after the last actually scanned. > > This doesn't mention the possible NULL result when the iteration > couldn't be done (only in the concurrent case). Of course, the > previous comment didn't describe the return value at all. Fixed. > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1RemSet.cpp > 923 assert(scanned_to >= mr.end(), "Scanned to " PTR_FORMAT " > less than range " PTR_FORMAT, p2i(scanned_to), p2i(mr.end())); > > I think this assertion isn't correct. > > Suppose we have a humongous non-objarray object that fits in a single > region. And suppose the range of dirty cards covers the entire > object, and beyond (because the object's size is not a multiple of > card size). The start of the scanned region includes (is) the start > of the object, and the end of the scan region is beyond the end of > the > object. The scan of the region will hit the clause in > do_oops_on_card_in_humongous that (with this change) returns obj end, > which is less than mr.end(), tripping the assertion. > > But see next comment. > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1RemSet.cpp > 296 return (HeapWord*)obj + size; > > Maybe return the greator of end of object and end of region? > Similarly to the dead case, there can't be other objects in this > region, and we've scanned the entire object. > > Doing so would also address the assert issue above. Fixed it the suggested way. > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1RemSet.cpp > 260 HeapWord* HeapRegion::do_oops_on_card_in_humongous(MemRegion mr, > ... > 281 if (!g1h->is_obj_dead(obj, sr)) { > ... > 298 } > 299 // The object is dead. There can be no other object in this > region, so return > 300 // the end of that region. > 301 return end(); > > The structure here has become somewhat confusing with the return > value change. It used to be that we always returned true at the end, > once we reached the is-dead check. But now we return different > values in every clause. I think it would be easier to read if the > control structure were flattened out to something like > > if (g1h->is_obj_dead(obj, sr)) { > ... > return ...; > } else if (obj->is_objArray() || (sr->bottom() < mr.start())) { > ... > return ...; > } else { > ... > return ...; > } Done. > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1RemSet.cpp > 349 bool is_imprecise = false; > > I think the variable name is backward, and should be is_precise. :( > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1RemSet.cpp > 367 return is_imprecise ? mr.end() : cur; > > s/mr.end()/end/ > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/heapRegion.hpp > 648 // Iterate over the objects overlapping part of a card, > applying cl > > [pre-existing] > > Before JDK-8213108, oops_on_card_seq_iterate_careful and its helpers > were applied to a MemRegion that covered no more than a single card. > 8213108 changed that, now possibly applying the iteration to a > MemRegion intersecting multiple cards. But comments for the iterate > functions haven't been updated. > Fixed. Updated webrevs: http://cr.openjdk.java.net/~tschatzl/8224741/webrev.0_to_1 (diff) http://cr.openjdk.java.net/~tschatzl/8224741/webrev.1 (full) testing: hs-tier1-3 almost done without issues Thanks, Thomas From nils.eliasson at oracle.com Tue Jul 2 13:07:54 2019 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Tue, 2 Jul 2019 15:07:54 +0200 Subject: RFR: 8226822: GC interface for C1 runtime calls In-Reply-To: <19eb6e9c-5d5b-da71-339c-2cff003ca83e@redhat.com> References: <19eb6e9c-5d5b-da71-339c-2cff003ca83e@redhat.com> Message-ID: Looks good, // Nils On 2019-07-01 16:51, Roman Kennke wrote: > Currently, it is not possible for GC to emit runtime calls in C1 > without touching shared code, because it needs to be known in > c1_Runtime.hpp/cpp. > > This patch adds a little hook to allow the GC to make runtime calls known. > > This is a prerequisite for: > https://bugs.openjdk.java.net/browse/JDK-8226695 > > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8226822 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8226822/webrev/ > > Can I please get reviews? > > Thanks, Roman > From poonam.bajaj at oracle.com Tue Jul 2 13:42:16 2019 From: poonam.bajaj at oracle.com (Poonam Parhar) Date: Tue, 2 Jul 2019 06:42:16 -0700 Subject: RFR 8215523: jstat reports incorrect values for OU for CMS GC In-Reply-To: <4e7eee4a06416761e1ee63be678c71f1426c9adb.camel@oracle.com> References: <64c5a88a-a39d-fedb-7738-998bca61b6a7@oracle.com> <4bb04022-9460-3956-cf86-447bafab0bf4@oracle.com> <3744fdba-2755-34fb-f0d5-76543a1faa68@redhat.com> <4e7eee4a06416761e1ee63be678c71f1426c9adb.camel@oracle.com> Message-ID: Hi Aleksey, Thomas, It wasn't meant to be non-public. I have opened it. Thanks, Poonam On 7/2/19 3:36 AM, Thomas Schatzl wrote: > Hi, > > On Tue, 2019-07-02 at 10:10 +0200, Aleksey Shipilev wrote: >> Hi, >> >> On 6/21/19 10:30 PM, Poonam Parhar wrote: >>> On 6/21/19 12:21 PM, Poonam Parhar wrote: >>>> Bug 8215523 : >>>> jstat reports incorrect values for >>>> OU for CMS GC >> This bug is non-public, was it really meant to be? >> > there does not seem to be anything confidential in the public areas > of the bug. Maybe Poonam can open it after looking at it again, and > eventually open it (and add a token "Description" ;) ). > > Thanks, > Thomas > From zgu at redhat.com Tue Jul 2 13:47:11 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 2 Jul 2019 09:47:11 -0400 Subject: RFR(T) 8227101: Shenandoah: Use ShenandoahSharedFlag for claimed flag in ShenandoahSerialRoot Message-ID: Please review this trivial patch, that uses ShenandoahSharedFlag for claimed flag in ShenandoahSerialRoot, to be consistent with ShenandoahWeakSerialRoot. Bug: https://bugs.openjdk.java.net/browse/JDK-8227101 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8227101/webrev.00/ Test: hotspot_gc_shenandoah (fastdebug and release) Thanks, -Zhengyu From stefan.karlsson at oracle.com Tue Jul 2 13:47:53 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 2 Jul 2019 15:47:53 +0200 Subject: RFR: 8227085: ZGC: Add on_weak load barrier verification Message-ID: Hi all, Please review this patch to add some verification to our on_weak load barriers. http://cr.openjdk.java.net/~stefank/8227085/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8227085 We've had a few bugs where on_weak load barriers have been generated instead of the normal, on_strong, load barriers. The on_weak barriers should only be used to read the referent fields of WeakReferences and SoftReferences. The given oop* is used to calculate the address of the containing object and check that it is a valid Reference object. This code was tested against the bugs in JDK-8225770 and JDK-8227083. Tested with tier 1-7 Thanks, StefanK From shade at redhat.com Tue Jul 2 13:50:53 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 2 Jul 2019 15:50:53 +0200 Subject: RFR 8215523: jstat reports incorrect values for OU for CMS GC In-Reply-To: References: <64c5a88a-a39d-fedb-7738-998bca61b6a7@oracle.com> <4bb04022-9460-3956-cf86-447bafab0bf4@oracle.com> <3744fdba-2755-34fb-f0d5-76543a1faa68@redhat.com> <4e7eee4a06416761e1ee63be678c71f1426c9adb.camel@oracle.com> Message-ID: <96ea1b19-df99-b5ba-7596-64703a99fdf6@redhat.com> On 7/2/19 3:42 PM, Poonam Parhar wrote: > It wasn't meant to be non-public. I have opened it. Thanks! -Aleksey From stefan.karlsson at oracle.com Tue Jul 2 13:53:14 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 2 Jul 2019 15:53:14 +0200 Subject: RFR: 8227086: Use AS_NO_KEEPALIVE loads in HeapDumper Message-ID: Hi all, Please review this patch to read objects with AS_NO_KEEPALIVE in the HeapDumper. http://cr.openjdk.java.net/~stefank/8227086/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8227086 Found this while running some extra verification code through our barriers. This is one place where we use ON_UNKNOWN_OOP_REF without AS_NO_KEEPALIVE. The current code isn't wrong, but we could just as well use the more light-weight load barrier here. Tested with tier 1-7 Thanks, StefanK From shade at redhat.com Tue Jul 2 13:57:46 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 2 Jul 2019 15:57:46 +0200 Subject: RFR(T) 8227101: Shenandoah: Use ShenandoahSharedFlag for claimed flag in ShenandoahSerialRoot In-Reply-To: References: Message-ID: On 7/2/19 3:47 PM, Zhengyu Gu wrote: > Please review this trivial patch, that uses ShenandoahSharedFlag for claimed flag in > ShenandoahSerialRoot, to be consistent with ShenandoahWeakSerialRoot. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8227101 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8227101/webrev.00/ Looks good. -- Thanks, -Aleksey From erik.osterlund at oracle.com Tue Jul 2 15:47:41 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 2 Jul 2019 17:47:41 +0200 Subject: RFR: 8227085: ZGC: Add on_weak load barrier verification In-Reply-To: References: Message-ID: <6a9753c1-1074-73b5-df2b-068e673e43ac@oracle.com> Hi Stefan, Looks good. Thanks, /Erik On 2019-07-02 15:47, Stefan Karlsson wrote: > Hi all, > > Please review this patch to add some verification to our on_weak load > barriers. > > http://cr.openjdk.java.net/~stefank/8227085/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8227085 > > We've had a few bugs where on_weak load barriers have been generated > instead of the normal, on_strong, load barriers. The on_weak barriers > should only be used to read the referent fields of WeakReferences and > SoftReferences. The given oop* is used to calculate the address of the > containing object and check that it is a valid Reference object. > > This code was tested against the bugs in JDK-8225770 and JDK-8227083. > > Tested with tier 1-7 > > Thanks, > StefanK From kim.barrett at oracle.com Tue Jul 2 17:06:16 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 2 Jul 2019 13:06:16 -0400 Subject: RFR (M): 8224741: Optimize the scan area during the Scan Heap Roots phase In-Reply-To: <0fcd10da4ffa77651b464ab528baf2d611a0adc4.camel@oracle.com> References: <7b434e2bc4dbe649a3795689073e1b9909abbdd0.camel@oracle.com> <7613C669-75F3-47A2-9071-EEAA46F1D129@oracle.com> <0fcd10da4ffa77651b464ab528baf2d611a0adc4.camel@oracle.com> Message-ID: <9C851B51-09D7-49D5-8237-B8CEC0A3401E@oracle.com> > On Jul 2, 2019, at 6:40 AM, Thomas Schatzl wrote: > Updated webrevs: > http://cr.openjdk.java.net/~tschatzl/8224741/webrev.0_to_1 (diff) > http://cr.openjdk.java.net/~tschatzl/8224741/webrev.1 (full) Looks good. A couple minor comment nits, and one (highly) optional suggestion. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/heapRegion.hpp Thanks for the "card" => "memregion" renamings. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/heapRegion.hpp 296 // Returns the address after the last actually scanned or NULL if the area could "the address after the last actually scanned" isn't really accurate. More accurate might be "the next unscanned address". ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/heapRegion.inline.hpp 296 // If obj is not an objArray and mr contains the start of the 297 // obj, then this could be an imprecise mark, and we need to 298 // process the entire object. 299 int size = obj->oop_iterate_size(cl); 300 return MAX2((HeapWord*)obj + size, mr.end()); Maybe mention here also that there aren't any objects after obj in the region. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/heapRegion.inline.hpp 300 return MAX2((HeapWord*)obj + size, mr.end()); obj + size could be in a region following this region. I think rather than the above result, we could instead return the end of the region containing obj + size. But I suspect this case also rarely occurs, and even then there's rarely going to be anything to be saved by that. ------------------------------------------------------------------------------ From kim.barrett at oracle.com Tue Jul 2 20:59:55 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 2 Jul 2019 16:59:55 -0400 Subject: RFR: 8224659: Parallel GC: Use WorkGang (1: PCRefProcTask) In-Reply-To: <08ed33c3-f6f7-8e59-d53c-e4f4162f8d4e@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <08ed33c3-f6f7-8e59-d53c-e4f4162f8d4e@oracle.com> Message-ID: > On Jun 28, 2019, at 12:30 PM, Leo Korinth wrote: > > Hi! > > Here are some updates on the patch series. > > http://cr.openjdk.java.net/~lkorinth/workgang/1/ > > Running some tests on them now. > > Thanks, > Leo http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224659-Parallel-GC-Use-WorkGang-1-PCRefProcTask/ http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224659-Parallel-GC-Use-WorkGang-1-PCRefProcTask-fixup-1/ This part looks good. From tprintezis at twitter.com Tue Jul 2 21:13:05 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Tue, 2 Jul 2019 14:13:05 -0700 Subject: RFR(S): 8223575: add subspace transitions to gc+metaspace=info log lines In-Reply-To: <356397d601e11b52ac73f2520e5a3a618b9cc609.camel@oracle.com> References: <356397d601e11b52ac73f2520e5a3a618b9cc609.camel@oracle.com> Message-ID: Hi Thomas, Thanks for taking a look. Please see inline. ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On July 2, 2019 at 6:34:16 AM, Thomas Schatzl (thomas.schatzl at oracle.com) wrote: Hi, On Wed, 2019-06-05 at 13:25 -0400, Tony Printezis wrote: > Thanks Thomas, still need to get clarification re: adding Twitter > copyright notice to the new files. I?ll report back when I have an > answer. > Some minor nits: - historically method that print log messages are prefixed with "print_" and not with "log_" like "log_metaspace_change" (which originally has also been called "print_metaspace_change"). There does not seem to be lots of reason to change this given that this method appears a few times in the vicinity of "print_" methods. For the sake of uniformity I would prefer if that would be kept; I am open to changing all of these at once (later). Just pointing out. OK, fair enough. Thomas (Stuefe) suggest to change it to log_ to be more consistent with what it actually does and I thought it was a good suggestion. But I can change it back. Worth doing a global rename of the print_ methods to log_ (as a separate change as you suggested)? - just a thought: the units of the output for metaspace logging are hardcoded with "K" (and previously were). It might be useful to adjust the unit as needed like other similar output. That's probably one for another CR though. I could do it as a follow-up (see below, though). I?m planning a couple of follow-up changes, to add sub-space size transitions for the young gen for parallel / serial / cms (as the log only shows the size transition of the young gen as a whole). I?m planning to re-use the HEAP_CHANGE_FORMAT macros for those changes too (which is why I put them in a shared file). Once they?re all done, we can switch them to using the other format, if you want. But, I have to say, I?m OK with leaving them as K. I almost never look at logs these days and I only look at charts / stats generated from the logs. So, the more granularity the log has, the more accurate the data is (IMHO). - it would be nice to have a test case for this, just checking the general format of the output for with/without coops. Can I do it as a follow-up (so I can parallelize the next few changes)? [... snipped long text...] > > > > -- > > > > > > > > I am not sure if this needs a CSR and/or a Release Note since > > > > we change the format of the Metaspace entgc log entries and > > > > that may confuse people. If it does, I can review the CSR for > > > > you and hopefully it goes fast. I wait for the second reviewers > > > > opinion on this (preferably someone from Oracles GC group). > > > > > > I have no idea if a CSR is needed. Was there a CSR for the > > > -Xlog:safepoint* changes? I?ll assume one is not needed unless > > > I?m told otherwise. ;-) Log messages are not part of the end user interface and can be changed as needed (but should not randomly). Potentially this change is worth mentioning in a release note though if you want to get word out. Thanks for the explanation! Yes, you?re right that we should not change log format for no reason. But, in this case, this change does add a lot of extra (and very helpful, IMHO) information. So, it?s worth it. As I said, I?m planning a couple of related changes (which I?ll finalize as soon as this is done). We can add a note to the release notes that covers all of them. Hopefully I did not overlook some important question in the very long email thread so far. The last point is the one we were unsure about! I think we?re all set. I?ll post a new webrev shortly. Tony Thanks, Thomas From tprintezis at twitter.com Tue Jul 2 21:19:18 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Tue, 2 Jul 2019 14:19:18 -0700 Subject: RFR(S): 8223575: add subspace transitions to gc+metaspace=info log lines In-Reply-To: References: <356397d601e11b52ac73f2520e5a3a618b9cc609.camel@oracle.com> Message-ID: Thomas and Thomas, Latest webrev here: http://cr.openjdk.java.net/~tonyp/8223575/webrev.4/ I reverted the method name back to print_metaspace_change(). I do have an action item to add a test for this, which I will do on a separate CR, if that?s OK. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On July 2, 2019 at 5:13:05 PM, Tony Printezis (tprintezis at twitter.com) wrote: Hi Thomas, Thanks for taking a look. Please see inline. ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On July 2, 2019 at 6:34:16 AM, Thomas Schatzl (thomas.schatzl at oracle.com) wrote: Hi, On Wed, 2019-06-05 at 13:25 -0400, Tony Printezis wrote: > Thanks Thomas, still need to get clarification re: adding Twitter > copyright notice to the new files. I?ll report back when I have an > answer. > Some minor nits: - historically method that print log messages are prefixed with "print_" and not with "log_" like "log_metaspace_change" (which originally has also been called "print_metaspace_change"). There does not seem to be lots of reason to change this given that this method appears a few times in the vicinity of "print_" methods. For the sake of uniformity I would prefer if that would be kept; I am open to changing all of these at once (later). Just pointing out. OK, fair enough. Thomas (Stuefe) suggest to change it to log_ to be more consistent with what it actually does and I thought it was a good suggestion. But I can change it back. Worth doing a global rename of the print_ methods to log_ (as a separate change as you suggested)? - just a thought: the units of the output for metaspace logging are hardcoded with "K" (and previously were). It might be useful to adjust the unit as needed like other similar output. That's probably one for another CR though. I could do it as a follow-up (see below, though). I?m planning a couple of follow-up changes, to add sub-space size transitions for the young gen for parallel / serial / cms (as the log only shows the size transition of the young gen as a whole). I?m planning to re-use the HEAP_CHANGE_FORMAT macros for those changes too (which is why I put them in a shared file). Once they?re all done, we can switch them to using the other format, if you want. But, I have to say, I?m OK with leaving them as K. I almost never look at logs these days and I only look at charts / stats generated from the logs. So, the more granularity the log has, the more accurate the data is (IMHO). - it would be nice to have a test case for this, just checking the general format of the output for with/without coops. Can I do it as a follow-up (so I can parallelize the next few changes)? [... snipped long text...] > > > > -- > > > > > > > > I am not sure if this needs a CSR and/or a Release Note since > > > > we change the format of the Metaspace entgc log entries and > > > > that may confuse people. If it does, I can review the CSR for > > > > you and hopefully it goes fast. I wait for the second reviewers > > > > opinion on this (preferably someone from Oracles GC group). > > > > > > I have no idea if a CSR is needed. Was there a CSR for the > > > -Xlog:safepoint* changes? I?ll assume one is not needed unless > > > I?m told otherwise. ;-) Log messages are not part of the end user interface and can be changed as needed (but should not randomly). Potentially this change is worth mentioning in a release note though if you want to get word out. Thanks for the explanation! Yes, you?re right that we should not change log format for no reason. But, in this case, this change does add a lot of extra (and very helpful, IMHO) information. So, it?s worth it. As I said, I?m planning a couple of related changes (which I?ll finalize as soon as this is done). We can add a note to the release notes that covers all of them. Hopefully I did not overlook some important question in the very long email thread so far. The last point is the one we were unsure about! I think we?re all set. I?ll post a new webrev shortly. Tony Thanks, Thomas From kim.barrett at oracle.com Tue Jul 2 21:39:16 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 2 Jul 2019 17:39:16 -0400 Subject: RFR: 8224660: Parallel GC: Use WorkGang (2: MarksFromRootsTask) In-Reply-To: <267462EA-DCC6-4829-966A-BB7D9C5857CA@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <90c03f6c-f5b1-bb39-f35b-e1de643ba652@oracle.com> <3FA47236-0E45-4B83-B92A-E822341F75C7@oracle.com> <0eb9e326-d675-f106-6372-5ba452516292@oracle.com> <267462EA-DCC6-4829-966A-BB7D9C5857CA@oracle.com> Message-ID: Leo and I discussed my issues with the task claiming in the first version. We agreed to use SequentialSubTasksDone in this WorkGang conversion project, rather than gating that project on fixing various issues we have with the existing subtask claiming mechanisms. We'll just squint and ignore those issues for this project, and file some RFEs. http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224660-Parallel-GC-Use-WorkGang-2-MarksFromRootsTask/ http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224660-Parallel-GC-Use-WorkGang-2-MarksFromRootsTask-fixup-1/ These look good. Just one minor naming change requested: ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psParallelCompact.cpp 2114 static void mark_from_roots_task(ParallelRootType::Value root_type, uint which) { Instead of "which", please stay with "worker_id" or "worker_i", as used in the caller and more or less everywhere else. ------------------------------------------------------------------------------ From kim.barrett at oracle.com Tue Jul 2 22:11:20 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 2 Jul 2019 18:11:20 -0400 Subject: RFR: 8227086: Use AS_NO_KEEPALIVE loads in HeapDumper In-Reply-To: References: Message-ID: > On Jul 2, 2019, at 9:53 AM, Stefan Karlsson wrote: > > Hi all, > > Please review this patch to read objects with AS_NO_KEEPALIVE in the HeapDumper. > > http://cr.openjdk.java.net/~stefank/8227086/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8227086 > > Found this while running some extra verification code through our barriers. This is one place where we use ON_UNKNOWN_OOP_REF without AS_NO_KEEPALIVE. The current code isn't wrong, but we could just as well use the more light-weight load barrier here. > > Tested with tier 1-7 > > Thanks, > StefanK Looks good. From kim.barrett at oracle.com Tue Jul 2 23:55:14 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 2 Jul 2019 19:55:14 -0400 Subject: RFR: 8224661: Parallel GC: Use WorkGang (3: UpdateDensePrefixAndCompactionTask) In-Reply-To: <1c1eff7f-5b13-5020-df2a-bb2f43813ba3@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <1c1eff7f-5b13-5020-df2a-bb2f43813ba3@oracle.com> Message-ID: > On May 27, 2019, at 4:21 AM, Leo Korinth wrote: > [?] > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8224661 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224661-Parallel-GC-Use-WorkGang-3-UpdateDensePrefixAndCompactionTask/ > http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ > > Testing (on the whole patch series): > mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC > gc test suite > > Thanks, > Leo http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224661-Parallel-GC-Use-WorkGang-3-UpdateDensePrefixAndCompactionTask/ http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224661-Parallel-GC-Use-WorkGang-3-UpdateDensePrefixAndCompactionTask-fixup-1/ Looks good. A few minor comments. ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psParallelCompact.hpp 33 #include "gc/shared/taskqueue.hpp" I don't see any changes to this file that need this #include. Presumably it's needed by some .cpp files, but they should be doing the #include. ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psParallelCompact.cpp 2467 guarantee(_backing_array != NULL, "alloc failure"); Not needed; NEW_C_HEAP_ARRAY uses EXIT_OOM for the allocation failure mode. ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psParallelCompact.cpp 2618 static void compaction_with_stealing_task(ParallelTaskTerminator* terminator, uint which) { s/which/worker_id/ ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psParallelCompact.cpp 2684 TaskQueue task_queue(last_space_id*(active_gc_threads * PAR_OLD_DENSE_PREFIX_OVER_PARTITIONING + 1)); Some breadcrumbs to explain that calculation would be really helpful. Maybe comments, or splitting up the calculation with helpful variable names, or something. Anyway, I think it's correct, but harder to verify than it could be. ------------------------------------------------------------------------------ From kim.barrett at oracle.com Wed Jul 3 01:06:37 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 2 Jul 2019 21:06:37 -0400 Subject: RFR: 8224662: Parallel GC: Use WorkGang (4: SharedRestorePreservedMarksTaskExecutor) In-Reply-To: <226f085b-6767-5ca8-a32e-76c0eea2de63@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <226f085b-6767-5ca8-a32e-76c0eea2de63@oracle.com> Message-ID: > On May 27, 2019, at 4:56 AM, Leo Korinth wrote: > [?] > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8224662 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224662-Parallel-GC-Use-WorkGang-4-SharedRestorePreservedMarksTaskExecutor/ > http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ > > Testing (on the whole patch series): > mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC > gc test suite > > Thanks, > Leo Looks good. From thomas.stuefe at gmail.com Wed Jul 3 07:23:10 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 3 Jul 2019 09:23:10 +0200 Subject: RFR(S): 8223575: add subspace transitions to gc+metaspace=info log lines In-Reply-To: References: <356397d601e11b52ac73f2520e5a3a618b9cc609.camel@oracle.com> Message-ID: Still looks good to me. Played with it and the numbers make sense. Cheers, Thomas On Tue, Jul 2, 2019 at 11:19 PM Tony Printezis wrote: > Thomas and Thomas, > > Latest webrev here: > > http://cr.openjdk.java.net/~tonyp/8223575/webrev.4/ > > I reverted the method name back to print_metaspace_change(). I do have an > action item to add a test for this, which I will do on a separate CR, if > that?s OK. > > Tony > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > On July 2, 2019 at 5:13:05 PM, Tony Printezis (tprintezis at twitter.com) > wrote: > > Hi Thomas, > > Thanks for taking a look. Please see inline. > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > On July 2, 2019 at 6:34:16 AM, Thomas Schatzl (thomas.schatzl at oracle.com) > wrote: > > Hi, > > On Wed, 2019-06-05 at 13:25 -0400, Tony Printezis wrote: > > Thanks Thomas, still need to get clarification re: adding Twitter > > copyright notice to the new files. I?ll report back when I have an > > answer. > > > > Some minor nits: > > - historically method that print log messages are prefixed with > "print_" and not with "log_" like "log_metaspace_change" (which > originally has also been called "print_metaspace_change"). There does > not seem to be lots of reason to change this given that this method > appears a few times in the vicinity of "print_" methods. > > For the sake of uniformity I would prefer if that would be kept; I am > open to changing all of these at once (later). Just pointing out. > > > OK, fair enough. Thomas (Stuefe) suggest to change it to log_ to be more > consistent with what it actually does and I thought it was a good > suggestion. But I can change it back. Worth doing a global rename of the > print_ methods to log_ (as a separate change as you suggested)? > > > > - just a thought: the units of the output for metaspace logging are > hardcoded with "K" (and previously were). It might be useful to adjust > the unit as needed like other similar output. > That's probably one for another CR though. > > > I could do it as a follow-up (see below, though). I?m planning a couple of > follow-up changes, to add sub-space size transitions for the young gen for > parallel / serial / cms (as the log only shows the size transition of the > young gen as a whole). I?m planning to re-use the HEAP_CHANGE_FORMAT macros > for those changes too (which is why I put them in a shared file). Once > they?re all done, we can switch them to using the other format, if you want. > > But, I have to say, I?m OK with leaving them as K. I almost never look at > logs these days and I only look at charts / stats generated from the logs. > So, the more granularity the log has, the more accurate the data is (IMHO). > > > > - it would be nice to have a test case for this, just checking the > general format of the output for with/without coops. > > > Can I do it as a follow-up (so I can parallelize the next few changes)? > > > > [... snipped long text...] > > > > > -- > > > > > > > > > > I am not sure if this needs a CSR and/or a Release Note since > > > > > we change the format of the Metaspace entgc log entries and > > > > > that may confuse people. If it does, I can review the CSR for > > > > > you and hopefully it goes fast. I wait for the second reviewers > > > > > opinion on this (preferably someone from Oracles GC group). > > > > > > > > I have no idea if a CSR is needed. Was there a CSR for the > > > > -Xlog:safepoint* changes? I?ll assume one is not needed unless > > > > I?m told otherwise. ;-) > > Log messages are not part of the end user interface and can be changed > as needed (but should not randomly). > Potentially this change is worth mentioning in a release note though if > you want to get word out. > > > Thanks for the explanation! Yes, you?re right that we should not change > log format for no reason. But, in this case, this change does add a lot of > extra (and very helpful, IMHO) information. So, it?s worth it. As I said, > I?m planning a couple of related changes (which I?ll finalize as soon as > this is done). We can add a note to the release notes that covers all of > them. > > > > Hopefully I did not overlook some important question in the very long > email thread so far. > > > The last point is the one we were unsure about! I think we?re all set. > I?ll post a new webrev shortly. > > > Tony > > > > > Thanks, > Thomas > > > From stefan.karlsson at oracle.com Wed Jul 3 07:39:34 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 3 Jul 2019 09:39:34 +0200 Subject: RFR: 8227086: Use AS_NO_KEEPALIVE loads in HeapDumper In-Reply-To: References: Message-ID: Thanks, Serguei. StefanK On 2019-07-02 17:57, serguei.spitsyn at oracle.com wrote: > Hi Stefan, > > It looks good. > > Thanks, > Serguei > > > > On 7/2/19 06:53, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to read objects with AS_NO_KEEPALIVE in the >> HeapDumper. >> >> http://cr.openjdk.java.net/~stefank/8227086/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8227086 >> >> Found this while running some extra verification code through our >> barriers. This is one place where we use ON_UNKNOWN_OOP_REF without >> AS_NO_KEEPALIVE. The current code isn't wrong, but we could just as >> well use the more light-weight load barrier here. >> >> Tested with tier 1-7 >> >> Thanks, >> StefanK > From stefan.karlsson at oracle.com Wed Jul 3 07:39:51 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 3 Jul 2019 09:39:51 +0200 Subject: RFR: 8227086: Use AS_NO_KEEPALIVE loads in HeapDumper In-Reply-To: References: Message-ID: Thanks, Kim. StefanK On 2019-07-03 00:11, Kim Barrett wrote: >> On Jul 2, 2019, at 9:53 AM, Stefan Karlsson wrote: >> >> Hi all, >> >> Please review this patch to read objects with AS_NO_KEEPALIVE in the HeapDumper. >> >> http://cr.openjdk.java.net/~stefank/8227086/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8227086 >> >> Found this while running some extra verification code through our barriers. This is one place where we use ON_UNKNOWN_OOP_REF without AS_NO_KEEPALIVE. The current code isn't wrong, but we could just as well use the more light-weight load barrier here. >> >> Tested with tier 1-7 >> >> Thanks, >> StefanK > Looks good. > From stefan.karlsson at oracle.com Wed Jul 3 08:07:52 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 3 Jul 2019 10:07:52 +0200 Subject: RFR: 8227085: ZGC: Add on_weak load barrier verification In-Reply-To: <6a9753c1-1074-73b5-df2b-068e673e43ac@oracle.com> References: <6a9753c1-1074-73b5-df2b-068e673e43ac@oracle.com> Message-ID: <035e2f86-7567-a79b-da57-45336ef12aa7@oracle.com> Thanks, Erik. StefanK On 2019-07-02 17:47, Erik ?sterlund wrote: > Hi Stefan, > > Looks good. > > Thanks, > /Erik > > On 2019-07-02 15:47, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to add some verification to our on_weak load >> barriers. >> >> http://cr.openjdk.java.net/~stefank/8227085/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8227085 >> >> We've had a few bugs where on_weak load barriers have been generated >> instead of the normal, on_strong, load barriers. The on_weak barriers >> should only be used to read the referent fields of WeakReferences and >> SoftReferences. The given oop* is used to calculate the address of >> the containing object and check that it is a valid Reference object. >> >> This code was tested against the bugs in JDK-8225770 and JDK-8227083. >> >> Tested with tier 1-7 >> >> Thanks, >> StefanK > From thomas.schatzl at oracle.com Wed Jul 3 08:29:10 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 03 Jul 2019 10:29:10 +0200 Subject: RFR(S): 8223575: add subspace transitions to gc+metaspace=info log lines In-Reply-To: References: <356397d601e11b52ac73f2520e5a3a618b9cc609.camel@oracle.com> Message-ID: <050c10ce07a57f3f73de2c3d27ed2a75d0df5a6e.camel@oracle.com> Hi, On Tue, 2019-07-02 at 14:19 -0700, Tony Printezis wrote: > Thomas and Thomas, > > Latest webrev here: > > http://cr.openjdk.java.net/~tonyp/8223575/webrev.4/ > > I reverted the method name back to print_metaspace_change(). I do > have an action item to add a test for this, which I will do on a > separate CR, if that?s OK. > Okay. > Tony > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > On July 2, 2019 at 5:13:05 PM, Tony Printezis (tprintezis at twitter.com > ) wrote: > > Hi Thomas, > > > > Thanks for taking a look. Please see inline. > > > > > > ????? > > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > > > > On July 2, 2019 at 6:34:16 AM, Thomas Schatzl ( > > thomas.schatzl at oracle.com) wrote: > > > Hi, > > > > > > On Wed, 2019-06-05 at 13:25 -0400, Tony Printezis wrote: > > > > Thanks Thomas, still need to get clarification re: adding > > > > Twitter copyright notice to the new files. I?ll report back > > > > when I have an answer. > > > > > > > > > > Some minor nits: > > > > > > - historically method that print log messages are prefixed with > > > "print_" and not with "log_" like "log_metaspace_change" (which > > > originally has also been called "print_metaspace_change"). There > > > does not seem to be lots of reason to change this given that this > > > method appears a few times in the vicinity of "print_" methods. > > > > > > For the sake of uniformity I would prefer if that would be kept; > > > I am open to changing all of these at once (later). Just pointing > > > out. > > > > OK, fair enough. Thomas (Stuefe) suggest to change it to log_ to be > > more consistent with what it actually does and I thought it was a > > good suggestion. But I can change it back. Worth doing a global > > rename of the print_ methods to log_ (as a separate change as you > > suggested)? >From my POV, not :) > > > > > - just a thought: the units of the output for metaspace logging > > > are hardcoded with "K" (and previously were). It might be useful > > > to adjust the unit as needed like other similar output. > > > That's probably one for another CR though. > > > > I could do it as a follow-up (see below, though). I?m planning a > > couple of follow-up changes, to add sub-space size transitions for > > the young gen for parallel / serial / cms (as the log only shows > > the size transition of the young gen as a whole). I?m planning to > > re-use the HEAP_CHANGE_FORMAT macros for those changes too (which > > is why I put them in a shared file). Once they?re all done, we can > > switch them to using the other format, if you want. > > But, I have to say, I?m OK with leaving them as K. I almost never > > look at logs these days and I only look at charts / stats generated > > from the logs. So, the more granularity the log has, the more > > accurate the data is (IMHO). Fine with me. This has merely been a suggestion. > > > > > - it would be nice to have a test case for this, just checking > > > the general format of the output for with/without coops. > > > > Can I do it as a follow-up (so I can parallelize the next few > > changes)? Okay. > > > > > > Log messages are not part of the end user interface and can be > > > changed as needed (but should not randomly). > > > Potentially this change is worth mentioning in a release note > > > though if you want to get word out. > > > > Thanks for the explanation! Yes, you?re right that we should not > > change log format for no reason. But, in this case, this change > > does add a lot of extra (and very helpful, IMHO) information. So, > > it?s worth it. As I said, I?m planning a couple of related changes > > (which I?ll finalize as soon as this is done). We can add a note to > > the release notes that covers all of them. I am totally fine with changing the log output as necessary. The release note is optional too. > > > > > Hopefully I did not overlook some important question in the very > > > long email thread so far. > > > > The last point is the one we were unsure about! I think we?re all > > set. I?ll post a new webrev shortly. > > Looks good. Thomas From shade at redhat.com Wed Jul 3 08:33:10 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 3 Jul 2019 10:33:10 +0200 Subject: RFR: JDK-8226757: Shenandoah: Make Traversal a separate mode In-Reply-To: References: <6d1681df-3ed9-4f25-08a4-5e2be80eb9c8@redhat.com> Message-ID: On 7/2/19 11:32 AM, Roman Kennke wrote: > There was a small mistake in one of the tests. Updated webrev: > > http://cr.openjdk.java.net/~rkennke/JDK-8226757/webrev.02/ *) Looks like the synopsis is incorrect. Should be e.g. "Make passive/normal/traversal modes explicit", or something. *) Please make sure it builds with --disable-precompiled-headers *) Changes like these are not exactly for making a separate mode? The condition seems superficial even without explicit mode: ShenandoahSATBBarrier should not be active with traversal? If so, might be worthwhile to split out, this would make current changeset a bit less denser. - if (ShenandoahSATBBarrier && !dest_uninitialized && !ShenandoahHeap::heap()->heuristics()->can_do_traversal_gc()) { + if (ShenandoahSATBBarrier && !dest_uninitialized) { *) Rewrite this: GCMode default_mode = heap->is_traversal_mode() ? concurrent_traversal : concurrent_normal; GCCause::Cause default_cause = default_mode == concurrent_traversal ? GCCause::_shenandoah_traversal_gc : GCCause::_shenandoah_concurrent_gc; Into this: GCMode default_mode = heap->is_traversal_mode() ? concurrent_traversal : concurrent_normal; GCCause::Cause default_cause = heap->is_traversal_mode() ? GCCause::_shenandoah_traversal_gc : GCCause::_shenandoah_concurrent_gc; *) shenandoah_globals.hpp still mention "passive" is heuristics, not mode. Happens in both ShenandoahGCHeuristics and ShenandoahGCMode option descriptions. *) TestObjItrWithHeapDump.java, TestClassLoaderLeak.java: indenting is confusing. Suggestion: String[][][] modeHeuristics = new String[][][] { {{"normal"}, {"adaptive", "compact", "static", "aggressive"}}, {{"traversal"}, {"adaptive"}}, {{"passive"}, {"passive"}} }; ...by the way, shouldn't "traversal" mode have "traversal" heuristics here? I see shenandoahTraversalMode equate "adaptive" to ShenandoahTraversalHeuristics, which I believe is confusing... *) @run lines indenting is not correct in TestRefprocSanity.java -- Thanks, -Aleksey From thomas.schatzl at oracle.com Wed Jul 3 09:17:34 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 03 Jul 2019 11:17:34 +0200 Subject: RFR (M): 8224741: Optimize the scan area during the Scan Heap Roots phase In-Reply-To: <9C851B51-09D7-49D5-8237-B8CEC0A3401E@oracle.com> References: <7b434e2bc4dbe649a3795689073e1b9909abbdd0.camel@oracle.com> <7613C669-75F3-47A2-9071-EEAA46F1D129@oracle.com> <0fcd10da4ffa77651b464ab528baf2d611a0adc4.camel@oracle.com> <9C851B51-09D7-49D5-8237-B8CEC0A3401E@oracle.com> Message-ID: <579b0c4c3bbeaf5b533d1d41c45aa851bd127474.camel@oracle.com> Hi, thanks for your review. On Tue, 2019-07-02 at 13:06 -0400, Kim Barrett wrote: > > On Jul 2, 2019, at 6:40 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > Updated webrevs: > > http://cr.openjdk.java.net/~tschatzl/8224741/webrev.0_to_1 (diff) > > http://cr.openjdk.java.net/~tschatzl/8224741/webrev.1 (full) > > Looks good. > > A couple minor comment nits, and one (highly) optional suggestion. > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/heapRegion.hpp > > Thanks for the "card" => "memregion" renamings. > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/heapRegion.hpp > 296 // Returns the address after the last actually scanned or NULL > if the area could > > "the address after the last actually scanned" isn't really accurate. > More accurate might be "the next unscanned address". > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/heapRegion.inline.hpp > 296 // If obj is not an objArray and mr contains the start of > the > 297 // obj, then this could be an imprecise mark, and we need to > 298 // process the entire object. > 299 int size = obj->oop_iterate_size(cl); > 300 return MAX2((HeapWord*)obj + size, mr.end()); > > Maybe mention here also that there aren't any objects after obj in > the region. Fixed. > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/heapRegion.inline.hpp > 300 return MAX2((HeapWord*)obj + size, mr.end()); > > obj + size could be in a region following this region. I think > rather > than the above result, we could instead return the end of the region > containing obj + size. But I suspect this case also rarely occurs, > and even then there's rarely going to be anything to be saved by > that. I kept the code as is. The result will most likely exceed the current chunk to avoid any further processing anyway. This does not seem to be worthwhile to optimize. http://cr.openjdk.java.net/~tschatzl/8224741/webrev.1_to_2 (diff) http://cr.openjdk.java.net/~tschatzl/8224741/webrev.2 (full) (These are only comment adjustments as suggested here, so no testing) Thanks, Thomas From thomas.schatzl at oracle.com Wed Jul 3 09:27:01 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 03 Jul 2019 11:27:01 +0200 Subject: RFR: 8224659: Parallel GC: Use WorkGang (1: PCRefProcTask) In-Reply-To: References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <08ed33c3-f6f7-8e59-d53c-e4f4162f8d4e@oracle.com> Message-ID: <8c98f9f3e489e92cd5773ff4b418f13494e0afd8.camel@oracle.com> Hi, On Tue, 2019-07-02 at 16:59 -0400, Kim Barrett wrote: > > On Jun 28, 2019, at 12:30 PM, Leo Korinth > > wrote: > > > > Hi! > > > > Here are some updates on the patch series. > > > > http://cr.openjdk.java.net/~lkorinth/workgang/1/ > > > > Running some tests on them now. > > > > Thanks, > > Leo > > http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224659-Parallel-GC-Use-WorkGang-1-PCRefProcTask/ > http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224659-Parallel-GC-Use-WorkGang-1-PCRefProcTask-fixup-1/ > > This part looks good. > some (very) minor nits. Sorry for all of them being indentation issues: parallelScavengeHeap.hpp:line 100+ - the indentation is broken, i.e. extra space. This is kind-of pre-existing, but the change continued with the broken indentation. psParallelCompact.cpp:1793: the arguments to the call to WorkerPolicy::calc_active_workers() should be indented according to it, not the outer method. An easy way to avoid all code moving to the right would be introducing a local temporary variable. E.g. it may also be useful to put the expression ParallelScavengeHeap::heap()->workers() into a local variable to avoid excessive spacing. psScavenge.cpp:342: same as above Looks good otherwise. Thanks, Thomas From shade at redhat.com Wed Jul 3 09:31:05 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 3 Jul 2019 11:31:05 +0200 Subject: RFR (S) 8224881: Shenandoah: trashing "Collection Set, Pinned" region during Degenerated GC In-Reply-To: <4146efa7-8165-c294-f889-92724e8d1e8f@redhat.com> References: <4146efa7-8165-c294-f889-92724e8d1e8f@redhat.com> Message-ID: <8e2151f0-f6a3-f5e3-f12d-3b6cde629be8@redhat.com> On 6/12/19 1:10 PM, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8224881 > > The failure is extremely hard to reproduce: I spent many hours trying, to no avail. It requires a > few unlikely things to happen at once. I managed to reproduce it a few times locally with the large > test, and the fix seems to help it. So, I would like to push the fix and see if it reproduces in the > upcoming months. > > Fix: > http://cr.openjdk.java.net/~shade/8224881/webrev.01/ Upon review, I realized this got into jdk/jdk after fork to jdk13: http://hg.openjdk.java.net/jdk/jdk/rev/2c47220ce9bb I would like to have it in jdk13 too. Okay to push to jdk/jdk13 too? -- Thanks, -Aleksey From thomas.schatzl at oracle.com Wed Jul 3 09:35:49 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 03 Jul 2019 11:35:49 +0200 Subject: RFR: 8224660: Parallel GC: Use WorkGang (2: MarksFromRootsTask) In-Reply-To: References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <90c03f6c-f5b1-bb39-f35b-e1de643ba652@oracle.com> <3FA47236-0E45-4B83-B92A-E822341F75C7@oracle.com> <0eb9e326-d675-f106-6372-5ba452516292@oracle.com> <267462EA-DCC6-4829-966A-BB7D9C5857CA@oracle.com> Message-ID: <101ed34f5409527fd26fb7570254af332c02df0a.camel@oracle.com> Hi, On Tue, 2019-07-02 at 17:39 -0400, Kim Barrett wrote: > Leo and I discussed my issues with the task claiming in the first > version. We agreed to use SequentialSubTasksDone in this WorkGang > conversion project, rather than gating that project on fixing various > issues we have with the existing subtask claiming mechanisms. We'll > just squint and ignore those issues for this project, and file some > RFEs. > > http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224660-Parallel-GC-Use-WorkGang-2-MarksFromRootsTask/ > http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224660-Parallel-GC-Use-WorkGang-2-MarksFromRootsTask-fixup-1/ > > These look good. > > Just one minor naming change requested: > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/parallel/psParallelCompact.cpp > 2114 static void mark_from_roots_task(ParallelRootType::Value > root_type, uint which) { > > Instead of "which", please stay with "worker_id" or "worker_i", as > used in the caller and more or less everywhere else. > Please use "worker_id". Almost everything is using "worker_id" by now, so let's avoid "worker_i" creeping back in. Looks good. No need to re-review for this renaming. Thanks, Thomas From leo.korinth at oracle.com Wed Jul 3 09:44:15 2019 From: leo.korinth at oracle.com (Leo Korinth) Date: Wed, 3 Jul 2019 11:44:15 +0200 Subject: RFR: 8224659: Parallel GC: Use WorkGang (1: PCRefProcTask) In-Reply-To: <8c98f9f3e489e92cd5773ff4b418f13494e0afd8.camel@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <08ed33c3-f6f7-8e59-d53c-e4f4162f8d4e@oracle.com> <8c98f9f3e489e92cd5773ff4b418f13494e0afd8.camel@oracle.com> Message-ID: On 03/07/2019 11:27, Thomas Schatzl wrote: > Hi, > > On Tue, 2019-07-02 at 16:59 -0400, Kim Barrett wrote: >>> On Jun 28, 2019, at 12:30 PM, Leo Korinth >>> wrote: >>> >>> Hi! >>> >>> Here are some updates on the patch series. >>> >>> http://cr.openjdk.java.net/~lkorinth/workgang/1/ >>> >>> Running some tests on them now. >>> >>> Thanks, >>> Leo >> >> > http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224659-Parallel-GC-Use-WorkGang-1-PCRefProcTask/ >> > http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224659-Parallel-GC-Use-WorkGang-1-PCRefProcTask-fixup-1/ >> >> This part looks good. >> > > some (very) minor nits. Sorry for all of them being indentation > issues: > > parallelScavengeHeap.hpp:line 100+ - the indentation is broken, i.e. > extra space. This is kind-of pre-existing, but the change continued > with the broken indentation. Will fix. > > psParallelCompact.cpp:1793: the arguments to the call to > WorkerPolicy::calc_active_workers() should be indented according to it, > not the outer method. > > An easy way to avoid all code moving to the right would be introducing > a local temporary variable. > > E.g. it may also be useful to put the expression > ParallelScavengeHeap::heap()->workers() into a local variable to avoid > excessive spacing. > > psScavenge.cpp:342: same as above Will fix (probably by indenting more if that is okay). > > Looks good otherwise. > > Thanks, > Thomas > Thanks Kim and Thomas, Leo From leo.korinth at oracle.com Wed Jul 3 09:49:10 2019 From: leo.korinth at oracle.com (Leo Korinth) Date: Wed, 3 Jul 2019 11:49:10 +0200 Subject: RFR: 8224660: Parallel GC: Use WorkGang (2: MarksFromRootsTask) In-Reply-To: <101ed34f5409527fd26fb7570254af332c02df0a.camel@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <90c03f6c-f5b1-bb39-f35b-e1de643ba652@oracle.com> <3FA47236-0E45-4B83-B92A-E822341F75C7@oracle.com> <0eb9e326-d675-f106-6372-5ba452516292@oracle.com> <267462EA-DCC6-4829-966A-BB7D9C5857CA@oracle.com> <101ed34f5409527fd26fb7570254af332c02df0a.camel@oracle.com> Message-ID: On 03/07/2019 11:35, Thomas Schatzl wrote: > Hi, > > On Tue, 2019-07-02 at 17:39 -0400, Kim Barrett wrote: >> Leo and I discussed my issues with the task claiming in the first >> version. We agreed to use SequentialSubTasksDone in this WorkGang >> conversion project, rather than gating that project on fixing various >> issues we have with the existing subtask claiming mechanisms. We'll >> just squint and ignore those issues for this project, and file some >> RFEs. >> >> > http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224660-Parallel-GC-Use-WorkGang-2-MarksFromRootsTask/ >> > http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224660-Parallel-GC-Use-WorkGang-2-MarksFromRootsTask-fixup-1/ >> >> These look good. >> >> Just one minor naming change requested: >> >> ------------------------------------------------------------------- >> ----------- >> src/hotspot/share/gc/parallel/psParallelCompact.cpp >> 2114 static void mark_from_roots_task(ParallelRootType::Value >> root_type, uint which) { >> >> Instead of "which", please stay with "worker_id" or "worker_i", as >> used in the caller and more or less everywhere else. >> > > Please use "worker_id". Almost everything is using "worker_id" by now, > so let's avoid "worker_i" creeping back in. > > Looks good. No need to re-review for this renaming. > > Thanks, > Thomas > I will fix to using worker_id in all places. My reason for using "which" was not to change the old code, but I agree that it ought to be fixed. Thanks, Leo From rkennke at redhat.com Wed Jul 3 09:49:15 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 3 Jul 2019 11:49:15 +0200 Subject: RFR (S) 8224881: Shenandoah: trashing "Collection Set, Pinned" region during Degenerated GC In-Reply-To: <8e2151f0-f6a3-f5e3-f12d-3b6cde629be8@redhat.com> References: <4146efa7-8165-c294-f889-92724e8d1e8f@redhat.com> <8e2151f0-f6a3-f5e3-f12d-3b6cde629be8@redhat.com> Message-ID: <46c4b62a-3962-0cd9-b637-a090be18bfc5@redhat.com> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8224881 >> >> The failure is extremely hard to reproduce: I spent many hours trying, to no avail. It requires a >> few unlikely things to happen at once. I managed to reproduce it a few times locally with the large >> test, and the fix seems to help it. So, I would like to push the fix and see if it reproduces in the >> upcoming months. >> >> Fix: >> http://cr.openjdk.java.net/~shade/8224881/webrev.01/ > > Upon review, I realized this got into jdk/jdk after fork to jdk13: > http://hg.openjdk.java.net/jdk/jdk/rev/2c47220ce9bb > > I would like to have it in jdk13 too. Okay to push to jdk/jdk13 too? Good to me. Roman From leo.korinth at oracle.com Wed Jul 3 11:46:45 2019 From: leo.korinth at oracle.com (Leo Korinth) Date: Wed, 3 Jul 2019 13:46:45 +0200 Subject: RFR: 8224661: Parallel GC: Use WorkGang (3: UpdateDensePrefixAndCompactionTask) In-Reply-To: References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <1c1eff7f-5b13-5020-df2a-bb2f43813ba3@oracle.com> Message-ID: <9652bd11-819f-661d-9099-bf4b72a98d3e@oracle.com> On 03/07/2019 01:55, Kim Barrett wrote: >> On May 27, 2019, at 4:21 AM, Leo Korinth wrote: >> [?] >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8224661 >> >> Webrev: >> http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224661-Parallel-GC-Use-WorkGang-3-UpdateDensePrefixAndCompactionTask/ >> http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ >> >> Testing (on the whole patch series): >> mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC >> gc test suite >> >> Thanks, >> Leo > > http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224661-Parallel-GC-Use-WorkGang-3-UpdateDensePrefixAndCompactionTask/ > http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224661-Parallel-GC-Use-WorkGang-3-UpdateDensePrefixAndCompactionTask-fixup-1/ > > Looks good. > > A few minor comments. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psParallelCompact.hpp > 33 #include "gc/shared/taskqueue.hpp" > > I don't see any changes to this file that need this #include. > Presumably it's needed by some .cpp files, but they should be doing > the #include. Old artifact from older version where I used the shared task queue, thanks for finding this, I will remove it as it is not needed. > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psParallelCompact.cpp > 2467 guarantee(_backing_array != NULL, "alloc failure"); > > Not needed; NEW_C_HEAP_ARRAY uses EXIT_OOM for the allocation failure > mode. Good catch, copied from other code that did it, will fix. I have (also) created https://bugs.openjdk.java.net/browse/JDK-8227168 to remove the source from where I got it. And fix some related problems. > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psParallelCompact.cpp > 2618 static void compaction_with_stealing_task(ParallelTaskTerminator* terminator, uint which) { > > s/which/worker_id/ will fix. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psParallelCompact.cpp > 2684 TaskQueue task_queue(last_space_id*(active_gc_threads * PAR_OLD_DENSE_PREFIX_OVER_PARTITIONING + 1)); > > Some breadcrumbs to explain that calculation would be really helpful. > Maybe comments, or splitting up the calculation with helpful variable > names, or something. will add some comment. > > Anyway, I think it's correct, but harder to verify than it could be. > > ------------------------------------------------------------------------------ > Thanks, Leo From tprintezis at twitter.com Wed Jul 3 14:06:57 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Wed, 3 Jul 2019 07:06:57 -0700 Subject: RFR(S): 8223575: add subspace transitions to gc+metaspace=info log lines In-Reply-To: <050c10ce07a57f3f73de2c3d27ed2a75d0df5a6e.camel@oracle.com> References: <356397d601e11b52ac73f2520e5a3a618b9cc609.camel@oracle.com> <050c10ce07a57f3f73de2c3d27ed2a75d0df5a6e.camel@oracle.com> Message-ID: Hi Thomas, I created JDK-8227179: "Test for new gc+metaspace=info output format? and I?ll start working on it today. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On July 3, 2019 at 4:29:21 AM, Thomas Schatzl (thomas.schatzl at oracle.com) wrote: Hi, On Tue, 2019-07-02 at 14:19 -0700, Tony Printezis wrote: > Thomas and Thomas, > > Latest webrev here: > > http://cr.openjdk.java.net/~tonyp/8223575/webrev.4/ > > I reverted the method name back to print_metaspace_change(). I do > have an action item to add a test for this, which I will do on a > separate CR, if that?s OK. > Okay. > Tony > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > On July 2, 2019 at 5:13:05 PM, Tony Printezis (tprintezis at twitter.com > ) wrote: > > Hi Thomas, > > > > Thanks for taking a look. Please see inline. > > > > > > ????? > > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > > > > On July 2, 2019 at 6:34:16 AM, Thomas Schatzl ( > > thomas.schatzl at oracle.com) wrote: > > > Hi, > > > > > > On Wed, 2019-06-05 at 13:25 -0400, Tony Printezis wrote: > > > > Thanks Thomas, still need to get clarification re: adding > > > > Twitter copyright notice to the new files. I?ll report back > > > > when I have an answer. > > > > > > > > > > Some minor nits: > > > > > > - historically method that print log messages are prefixed with > > > "print_" and not with "log_" like "log_metaspace_change" (which > > > originally has also been called "print_metaspace_change"). There > > > does not seem to be lots of reason to change this given that this > > > method appears a few times in the vicinity of "print_" methods. > > > > > > For the sake of uniformity I would prefer if that would be kept; > > > I am open to changing all of these at once (later). Just pointing > > > out. > > > > OK, fair enough. Thomas (Stuefe) suggest to change it to log_ to be > > more consistent with what it actually does and I thought it was a > > good suggestion. But I can change it back. Worth doing a global > > rename of the print_ methods to log_ (as a separate change as you > > suggested)? >From my POV, not :) > > > > > - just a thought: the units of the output for metaspace logging > > > are hardcoded with "K" (and previously were). It might be useful > > > to adjust the unit as needed like other similar output. > > > That's probably one for another CR though. > > > > I could do it as a follow-up (see below, though). I?m planning a > > couple of follow-up changes, to add sub-space size transitions for > > the young gen for parallel / serial / cms (as the log only shows > > the size transition of the young gen as a whole). I?m planning to > > re-use the HEAP_CHANGE_FORMAT macros for those changes too (which > > is why I put them in a shared file). Once they?re all done, we can > > switch them to using the other format, if you want. > > But, I have to say, I?m OK with leaving them as K. I almost never > > look at logs these days and I only look at charts / stats generated > > from the logs. So, the more granularity the log has, the more > > accurate the data is (IMHO). Fine with me. This has merely been a suggestion. > > > > > - it would be nice to have a test case for this, just checking > > > the general format of the output for with/without coops. > > > > Can I do it as a follow-up (so I can parallelize the next few > > changes)? Okay. > > > > > > Log messages are not part of the end user interface and can be > > > changed as needed (but should not randomly). > > > Potentially this change is worth mentioning in a release note > > > though if you want to get word out. > > > > Thanks for the explanation! Yes, you?re right that we should not > > change log format for no reason. But, in this case, this change > > does add a lot of extra (and very helpful, IMHO) information. So, > > it?s worth it. As I said, I?m planning a couple of related changes > > (which I?ll finalize as soon as this is done). We can add a note to > > the release notes that covers all of them. I am totally fine with changing the log output as necessary. The release note is optional too. > > > > > Hopefully I did not overlook some important question in the very > > > long email thread so far. > > > > The last point is the one we were unsure about! I think we?re all > > set. I?ll post a new webrev shortly. > > Looks good. Thomas From tprintezis at twitter.com Wed Jul 3 14:08:19 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Wed, 3 Jul 2019 07:08:19 -0700 Subject: RFR(S): 8223575: add subspace transitions to gc+metaspace=info log lines In-Reply-To: References: <356397d601e11b52ac73f2520e5a3a618b9cc609.camel@oracle.com> Message-ID: Thanks Thomas! ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On July 3, 2019 at 3:23:22 AM, Thomas St?fe (thomas.stuefe at gmail.com) wrote: Still looks good to me. Played with it and the numbers make sense. Cheers, Thomas On Tue, Jul 2, 2019 at 11:19 PM Tony Printezis wrote: > Thomas and Thomas, > > Latest webrev here: > > http://cr.openjdk.java.net/~tonyp/8223575/webrev.4/ > > I reverted the method name back to print_metaspace_change(). I do have an > action item to add a test for this, which I will do on a separate CR, if > that?s OK. > > Tony > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > On July 2, 2019 at 5:13:05 PM, Tony Printezis (tprintezis at twitter.com) > wrote: > > Hi Thomas, > > Thanks for taking a look. Please see inline. > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > On July 2, 2019 at 6:34:16 AM, Thomas Schatzl (thomas.schatzl at oracle.com) > wrote: > > Hi, > > On Wed, 2019-06-05 at 13:25 -0400, Tony Printezis wrote: > > Thanks Thomas, still need to get clarification re: adding Twitter > > copyright notice to the new files. I?ll report back when I have an > > answer. > > > > Some minor nits: > > - historically method that print log messages are prefixed with > "print_" and not with "log_" like "log_metaspace_change" (which > originally has also been called "print_metaspace_change"). There does > not seem to be lots of reason to change this given that this method > appears a few times in the vicinity of "print_" methods. > > For the sake of uniformity I would prefer if that would be kept; I am > open to changing all of these at once (later). Just pointing out. > > > OK, fair enough. Thomas (Stuefe) suggest to change it to log_ to be more > consistent with what it actually does and I thought it was a good > suggestion. But I can change it back. Worth doing a global rename of the > print_ methods to log_ (as a separate change as you suggested)? > > > > - just a thought: the units of the output for metaspace logging are > hardcoded with "K" (and previously were). It might be useful to adjust > the unit as needed like other similar output. > That's probably one for another CR though. > > > I could do it as a follow-up (see below, though). I?m planning a couple of > follow-up changes, to add sub-space size transitions for the young gen for > parallel / serial / cms (as the log only shows the size transition of the > young gen as a whole). I?m planning to re-use the HEAP_CHANGE_FORMAT macros > for those changes too (which is why I put them in a shared file). Once > they?re all done, we can switch them to using the other format, if you want. > > But, I have to say, I?m OK with leaving them as K. I almost never look at > logs these days and I only look at charts / stats generated from the logs. > So, the more granularity the log has, the more accurate the data is (IMHO). > > > > - it would be nice to have a test case for this, just checking the > general format of the output for with/without coops. > > > Can I do it as a follow-up (so I can parallelize the next few changes)? > > > > [... snipped long text...] > > > > > -- > > > > > > > > > > I am not sure if this needs a CSR and/or a Release Note since > > > > > we change the format of the Metaspace entgc log entries and > > > > > that may confuse people. If it does, I can review the CSR for > > > > > you and hopefully it goes fast. I wait for the second reviewers > > > > > opinion on this (preferably someone from Oracles GC group). > > > > > > > > I have no idea if a CSR is needed. Was there a CSR for the > > > > -Xlog:safepoint* changes? I?ll assume one is not needed unless > > > > I?m told otherwise. ;-) > > Log messages are not part of the end user interface and can be changed > as needed (but should not randomly). > Potentially this change is worth mentioning in a release note though if > you want to get word out. > > > Thanks for the explanation! Yes, you?re right that we should not change > log format for no reason. But, in this case, this change does add a lot of > extra (and very helpful, IMHO) information. So, it?s worth it. As I said, > I?m planning a couple of related changes (which I?ll finalize as soon as > this is done). We can add a note to the release notes that covers all of > them. > > > > Hopefully I did not overlook some important question in the very long > email thread so far. > > > The last point is the one we were unsure about! I think we?re all set. > I?ll post a new webrev shortly. > > > Tony > > > > > Thanks, > Thomas > > > From poonam.bajaj at oracle.com Wed Jul 3 14:24:23 2019 From: poonam.bajaj at oracle.com (Poonam Parhar) Date: Wed, 3 Jul 2019 07:24:23 -0700 Subject: RFR 8227178: Backout of 8215523 Message-ID: Hello, Please review the changeset that reverses the changes done with JDK-8215523 . JDK-8227178 : Backout of 8215523 webrev: http://cr.openjdk.java.net/~poonam/8227178/webrev.00/ The fix for JDK-8215523 is causing a lot of timeout failures for CMS tests. We are backing out that change for now. Thanks, Poonam From thomas.schatzl at oracle.com Wed Jul 3 14:29:53 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 03 Jul 2019 16:29:53 +0200 Subject: RFR 8227178: Backout of 8215523 In-Reply-To: References: Message-ID: Hi, On Wed, 2019-07-03 at 07:24 -0700, Poonam Parhar wrote: > Hello, > > Please review the changeset that reverses the changes done with > JDK-8215523 . > > JDK-8227178 : > Backout > of 8215523 > webrev: http://cr.openjdk.java.net/~poonam/8227178/webrev.00/ > > The fix for JDK-8215523 is causing a lot of timeout failures for CMS > tests. We are backing out that change for now. > looks like a clean anti-delta. Ship it. Thanks, Thomas From serguei.spitsyn at oracle.com Wed Jul 3 14:32:01 2019 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 3 Jul 2019 07:32:01 -0700 Subject: RFR 8227178: Backout of 8215523 In-Reply-To: References: Message-ID: <74d92e53-0240-f361-148b-426035bdbf54@oracle.com> Hi Poonam, +1 Thanks, Serguei On 7/3/19 07:29, Thomas Schatzl wrote: > Hi, > > On Wed, 2019-07-03 at 07:24 -0700, Poonam Parhar wrote: >> Hello, >> >> Please review the changeset that reverses the changes done with >> JDK-8215523 . >> >> JDK-8227178 : >> Backout >> of 8215523 >> webrev: http://cr.openjdk.java.net/~poonam/8227178/webrev.00/ >> >> The fix for JDK-8215523 is causing a lot of timeout failures for CMS >> tests. We are backing out that change for now. >> > looks like a clean anti-delta. Ship it. > > Thanks, > Thomas > From rkennke at redhat.com Wed Jul 3 15:35:31 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 3 Jul 2019 17:35:31 +0200 Subject: RFR: JDK-8227199: Shenandoah: Remove superfluous calls to can_do_traversal_gc() Message-ID: <7ea0429f-5b84-ef47-fc17-c53799c78dbe@redhat.com> We've got a few calls to can_do_traversal_gc() in assembler which are superfluous because ShenandoahSATBBarrier is already checked and covers the traversal case. Let's remove them. Bug: https://bugs.openjdk.java.net/browse/JDK-8227199 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8227199/webrev.00/ Testing: hotspot_gc_shenandoah Ok? Roman From shade at redhat.com Wed Jul 3 15:39:28 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 3 Jul 2019 17:39:28 +0200 Subject: RFR: JDK-8227199: Shenandoah: Remove superfluous calls to can_do_traversal_gc() In-Reply-To: <7ea0429f-5b84-ef47-fc17-c53799c78dbe@redhat.com> References: <7ea0429f-5b84-ef47-fc17-c53799c78dbe@redhat.com> Message-ID: <45facff5-113c-7f59-4007-8cb6c61ee995@redhat.com> On 7/3/19 5:35 PM, Roman Kennke wrote: > We've got a few calls to can_do_traversal_gc() in assembler which are > superfluous because ShenandoahSATBBarrier is already checked and covers > the traversal case. Let's remove them. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8227199 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8227199/webrev.00/ OK. Thanks! -Aleksey From rkennke at redhat.com Wed Jul 3 16:16:34 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 3 Jul 2019 18:16:34 +0200 Subject: RFR: JDK-8226757: Shenandoah: Make Traversal a separate mode In-Reply-To: References: <6d1681df-3ed9-4f25-08a4-5e2be80eb9c8@redhat.com> Message-ID: Hi Aleksey, Thanks for reviewing! >> There was a small mistake in one of the tests. Updated webrev: >> >> http://cr.openjdk.java.net/~rkennke/JDK-8226757/webrev.02/ > *) Looks like the synopsis is incorrect. Should be e.g. "Make passive/normal/traversal modes > explicit", or something. Changed it to: "Shenandoah: Make traversal and passive modes explicit" > *) Please make sure it builds with --disable-precompiled-headers Done. Required logging includes. > *) Changes like these are not exactly for making a separate mode? The condition seems superficial > even without explicit mode: ShenandoahSATBBarrier should not be active with traversal? If so, might > be worthwhile to split out, this would make current changeset a bit less denser. > > - if (ShenandoahSATBBarrier && !dest_uninitialized && > !ShenandoahHeap::heap()->heuristics()->can_do_traversal_gc()) { > + if (ShenandoahSATBBarrier && !dest_uninitialized) { Split out and pushed under: https://bugs.openjdk.java.net/browse/JDK-8227199 > *) Rewrite this: > > GCMode default_mode = heap->is_traversal_mode() ? concurrent_traversal : concurrent_normal; > GCCause::Cause default_cause = default_mode == concurrent_traversal ? > GCCause::_shenandoah_traversal_gc : GCCause::_shenandoah_concurrent_gc; > > Into this: > > GCMode default_mode = heap->is_traversal_mode() ? > concurrent_traversal : concurrent_normal; > GCCause::Cause default_cause = heap->is_traversal_mode() ? > GCCause::_shenandoah_traversal_gc : GCCause::_shenandoah_concurrent_gc; Done. > *) shenandoah_globals.hpp still mention "passive" is heuristics, not mode. Happens in both > ShenandoahGCHeuristics and ShenandoahGCMode option descriptions. Fixed. > *) TestObjItrWithHeapDump.java, TestClassLoaderLeak.java: indenting is confusing. Suggestion: > > String[][][] modeHeuristics = new String[][][] { > {{"normal"}, {"adaptive", "compact", "static", "aggressive"}}, > {{"traversal"}, {"adaptive"}}, > {{"passive"}, {"passive"}} > }; Done. > ...by the way, shouldn't "traversal" mode have "traversal" heuristics here? I see > shenandoahTraversalMode equate "adaptive" to ShenandoahTraversalHeuristics, which I believe is > confusing... Well, the default traversal heuristics mimics the normal adaptive heuristics as much as possible, and it is also adaptive. IMO it makes sense to call it adaptive heuristics. I am about to also add an aggressive heuristic to traversal mode. I'd rather keep it. > *) @run lines indenting is not correct in TestRefprocSanity.java Fixed. Incremental: http://cr.openjdk.java.net/~rkennke/JDK-8226757/webrev.03.diff/ Full: http://cr.openjdk.java.net/~rkennke/JDK-8226757/webrev.03/ Good? Roman From tprintezis at twitter.com Wed Jul 3 19:57:50 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Wed, 3 Jul 2019 12:57:50 -0700 Subject: RFR (S): 8227179: Test for new gc+metaspace=info output format Message-ID: Hi all, This is a follow-up to: 8223575: add subspace transitions to gc+metaspace=info log lines (that I just pushed) and adds a test to sanity check the gc+metaspace=info output. Webrev here: http://cr.openjdk.java.net/~tonyp/8227179/webrev.0/ Thanks, Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com From kim.barrett at oracle.com Wed Jul 3 21:06:18 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 3 Jul 2019 17:06:18 -0400 Subject: RFR (M): 8224741: Optimize the scan area during the Scan Heap Roots phase In-Reply-To: <579b0c4c3bbeaf5b533d1d41c45aa851bd127474.camel@oracle.com> References: <7b434e2bc4dbe649a3795689073e1b9909abbdd0.camel@oracle.com> <7613C669-75F3-47A2-9071-EEAA46F1D129@oracle.com> <0fcd10da4ffa77651b464ab528baf2d611a0adc4.camel@oracle.com> <9C851B51-09D7-49D5-8237-B8CEC0A3401E@oracle.com> <579b0c4c3bbeaf5b533d1d41c45aa851bd127474.camel@oracle.com> Message-ID: > On Jul 3, 2019, at 5:17 AM, Thomas Schatzl wrote: >> src/hotspot/share/gc/g1/heapRegion.inline.hpp >> 296 // If obj is not an objArray and mr contains the start of >> the >> 297 // obj, then this could be an imprecise mark, and we need to >> 298 // process the entire object. >> 299 int size = obj->oop_iterate_size(cl); >> 300 return MAX2((HeapWord*)obj + size, mr.end()); >> >> Maybe mention here also that there aren't any objects after obj in >> the region. > > Fixed. 300 // There are no objects after this humongous object in the region, so we 301 // can return the end of the object. That?s not quite what I had in mind. I was thinking something like We've scanned to the end of the object, but since there are no objects after this humongous object in the region, we can return the end of the region of that's greater. >> src/hotspot/share/gc/g1/heapRegion.inline.hpp >> 300 return MAX2((HeapWord*)obj + size, mr.end()); >> >> obj + size could be in a region following this region. I think >> rather >> than the above result, we could instead return the end of the region >> containing obj + size. But I suspect this case also rarely occurs, >> and even then there's rarely going to be anything to be saved by >> that. > > I kept the code as is. The result will most likely exceed the current > chunk to avoid any further processing anyway. This does not seem to be > worthwhile to optimize. Agreed. I think it could only (possibly) help if there are stale cards in the last region. > http://cr.openjdk.java.net/~tschatzl/8224741/webrev.1_to_2 (diff) > http://cr.openjdk.java.net/~tschatzl/8224741/webrev.2 (full) > > (These are only comment adjustments as suggested here, so no testing) Looks good, other than the above suggested further comment tweak. From kim.barrett at oracle.com Thu Jul 4 01:46:03 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 3 Jul 2019 21:46:03 -0400 Subject: RFR[trivial]: 8226793: Replace OopStorage dup_name with os::strdup Message-ID: <57090ED6-BC48-468E-9447-919C28C93A9F@oracle.com> Please review this trivial change to OopStorage's name handling. Rather than bespoke duplicatation (with dup_name unintentionally having external linkage), it should just be using os::strdup. CR: https://bugs.openjdk.java.net/browse/JDK-8226793 Webrev: http://cr.openjdk.java.net/~kbarrett/8226793/open.00/ Testing: mach5 tier1 From thomas.schatzl at oracle.com Thu Jul 4 08:30:39 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 04 Jul 2019 10:30:39 +0200 Subject: RFR[trivial]: 8226793: Replace OopStorage dup_name with os::strdup In-Reply-To: <57090ED6-BC48-468E-9447-919C28C93A9F@oracle.com> References: <57090ED6-BC48-468E-9447-919C28C93A9F@oracle.com> Message-ID: <53fed72b1d7f5fd471bc9df848376921ca96fbae.camel@oracle.com> Hi, On Wed, 2019-07-03 at 21:46 -0400, Kim Barrett wrote: > Please review this trivial change to OopStorage's name handling. > Rather than bespoke duplicatation (with dup_name unintentionally > having external linkage), it should just be using os::strdup. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8226793 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8226793/open.00/ > > Testing: > mach5 tier1 > looks good and trivial. Thomas From shade at redhat.com Thu Jul 4 08:38:46 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 4 Jul 2019 10:38:46 +0200 Subject: RFR: JDK-8226757: Shenandoah: Make Traversal a separate mode In-Reply-To: References: <6d1681df-3ed9-4f25-08a4-5e2be80eb9c8@redhat.com> Message-ID: <6f409488-82c3-fd0c-8872-afd36165e0a0@redhat.com> On 7/3/19 6:16 PM, Roman Kennke wrote: >> GCMode default_mode = heap->is_traversal_mode() ? >> concurrent_traversal : concurrent_normal; >> GCCause::Cause default_cause = heap->is_traversal_mode() ? >> GCCause::_shenandoah_traversal_gc : GCCause::_shenandoah_concurrent_gc; > > Done. Note heap->is_traversal_mode() in the second statement. There is no need to introduce superficial dependency between two statements by using default_mode in the second one. >> *) shenandoah_globals.hpp still mention "passive" is heuristics, not mode. Happens in both >> ShenandoahGCHeuristics and ShenandoahGCMode option descriptions. > > Fixed. Not accurate, because we would mostly run degen GC in this mode: 78 " *) passive - do not start concurrent GC, wait for Full GC; ") \ Let's say "passive - disable concurrent GC, do stop-the-world GC". >> ...by the way, shouldn't "traversal" mode have "traversal" heuristics here? I see >> shenandoahTraversalMode equate "adaptive" to ShenandoahTraversalHeuristics, which I believe is >> confusing... > > Well, the default traversal heuristics mimics the normal adaptive > heuristics as much as possible, and it is also adaptive. IMO it makes > sense to call it adaptive heuristics. I am about to also add an > aggressive heuristic to traversal mode. I'd rather keep it. Right, make sense. > Full: > http://cr.openjdk.java.net/~rkennke/JDK-8226757/webrev.03/ Looks good, modulo changes above. -- Thanks, -Aleksey From thomas.schatzl at oracle.com Thu Jul 4 09:28:50 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 04 Jul 2019 11:28:50 +0200 Subject: RFR (M): 8224741: Optimize the scan area during the Scan Heap Roots phase In-Reply-To: References: <7b434e2bc4dbe649a3795689073e1b9909abbdd0.camel@oracle.com> <7613C669-75F3-47A2-9071-EEAA46F1D129@oracle.com> <0fcd10da4ffa77651b464ab528baf2d611a0adc4.camel@oracle.com> <9C851B51-09D7-49D5-8237-B8CEC0A3401E@oracle.com> <579b0c4c3bbeaf5b533d1d41c45aa851bd127474.camel@oracle.com> Message-ID: Hi Kim, On Wed, 2019-07-03 at 17:06 -0400, Kim Barrett wrote: > > On Jul 3, 2019, at 5:17 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > src/hotspot/share/gc/g1/heapRegion.inline.hpp > > > 296 // If obj is not an objArray and mr contains the start of > > > the > > > 297 // obj, then this could be an imprecise mark, and we need > > > to > > > 298 // process the entire object. > > > 299 int size = obj->oop_iterate_size(cl); > > > 300 return MAX2((HeapWord*)obj + size, mr.end()); > > > > > > Maybe mention here also that there aren't any objects after obj > > > in the region. > > > > Fixed. > > 300 // There are no objects after this humongous object in the > region, so we > 301 // can return the end of the object. > > That?s not quite what I had in mind. I was thinking something like > > We've scanned to the end of the object, but since there are no > objects after this humongous object in the region, we can return the > end of the region of that's greater. Fixed according to your suggestion. > > > http://cr.openjdk.java.net/~tschatzl/8224741/webrev.1_to_2 (diff) > > http://cr.openjdk.java.net/~tschatzl/8224741/webrev.2 (full) > > > > (These are only comment adjustments as suggested here, so no > > testing) > > Looks good, other than the above suggested further comment tweak. Leo also found a superfluous size_t cast that I removed in the webrevs below: http://cr.openjdk.java.net/~tschatzl/8224741/webrev.2_to_3 (diff) http://cr.openjdk.java.net/~tschatzl/8224741/webrev.3 (full) Thanks, Thomas From leo.korinth at oracle.com Thu Jul 4 10:06:29 2019 From: leo.korinth at oracle.com (Leo Korinth) Date: Thu, 4 Jul 2019 12:06:29 +0200 Subject: RFR (M): 8224741: Optimize the scan area during the Scan Heap Roots phase In-Reply-To: References: <7b434e2bc4dbe649a3795689073e1b9909abbdd0.camel@oracle.com> <7613C669-75F3-47A2-9071-EEAA46F1D129@oracle.com> <0fcd10da4ffa77651b464ab528baf2d611a0adc4.camel@oracle.com> <9C851B51-09D7-49D5-8237-B8CEC0A3401E@oracle.com> <579b0c4c3bbeaf5b533d1d41c45aa851bd127474.camel@oracle.com> Message-ID: Looks great. Thanks, Leo On 04/07/2019 11:28, Thomas Schatzl wrote: > Hi Kim, > > On Wed, 2019-07-03 at 17:06 -0400, Kim Barrett wrote: >>> On Jul 3, 2019, at 5:17 AM, Thomas Schatzl < >>> thomas.schatzl at oracle.com> wrote: >>>> src/hotspot/share/gc/g1/heapRegion.inline.hpp >>>> 296 // If obj is not an objArray and mr contains the start of >>>> the >>>> 297 // obj, then this could be an imprecise mark, and we need >>>> to >>>> 298 // process the entire object. >>>> 299 int size = obj->oop_iterate_size(cl); >>>> 300 return MAX2((HeapWord*)obj + size, mr.end()); >>>> >>>> Maybe mention here also that there aren't any objects after obj >>>> in the region. >>> >>> Fixed. >> >> 300 // There are no objects after this humongous object in the >> region, so we >> 301 // can return the end of the object. >> >> That?s not quite what I had in mind. I was thinking something like >> >> We've scanned to the end of the object, but since there are no >> objects after this humongous object in the region, we can return the >> end of the region of that's greater. > > Fixed according to your suggestion. > >> >>> http://cr.openjdk.java.net/~tschatzl/8224741/webrev.1_to_2 (diff) >>> http://cr.openjdk.java.net/~tschatzl/8224741/webrev.2 (full) >>> >>> (These are only comment adjustments as suggested here, so no >>> testing) >> >> Looks good, other than the above suggested further comment tweak. > > Leo also found a superfluous size_t cast that I removed in the webrevs > below: > > http://cr.openjdk.java.net/~tschatzl/8224741/webrev.2_to_3 (diff) > http://cr.openjdk.java.net/~tschatzl/8224741/webrev.3 (full) > > > Thanks, > Thomas > From martin.doerr at sap.com Thu Jul 4 10:35:10 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 4 Jul 2019 10:35:10 +0000 Subject: RFR(XS): 8226302: Test failures on IBM platforms (power and s/390) after JDK-8223837 Message-ID: Hi, I'd like to fix the heap size related issues described in https://bugs.openjdk.java.net/browse/JDK-8226302 by adjusting the sizes. I prefer this simple approach for jdk13. We could change the tests such that they determine appropriate sizes depending on the HeapAlignment (retrieved by whitebox api) for jdk14. Please review: http://cr.openjdk.java.net/~mdoerr/8226302_TestHeapSize/webrev.00/ Best regards, Martin From rwestrel at redhat.com Thu Jul 4 11:08:24 2019 From: rwestrel at redhat.com (Roland Westrelin) Date: Thu, 04 Jul 2019 13:08:24 +0200 Subject: RFR: 8226823: Adjust BarrierSetC2 for C2 runtime calls In-Reply-To: References: Message-ID: <87k1cy83br.fsf@redhat.com> > http://cr.openjdk.java.net/~rkennke/JDK-8226823/webrev.00/ Looks good to me. Roland. From rwestrel at redhat.com Thu Jul 4 11:10:54 2019 From: rwestrel at redhat.com (Roland Westrelin) Date: Thu, 04 Jul 2019 13:10:54 +0200 Subject: RFR: 8226822: GC interface for C1 runtime calls In-Reply-To: <19eb6e9c-5d5b-da71-339c-2cff003ca83e@redhat.com> References: <19eb6e9c-5d5b-da71-339c-2cff003ca83e@redhat.com> Message-ID: <87h882837l.fsf@redhat.com> > http://cr.openjdk.java.net/~rkennke/JDK-8226822/webrev/ Looks good to me. Roland. From rkennke at redhat.com Thu Jul 4 12:29:22 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 4 Jul 2019 14:29:22 +0200 Subject: RFR: JDK-8226757: Shenandoah: Make Traversal a separate mode In-Reply-To: <6f409488-82c3-fd0c-8872-afd36165e0a0@redhat.com> References: <6d1681df-3ed9-4f25-08a4-5e2be80eb9c8@redhat.com> <6f409488-82c3-fd0c-8872-afd36165e0a0@redhat.com> Message-ID: Hi Aleksey, >>> GCMode default_mode = heap->is_traversal_mode() ? >>> concurrent_traversal : concurrent_normal; >>> GCCause::Cause default_cause = heap->is_traversal_mode() ? >>> GCCause::_shenandoah_traversal_gc : GCCause::_shenandoah_concurrent_gc; >> >> Done. > > Note heap->is_traversal_mode() in the second statement. There is no need to introduce superficial > dependency between two statements by using default_mode in the second one. > >>> *) shenandoah_globals.hpp still mention "passive" is heuristics, not mode. Happens in both >>> ShenandoahGCHeuristics and ShenandoahGCMode option descriptions. >> >> Fixed. > > Not accurate, because we would mostly run degen GC in this mode: > 78 " *) passive - do not start concurrent GC, wait for Full GC; ") \ > > Let's say "passive - disable concurrent GC, do stop-the-world GC". > > >>> ...by the way, shouldn't "traversal" mode have "traversal" heuristics here? I see >>> shenandoahTraversalMode equate "adaptive" to ShenandoahTraversalHeuristics, which I believe is >>> confusing... >> >> Well, the default traversal heuristics mimics the normal adaptive >> heuristics as much as possible, and it is also adaptive. IMO it makes >> sense to call it adaptive heuristics. I am about to also add an >> aggressive heuristic to traversal mode. I'd rather keep it. > > Right, make sense. > >> Full: >> http://cr.openjdk.java.net/~rkennke/JDK-8226757/webrev.03/ > > Looks good, modulo changes above. Did the changes and pushed. Thanks! Roman From nils.eliasson at oracle.com Thu Jul 4 13:27:19 2019 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Thu, 4 Jul 2019 15:27:19 +0200 Subject: [13] RFR(S): 8226536: Catch OOM from deopt that fails rematerializing objects Message-ID: Hi, This is a patch of GarbageUtils.java used by a bunch of GC tests. This issue was diagnosed by Stefan Karlsson and passed to the compiler team. The core problem is that the test relies on successfully catching OOM errors. But the OOM causes a deopt, and the deopt must rematerialize some objects before passing the execution on to the interpreter to run the catch-block. If the rematerialization can't allocate, it will pass the OOM on, without having run the catch-block. With this patch I fix the tests by wrapping the faulting code with an extra layer of try-catch-OOM. It relies on not inlining the inner method, because then it would be part of the compiled frame being deoptimized. In general this is a problem for any kind of deopt when we are out of heap. Another bug will be opened to keep tracking this issue. Webrev:http://cr.openjdk.java.net/~neliasso/8226536/webrev.02/ Bug: https://bugs.openjdk.java.net/browse/JDK-8226536 // Nils From leo.korinth at oracle.com Thu Jul 4 15:21:44 2019 From: leo.korinth at oracle.com (Leo Korinth) Date: Thu, 4 Jul 2019 17:21:44 +0200 Subject: RFR: 8224659: Parallel GC: Use WorkGang (1: PCRefProcTask) In-Reply-To: <08ed33c3-f6f7-8e59-d53c-e4f4162f8d4e@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <08ed33c3-f6f7-8e59-d53c-e4f4162f8d4e@oracle.com> Message-ID: Hi! Latest fixes at http://cr.openjdk.java.net/~lkorinth/workgang/2 , not tested, but compiles. I will run testing when everything is fixed. In addition to your comments I added _subtasks.all_tasks_completed(). Thanks, Leo On 28/06/2019 18:30, Leo Korinth wrote: Hi! Here are some updates on the patch series. > > http://cr.openjdk.java.net/~lkorinth/workgang/1/ > > Running some tests on them now. > > Thanks, > Leo > > From kim.barrett at oracle.com Thu Jul 4 15:40:04 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 4 Jul 2019 11:40:04 -0400 Subject: RFR[trivial]: 8226793: Replace OopStorage dup_name with os::strdup In-Reply-To: <53fed72b1d7f5fd471bc9df848376921ca96fbae.camel@oracle.com> References: <57090ED6-BC48-468E-9447-919C28C93A9F@oracle.com> <53fed72b1d7f5fd471bc9df848376921ca96fbae.camel@oracle.com> Message-ID: <67448B23-07BB-4E45-B5AA-8BA3CE27CBAA@oracle.com> > On Jul 4, 2019, at 4:30 AM, Thomas Schatzl wrote: > > Hi, > > On Wed, 2019-07-03 at 21:46 -0400, Kim Barrett wrote: >> Please review this trivial change to OopStorage's name handling. >> Rather than bespoke duplicatation (with dup_name unintentionally >> having external linkage), it should just be using os::strdup. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8226793 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8226793/open.00/ >> >> Testing: >> mach5 tier1 >> > > looks good and trivial. > > Thomas Thanks. From kim.barrett at oracle.com Thu Jul 4 15:42:31 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 4 Jul 2019 11:42:31 -0400 Subject: RFR (M): 8224741: Optimize the scan area during the Scan Heap Roots phase In-Reply-To: References: <7b434e2bc4dbe649a3795689073e1b9909abbdd0.camel@oracle.com> <7613C669-75F3-47A2-9071-EEAA46F1D129@oracle.com> <0fcd10da4ffa77651b464ab528baf2d611a0adc4.camel@oracle.com> <9C851B51-09D7-49D5-8237-B8CEC0A3401E@oracle.com> <579b0c4c3bbeaf5b533d1d41c45aa851bd127474.camel@oracle.com> Message-ID: > On Jul 4, 2019, at 5:28 AM, Thomas Schatzl wrote: >>> http://cr.openjdk.java.net/~tschatzl/8224741/webrev.1_to_2 (diff) >>> http://cr.openjdk.java.net/~tschatzl/8224741/webrev.2 (full) >>> >>> (These are only comment adjustments as suggested here, so no >>> testing) >> >> Looks good, other than the above suggested further comment tweak. > > Leo also found a superfluous size_t cast that I removed in the webrevs > below: > > http://cr.openjdk.java.net/~tschatzl/8224741/webrev.2_to_3 (diff) > http://cr.openjdk.java.net/~tschatzl/8224741/webrev.3 (full) > > > Thanks, > Thomas Ship it! From rkennke at redhat.com Thu Jul 4 18:37:02 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 4 Jul 2019 20:37:02 +0200 Subject: RFR: JDK-8226695: Shenandoah: Wire C1 and C2 IN_NATIVE barrier Message-ID: IN_NATIVE accesses in compiled code should go to runtime, in order to be able to check reachability of accessed objects, and possibly offer NULL for unreachable objects. Bug: https://bugs.openjdk.java.net/browse/JDK-8226695 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8226695/webrev.00/ Testing: hotspot_gc_dev Ok? Roman From rkennke at redhat.com Thu Jul 4 18:43:27 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 4 Jul 2019 20:43:27 +0200 Subject: RFR: JDK-8227039: Shenandoah: Aggressive heuristics for Traversal mode Message-ID: We have aggressive heuristic in normal/default mode which is great for testing by running GC continuosly and using large/maximum collection set. We should add similar heuristics for Traversal mode in order to ramp up testing of it. The traversal-aggressive heuristic mimics the normal-aggressive: it runs GCs back-to-back and selects any region with garbage into the cset. I added traversal-aggressive test configs whereever normal-aggressive test configs are present. Bug: https://bugs.openjdk.java.net/browse/JDK-8227039 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8227039/webrev.00/ Testing: hotspot_gc_shenandoah Can I please get reviews? Thanks, Roman From shade at redhat.com Thu Jul 4 18:59:06 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 4 Jul 2019 20:59:06 +0200 Subject: RFR: JDK-8227039: Shenandoah: Aggressive heuristics for Traversal mode In-Reply-To: References: Message-ID: On 7/4/19 8:43 PM, Roman Kennke wrote: > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8227039/webrev.00/ *) Indenting at L52: 49 if (strcmp(ShenandoahGCHeuristics, "adaptive") == 0) { 50 return new ShenandoahTraversalHeuristics(); 51 } else if (strcmp(ShenandoahGCHeuristics, "aggressive") == 0) { 52 return new ShenandoahTraversalAggressiveHeuristics(); 53 } else { *) I'd say in test configuration the mode should come first, then heuristics. See for example: http://cr.openjdk.java.net/~rkennke/JDK-8227039/webrev.00/test/hotspot/jtreg/gc/shenandoah/TestGCThreadGroups.java.sdiff.html Otherwise looks fine. -- Thanks, -Aleksey From kim.barrett at oracle.com Fri Jul 5 04:50:21 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 5 Jul 2019 00:50:21 -0400 Subject: RFR: 8162929: Enqueuing dirty cards into a single DCQS during GC does not scale Message-ID: Please review this refactoring of G1DirtyCardQueue[Set], splitting into two sets of classes that each better support the purpose they are used for. The existing G1DirtyCardQueue[Set] is (and remains) the mechanism for communicating between the logging post-write barrier and the remset, via refinement (both concurrent and STW). The new G1RedirtyCardsQueue[Set] are used during collections to collect and en masse dump old remset information back into that cycle for later reconsideration. There is a G1RedirtyCardsQueueSet associated with the collector object, replacing the second (non-refinement related) G1DirtyCardQueueSet. During certain collection phases we now allocate G1RedirtyCardsQueue objects local to worker threads, with the collected buffers sent to that new qset. Because the collection and processing of those buffers happen in distinct non-overlapping phases, we can easily use a lock-free stack as the qset's completed buffer list. We don't even need to track the number of buffers. This splitting allowed a little bit of immediate simplification of G1DirtyCardQueueSet. It will also make it easier to make future improvements in that area. Although the CR describes a performance problem, it seems that other improvements in the intervening 3 years have substantially reduced the described problems. (In particular, substantially fewer cards go through this process.) On various performance tests this change has not shown significant improvements, though also no regressions. It might be that with some workloads and with enough worker threads this change could still have significant performance benefit though. But the main point of making this change now is the resultant code simplification. CR: https://bugs.openjdk.java.net/browse/JDK-8162929 Webrev: http://cr.openjdk.java.net/~kbarrett/8162929/open.00/ Testing: mach5 tier1-5 aurora perf tests for G1 locally BigRamTester From rkennke at redhat.com Fri Jul 5 08:59:33 2019 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 5 Jul 2019 10:59:33 +0200 Subject: RFR: JDK-8227039: Shenandoah: Aggressive heuristics for Traversal mode In-Reply-To: References: Message-ID: <755cf0e0-10df-201b-97b4-688fbe561a21@redhat.com> >> Webrev: >> http://cr.openjdk.java.net/~rkennke/JDK-8227039/webrev.00/ > > *) Indenting at L52: > > 49 if (strcmp(ShenandoahGCHeuristics, "adaptive") == 0) { > 50 return new ShenandoahTraversalHeuristics(); > 51 } else if (strcmp(ShenandoahGCHeuristics, "aggressive") == 0) { > 52 return new ShenandoahTraversalAggressiveHeuristics(); > 53 } else { Good catch! Fixed! > *) I'd say in test configuration the mode should come first, then heuristics. See for example: > > http://cr.openjdk.java.net/~rkennke/JDK-8227039/webrev.00/test/hotspot/jtreg/gc/shenandoah/TestGCThreadGroups.java.sdiff.html Right! Fixed it there and all other instances that I could find. Incremental: http://cr.openjdk.java.net/~rkennke/JDK-8227039/webrev.01/ Full: http://cr.openjdk.java.net/~rkennke/JDK-8227039/webrev.01.full/ Testing: same as always ;-) Ok? Roman From shade at redhat.com Fri Jul 5 09:05:28 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 5 Jul 2019 11:05:28 +0200 Subject: RFR: JDK-8227039: Shenandoah: Aggressive heuristics for Traversal mode In-Reply-To: <755cf0e0-10df-201b-97b4-688fbe561a21@redhat.com> References: <755cf0e0-10df-201b-97b4-688fbe561a21@redhat.com> Message-ID: <22b90f86-9507-febd-7e0c-7dfe752d230d@redhat.com> On 7/5/19 10:59 AM, Roman Kennke wrote: > Incremental: > http://cr.openjdk.java.net/~rkennke/JDK-8227039/webrev.01/ > Full: > http://cr.openjdk.java.net/~rkennke/JDK-8227039/webrev.01.full/ Looks good to me. -- Thanks, -Aleksey From nils.eliasson at oracle.com Fri Jul 5 09:42:17 2019 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Fri, 5 Jul 2019 11:42:17 +0200 Subject: [TESTBUG] Re: [13] RFR(S): 8226536: Catch OOM from deopt that fails rematerializing objects In-Reply-To: References: Message-ID: Adding the TESTBUG label in hope of getting some more attention. This bug is creating a lot of noise in our testing. // Nils On 2019-07-04 15:27, Nils Eliasson wrote: > Hi, > > This is a patch of GarbageUtils.java used by a bunch of GC tests. > > This issue was diagnosed by Stefan Karlsson and passed to the compiler > team. > > The core problem is that the test relies on successfully catching OOM > errors. But the OOM causes a deopt, and the deopt must rematerialize > some objects before passing the execution on to the interpreter to run > the catch-block. If the rematerialization can't allocate, it will pass > the OOM on, without having run the catch-block. > > With this patch I fix the tests by wrapping the faulting code with an > extra layer of try-catch-OOM. It relies on not inlining the inner > method, because then it would be part of the compiled frame being > deoptimized. > > In general this is a problem for any kind of deopt when we are out of > heap. Another bug will be opened to keep tracking this issue. > > Webrev:http://cr.openjdk.java.net/~neliasso/8226536/webrev.02/ > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8226536 > > // Nils > From erik.osterlund at oracle.com Fri Jul 5 09:57:59 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 5 Jul 2019 11:57:59 +0200 Subject: [13] RFR(S): 8226536: Catch OOM from deopt that fails rematerializing objects In-Reply-To: References: Message-ID: <72937448-36ab-a1a2-2d03-4cc1c8f6f957@oracle.com> Hi Nils, Looks good. Thanks, /Erik On 2019-07-04 15:27, Nils Eliasson wrote: > Hi, > > This is a patch of GarbageUtils.java used by a bunch of GC tests. > > This issue was diagnosed by Stefan Karlsson and passed to the compiler > team. > > The core problem is that the test relies on successfully catching OOM > errors. But the OOM causes a deopt, and the deopt must rematerialize > some objects before passing the execution on to the interpreter to run > the catch-block. If the rematerialization can't allocate, it will pass > the OOM on, without having run the catch-block. > > With this patch I fix the tests by wrapping the faulting code with an > extra layer of try-catch-OOM. It relies on not inlining the inner > method, because then it would be part of the compiled frame being > deoptimized. > > In general this is a problem for any kind of deopt when we are out of > heap. Another bug will be opened to keep tracking this issue. > > Webrev:http://cr.openjdk.java.net/~neliasso/8226536/webrev.02/ > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8226536 > > // Nils > From vladimir.x.ivanov at oracle.com Fri Jul 5 10:00:47 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 5 Jul 2019 13:00:47 +0300 Subject: [13] RFR(S): 8226536: Catch OOM from deopt that fails rematerializing objects In-Reply-To: References: Message-ID: <94bfdef5-5cbf-86e9-93ce-ed7d900adc76@oracle.com> test/hotspot/jtreg/vmTestbase/nsk/share/gc/gp/GarbageUtils.java: + * It is Important that the impl is not inlined. What do you think about putting @DontInline on eatMemoryImpl to guarantee that invariant? Best regards, Vladimir Ivanov On 04/07/2019 16:27, Nils Eliasson wrote: > Hi, > > This is a patch of GarbageUtils.java used by a bunch of GC tests. > > This issue was diagnosed by Stefan Karlsson and passed to the compiler > team. > > The core problem is that the test relies on successfully catching OOM > errors. But the OOM causes a deopt, and the deopt must rematerialize > some objects before passing the execution on to the interpreter to run > the catch-block. If the rematerialization can't allocate, it will pass > the OOM on, without having run the catch-block. > > With this patch I fix the tests by wrapping the faulting code with an > extra layer of try-catch-OOM. It relies on not inlining the inner > method, because then it would be part of the compiled frame being > deoptimized. > > In general this is a problem for any kind of deopt when we are out of > heap. Another bug will be opened to keep tracking this issue. > > Webrev:http://cr.openjdk.java.net/~neliasso/8226536/webrev.02/ > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8226536 > > // Nils > From nils.eliasson at oracle.com Fri Jul 5 10:14:17 2019 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Fri, 5 Jul 2019 12:14:17 +0200 Subject: [13] RFR(S): 8226536: Catch OOM from deopt that fails rematerializing objects In-Reply-To: <94bfdef5-5cbf-86e9-93ce-ed7d900adc76@oracle.com> References: <94bfdef5-5cbf-86e9-93ce-ed7d900adc76@oracle.com> Message-ID: Doesn't that only work on classes loaded by the Bootstrap classloader? // Nils On 2019-07-05 12:00, Vladimir Ivanov wrote: > test/hotspot/jtreg/vmTestbase/nsk/share/gc/gp/GarbageUtils.java: > > +????????? * It is Important that the impl is not inlined. > > What do you think about putting @DontInline on eatMemoryImpl to > guarantee that invariant? > > Best regards, > Vladimir Ivanov > > On 04/07/2019 16:27, Nils Eliasson wrote: >> Hi, >> >> This is a patch of GarbageUtils.java used by a bunch of GC tests. >> >> This issue was diagnosed by Stefan Karlsson and passed to the >> compiler team. >> >> The core problem is that the test relies on successfully catching OOM >> errors. But the OOM causes a deopt, and the deopt must rematerialize >> some objects before passing the execution on to the interpreter to >> run the catch-block. If the rematerialization can't allocate, it will >> pass the OOM on, without having run the catch-block. >> >> With this patch I fix the tests by wrapping the faulting code with an >> extra layer of try-catch-OOM. It relies on not inlining the inner >> method, because then it would be part of the compiled frame being >> deoptimized. >> >> In general this is a problem for any kind of deopt when we are out of >> heap. Another bug will be opened to keep tracking this issue. >> >> Webrev:http://cr.openjdk.java.net/~neliasso/8226536/webrev.02/ >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8226536 >> >> // Nils >> From vladimir.x.ivanov at oracle.com Fri Jul 5 10:20:18 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 5 Jul 2019 13:20:18 +0300 Subject: [13] RFR(S): 8226536: Catch OOM from deopt that fails rematerializing objects In-Reply-To: References: <94bfdef5-5cbf-86e9-93ce-ed7d900adc76@oracle.com> Message-ID: > Doesn't that only work on classes loaded by the Bootstrap classloader? Yes, that would require putting GarbageUtils on boot class path or running affected tests with main/bootclasspath/othervm. Best regards, Vladimir Ivanov > > // Nils > > On 2019-07-05 12:00, Vladimir Ivanov wrote: >> test/hotspot/jtreg/vmTestbase/nsk/share/gc/gp/GarbageUtils.java: >> >> +????????? * It is Important that the impl is not inlined. >> >> What do you think about putting @DontInline on eatMemoryImpl to >> guarantee that invariant? >> >> Best regards, >> Vladimir Ivanov >> >> On 04/07/2019 16:27, Nils Eliasson wrote: >>> Hi, >>> >>> This is a patch of GarbageUtils.java used by a bunch of GC tests. >>> >>> This issue was diagnosed by Stefan Karlsson and passed to the >>> compiler team. >>> >>> The core problem is that the test relies on successfully catching OOM >>> errors. But the OOM causes a deopt, and the deopt must rematerialize >>> some objects before passing the execution on to the interpreter to >>> run the catch-block. If the rematerialization can't allocate, it will >>> pass the OOM on, without having run the catch-block. >>> >>> With this patch I fix the tests by wrapping the faulting code with an >>> extra layer of try-catch-OOM. It relies on not inlining the inner >>> method, because then it would be part of the compiled frame being >>> deoptimized. >>> >>> In general this is a problem for any kind of deopt when we are out of >>> heap. Another bug will be opened to keep tracking this issue. >>> >>> Webrev:http://cr.openjdk.java.net/~neliasso/8226536/webrev.02/ >>> >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8226536 >>> >>> // Nils >>> From nils.eliasson at oracle.com Fri Jul 5 10:27:25 2019 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Fri, 5 Jul 2019 12:27:25 +0200 Subject: [13] RFR(S): 8226536: Catch OOM from deopt that fails rematerializing objects In-Reply-To: References: <94bfdef5-5cbf-86e9-93ce-ed7d900adc76@oracle.com> Message-ID: <69deaeba-3afa-ce91-59c2-54deab207a1a@oracle.com> Actually the tests do run with /othervm I'll try it out. // N On 2019-07-05 12:20, Vladimir Ivanov wrote: >> Doesn't that only work on classes loaded by the Bootstrap classloader? > > Yes, that would require putting GarbageUtils on boot class path or > running affected tests with main/bootclasspath/othervm. > > Best regards, > Vladimir Ivanov > >> >> // Nils >> >> On 2019-07-05 12:00, Vladimir Ivanov wrote: >>> test/hotspot/jtreg/vmTestbase/nsk/share/gc/gp/GarbageUtils.java: >>> >>> +????????? * It is Important that the impl is not inlined. >>> >>> What do you think about putting @DontInline on eatMemoryImpl to >>> guarantee that invariant? >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> On 04/07/2019 16:27, Nils Eliasson wrote: >>>> Hi, >>>> >>>> This is a patch of GarbageUtils.java used by a bunch of GC tests. >>>> >>>> This issue was diagnosed by Stefan Karlsson and passed to the >>>> compiler team. >>>> >>>> The core problem is that the test relies on successfully catching >>>> OOM errors. But the OOM causes a deopt, and the deopt must >>>> rematerialize some objects before passing the execution on to the >>>> interpreter to run the catch-block. If the rematerialization can't >>>> allocate, it will pass the OOM on, without having run the catch-block. >>>> >>>> With this patch I fix the tests by wrapping the faulting code with >>>> an extra layer of try-catch-OOM. It relies on not inlining the >>>> inner method, because then it would be part of the compiled frame >>>> being deoptimized. >>>> >>>> In general this is a problem for any kind of deopt when we are out >>>> of heap. Another bug will be opened to keep tracking this issue. >>>> >>>> Webrev:http://cr.openjdk.java.net/~neliasso/8226536/webrev.02/ >>>> >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8226536 >>>> >>>> // Nils >>>> From erik.osterlund at oracle.com Fri Jul 5 10:42:51 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 5 Jul 2019 12:42:51 +0200 Subject: [13] RFR(S): 8226536: Catch OOM from deopt that fails rematerializing objects In-Reply-To: References: <94bfdef5-5cbf-86e9-93ce-ed7d900adc76@oracle.com> Message-ID: Hi, Sounds like properly annotating this will clobber a lot of tests (~40?). Not sure it's worth the hassle when this RFE is only about silencing the tests (which it does as the method is too large to get inlined), as opposed to fixing the underlying issue. The worst that could happen is that the tests start moaning again with new inlining heuristics, which is okay IMO. /Erik On 2019-07-05 12:20, Vladimir Ivanov wrote: >> Doesn't that only work on classes loaded by the Bootstrap classloader? > > Yes, that would require putting GarbageUtils on boot class path or > running affected tests with main/bootclasspath/othervm. > > Best regards, > Vladimir Ivanov > >> >> // Nils >> >> On 2019-07-05 12:00, Vladimir Ivanov wrote: >>> test/hotspot/jtreg/vmTestbase/nsk/share/gc/gp/GarbageUtils.java: >>> >>> +????????? * It is Important that the impl is not inlined. >>> >>> What do you think about putting @DontInline on eatMemoryImpl to >>> guarantee that invariant? >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> On 04/07/2019 16:27, Nils Eliasson wrote: >>>> Hi, >>>> >>>> This is a patch of GarbageUtils.java used by a bunch of GC tests. >>>> >>>> This issue was diagnosed by Stefan Karlsson and passed to the >>>> compiler team. >>>> >>>> The core problem is that the test relies on successfully catching >>>> OOM errors. But the OOM causes a deopt, and the deopt must >>>> rematerialize some objects before passing the execution on to the >>>> interpreter to run the catch-block. If the rematerialization can't >>>> allocate, it will pass the OOM on, without having run the catch-block. >>>> >>>> With this patch I fix the tests by wrapping the faulting code with >>>> an extra layer of try-catch-OOM. It relies on not inlining the >>>> inner method, because then it would be part of the compiled frame >>>> being deoptimized. >>>> >>>> In general this is a problem for any kind of deopt when we are out >>>> of heap. Another bug will be opened to keep tracking this issue. >>>> >>>> Webrev:http://cr.openjdk.java.net/~neliasso/8226536/webrev.02/ >>>> >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8226536 >>>> >>>> // Nils >>>> From vladimir.x.ivanov at oracle.com Fri Jul 5 11:18:03 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 5 Jul 2019 14:18:03 +0300 Subject: [13] RFR(S): 8226536: Catch OOM from deopt that fails rematerializing objects In-Reply-To: References: <94bfdef5-5cbf-86e9-93ce-ed7d900adc76@oracle.com> Message-ID: <246fc050-bac8-9a3d-5671-55b23826cf6f@oracle.com> > Sounds like properly annotating this will clobber a lot of tests (~40?). > Not sure it's worth the hassle when this RFE is only about silencing the > tests (which it does as the method is too large to get inlined), as > opposed to fixing the underlying issue. > The worst that could happen is that the tests start moaning again with > new inlining heuristics, which is okay IMO. I'd prefer to introduce a reliable fix and not bother with it again. If the concern is the amount of changes needed, calling eatMemoryImpl using a non-constant MethodHandle (cached in a non-static non-final field) would block inlining as well. Best regards, Vladimir Ivanov > On 2019-07-05 12:20, Vladimir Ivanov wrote: >>> Doesn't that only work on classes loaded by the Bootstrap classloader? >> >> Yes, that would require putting GarbageUtils on boot class path or >> running affected tests with main/bootclasspath/othervm. >> >> Best regards, >> Vladimir Ivanov >> >>> >>> // Nils >>> >>> On 2019-07-05 12:00, Vladimir Ivanov wrote: >>>> test/hotspot/jtreg/vmTestbase/nsk/share/gc/gp/GarbageUtils.java: >>>> >>>> +????????? * It is Important that the impl is not inlined. >>>> >>>> What do you think about putting @DontInline on eatMemoryImpl to >>>> guarantee that invariant? >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>> On 04/07/2019 16:27, Nils Eliasson wrote: >>>>> Hi, >>>>> >>>>> This is a patch of GarbageUtils.java used by a bunch of GC tests. >>>>> >>>>> This issue was diagnosed by Stefan Karlsson and passed to the >>>>> compiler team. >>>>> >>>>> The core problem is that the test relies on successfully catching >>>>> OOM errors. But the OOM causes a deopt, and the deopt must >>>>> rematerialize some objects before passing the execution on to the >>>>> interpreter to run the catch-block. If the rematerialization can't >>>>> allocate, it will pass the OOM on, without having run the catch-block. >>>>> >>>>> With this patch I fix the tests by wrapping the faulting code with >>>>> an extra layer of try-catch-OOM. It relies on not inlining the >>>>> inner method, because then it would be part of the compiled frame >>>>> being deoptimized. >>>>> >>>>> In general this is a problem for any kind of deopt when we are out >>>>> of heap. Another bug will be opened to keep tracking this issue. >>>>> >>>>> Webrev:http://cr.openjdk.java.net/~neliasso/8226536/webrev.02/ >>>>> >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8226536 >>>>> >>>>> // Nils >>>>> > From nils.eliasson at oracle.com Fri Jul 5 14:50:50 2019 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Fri, 5 Jul 2019 16:50:50 +0200 Subject: [13] RFR(S): 8226536: Catch OOM from deopt that fails rematerializing objects In-Reply-To: <246fc050-bac8-9a3d-5671-55b23826cf6f@oracle.com> References: <94bfdef5-5cbf-86e9-93ce-ed7d900adc76@oracle.com> <246fc050-bac8-9a3d-5671-55b23826cf6f@oracle.com> Message-ID: <08cff491-b348-1782-1cd5-826c4c83b47b@oracle.com> ok, I first tried the @DontInline annotation since that captures the intention clearly. That failed spectacularly, the GarbageUtil was used by an immense amounts of tests that all would have required the @module tag. So here is a version using methodhandles instead: http://cr.openjdk.java.net/~neliasso/8226536/webrev.04/ // Nils On 2019-07-05 13:18, Vladimir Ivanov wrote: > >> Sounds like properly annotating this will clobber a lot of tests (~40?). >> Not sure it's worth the hassle when this RFE is only about silencing >> the tests (which it does as the method is too large to get inlined), >> as opposed to fixing the underlying issue. >> The worst that could happen is that the tests start moaning again >> with new inlining heuristics, which is okay IMO. > > I'd prefer to introduce a reliable fix and not bother with it again. > > If the concern is the amount of changes needed, calling eatMemoryImpl > using a non-constant MethodHandle (cached in a non-static non-final > field) would block inlining as well. > > Best regards, > Vladimir Ivanov > >> On 2019-07-05 12:20, Vladimir Ivanov wrote: >>>> Doesn't that only work on classes loaded by the Bootstrap classloader? >>> >>> Yes, that would require putting GarbageUtils on boot class path or >>> running affected tests with main/bootclasspath/othervm. >>> >>> Best regards, >>> Vladimir Ivanov >>> >>>> >>>> // Nils >>>> >>>> On 2019-07-05 12:00, Vladimir Ivanov wrote: >>>>> test/hotspot/jtreg/vmTestbase/nsk/share/gc/gp/GarbageUtils.java: >>>>> >>>>> +????????? * It is Important that the impl is not inlined. >>>>> >>>>> What do you think about putting @DontInline on eatMemoryImpl to >>>>> guarantee that invariant? >>>>> >>>>> Best regards, >>>>> Vladimir Ivanov >>>>> >>>>> On 04/07/2019 16:27, Nils Eliasson wrote: >>>>>> Hi, >>>>>> >>>>>> This is a patch of GarbageUtils.java used by a bunch of GC tests. >>>>>> >>>>>> This issue was diagnosed by Stefan Karlsson and passed to the >>>>>> compiler team. >>>>>> >>>>>> The core problem is that the test relies on successfully catching >>>>>> OOM errors. But the OOM causes a deopt, and the deopt must >>>>>> rematerialize some objects before passing the execution on to the >>>>>> interpreter to run the catch-block. If the rematerialization >>>>>> can't allocate, it will pass the OOM on, without having run the >>>>>> catch-block. >>>>>> >>>>>> With this patch I fix the tests by wrapping the faulting code >>>>>> with an extra layer of try-catch-OOM. It relies on not inlining >>>>>> the inner method, because then it would be part of the compiled >>>>>> frame being deoptimized. >>>>>> >>>>>> In general this is a problem for any kind of deopt when we are >>>>>> out of heap. Another bug will be opened to keep tracking this issue. >>>>>> >>>>>> Webrev:http://cr.openjdk.java.net/~neliasso/8226536/webrev.02/ >>>>>> >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8226536 >>>>>> >>>>>> // Nils >>>>>> >> From sci at amazon.com Fri Jul 5 15:18:50 2019 From: sci at amazon.com (Sciampacone, Ryan) Date: Fri, 5 Jul 2019 15:18:50 +0000 Subject: RFR: 8227226: Segmented array clearing for ZGC Message-ID: <4443C38C-7E12-4675-A81B-616EB7D1D038@amazon.com> http://cr.openjdk.java.net/~phh/8227226/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8227226 This patch introduces safe point checks into array clearing during allocation for ZGC. The patch isolates the changes to ZGC as (in particular with the more modern collectors) the approach to incrementalizing or respecting safe point checks is going to be different. The approach is to keep the region holding the array in the allocating state (pin logic) while updating the color to the array after checks. Can I get a review? Thanks. Ryan From tprintezis at twitter.com Fri Jul 5 18:06:57 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Fri, 5 Jul 2019 11:06:57 -0700 Subject: RFR(S): 8227225: ParallelGC: add subspace transitions for young gen for gc+heap=info log lines Message-ID: Hi all, This is a follow-up to 8223575 and adds subspace info for eden and from space to the gc+heap=info output for ParallelGC. I also added the before capacity, similar to 8223575, as it can change during the GC (see, in fact, example below). Before: [11.123s][info][gc,heap ] GC(17) PSYoungGen: 4759616K->1664K(4545536K) [11.123s][info][gc,heap ] GC(17) ParOldGen: 1280K->1280K(1376256K) After: [12.133s][info][gc,heap ] GC(25) PSYoungGen: 2065952K(2066432K)->1536K(1983488K) Eden: 2064384K(2064384K)->0K(1981952K) From: 1568K(2048K)->1536K(1536K) [12.133s][info][gc,heap ] GC(25) ParOldGen: 1272K(1376256K)->1272K(1376256K) http://cr.openjdk.java.net/~tonyp/8227225/webrev.0/ I have a similar change ready to go for GenCollectedHeap (Serial / CMS). I?ll see what the feedback is on this before I open that for code review. A couple of notes: - I moved the PreGCValues class to gc/shared as I?d like to re-use it for the GenCollectedHeap changes too. - Is it worth also adding info about To space to the output? Tony PS Should I add the gc-pending-review label to any CR I open for code review? ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com From shade at redhat.com Fri Jul 5 18:40:59 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 5 Jul 2019 20:40:59 +0200 Subject: RFR (L) 8227327: Shenandoah: Faster and more parallel tests Message-ID: <8c020ff2-c06e-9f02-7718-cb5a3bcb699d@redhat.com> Current hotspot_gc_shenandoah suite runs in fastdebug like this: real 31m31.172s user 287m50.380s sys 5m28.277s This amounts to 9.1x parallelism on 16-thread machine. Clearly, something big is on the critical path. Profiling shows the critical path is largely dominated by GCBasherWithShenandoah, which takes 26 minutes to run through all configurations. Other huge tests also run for a long time. We can improve this by splitting @run into several @test groups -- this would execute @test-s in parallel, reducing the impact on critical path. The convenient way to split is by mode: passive, normal, traversal. Also, we can remove some the useless configurations from the tests. This becomes apparent once we reformat @run sections to be more digestible and following the same format. Webrev: https://cr.openjdk.java.net/~shade/8227327/webrev.01/ Patched version runs like this: real 17m53.136s user 247m50.750s sys 5m33.051s So, it yields ~13.7x parallelism on 16-thread machine, and does less work. Testing: hotspot_gc_shenandoah (d'uh) -- Thanks, -Aleksey From rkennke at redhat.com Fri Jul 5 20:37:14 2019 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 5 Jul 2019 22:37:14 +0200 Subject: RFR (L) 8227327: Shenandoah: Faster and more parallel tests In-Reply-To: <8c020ff2-c06e-9f02-7718-cb5a3bcb699d@redhat.com> References: <8c020ff2-c06e-9f02-7718-cb5a3bcb699d@redhat.com> Message-ID: <46374c15-8a09-f449-46b8-697c420940b8@redhat.com> Very nice! Patch looks good! Roman > Current hotspot_gc_shenandoah suite runs in fastdebug like this: > > real 31m31.172s > user 287m50.380s > sys 5m28.277s > > This amounts to 9.1x parallelism on 16-thread machine. Clearly, something big is on the critical > path. Profiling shows the critical path is largely dominated by GCBasherWithShenandoah, which takes > 26 minutes to run through all configurations. Other huge tests also run for a long time. > > We can improve this by splitting @run into several @test groups -- this would execute @test-s in > parallel, reducing the impact on critical path. The convenient way to split is by mode: passive, > normal, traversal. > > Also, we can remove some the useless configurations from the tests. This becomes apparent once we > reformat @run sections to be more digestible and following the same format. > > Webrev: > https://cr.openjdk.java.net/~shade/8227327/webrev.01/ > > Patched version runs like this: > > real 17m53.136s > user 247m50.750s > sys 5m33.051s > > So, it yields ~13.7x parallelism on 16-thread machine, and does less work. > > Testing: hotspot_gc_shenandoah (d'uh) > From kim.barrett at oracle.com Sat Jul 6 21:45:12 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 6 Jul 2019 17:45:12 -0400 Subject: RFR: 8217170: gc/arguments/TestUseCompressedOopsErgo.java timed out Message-ID: <235C3194-422B-4028-83C4-44D1E4BC599B@oracle.com> Please review this change to many of the tests in gc/arguments to reduce the chances of some of them timing out on some platform configurations, especially when run with debug builds. These tests typically launch a series of one or more child processes with specific command line options chosen to test that specific configuration. In debug builds there is some extra work done by default by each of those child processes that is unnecessary for the use at hand, and may (in some platform configurations) add a lot to the time taken by the child. ZapUnusedHeapArea has been identified as the major offender, with VerifyBeforeExit being a lesser contributor. Both of these are enabled by default in debug builds. To address the problem there is a new helper class, GCArguments, which provides wrappers around ProcessTools.createJavaProcessBuilder to add options to disable those problematic features. The options are added at the front, so they can be re-enabled by later options from a test if desired (such as the test that checks VerifyBeforeExit). The tests have been updated to call that new wrapper. There is still some residual odd slowness to at least some of these tests. The child processes usually take 10's to low 100's of milliseconds to run, but sometimes 3-5 seconds (or 20+ seconds when using CMS; don't know what's up with that). So there's further investigation to be done, but the changes being made here at least reduce the spurious timeout noise in regression testing. CR: https://bugs.openjdk.java.net/browse/JDK-8217170 Webrev: http://cr.openjdk.java.net/~kbarrett/8217170/open.00/ Testing: mach5 tier1-3 Ran tier1 entirely on a machine that is known for having reliable timeouts without this change. From tprintezis at twitter.com Sun Jul 7 12:35:31 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Sun, 7 Jul 2019 05:35:31 -0700 Subject: RFR: 8217170: gc/arguments/TestUseCompressedOopsErgo.java timed out In-Reply-To: <235C3194-422B-4028-83C4-44D1E4BC599B@oracle.com> References: <235C3194-422B-4028-83C4-44D1E4BC599B@oracle.com> Message-ID: Kim, If the tests start with a very small initial heap, CMS will set the young gen size to something very small and then not resize it during the run. This results in an unnecessarily high young GC overhead. We?ve seen that in quite a few occasions. Could this be the reason for the child processes taking a long time with CMS? Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On July 6, 2019 at 5:47:49 PM, Kim Barrett (kim.barrett at oracle.com) wrote: Please review this change to many of the tests in gc/arguments to reduce the chances of some of them timing out on some platform configurations, especially when run with debug builds. These tests typically launch a series of one or more child processes with specific command line options chosen to test that specific configuration. In debug builds there is some extra work done by default by each of those child processes that is unnecessary for the use at hand, and may (in some platform configurations) add a lot to the time taken by the child. ZapUnusedHeapArea has been identified as the major offender, with VerifyBeforeExit being a lesser contributor. Both of these are enabled by default in debug builds. To address the problem there is a new helper class, GCArguments, which provides wrappers around ProcessTools.createJavaProcessBuilder to add options to disable those problematic features. The options are added at the front, so they can be re-enabled by later options from a test if desired (such as the test that checks VerifyBeforeExit). The tests have been updated to call that new wrapper. There is still some residual odd slowness to at least some of these tests. The child processes usually take 10's to low 100's of milliseconds to run, but sometimes 3-5 seconds (or 20+ seconds when using CMS; don't know what's up with that). So there's further investigation to be done, but the changes being made here at least reduce the spurious timeout noise in regression testing. CR: https://bugs.openjdk.java.net/browse/JDK-8217170 Webrev: http://cr.openjdk.java.net/~kbarrett/8217170/open.00/ Testing: mach5 tier1-3 Ran tier1 entirely on a machine that is known for having reliable timeouts without this change. From thomas.schatzl at oracle.com Mon Jul 8 07:54:43 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 08 Jul 2019 09:54:43 +0200 Subject: RFR (S): 8227179: Test for new gc+metaspace=info output format In-Reply-To: References: Message-ID: Hi Tony, On Wed, 2019-07-03 at 12:57 -0700, Tony Printezis wrote: > Hi all, > > This is a follow-up to: > > 8223575: add subspace transitions to gc+metaspace=info log lines > > (that I just pushed) and adds a test to sanity check the > gc+metaspace=info > output. Webrev here: > > http://cr.openjdk.java.net/~tonyp/8227179/webrev.0/ > test looks good except for some issue with builds that do not build all GCs. I.e. you need to split the single @test block into multiple, for each collector one and add a @requires tag. E.g. gc/TestSystemGC.java shows how to do that. Thanks, Thomas From goetz.lindenmaier at sap.com Mon Jul 8 08:18:11 2019 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 8 Jul 2019 08:18:11 +0000 Subject: RFR(XS): 8226302: Test failures on IBM platforms (power and s/390) after JDK-8223837 In-Reply-To: References: Message-ID: Hi Martin, thanks for the fix. Looks good. It's the simple way to fix it, but as new page sizes come up rarely, this is fine. Best regards, Goetz. > -----Original Message----- > From: Doerr, Martin > Sent: Donnerstag, 4. Juli 2019 12:35 > To: hotspot-gc-dev at openjdk.java.net > Cc: Lindenmaier, Goetz > Subject: RFR(XS): 8226302: Test failures on IBM platforms (power and s/390) > after JDK-8223837 > > Hi, > > > > I'd like to fix the heap size related issues described in > > https://bugs.openjdk.java.net/browse/JDK-8226302 > > by adjusting the sizes. I prefer this simple approach for jdk13. > > > > We could change the tests such that they determine appropriate sizes > depending on the HeapAlignment (retrieved by whitebox api) for jdk14. > > > > Please review: > > http://cr.openjdk.java.net/~mdoerr/8226302_TestHeapSize/webrev.00/ > > > > Best regards, > > Martin > > From thomas.schatzl at oracle.com Mon Jul 8 08:44:11 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 08 Jul 2019 10:44:11 +0200 Subject: RFR: 8217170: gc/arguments/TestUseCompressedOopsErgo.java timed out In-Reply-To: <235C3194-422B-4028-83C4-44D1E4BC599B@oracle.com> References: <235C3194-422B-4028-83C4-44D1E4BC599B@oracle.com> Message-ID: Hi, On Sat, 2019-07-06 at 17:45 -0400, Kim Barrett wrote: > Please review this change to many of the tests in gc/arguments to > reduce the chances of some of them timing out on some platform > configurations, especially when run with debug builds. > [..] > To address the problem there is a new helper class, GCArguments, > which provides wrappers around ProcessTools.createJavaProcessBuilder > to add options to disable those problematic features. The options > are added at the front, so they can be re-enabled by later options > from a test if desired (such as the test that checks > VerifyBeforeExit). The tests have been updated to call that new > wrapper. > > There is still some residual odd slowness to at least some of these > tests. The child processes usually take 10's to low 100's of > milliseconds to run, but sometimes 3-5 seconds (or 20+ seconds when > using CMS; don't know what's up with that). So there's further > investigation to be done, but the changes being made here at least > reduce the spurious timeout noise in regression testing. There are a few other trueInDebug flags, but apart from ZapFillerObjects there is nothing obvious. And even that seems doubtful to cause issues. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8217170 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8217170/open.00/ > looks good. Thomas From ralf.schmelter at sap.com Mon Jul 8 08:52:38 2019 From: ralf.schmelter at sap.com (Schmelter, Ralf) Date: Mon, 8 Jul 2019 08:52:38 +0000 Subject: RFR(XS): 8226302: Test failures on IBM platforms (power and s/390) after JDK-8223837 In-Reply-To: References: Message-ID: Hi Martin, looks good. It would be nicer, if one could do it programmatically, but it's probably not worth the effort. Best regards, Ralf From thomas.schatzl at oracle.com Mon Jul 8 08:53:58 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 08 Jul 2019 10:53:58 +0200 Subject: RFR(XS): 8226302: Test failures on IBM platforms (power and s/390) after JDK-8223837 In-Reply-To: References: Message-ID: <75575f40602412341069d48d997061a9578ec564.camel@oracle.com> On Mon, 2019-07-08 at 08:52 +0000, Schmelter, Ralf wrote: > Hi Martin, > > looks good. It would be nicer, if one could do it programmatically, > but it's probably not worth the effort. > +1 Thomas :) > Best regards, > Ralf From martin.doerr at sap.com Mon Jul 8 08:58:57 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 8 Jul 2019 08:58:57 +0000 Subject: RFR(XS): 8226302: Test failures on IBM platforms (power and s/390) after JDK-8223837 In-Reply-To: <75575f40602412341069d48d997061a9578ec564.camel@oracle.com> References: <75575f40602412341069d48d997061a9578ec564.camel@oracle.com> Message-ID: Hi Thomas, Ralf and G?tz, thank you for the reviews. Pushed. Best regards, Martin > -----Original Message----- > From: Thomas Schatzl > Sent: Montag, 8. Juli 2019 10:54 > To: Schmelter, Ralf ; Doerr, Martin > ; hotspot-gc-dev at openjdk.java.net > Subject: Re: RFR(XS): 8226302: Test failures on IBM platforms (power and > s/390) after JDK-8223837 > > On Mon, 2019-07-08 at 08:52 +0000, Schmelter, Ralf wrote: > > Hi Martin, > > > > looks good. It would be nicer, if one could do it programmatically, > > but it's probably not worth the effort. > > > > +1 > > Thomas :) > > > Best regards, > > Ralf > > From thomas.schatzl at oracle.com Mon Jul 8 09:43:47 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 08 Jul 2019 11:43:47 +0200 Subject: How do I know how much memory is reclaimed in major GC? In-Reply-To: References: Message-ID: Hi, On Thu, 2019-06-27 at 21:32 +0000, shang xinli wrote: > Hi all, > > I know young GC and full GC logs show that information like xxx->yyy. > But in the major GC log, I can only see that information in 'initial > mark' and 'final mark', but not in 'sweep'. So do I know in the end > of a major GC, how much memory in the old generation is claimed? The > logs are copied below. > > 2019-06-27T17:23:18.139+0000: 4910103.818: [GC (CMS Initial Mark) [1 > CMS-initial-mark: 153077851K(192937984K)] 163324888K(208037504K), > 0.4523961 secs] [Times: user=9.72 sys=0.00, real=0.45 secs] > > [...] > > 2019-06-27T17:24:52.150+0000: 4910197.830: [GC (CMS Final Remark) [YG > occupancy: 11527850 K (15099520 K)]2019-06-27T17:24:52.150+0000: > 4910197.830: [Rescan (parallel) , 0.4904597 secs]2019-06- > 27T17:24:52.641+0000: 4910198.320: [weak refs processing, 0.0000493 > secs]2019-06-27T17:24:52.641+0000: 4910198.320: [class unloading, > 0.0275684 secs]2019-06-27T17:24:52.668+0000: 4910198.348: [scrub > symbol table, 0.0065112 secs]2019-06-27T17:24:52.675+0000: > 4910198.354: [scrub string table, 0.0010931 secs][1 CMS-remark: > 153467633K(192937984K)] 164995484K(208037504K), 0.5259280 secs] > [Times: user=11.21 sys=0.00, real=0.53 secs] Sweep does not reevalute the heap, looking for newly reclaimable memory. It only removes the unnecesssary memory found to be dead during Final mark. I.e. you need to look at Final Mark (vs. Initial Mark) to get an idea how much of the old generation has been reclaimed. The "[1 CMS-initial-mark: 153077851K(192937984K)]" and "[1 CMS-remark: > 153467633K(192937984K)]" give old gen usage/occupancy. Note that allocation between initial and final mark (either direct or during GCs) also changes the occupancy in the old gen, so you can't directly subtract the values. This is the most accurate way of getting this data from the logs I can think of though. Thanks, Thomas From erik.osterlund at oracle.com Mon Jul 8 10:42:18 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 8 Jul 2019 12:42:18 +0200 Subject: RFR: 8227226: Segmented array clearing for ZGC In-Reply-To: <4443C38C-7E12-4675-A81B-616EB7D1D038@amazon.com> References: <4443C38C-7E12-4675-A81B-616EB7D1D038@amazon.com> Message-ID: <49ec6d28-56f8-8227-d4e9-c9c8a4607050@oracle.com> Hi Ryan, This looks good in general. Just some stylistic things... 1) In the ZGC project we like the letter 'Z' so much that we put it in front of everything we possibly can, including all class names. 2) We also explicitly state things are private even though it's bleedingly obvious. So: 39 class PinAllocating { 40 HeapWord* _mem; 41 public: -> 39 class ZPinAllocating { 40 private: 41 HeapWord* _mem; 42 41 public: I can be your sponsor and push this change for you. I don't think there is a need for another webrev for my small stylistic remarks, so I can just fix that before pushing this for you. On that note, I'll add me and StefanK to the contributed-by section as we all worked out the right solution to this problem collaboratively. I have run through mach5 tier1-5, and found no issues with this patch. Thanks, /Erik On 2019-07-05 17:18, Sciampacone, Ryan wrote: > http://cr.openjdk.java.net/~phh/8227226/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8227226 > > This patch introduces safe point checks into array clearing during allocation for ZGC. The patch isolates the changes to ZGC as (in particular with the more modern collectors) the approach to incrementalizing or respecting safe point checks is going to be different. > > The approach is to keep the region holding the array in the allocating state (pin logic) while updating the color to the array after checks. > > Can I get a review? Thanks. > > Ryan From per.liden at oracle.com Mon Jul 8 12:48:34 2019 From: per.liden at oracle.com (Per Liden) Date: Mon, 8 Jul 2019 14:48:34 +0200 Subject: RFR: 8227226: Segmented array clearing for ZGC In-Reply-To: <49ec6d28-56f8-8227-d4e9-c9c8a4607050@oracle.com> References: <4443C38C-7E12-4675-A81B-616EB7D1D038@amazon.com> <49ec6d28-56f8-8227-d4e9-c9c8a4607050@oracle.com> Message-ID: Hi Ryan, A few general comments: 1) It looks like this still only work for large pages? 2) The log_info stuff should be removed. 3) I'm not a huge fan of single-use utilities like PinAllocating, at least not when, IMO, the alternative is more straight forward and less code. 4) Please make locals const when possible. 5) Duplicating _do_zero looks odd. Injecting a "mem clearer", similar to what Stefans original patch did, seems worth exploring. cheers, /Per (Btw, I'm on vacation so I might not be super-responsive to emails) On 2019-07-08 12:42, Erik ?sterlund wrote: > Hi Ryan, > > This looks good in general. Just some stylistic things... > > 1) In the ZGC project we like the letter 'Z' so much that we put it in > front of everything we possibly can, including all class names. > 2) We also explicitly state things are private even though it's > bleedingly obvious. > > So: > > 39 class PinAllocating { > 40 HeapWord* _mem; > 41 public: -> 39 class ZPinAllocating { 40 private:??? 41 HeapWord* _mem; > ? 42 > ?41 public: I can be your sponsor and push this change for you. I don't > think there is a need for another webrev for my small stylistic remarks, > so I can just fix that before pushing this for you. On that note, I'll > add me and StefanK to the contributed-by section as we all worked out > the right solution to this problem collaboratively. I have run through > mach5 tier1-5, and found no issues with this patch. Thanks, /Erik > > On 2019-07-05 17:18, Sciampacone, Ryan wrote: >> http://cr.openjdk.java.net/~phh/8227226/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8227226 >> >> This patch introduces safe point checks into array clearing during >> allocation for ZGC.? The patch isolates the changes to ZGC as (in >> particular with the more modern collectors) the approach to >> incrementalizing or respecting safe point checks is going to be >> different. >> >> The approach is to keep the region holding the array in the allocating >> state (pin logic) while updating the color to the array after checks. >> >> Can I get a review?? Thanks. >> >> Ryan > From sci at amazon.com Mon Jul 8 13:35:28 2019 From: sci at amazon.com (Sciampacone, Ryan) Date: Mon, 8 Jul 2019 13:35:28 +0000 Subject: 8227226: Segmented array clearing for ZGC Message-ID: <5809FFE3-ED37-429B-9189-49D8FD14D092@amazon.com> 1) Yes this was a conscious decision. There was discussion on determining the optimal point for breakup but given the existing sizes this seemed sufficient. This doesn't preclude the ability to go down that path if its deemed absolutely necessary. The path for more complex decisions is now available. 2) Agree 3) I'm not clear here. Do you mean effectively going direct to ZHeap and bypassing the single function PinAllocating? Agree. Otherwise I'll ask you to be a bit clearer. 4) Agree 5) I initially had the exact same reaction but I played around with a few other versions (including breaking up initialization points between header and body to get the desired results) and this ended up looking correct. I'll try mixing in the mem clearer function again (fresh start) to see if it looks any better. ?On 7/8/19, 5:49 AM, "Per Liden" wrote: Hi Ryan, A few general comments: 1) It looks like this still only work for large pages? 2) The log_info stuff should be removed. 3) I'm not a huge fan of single-use utilities like PinAllocating, at least not when, IMO, the alternative is more straight forward and less code. 4) Please make locals const when possible. 5) Duplicating _do_zero looks odd. Injecting a "mem clearer", similar to what Stefans original patch did, seems worth exploring. cheers, /Per (Btw, I'm on vacation so I might not be super-responsive to emails) On 2019-07-08 12:42, Erik ?sterlund wrote: > Hi Ryan, > > This looks good in general. Just some stylistic things... > > 1) In the ZGC project we like the letter 'Z' so much that we put it in > front of everything we possibly can, including all class names. > 2) We also explicitly state things are private even though it's > bleedingly obvious. > > So: > > 39 class PinAllocating { > 40 HeapWord* _mem; > 41 public: -> 39 class ZPinAllocating { 40 private: 41 HeapWord* _mem; > 42 > 41 public: I can be your sponsor and push this change for you. I don't > think there is a need for another webrev for my small stylistic remarks, > so I can just fix that before pushing this for you. On that note, I'll > add me and StefanK to the contributed-by section as we all worked out > the right solution to this problem collaboratively. I have run through > mach5 tier1-5, and found no issues with this patch. Thanks, /Erik > > On 2019-07-05 17:18, Sciampacone, Ryan wrote: >> http://cr.openjdk.java.net/~phh/8227226/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8227226 >> >> This patch introduces safe point checks into array clearing during >> allocation for ZGC. The patch isolates the changes to ZGC as (in >> particular with the more modern collectors) the approach to >> incrementalizing or respecting safe point checks is going to be >> different. >> >> The approach is to keep the region holding the array in the allocating >> state (pin logic) while updating the color to the array after checks. >> >> Can I get a review? Thanks. >> >> Ryan > From tprintezis at twitter.com Mon Jul 8 15:09:57 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Mon, 8 Jul 2019 08:09:57 -0700 Subject: RFR (S): 8227179: Test for new gc+metaspace=info output format In-Reply-To: References: Message-ID: Thanks for pointing that out, Thomas. Updated webrev (the only change is the @test comments at the top): http://cr.openjdk.java.net/~tonyp/8227179/webrev.1/ I tested it with a ?normal? build (all GCs built) and one with no CMS and the test seemed to have done the right thing in both occasions. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On July 8, 2019 at 3:54:50 AM, Thomas Schatzl (thomas.schatzl at oracle.com) wrote: Hi Tony, On Wed, 2019-07-03 at 12:57 -0700, Tony Printezis wrote: > Hi all, > > This is a follow-up to: > > 8223575: add subspace transitions to gc+metaspace=info log lines > > (that I just pushed) and adds a test to sanity check the > gc+metaspace=info > output. Webrev here: > > http://cr.openjdk.java.net/~tonyp/8227179/webrev.0/ > test looks good except for some issue with builds that do not build all GCs. I.e. you need to split the single @test block into multiple, for each collector one and add a @requires tag. E.g. gc/TestSystemGC.java shows how to do that. Thanks, Thomas From sci at amazon.com Mon Jul 8 17:39:03 2019 From: sci at amazon.com (Sciampacone, Ryan) Date: Mon, 8 Jul 2019 17:39:03 +0000 Subject: 8227226: Segmented array clearing for ZGC In-Reply-To: <5809FFE3-ED37-429B-9189-49D8FD14D092@amazon.com> References: <5809FFE3-ED37-429B-9189-49D8FD14D092@amazon.com> Message-ID: <625131FC-8B02-4BEC-80B5-F1757232277A@amazon.com> http://cr.openjdk.java.net/~phh/8227226/webrev.01/ This shifts away from abusing the constructor do_zero magic in exchange for virtualizing mem_clear() and specializing for the Z version. It does create a change in mem_clear in that it returns an updated version of mem. It does create change outside of the Z code however it does feel cleaner. I didn't make a change to PinAllocating - looking at it, the safety of keeping it constructor / destructor based still seemed appropriate to me. If the objection is to using the sequence numbers to pin (and instead using handles to update) - this to me seems less error prone. I had originally discussed handles with Stefan but the proposal came down to this which looks much cleaner. ?On 7/8/19, 6:36 AM, "hotspot-gc-dev on behalf of Sciampacone, Ryan" wrote: 1) Yes this was a conscious decision. There was discussion on determining the optimal point for breakup but given the existing sizes this seemed sufficient. This doesn't preclude the ability to go down that path if its deemed absolutely necessary. The path for more complex decisions is now available. 2) Agree 3) I'm not clear here. Do you mean effectively going direct to ZHeap and bypassing the single function PinAllocating? Agree. Otherwise I'll ask you to be a bit clearer. 4) Agree 5) I initially had the exact same reaction but I played around with a few other versions (including breaking up initialization points between header and body to get the desired results) and this ended up looking correct. I'll try mixing in the mem clearer function again (fresh start) to see if it looks any better. On 7/8/19, 5:49 AM, "Per Liden" wrote: Hi Ryan, A few general comments: 1) It looks like this still only work for large pages? 2) The log_info stuff should be removed. 3) I'm not a huge fan of single-use utilities like PinAllocating, at least not when, IMO, the alternative is more straight forward and less code. 4) Please make locals const when possible. 5) Duplicating _do_zero looks odd. Injecting a "mem clearer", similar to what Stefans original patch did, seems worth exploring. cheers, /Per (Btw, I'm on vacation so I might not be super-responsive to emails) On 2019-07-08 12:42, Erik ?sterlund wrote: > Hi Ryan, > > This looks good in general. Just some stylistic things... > > 1) In the ZGC project we like the letter 'Z' so much that we put it in > front of everything we possibly can, including all class names. > 2) We also explicitly state things are private even though it's > bleedingly obvious. > > So: > > 39 class PinAllocating { > 40 HeapWord* _mem; > 41 public: -> 39 class ZPinAllocating { 40 private: 41 HeapWord* _mem; > 42 > 41 public: I can be your sponsor and push this change for you. I don't > think there is a need for another webrev for my small stylistic remarks, > so I can just fix that before pushing this for you. On that note, I'll > add me and StefanK to the contributed-by section as we all worked out > the right solution to this problem collaboratively. I have run through > mach5 tier1-5, and found no issues with this patch. Thanks, /Erik > > On 2019-07-05 17:18, Sciampacone, Ryan wrote: >> http://cr.openjdk.java.net/~phh/8227226/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8227226 >> >> This patch introduces safe point checks into array clearing during >> allocation for ZGC. The patch isolates the changes to ZGC as (in >> particular with the more modern collectors) the approach to >> incrementalizing or respecting safe point checks is going to be >> different. >> >> The approach is to keep the region holding the array in the allocating >> state (pin logic) while updating the color to the array after checks. >> >> Can I get a review? Thanks. >> >> Ryan > From tprintezis at twitter.com Mon Jul 8 18:42:40 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Mon, 8 Jul 2019 11:42:40 -0700 Subject: RFR: 8217170: gc/arguments/TestUseCompressedOopsErgo.java timed out In-Reply-To: <235C3194-422B-4028-83C4-44D1E4BC599B@oracle.com> References: <235C3194-422B-4028-83C4-44D1E4BC599B@oracle.com> Message-ID: Looks good to me too. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On July 6, 2019 at 5:47:49 PM, Kim Barrett (kim.barrett at oracle.com) wrote: Please review this change to many of the tests in gc/arguments to reduce the chances of some of them timing out on some platform configurations, especially when run with debug builds. These tests typically launch a series of one or more child processes with specific command line options chosen to test that specific configuration. In debug builds there is some extra work done by default by each of those child processes that is unnecessary for the use at hand, and may (in some platform configurations) add a lot to the time taken by the child. ZapUnusedHeapArea has been identified as the major offender, with VerifyBeforeExit being a lesser contributor. Both of these are enabled by default in debug builds. To address the problem there is a new helper class, GCArguments, which provides wrappers around ProcessTools.createJavaProcessBuilder to add options to disable those problematic features. The options are added at the front, so they can be re-enabled by later options from a test if desired (such as the test that checks VerifyBeforeExit). The tests have been updated to call that new wrapper. There is still some residual odd slowness to at least some of these tests. The child processes usually take 10's to low 100's of milliseconds to run, but sometimes 3-5 seconds (or 20+ seconds when using CMS; don't know what's up with that). So there's further investigation to be done, but the changes being made here at least reduce the spurious timeout noise in regression testing. CR: https://bugs.openjdk.java.net/browse/JDK-8217170 Webrev: http://cr.openjdk.java.net/~kbarrett/8217170/open.00/ Testing: mach5 tier1-3 Ran tier1 entirely on a machine that is known for having reliable timeouts without this change. From kim.barrett at oracle.com Mon Jul 8 20:55:58 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 8 Jul 2019 16:55:58 -0400 Subject: RFR: 8217170: gc/arguments/TestUseCompressedOopsErgo.java timed out In-Reply-To: References: <235C3194-422B-4028-83C4-44D1E4BC599B@oracle.com> Message-ID: <9E49FB51-4F39-4F5D-965C-B208FA466C8A@oracle.com> > On Jul 8, 2019, at 4:44 AM, Thomas Schatzl wrote: > > Hi, > > On Sat, 2019-07-06 at 17:45 -0400, Kim Barrett wrote: >> Please review this change to many of the tests in gc/arguments to >> reduce the chances of some of them timing out on some platform >> configurations, especially when run with debug builds. >> > [..] >> To address the problem there is a new helper class, GCArguments, >> which provides wrappers around ProcessTools.createJavaProcessBuilder >> to add options to disable those problematic features. The options >> are added at the front, so they can be re-enabled by later options >> from a test if desired (such as the test that checks >> VerifyBeforeExit). The tests have been updated to call that new >> wrapper. >> >> There is still some residual odd slowness to at least some of these >> tests. The child processes usually take 10's to low 100's of >> milliseconds to run, but sometimes 3-5 seconds (or 20+ seconds when >> using CMS; don't know what's up with that). So there's further >> investigation to be done, but the changes being made here at least >> reduce the spurious timeout noise in regression testing. > > There are a few other trueInDebug flags, but apart from > ZapFillerObjects there is nothing obvious. And even that seems doubtful > to cause issues. Yes, I didn?t see anything obvious when looking at trueInDebug flags either. Getting relevant information out of these child processes seems to be somewhat tricky, but Markus Gronlund is helping me with that. We?ll see what shows up; I?m guessing that once we get the proper state information it will be completely obvious and we?ll wonder why we didn?t think of that... >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8217170 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8217170/open.00/ >> > > looks good. > > Thomas Thanks. From kim.barrett at oracle.com Mon Jul 8 21:08:15 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 8 Jul 2019 17:08:15 -0400 Subject: RFR: 8217170: gc/arguments/TestUseCompressedOopsErgo.java timed out In-Reply-To: References: <235C3194-422B-4028-83C4-44D1E4BC599B@oracle.com> Message-ID: <3A7F4E5C-E4D1-4CEE-B9F2-7C1840A54E39@oracle.com> > On Jul 7, 2019, at 8:35 AM, Tony Printezis wrote: > > Kim, > > If the tests start with a very small initial heap, CMS will set the young gen size to something very small and then not resize it during the run. This results in an unnecessarily high young GC overhead. We?ve seen that in quite a few occasions. Could this be the reason for the child processes taking a long time with CMS? Maybe? These tests mostly don?t specify minimum or initial heap sizes, except where doing so is part of what is being tested. But I don?t know if we?re even doing any GCs in the child processes; most of them are just ?java -version?. Having some default option values or behaviors that sometimes makes that really slow seems like a problem, right? But maybe part of the answer might be that CMS is the wrong GC to use in production for small, short-lived applications? I?m not sure how much effort I?ll be able to put into running to ground the CMS-specific part of this problem. CMS has been deprecated for a while, and isn?t getting a lot of resources allocated to non-critical problems. From kim.barrett at oracle.com Mon Jul 8 21:08:30 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 8 Jul 2019 17:08:30 -0400 Subject: RFR: 8217170: gc/arguments/TestUseCompressedOopsErgo.java timed out In-Reply-To: References: <235C3194-422B-4028-83C4-44D1E4BC599B@oracle.com> Message-ID: <5DF87551-A24E-464B-B80A-CFF863F96A29@oracle.com> > On Jul 8, 2019, at 2:42 PM, Tony Printezis wrote: > > Looks good to me too. Thanks. From tobias.hartmann at oracle.com Tue Jul 9 06:29:45 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 9 Jul 2019 08:29:45 +0200 Subject: [13] RFR(S): 8226536: Catch OOM from deopt that fails rematerializing objects In-Reply-To: References: Message-ID: <3047181d-bb65-ed3d-9dc0-1faa4d2737f5@oracle.com> Hi Nils, On 04.07.19 15:27, Nils Eliasson wrote: > It relies on not inlining the inner method, because then it would be part of the compiled frame > being deoptimized. Shouldn't the tests then be adapter to exclude inling of that method via a compile command? Also noticed that the indentation is wrong in line 208. Best regards, Tobias From tobias.hartmann at oracle.com Tue Jul 9 06:36:32 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 9 Jul 2019 08:36:32 +0200 Subject: [13] RFR(S): 8226536: Catch OOM from deopt that fails rematerializing objects In-Reply-To: <08cff491-b348-1782-1cd5-826c4c83b47b@oracle.com> References: <94bfdef5-5cbf-86e9-93ce-ed7d900adc76@oracle.com> <246fc050-bac8-9a3d-5671-55b23826cf6f@oracle.com> <08cff491-b348-1782-1cd5-826c4c83b47b@oracle.com> Message-ID: Only now noticed that there is already is an ongoing discussion, please ignore my other review. On 05.07.19 16:50, Nils Eliasson wrote: > http://cr.openjdk.java.net/~neliasso/8226536/webrev.04/ Looks good to me assuming that we can never inline that method handle call. Best regards, Tobias From vladimir.x.ivanov at oracle.com Tue Jul 9 10:46:24 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 9 Jul 2019 13:46:24 +0300 Subject: [13] RFR(S): 8226536: Catch OOM from deopt that fails rematerializing objects In-Reply-To: <08cff491-b348-1782-1cd5-826c4c83b47b@oracle.com> References: <94bfdef5-5cbf-86e9-93ce-ed7d900adc76@oracle.com> <246fc050-bac8-9a3d-5671-55b23826cf6f@oracle.com> <08cff491-b348-1782-1cd5-826c4c83b47b@oracle.com> Message-ID: <3b1558dc-ee5d-961c-9282-5170c367f16b@oracle.com> > So here is a version using methodhandles instead: > > http://cr.openjdk.java.net/~neliasso/8226536/webrev.04/ You could do MethodHandle lookup just once (e.g., as part of initialization and put MethodHandle instance into a field), but current version looks good as well. Best regards, Vladimir Ivanov > > // Nils > > On 2019-07-05 13:18, Vladimir Ivanov wrote: >> >>> Sounds like properly annotating this will clobber a lot of tests (~40?). >>> Not sure it's worth the hassle when this RFE is only about silencing >>> the tests (which it does as the method is too large to get inlined), >>> as opposed to fixing the underlying issue. >>> The worst that could happen is that the tests start moaning again >>> with new inlining heuristics, which is okay IMO. >> >> I'd prefer to introduce a reliable fix and not bother with it again. >> >> If the concern is the amount of changes needed, calling eatMemoryImpl >> using a non-constant MethodHandle (cached in a non-static non-final >> field) would block inlining as well. >> >> Best regards, >> Vladimir Ivanov >> >>> On 2019-07-05 12:20, Vladimir Ivanov wrote: >>>>> Doesn't that only work on classes loaded by the Bootstrap classloader? >>>> >>>> Yes, that would require putting GarbageUtils on boot class path or >>>> running affected tests with main/bootclasspath/othervm. >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>>> >>>>> // Nils >>>>> >>>>> On 2019-07-05 12:00, Vladimir Ivanov wrote: >>>>>> test/hotspot/jtreg/vmTestbase/nsk/share/gc/gp/GarbageUtils.java: >>>>>> >>>>>> +????????? * It is Important that the impl is not inlined. >>>>>> >>>>>> What do you think about putting @DontInline on eatMemoryImpl to >>>>>> guarantee that invariant? >>>>>> >>>>>> Best regards, >>>>>> Vladimir Ivanov >>>>>> >>>>>> On 04/07/2019 16:27, Nils Eliasson wrote: >>>>>>> Hi, >>>>>>> >>>>>>> This is a patch of GarbageUtils.java used by a bunch of GC tests. >>>>>>> >>>>>>> This issue was diagnosed by Stefan Karlsson and passed to the >>>>>>> compiler team. >>>>>>> >>>>>>> The core problem is that the test relies on successfully catching >>>>>>> OOM errors. But the OOM causes a deopt, and the deopt must >>>>>>> rematerialize some objects before passing the execution on to the >>>>>>> interpreter to run the catch-block. If the rematerialization >>>>>>> can't allocate, it will pass the OOM on, without having run the >>>>>>> catch-block. >>>>>>> >>>>>>> With this patch I fix the tests by wrapping the faulting code >>>>>>> with an extra layer of try-catch-OOM. It relies on not inlining >>>>>>> the inner method, because then it would be part of the compiled >>>>>>> frame being deoptimized. >>>>>>> >>>>>>> In general this is a problem for any kind of deopt when we are >>>>>>> out of heap. Another bug will be opened to keep tracking this issue. >>>>>>> >>>>>>> Webrev:http://cr.openjdk.java.net/~neliasso/8226536/webrev.02/ >>>>>>> >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8226536 >>>>>>> >>>>>>> // Nils >>>>>>> >>> From thomas.schatzl at oracle.com Tue Jul 9 12:36:15 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 09 Jul 2019 14:36:15 +0200 Subject: RFR (S): 8227179: Test for new gc+metaspace=info output format In-Reply-To: References: Message-ID: Hi, On Mon, 2019-07-08 at 08:09 -0700, Tony Printezis wrote: > Thanks for pointing that out, Thomas. Updated webrev (the only change > is the @test comments at the top): > > http://cr.openjdk.java.net/~tonyp/8227179/webrev.1/ > > I tested it with a ?normal? build (all GCs built) and one with no CMS > and the test seemed to have done the right thing in both occasions. > looks good to me. Thomas From sgehwolf at redhat.com Tue Jul 9 12:37:09 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 09 Jul 2019 14:37:09 +0200 Subject: [8u] RFR: 8135318: CMS wrong max_eden_size for check_gc_overhead_limit Message-ID: Hi, Could I please get a review for this OpenJDK 8u backport? It's essentially the same patch as in JDK 9+, but it didn't apply cleanly since the surrounding code is different and the files were moved around. Adjustments I've done: JDK 8u uses a different file layout: src/share/vm/gc/cms/concurrentMarkSweepGeneration.cpp => src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp JDK 8u doesn't have JDK-8065972 and JDK-8065992, which account for the surrounding code changes. Note that JDK-8065992 makes the _young_gen instance variable ParNewGEneration* from Generation*. Bug: https://bugs.openjdk.java.net/browse/JDK-8135318 webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8135318/01/webrev/ JDK 9 change: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/dd0c55eac358 Testing: "tier1" testing of Linux x86_64 which showed no new regressions. Thoughts? Thanks, Severin From tprintezis at twitter.com Tue Jul 9 17:18:50 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Tue, 9 Jul 2019 19:18:50 +0200 Subject: RFR: 8217170: gc/arguments/TestUseCompressedOopsErgo.java timed out In-Reply-To: <3A7F4E5C-E4D1-4CEE-B9F2-7C1840A54E39@oracle.com> References: <235C3194-422B-4028-83C4-44D1E4BC599B@oracle.com> <3A7F4E5C-E4D1-4CEE-B9F2-7C1840A54E39@oracle.com> Message-ID: Hi Kim, I?ll be happy to try trouble-shooting this. It's clearly of interest to us as we do use CMS and it?d be nice to get the tests to finish quicker. I tried reproducing the 20sec durations for the child processes with CMS but I was not able to. They seem to have more or less the same duration as with the other GCs. Do you remember which tests you saw this issue in? Also, what type of machine did you run on (?small? laptop, ?larger? workstation, etc)? BTW, meant to say along with my review: I tried your patch locally and it shaved off around 35% of the elapsed time for the gc/arguments tests on my workstation. So a nice improvement! Thanks! Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On July 8, 2019 at 5:08:21 PM, Kim Barrett (kim.barrett at oracle.com) wrote: > On Jul 7, 2019, at 8:35 AM, Tony Printezis wrote: > > Kim, > > If the tests start with a very small initial heap, CMS will set the young gen size to something very small and then not resize it during the run. This results in an unnecessarily high young GC overhead. We?ve seen that in quite a few occasions. Could this be the reason for the child processes taking a long time with CMS? Maybe? These tests mostly don?t specify minimum or initial heap sizes, except where doing so is part of what is being tested. But I don?t know if we?re even doing any GCs in the child processes; most of them are just ?java -version?. Having some default option values or behaviors that sometimes makes that really slow seems like a problem, right? But maybe part of the answer might be that CMS is the wrong GC to use in production for small, short-lived applications? I?m not sure how much effort I?ll be able to put into running to ground the CMS-specific part of this problem. CMS has been deprecated for a while, and isn?t getting a lot of resources allocated to non-critical problems. From kim.barrett at oracle.com Tue Jul 9 21:54:19 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 9 Jul 2019 17:54:19 -0400 Subject: RFR: 8217170: gc/arguments/TestUseCompressedOopsErgo.java timed out In-Reply-To: References: <235C3194-422B-4028-83C4-44D1E4BC599B@oracle.com> <3A7F4E5C-E4D1-4CEE-B9F2-7C1840A54E39@oracle.com> Message-ID: <8F2C5A99-A199-47C1-A0C4-1E5A3D59B6AC@oracle.com> > On Jul 9, 2019, at 1:18 PM, Tony Printezis wrote: > > Hi Kim, > > I?ll be happy to try trouble-shooting this. It's clearly of interest to us as we do use CMS and it?d be nice to get the tests to finish quicker. I tried reproducing the 20sec durations for the child processes with CMS but I was not able to. They seem to have more or less the same duration as with the other GCs. Do you remember which tests you saw this issue in? Also, what type of machine did you run on (?small? laptop, ?larger? workstation, etc)? I'm in the middle of a couple of other things right now, so would be really happy to have someone else poke at this for a bit. I filed JDK-8227414 for followup investigation. Markus Gronlund recently attached a hacky patch (windows_process_abort.patch) to 8217170 to help with the investigation. That probably ought to get copied to JDK-8227414. The test I've mostly been looking at is TestUseCompressedOopsErgo, which runs over a dozen subprocesses. There's a fair amount of information in JDK-8217170 comments, but I'll try to summarize some of it here to save you from lots of scrolling. The machines where I've been able to fairly reliably provoke the problem are a couple of Windows VMs in our build&test farm. They are configured with 8 cores and 60G of memory. They are known to be "slow", so we don't use them for builds, only for tests. jtreg test concurrency is 4, I think. Because they are VMs, the hypervisor might be running other VMs that are also running tests. (Indeed, those two machines are on the same hypervisor.) Some of the issue may be related to load on the VM and/or hypervisor; at least, load seems to be useful in provocation. I've seen the problem to a similar or lesser degree on other Windows machines, e.g. the reported waitFor time is unusually long (at least seconds rather than (usually significantly) subsecond). There's a bit of logging spew in the .jtr files to help spot that, added by JDK-8219149. I *think* I've seen the problem to a lesser degree on Linux. I've not looked at Mac or Solaris at all. Something that I find puzzling is that I think in all the runs of TestUseCompressedOopsErgo that I've looked at, either all of the subprocesses were quick, or all were multi-second slow. > BTW, meant to say along with my review: I tried your patch locally and it shaved off around 35% of the elapsed time for the gc/arguments tests on my workstation. So a nice improvement! Thanks! Oh, that?s nice! I hadn?t measured that. From thomas.schatzl at oracle.com Wed Jul 10 15:19:27 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 10 Jul 2019 17:19:27 +0200 Subject: RFR(S): 8227225: ParallelGC: add subspace transitions for young gen for gc+heap=info log lines In-Reply-To: References: Message-ID: <877bd17cff3be20bf8acd95d9eddbb5c9cfb7cf5.camel@oracle.com> On Fri, 2019-07-05 at 11:06 -0700, Tony Printezis wrote: > Hi all, > > This is a follow-up to 8223575 and adds subspace info for eden and > from space to the gc+heap=info output for ParallelGC. I also added > the before capacity, similar to 8223575, as it can change during the > GC (see, in fact, example below). > > Before: > > [11.123s][info][gc,heap ] GC(17) PSYoungGen: 4759616K- > >1664K(4545536K) > [11.123s][info][gc,heap ] GC(17) ParOldGen: 1280K- > >1280K(1376256K) > > After: > > [12.133s][info][gc,heap ] GC(25) PSYoungGen: > 2065952K(2066432K)->1536K(1983488K) Eden: 2064384K(2064384K)- > >0K(1981952K) > From: 1568K(2048K)->1536K(1536K) Note that I think this is confusing, or the "From" label should be renamed. The second part of this heap transition actually prints the "To" space from what I understand. What does it print on evacuation failure? It would be cleaner imho to print From and To space separately. > > [12.133s][info][gc,heap ] GC(25) ParOldGen: > 1272K(1376256K)->1272K(1376256K) > > http://cr.openjdk.java.net/~tonyp/8227225/webrev.0/ > > I have a similar change ready to go for GenCollectedHeap (Serial / > CMS). > I?ll see what the feedback is on this before I open that for code > review. > > A couple of notes: > > - I moved the PreGCValues class to gc/shared as I?d like to re-use > it for the GenCollectedHeap changes too. It's a bit awkwards as there are non-generational GCs now, but I do not have a better idea. Maybe somebody else has. > - Is it worth also adding info about To space to the output? I think so. (Note that I did not really review this change yet, just answering questions) Thanks, Thomas > > Tony > > PS Should I add the gc-pending-review label to any CR I open for code > review? As discussed, feel free to do so, but it may or may not help. Note that there is no need to remove that label manually: if you set your JIRA filter to ignore already fixed/resolved issues they will not be shown automatically (if you filter). Once or twice a year we typically bulk-remove these labels from already resolved/closed issues. From thomas.schatzl at oracle.com Wed Jul 10 15:30:27 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 10 Jul 2019 17:30:27 +0200 Subject: RFR (S): 8226232: Move merge heap roots code out from G1RemSetScanState Message-ID: Hi all, can I have reviews for this small change that moves code related to the merge heap roots phase out of G1RemSetScanState? This imho significantly increases readability of G1RemSetScanState. More minor cleanups to improve the code are planned, but I think it is good to take it in steps. CR: https://bugs.openjdk.java.net/browse/JDK-8226232 Webrev: http://cr.openjdk.java.net/~tschatzl/8226232/webrev/ Testing: hs-tier1-5 Thanks, Thomas From thomas.schatzl at oracle.com Wed Jul 10 15:36:03 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 10 Jul 2019 17:36:03 +0200 Subject: RFR (S): 8227084: Add timing information for merge heap root preparation Message-ID: Hi all, can I have reviews for this change that adds timing information for parts of the Merge Heap Roots phase as preparation for JDK-8226913. CR: https://bugs.openjdk.java.net/browse/JDK-8227084 Webrev: http://cr.openjdk.java.net/~tschatzl/8227084 Testing: hs-tier1-5 Thanks, Thomas From thomas.schatzl at oracle.com Wed Jul 10 15:40:02 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 10 Jul 2019 17:40:02 +0200 Subject: RFR (S): 8227089: Add timing information for merging humongous remembered sets Message-ID: <7eafbdb5dc7c9906021c1fbc43f5912d8ddc267f.camel@oracle.com> Hi all, can I have reviews for this change that adds timing information for the part of the merge heap roots phase that dumps eager reclaim candidates onto the card table? I am aware that the eager reclaim phase is actually single threaded - there is an extra CR for fixing this (JDK-8141637). At least with recent change that effort is hidden by the other work in this phase; but until now we did not know that there may be an issue. CR: https://bugs.openjdk.java.net/browse/JDK-8227089 Webrev: http://cr.openjdk.java.net/~tschatzl/8227089/webrev Testing: hs-tier1-5 Thanks, Thomas From thomas.schatzl at oracle.com Wed Jul 10 15:42:20 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 10 Jul 2019 17:42:20 +0200 Subject: RFR (XXS): 8227090: G1 does not account the OptMergeRS in cost per card calculation Message-ID: <80247f44da3ad38ed6c66d13a89ff52b17e69e34.camel@oracle.com> Hi all, can I have reviews for this (almost-)one-liner that makes sure that G1 accounts the OptMergeRS time in the cost per card calculation? CR: https://bugs.openjdk.java.net/browse/JDK-8227090 Webrev: http://cr.openjdk.java.net/~tschatzl/8227090/webrev Testing: hs-tier1-5 Thanks, Thomas From thomas.schatzl at oracle.com Wed Jul 10 15:48:45 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 10 Jul 2019 17:48:45 +0200 Subject: RFR (M): 8226913: Scale cards per chunk used during heap root scanning with region Message-ID: <4ec44e39449e8aba219a491afbfc5871e109faee.camel@oracle.com> Hi all, can I have reviews for this change that scales the amount of cards processed per chunk depending on heap/heap region size to decrease some unnecessary overhead when clearing it on larger heaps. It is basically the answer to this comment in the code: 96 // Random power of two number of cards we want to claim per thread. This corresponds 97 // to a 64k of memory work chunk area for every thread. 98 // We use the same claim size as Parallel GC. No particular measurements have been 99 // performed to determine an optimal number. I.e. the chosen number simply seemed suboptimal, and should actually be determined adaptively. CR: https://bugs.openjdk.java.net/browse/JDK-8226913 Webrev: http://cr.openjdk.java.net/~tschatzl/8226913/webrev Testing: hs-tier1-5, manually checking that correct chunk sizes are returned for all possible region sizes Thanks, Thomas From nils.eliasson at oracle.com Wed Jul 10 16:52:58 2019 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Wed, 10 Jul 2019 18:52:58 +0200 Subject: [13] RFR(S): 8226536: Catch OOM from deopt that fails rematerializing objects In-Reply-To: <3b1558dc-ee5d-961c-9282-5170c367f16b@oracle.com> References: <94bfdef5-5cbf-86e9-93ce-ed7d900adc76@oracle.com> <246fc050-bac8-9a3d-5671-55b23826cf6f@oracle.com> <08cff491-b348-1782-1cd5-826c4c83b47b@oracle.com> <3b1558dc-ee5d-961c-9282-5170c367f16b@oracle.com> Message-ID: <09664b6f-8691-af28-f060-6773fbea0049@oracle.com> Thank you Vladimir and Tobias! // Nils On 2019-07-09 12:46, Vladimir Ivanov wrote: > > >> So here is a version using methodhandles instead: >> >> http://cr.openjdk.java.net/~neliasso/8226536/webrev.04/ > > You could do MethodHandle lookup just once (e.g., as part of > initialization and put MethodHandle instance into a field), but > current version looks good as well. > > Best regards, > Vladimir Ivanov > >> >> // Nils >> >> On 2019-07-05 13:18, Vladimir Ivanov wrote: >>> >>>> Sounds like properly annotating this will clobber a lot of tests >>>> (~40?). >>>> Not sure it's worth the hassle when this RFE is only about >>>> silencing the tests (which it does as the method is too large to >>>> get inlined), as opposed to fixing the underlying issue. >>>> The worst that could happen is that the tests start moaning again >>>> with new inlining heuristics, which is okay IMO. >>> >>> I'd prefer to introduce a reliable fix and not bother with it again. >>> >>> If the concern is the amount of changes needed, calling >>> eatMemoryImpl using a non-constant MethodHandle (cached in a >>> non-static non-final field) would block inlining as well. >>> >>> Best regards, >>> Vladimir Ivanov >>> >>>> On 2019-07-05 12:20, Vladimir Ivanov wrote: >>>>>> Doesn't that only work on classes loaded by the Bootstrap >>>>>> classloader? >>>>> >>>>> Yes, that would require putting GarbageUtils on boot class path or >>>>> running affected tests with main/bootclasspath/othervm. >>>>> >>>>> Best regards, >>>>> Vladimir Ivanov >>>>> >>>>>> >>>>>> // Nils >>>>>> >>>>>> On 2019-07-05 12:00, Vladimir Ivanov wrote: >>>>>>> test/hotspot/jtreg/vmTestbase/nsk/share/gc/gp/GarbageUtils.java: >>>>>>> >>>>>>> +????????? * It is Important that the impl is not inlined. >>>>>>> >>>>>>> What do you think about putting @DontInline on eatMemoryImpl to >>>>>>> guarantee that invariant? >>>>>>> >>>>>>> Best regards, >>>>>>> Vladimir Ivanov >>>>>>> >>>>>>> On 04/07/2019 16:27, Nils Eliasson wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> This is a patch of GarbageUtils.java used by a bunch of GC tests. >>>>>>>> >>>>>>>> This issue was diagnosed by Stefan Karlsson and passed to the >>>>>>>> compiler team. >>>>>>>> >>>>>>>> The core problem is that the test relies on successfully >>>>>>>> catching OOM errors. But the OOM causes a deopt, and the deopt >>>>>>>> must rematerialize some objects before passing the execution on >>>>>>>> to the interpreter to run the catch-block. If the >>>>>>>> rematerialization can't allocate, it will pass the OOM on, >>>>>>>> without having run the catch-block. >>>>>>>> >>>>>>>> With this patch I fix the tests by wrapping the faulting code >>>>>>>> with an extra layer of try-catch-OOM. It relies on not inlining >>>>>>>> the inner method, because then it would be part of the compiled >>>>>>>> frame being deoptimized. >>>>>>>> >>>>>>>> In general this is a problem for any kind of deopt when we are >>>>>>>> out of heap. Another bug will be opened to keep tracking this >>>>>>>> issue. >>>>>>>> >>>>>>>> Webrev:http://cr.openjdk.java.net/~neliasso/8226536/webrev.02/ >>>>>>>> >>>>>>>> >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8226536 >>>>>>>> >>>>>>>> // Nils >>>>>>>> >>>> From rkennke at redhat.com Wed Jul 10 18:14:19 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 10 Jul 2019 20:14:19 +0200 Subject: RFR: JDK-8226695: Shenandoah: Wire C1 and C2 IN_NATIVE barrier In-Reply-To: References: Message-ID: Ping? Roman > IN_NATIVE accesses in compiled code should go to runtime, in order to be > able to check reachability of accessed objects, and possibly offer NULL > for unreachable objects. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8226695 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8226695/webrev.00/ > > Testing: hotspot_gc_dev > > Ok? > > Roman > From kim.barrett at oracle.com Wed Jul 10 22:43:44 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 10 Jul 2019 18:43:44 -0400 Subject: RFR (XXS): 8227090: G1 does not account the OptMergeRS in cost per card calculation In-Reply-To: <80247f44da3ad38ed6c66d13a89ff52b17e69e34.camel@oracle.com> References: <80247f44da3ad38ed6c66d13a89ff52b17e69e34.camel@oracle.com> Message-ID: <23BD546E-8327-480E-90D8-AA4514CA4BDB@oracle.com> > On Jul 10, 2019, at 11:42 AM, Thomas Schatzl wrote: > > Hi all, > > can I have reviews for this (almost-)one-liner that makes sure that > G1 accounts the OptMergeRS time in the cost per card calculation? > > CR: > https://bugs.openjdk.java.net/browse/JDK-8227090 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8227090/webrev > Testing: > hs-tier1-5 > > Thanks, > Thomas Looks good. From thomas.schatzl at oracle.com Thu Jul 11 07:29:26 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 11 Jul 2019 09:29:26 +0200 Subject: RFR (XXS): 8227090: G1 does not account the OptMergeRS in cost per card calculation In-Reply-To: <23BD546E-8327-480E-90D8-AA4514CA4BDB@oracle.com> References: <80247f44da3ad38ed6c66d13a89ff52b17e69e34.camel@oracle.com> <23BD546E-8327-480E-90D8-AA4514CA4BDB@oracle.com> Message-ID: Hi, On Wed, 2019-07-10 at 18:43 -0400, Kim Barrett wrote: > > On Jul 10, 2019, at 11:42 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > Hi all, > > > > can I have reviews for this (almost-)one-liner that makes sure > > that G1 accounts the OptMergeRS time in the cost per card > > calculation? > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8227090 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8227090/webrev > > Testing: > > hs-tier1-5 > > > > Thanks, > > Thomas > > Looks good. > thanks for your review :) Thomas From rwestrel at redhat.com Thu Jul 11 07:51:37 2019 From: rwestrel at redhat.com (Roland Westrelin) Date: Thu, 11 Jul 2019 09:51:37 +0200 Subject: RFR: JDK-8226695: Shenandoah: Wire C1 and C2 IN_NATIVE barrier In-Reply-To: References: Message-ID: <877e8p57qu.fsf@redhat.com> >> http://cr.openjdk.java.net/~rkennke/JDK-8226695/webrev.00/ Looks good to me. Roland. From thomas.schatzl at oracle.com Thu Jul 11 09:38:30 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 11 Jul 2019 11:38:30 +0200 Subject: RFR (XS): Optimize branch frequency of G1's write post-barrier in C2 In-Reply-To: References: Message-ID: Hi, On Thu, 2019-06-13 at 22:13 -0700, Man Cao wrote: > Hi all, > Can I have reviews for this improvement for G1's write post-barrier? > More details are stated on the JBS page. Highlight: it reduces CPU- > cost-per-query by 1% for Google search frontend's production > workload. > Webrev: https://cr.openjdk.java.net/~manc/8225776/webrev.00/ > RFE: https://bugs.openjdk.java.net/browse/JDK-8225776 > > Some notes: > The RFE could be a duplicate of JDK-8130918. However, this patch does > not improve the performance of the microbenchmark in JDK-8130918. I'm > not sure if this patch fully addresses JDK-8130918. > Chuck Rasbold helped me to figure out the proper fix for the basic > block ordering by looking at the CFG before and after C2's > PhaseBlockLayout. The out-most if branch (xor) has to have a > frequency greater than 0.5 to make the BBs laid out correctly. It is > also more conventional to use PROB_LIKELY_MAG than PROB_LIKELY in C2. - the changes do what you would expect, laying out the xor/null check in the fast path and the rest in the slow path. I also think the use of PROB_LIKELY_MAG is okay and equivalent to the original code. CC'ing compiler-dev to look over it again just in case. - can you share the code changes to generate the statistics? It would be nice to confirm these on a few more applications and play around with them a bit :) I would like to confirm some very old numbers we have for other older benchmarks that this is indeed the best probabibility distribution. Particularly I do not understand that from these numbers we did not change the probabilities as you suggested :( There were other changes mostly related to barrier elision in that time frame, but it seems likelihood changes were not attempted. - these numbers (and yours) also indicate that the not-young check is very likely to be not taken (i.e. you jump over the storeload). Did you also perform some experiments changing the order a bit? It might be detrimental for this particular case where the StoreLoad is expensive, and the xor/non-null filter out at least some additional of those, but maybe if (young) -> exit if (different-region) -> exit if (non-null) -> exit StoreLoad ... may be better to do? I am aware that the "young" check adds a load, which is also expensive (but not as much as the StoreLoad), but it seems to be an interesting case to look at. In our old results (as far as I can interpret them) it did not seem to have any advantage/disadvantage, so I am just curious whether you did such tests and their conclusion. - internal (quick) perf testing showed no overall score changes, except that maxJOPS on SpecJBB2015 seemed to improve by ~1.2% (only had time for very few experiments at this time, will rerun, so there is some chance that this has been a fluke) which is definitely nice. So, looks good to me :) Thanks, Thomas From zgu at redhat.com Thu Jul 11 13:43:45 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 11 Jul 2019 09:43:45 -0400 Subject: RFR 8225483: Shenandoah: Enhance native access barrier In-Reply-To: References: <9d9f5bb2-eff2-e117-d1a4-49339a2cd124@redhat.com> <8a2d332d-6005-64ed-9fbc-7acf388f95b4@redhat.com> <5605fa8e-4e19-5a7a-04cd-e480fd35fa5f@redhat.com> <606DB543-B668-4530-BD8F-5B26BC9AFFE3@redhat.com> <9ccaef06-f6b1-7e8c-8f7f-f23063354290@redhat.com> <13F7044D-4E98-4959-9F2E-8D7ED2D453EA@redhat.com> Message-ID: <9fd9acf2-446e-0f0f-bccf-e1a21ee88c0b@redhat.com> On 6/12/19 4:32 AM, Aleksey Shipilev wrote: > On 6/11/19 5:51 PM, Roman Kennke wrote: >> Am 11. Juni 2019 16:25:21 MESZ schrieb Zhengyu Gu : >> Updated: http://cr.openjdk.java.net/~zgu/JDK-8225483/webrev.02/ >> Okay now? > > I still do not like ASSERT around the declaration/definition. What prevents us from making that > method available in both release/debug? That would certainly simplify debugging: we would execute > the same code in both configurations. Rebased on Roman's JDK-8226695 fix and made SHBS::oop_store_not_in_heap() available for both configurations, as suggested by Aleksey. http://cr.openjdk.java.net/~zgu/JDK-8225483/webrev.03/ Reran hotspot_gc_shenandoah (fastdebug and release) Thanks, -Zhengyu > > -Aleksey > From kim.barrett at oracle.com Thu Jul 11 19:12:59 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 11 Jul 2019 15:12:59 -0400 Subject: RFR (S): 8226232: Move merge heap roots code out from G1RemSetScanState In-Reply-To: References: Message-ID: > On Jul 10, 2019, at 11:30 AM, Thomas Schatzl wrote: > > Hi all, > > can I have reviews for this small change that moves code related to > the merge heap roots phase out of G1RemSetScanState? This imho > significantly increases readability of G1RemSetScanState. > > More minor cleanups to improve the code are planned, but I think it is > good to take it in steps. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8226232 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8226232/webrev/ > Testing: > hs-tier1-5 > > Thanks, > Thomas Looks good. One small item: ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1RemSet.cpp 1166 { 1167 _scan_state->prepare_for_merge_heap_roots(); 1168 } Leftover extra scope nesting? ------------------------------------------------------------------------------ From kim.barrett at oracle.com Thu Jul 11 19:17:28 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 11 Jul 2019 15:17:28 -0400 Subject: RFR (S): 8227179: Test for new gc+metaspace=info output format In-Reply-To: References: Message-ID: <44B40714-1AE7-47BD-8D53-D5C8CFF267C1@oracle.com> > On Jul 8, 2019, at 11:09 AM, Tony Printezis wrote: > > Thanks for pointing that out, Thomas. Updated webrev (the only change is > the @test comments at the top): > > http://cr.openjdk.java.net/~tonyp/8227179/webrev.1/ > > I tested it with a ?normal? build (all GCs built) and one with no CMS and > the test seemed to have done the right thing in both occasions. > > Tony Looks good. From kim.barrett at oracle.com Thu Jul 11 21:06:42 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 11 Jul 2019 17:06:42 -0400 Subject: RFR (S): 8227089: Add timing information for merging humongous remembered sets In-Reply-To: <7eafbdb5dc7c9906021c1fbc43f5912d8ddc267f.camel@oracle.com> References: <7eafbdb5dc7c9906021c1fbc43f5912d8ddc267f.camel@oracle.com> Message-ID: <3DA35B5A-FC3D-4F4A-A08B-2F65A498493C@oracle.com> > On Jul 10, 2019, at 11:40 AM, Thomas Schatzl wrote: > > Hi all, > > can I have reviews for this change that adds timing information for > the part of the merge heap roots phase that dumps eager reclaim > candidates onto the card table? > > I am aware that the eager reclaim phase is actually single threaded - > there is an extra CR for fixing this (JDK-8141637). At least with > recent change that effort is hidden by the other work in this phase; > but until now we did not know that there may be an issue. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8227089 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8227089/webrev > Testing: > hs-tier1-5 > > Thanks, > Thomas Looks good. From kim.barrett at oracle.com Thu Jul 11 21:16:23 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 11 Jul 2019 17:16:23 -0400 Subject: RFR (S): 8226232: Move merge heap roots code out from G1RemSetScanState In-Reply-To: References: Message-ID: <41D3DF69-6516-45E9-BCB2-18E83496D683@oracle.com> > On Jul 11, 2019, at 3:12 PM, Kim Barrett wrote: > >> On Jul 10, 2019, at 11:30 AM, Thomas Schatzl wrote: >> [?] >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8226232 >> Webrev: >> http://cr.openjdk.java.net/~tschatzl/8226232/webrev/ >> Testing: >> hs-tier1-5 >> > > Looks good. > > One small item: > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1RemSet.cpp > 1166 { > 1167 _scan_state->prepare_for_merge_heap_roots(); > 1168 } > > Leftover extra scope nesting? > > ------------------------------------------------------------------------------ Looking at the RFR for 8227084, that extra scope is used for timing-related code. So never mind, the extra scope nesting is fine. From kim.barrett at oracle.com Thu Jul 11 21:30:42 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 11 Jul 2019 17:30:42 -0400 Subject: RFR (S): 8227084: Add timing information for merge heap root preparation In-Reply-To: References: Message-ID: > On Jul 10, 2019, at 11:36 AM, Thomas Schatzl wrote: > > Hi all, > > can I have reviews for this change that adds timing information for > parts of the Merge Heap Roots phase as preparation for JDK-8226913. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8227084 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8227084 > Testing: > hs-tier1-5 > > Thanks, > Thomas Looks good. Just one minor thing that you can change or not, as you prefer. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1RemSet.cpp 1129 assert(merge_remset_phase == G1GCPhaseTimes::MergeRS, "Wrong merge phase"); 1137 assert(merge_remset_phase == G1GCPhaseTimes::MergeRS, "Wrong merge phase"); I'm not sure these assertions carry their weight anymore. They used to verify consistency between two arguments, but now merge_remset_phase is computed locally from the argument whose value is used in the immediately enclosing if-statement. ------------------------------------------------------------------------------ An aside: We really need a helper somewhere for expressions like total.seconds() * 1000.0 I know that Tickspan::milliseconds() doesn't do what we want, because it returns an integral value. From manc at google.com Thu Jul 11 23:35:26 2019 From: manc at google.com (Man Cao) Date: Thu, 11 Jul 2019 16:35:26 -0700 Subject: RFR (XS): Optimize branch frequency of G1's write post-barrier in C2 In-Reply-To: References: Message-ID: Thanks Thomas for the review and running experiments! > - can you share the code changes to generate the statistics? It would > be nice to confirm these on a few more applications and play around > with them a bit :) > I would like to confirm some very old numbers we have for other older > benchmarks that this is indeed the best probabibility distribution. > Particularly I do not understand that from these numbers we did not > change the probabilities as you suggested :( There were other changes > mostly related to barrier elision in that time frame, but it seems > likelihood changes were not attempted. It is here: http://cr.openjdk.java.net/~manc/8225776/branch_profiling/ I also added a comment in https://bugs.openjdk.java.net/browse/JDK-8225776 to clarify the methodology. > - these numbers (and yours) also indicate that the not-young check is > very likely to be not taken (i.e. you jump over the storeload). Did you > also perform some experiments changing the order a bit? > It might be detrimental for this particular case where the StoreLoad is > expensive, and the xor/non-null filter out at least some additional of > those, but maybe > if (young) -> exit > if (different-region) -> exit > if (non-null) -> exit > StoreLoad > ... > may be better to do? I am aware that the "young" check adds a load, > which is also expensive (but not as much as the StoreLoad), but it > seems to be an interesting case to look at. > > In our old results (as far as I can interpret them) it did not seem to > have any advantage/disadvantage, so I am just curious whether you did > such tests and their conclusion. Yes, I did this experiment. The load from card table on the fast path turns out to be expensive for several benchmarks: https://cr.openjdk.java.net/~manc/8225776/20190516-jdk11G1WriteBarrier-dacapoDefault4G-YoungCheckFirst.html For this experiment, I was setting 4G heap with -XX:NewRatio=1, so most writes happen to young object, and GC happens very infrequently. The implementation had some bug that some benchmarks crashed while running. I didn't look into fixing the bug, as this direction does not seem worthwhile. > - internal (quick) perf testing showed no overall score changes, except > that maxJOPS on SpecJBB2015 seemed to improve by ~1.2% (only had time > for very few experiments at this time, will rerun, so there is some > chance that this has been a fluke) which is definitely nice. Good to hear that! -Man From thomas.schatzl at oracle.com Fri Jul 12 08:25:56 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 12 Jul 2019 10:25:56 +0200 Subject: RFR (S): 8226232: Move merge heap roots code out from G1RemSetScanState In-Reply-To: <41D3DF69-6516-45E9-BCB2-18E83496D683@oracle.com> References: <41D3DF69-6516-45E9-BCB2-18E83496D683@oracle.com> Message-ID: Hi Kim, On Thu, 2019-07-11 at 17:16 -0400, Kim Barrett wrote: > > On Jul 11, 2019, at 3:12 PM, Kim Barrett > > wrote: > > > > > On Jul 10, 2019, at 11:30 AM, Thomas Schatzl < > > > thomas.schatzl at oracle.com> wrote: > > > [?] > > > CR: > > > https://bugs.openjdk.java.net/browse/JDK-8226232 > > > Webrev: > > > http://cr.openjdk.java.net/~tschatzl/8226232/webrev/ > > > Testing: > > > [...] > > One small item: > > > > ----------------------------------------------------------------- > > ------------- > > src/hotspot/share/gc/g1/g1RemSet.cpp > > 1166 { > > 1167 _scan_state->prepare_for_merge_heap_roots(); > > 1168 } > > > > Leftover extra scope nesting? > > > > ----------------------------------------------------------------- > > ------------- > > Looking at the RFR for 8227084, that extra scope is used for timing- > related code. > So never mind, the extra scope nesting is fine. > > yes. Thanks for your review. Thomas From thomas.schatzl at oracle.com Fri Jul 12 08:31:36 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 12 Jul 2019 10:31:36 +0200 Subject: RFR (S): 8227084: Add timing information for merge heap root preparation In-Reply-To: References: Message-ID: <075a4e93144ff7736610f14c11df4c33749c25eb.camel@oracle.com> Hi, On Thu, 2019-07-11 at 17:30 -0400, Kim Barrett wrote: > > On Jul 10, 2019, at 11:36 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > Hi all, > > > > can I have reviews for this change that adds timing information > > for parts of the Merge Heap Roots phase as preparation for JDK- > > 8226913. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8227084 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8227084 > > Testing: > > hs-tier1-5 > > > > Thanks, > > Thomas > > Looks good. > > Just one minor thing that you can change or not, as you prefer. > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1RemSet.cpp > 1129 assert(merge_remset_phase == G1GCPhaseTimes::MergeRS, > "Wrong merge phase"); > 1137 assert(merge_remset_phase == G1GCPhaseTimes::MergeRS, > "Wrong merge phase"); > > I'm not sure these assertions carry their weight anymore. They used > to verify consistency between two arguments, but now > merge_remset_phase is computed locally from the argument whose value > is used in the immediately enclosing if-statement. I regenerated the webrev with those removed. > > An aside: We really need a helper somewhere for expressions like > > total.seconds() * 1000.0 > > I know that Tickspan::milliseconds() doesn't do what we want, because > it returns an integral value. > I know, it's getting annoying for me as well. It also causes unnecessary bugs. I will see if there is anything that can be extracted from the unfinished changes of JDK-8201313 I have. Thanks, Thomas From thomas.schatzl at oracle.com Fri Jul 12 08:32:17 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 12 Jul 2019 10:32:17 +0200 Subject: RFR (S): 8227089: Add timing information for merging humongous remembered sets In-Reply-To: <3DA35B5A-FC3D-4F4A-A08B-2F65A498493C@oracle.com> References: <7eafbdb5dc7c9906021c1fbc43f5912d8ddc267f.camel@oracle.com> <3DA35B5A-FC3D-4F4A-A08B-2F65A498493C@oracle.com> Message-ID: Hi, On Thu, 2019-07-11 at 17:06 -0400, Kim Barrett wrote: > > On Jul 10, 2019, at 11:40 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > Hi all, > > > > can I have reviews for this change that adds timing information > > for the part of the merge heap roots phase that dumps eager reclaim > > candidates onto the card table? > > > > I am aware that the eager reclaim phase is actually single threaded > > - there is an extra CR for fixing this (JDK-8141637). At least with > > recent change that effort is hidden by the other work in this > > phase; but until now we did not know that there may be an issue. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8227089 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8227089/webrev > > Testing: > > hs-tier1-5 > > > > Thanks, > > Thomas > > Looks good. > thanks for your review. Thomas From thomas.schatzl at oracle.com Fri Jul 12 12:36:23 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 12 Jul 2019 14:36:23 +0200 Subject: RFR: 8162929: Enqueuing dirty cards into a single DCQS during GC does not scale In-Reply-To: References: Message-ID: Hi, On Fri, 2019-07-05 at 00:50 -0400, Kim Barrett wrote: > Please review this refactoring of G1DirtyCardQueue[Set], splitting > into two sets of classes that each better support the purpose they > are used for. The existing G1DirtyCardQueue[Set] is (and remains) > the mechanism for communicating between the logging post-write > barrier and the remset, via refinement (both concurrent and > STW). The new G1RedirtyCardsQueue[Set] are used during collections > to collect and en masse dump old remset information back into that > cycle for later reconsideration. > > There is a G1RedirtyCardsQueueSet associated with the collector > object, replacing the second (non-refinement related) > G1DirtyCardQueueSet. During certain collection phases we now > allocate G1RedirtyCardsQueue objects local to worker threads, with > the collected buffers sent to that new qset. Because the collection > and processing of those buffers happen in distinct non-overlapping > phases, we can easily use a lock-free stack as the qset's completed > buffer list. We don't even need to track the number of buffers. > > This splitting allowed a little bit of immediate simplification of > G1DirtyCardQueueSet. It will also make it easier to make future > improvements in that area. > > Although the CR describes a performance problem, it seems that other > improvements in the intervening 3 years have substantially reduced > the described problems. (In particular, substantially fewer cards go > through this process.) On various performance tests this change has > not shown significant improvements, though also no regressions. It > might be that with some workloads and with enough worker threads this > change could still have significant performance benefit though. But > the main point of making this change now is the resultant code > simplification. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8162929 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8162929/open.00/ > some comments: - pre-existing: I would prefer if PtrQueue::~PtrQueue would be virtual to avoid the risk of object slicing. There is no problem right now in the code afaics though, but we've been bitten by object slicing issues before, and it's imho okay to use a virtual destructor here. - I would prefer to add an "is_" prefix to the G1RedirtyCardsQueueSet::empty() member as this reads better. - maybe rename the local "QSet" to "LocalPtrQueueSet" to make it more self-describing. At least something a bit more descriptive if possible. - it's a bit unfortunate that on the way from the redirty qset to the global qsets (in G1DirtyCardQueueSet::merge_bufferlists) we can not easily keep a "tail" pointer and count, and then when merging the redirty qset with the dirty qset have to iterate over the BufferNodes to determine it. This might have some minor (seemingly unnecessary) impact on huge loads like we discussed internally when doing perf testing. Probably not worth the effort dealing with here, given that you may simply increase the buffer size there. I will do some cursory tests, with probably no detrimental outcome though. Unfortunately there is no good way to just move the data between them other than the one actually implemented. - in G1DirtyCardQueueSet::merge_bufferlists() I am not sure whether that loop used to determine the tail and the count is much favorable (e.g. easiert to read) than this due to endless loop + break "somewhere" within the loop body: BufferNode cur = src_tail->next(); while (cur != NULL) { src_tail = cur; ++src_count; cur = cur->next(); } but it's personal style :), as it does add a "cur" variable in the method scope, so probably ignore this comment. - I am good with the "base from member" idiom and the multiple inheritance it causes. Thanks for looking into this, even if it does not give the expected performance gains any more ;) Thanks, Thomas From thomas.schatzl at oracle.com Fri Jul 12 13:52:35 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 12 Jul 2019 15:52:35 +0200 Subject: RFR: 8162929: Enqueuing dirty cards into a single DCQS during GC does not scale In-Reply-To: References: Message-ID: <65125e989f9746cc17f26b6ab8c419e3911469ad.camel@oracle.com> Hi, On Fri, 2019-07-12 at 14:36 +0200, Thomas Schatzl wrote: > Hi, > > On Fri, 2019-07-05 at 00:50 -0400, Kim Barrett wrote: > > Please review this refactoring of G1DirtyCardQueue[Set], splitting > > into two sets of classes that each better support the purpose they > > are used for. The existing G1DirtyCardQueue[Set] is (and remains) > > the mechanism for communicating between the logging post-write > > barrier and the remset, via refinement (both concurrent and > > STW). The new G1RedirtyCardsQueue[Set] are used during collections > > to collect and en masse dump old remset information back into that > > cycle for later reconsideration. > > [...] > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8162929 > > > > Webrev: > > http://cr.openjdk.java.net/~kbarrett/8162929/open.00/ > > > > some comments: > > > [...] > > - it's a bit unfortunate that on the way from the redirty qset to the > global qsets (in G1DirtyCardQueueSet::merge_bufferlists) we can not > easily keep a "tail" pointer and count, and then when merging the > redirty qset with the dirty qset have to iterate over the BufferNodes > to determine it. > > This might have some minor (seemingly unnecessary) impact on huge > loads like we discussed internally when doing perf testing. Probably > not worth the effort dealing with here, given that you may simply > increase the buffer size there. > I will do some cursory tests, with probably no detrimental outcome > though. Merging the bufferlists (adding a GCtraceTime scoped object before the call) adds up to ~2.5ms, that results in up to 0.5% of total pause time, on BigRAMTester 50G with 1M regions (a somewhat contrived example in many ways). It's not much better with 32M regions either unfortunately, as GC itself is faster there (around ~0.3% of total pause time, little bit less contrived). Is there a way to improve that? (The previous merge code took in the single digit us range) Thanks, Thomas From rkennke at redhat.com Fri Jul 12 14:15:54 2019 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 12 Jul 2019 16:15:54 +0200 Subject: RFR: 8227637: Adjust Shenandoah C2 verifier to recognize IN_NATIVE barriers Message-ID: We get test failures in CTW tests because Shenandoah C2 verifier doesn't recognize the new IN_NATIVE barriers. Bug: https://bugs.openjdk.java.net/browse/JDK-8227637 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8227637/webrev.00/ Testing: hotspot_gc_shenandoah, ctw-tests Can I please get a review? (Aside: all this makes me think if this should be a special case of the regular LRB instead, and become subject to the usual optos, etc. I need to think this through. Let's fix the nightlies first.) Roman From zgu at redhat.com Fri Jul 12 17:21:38 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Fri, 12 Jul 2019 13:21:38 -0400 Subject: RFR 8227635: Shenandoah: SHBSA::load_at() needs to deal IN_NATIVE load Message-ID: <263efc0d-3a10-bcbc-8ff1-065a6d161947@redhat.com> Please review this patch that handles IN_NATIVE load in ShenandoahBarrierSetAssembler::load_at(). Bug: https://bugs.openjdk.java.net/browse/JDK-8227635 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8227635/webrev.00/index.html Test: hotspot_gc_shenandoah (fastdebug and release) on Linux x64 and arrach64. Thanks, -Zhengyu From sangheon.kim at oracle.com Fri Jul 12 20:07:54 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Fri, 12 Jul 2019 13:07:54 -0700 Subject: RFR (S): 8226232: Move merge heap roots code out from G1RemSetScanState In-Reply-To: References: Message-ID: <5b6ee803-05e3-9272-4b30-98f8cd5ba689@oracle.com> Hi Thomas, On 7/10/19 8:30 AM, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this small change that moves code related to > the merge heap roots phase out of G1RemSetScanState? This imho > significantly increases readability of G1RemSetScanState. > > More minor cleanups to improve the code are planned, but I think it is > good to take it in steps. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8226232 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8226232/webrev/ Looks good. Thanks, Sangheon > Testing: > hs-tier1-5 > > Thanks, > Thomas > From kim.barrett at oracle.com Sat Jul 13 23:59:33 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 13 Jul 2019 19:59:33 -0400 Subject: RFR (M): 8226913: Scale cards per chunk used during heap root scanning with region In-Reply-To: <4ec44e39449e8aba219a491afbfc5871e109faee.camel@oracle.com> References: <4ec44e39449e8aba219a491afbfc5871e109faee.camel@oracle.com> Message-ID: > On Jul 10, 2019, at 11:48 AM, Thomas Schatzl wrote: > > Hi all, > > can I have reviews for this change that scales the amount of cards > processed per chunk depending on heap/heap region size to decrease some > unnecessary overhead when clearing it on larger heaps. > > It is basically the answer to this comment in the code: > 96 // Random power of two number of cards we want to claim per > thread. This corresponds > 97 // to a 64k of memory work chunk area for every thread. > 98 // We use the same claim size as Parallel GC. No particular > measurements have been > 99 // performed to determine an optimal number. > > I.e. the chosen number simply seemed suboptimal, and should actually be > determined adaptively. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8226913 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8226913/webrev > Testing: > hs-tier1-5, manually checking that correct chunk sizes are returned for > all possible region sizes > > Thanks, > Thomas ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1RemSet.cpp 98 // size we limit the total number of chunks to limit memory usage and maintenance I think there should be a sentence break in "size we". ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1RemSet.cpp 100 // Testing showed that 8 for 1M/2M region, 16 for 4M/8M regions, 32 for 16/32M regions 101 // seems to be such a good trade-off. ... 105 // The original formula is y = 0.4 * x - 5. The below formula is tweaked so I'm not sure how that formula was derived (some linear fit?), but 1u << ((log_region_size / 2) - 7) seems to give equivalent results for the given range and seems (to me) more "obviously" based on the given numbers. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1RemSet.cpp 103 assert(log_region_size >= 20 && log_region_size <= 25, 104 "expected value in [20,25], but got %u", log_region_size); This is hard-coded to the current region size range limits, and doesn't seem to be related to any requirements of the calcuation being done. Any future expansion of the permitted region size range would fail here for no immediately obvious reason. The only reason to choose those bounds here seems to be that those are what's actually been tested, so as to not make any assumptions about what might be good outside that range. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1RemSet.cpp 370 return result * (HeapRegion::CardsPerRegion / get_chunks_per_region(HeapRegion::LogOfHRGrainBytes)); Instead of get_chunks_per_region(...), why not _scan_chunks_per_region? ------------------------------------------------------------------------------ From rwestrel at redhat.com Mon Jul 15 07:15:29 2019 From: rwestrel at redhat.com (Roland Westrelin) Date: Mon, 15 Jul 2019 09:15:29 +0200 Subject: RFR: 8227637: Adjust Shenandoah C2 verifier to recognize IN_NATIVE barriers In-Reply-To: References: Message-ID: <87y30z4vla.fsf@redhat.com> > http://cr.openjdk.java.net/~rkennke/JDK-8227637/webrev.00/ Good. Roland. From rwestrel at redhat.com Mon Jul 15 07:17:49 2019 From: rwestrel at redhat.com (Roland Westrelin) Date: Mon, 15 Jul 2019 09:17:49 +0200 Subject: RFR 8227635: Shenandoah: SHBSA::load_at() needs to deal IN_NATIVE load In-Reply-To: <263efc0d-3a10-bcbc-8ff1-065a6d161947@redhat.com> References: <263efc0d-3a10-bcbc-8ff1-065a6d161947@redhat.com> Message-ID: <87v9w34vhe.fsf@redhat.com> > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8227635/webrev.00/index.html Do you really need to save vector registers? Isn't this called from the interpreter only? Roland. From rkennke at redhat.com Mon Jul 15 10:14:38 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 15 Jul 2019 12:14:38 +0200 Subject: RFR: 8227637: Adjust Shenandoah C2 verifier to recognize IN_NATIVE barriers In-Reply-To: <87y30z4vla.fsf@redhat.com> References: <87y30z4vla.fsf@redhat.com> Message-ID: <5bd64262-81ce-75cd-98cb-fe005d4ccb99@redhat.com> Hi Roland, Thank you! What do you think about making the IN_NATIVE barrier a special-case of the general LRB? I would think that adding a flag to the LRBNode, and calling to a different runtime entry when expanding the LRB should do the trick. It would probably require some extra care to not accidentally coalesce normal LRB with native-LRB. Are there other complications that I should be aware of when I follow that route? One optimization that comes to mind that is currently inhibited by the IN_NATIVE barrier is replacing mirror->Klass*->mirror or Klass*->mirror->Klas* load-chains by the simple variants. I have a patch that implements is_gc_barrier() and step_over_gc_barrier() which solves this for the current call to runtime, but it seems rather crude, and dealing with IN_NATIVE just like normal LRB seems to cover all possible scenarios of failing optimizations better. What do you think? Roman >> http://cr.openjdk.java.net/~rkennke/JDK-8227637/webrev.00/ > > Good. > > Roland. > From fweimer at redhat.com Mon Jul 15 11:56:18 2019 From: fweimer at redhat.com (Florian Weimer) Date: Mon, 15 Jul 2019 13:56:18 +0200 Subject: RFR(XS): 8226302: Test failures on IBM platforms (power and s/390) after JDK-8223837 In-Reply-To: (Martin Doerr's message of "Thu, 4 Jul 2019 10:35:10 +0000") References: Message-ID: <874l3ntst9.fsf@oldenburg2.str.redhat.com> * Martin Doerr: > I'd like to fix the heap size related issues described in > https://bugs.openjdk.java.net/browse/JDK-8226302 > by adjusting the sizes. I prefer this simple approach for jdk13. Where do you see 16 KiB pages on s390? I think s390 is thoroughly 4 KiB only. Unlike POWER, where some distributions boot with 4 KiB pages, and others with 64 KiB pages (likewise for AArch64). Thanks, Florian From martin.doerr at sap.com Mon Jul 15 12:43:27 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 15 Jul 2019 12:43:27 +0000 Subject: RFR(XS): 8226302: Test failures on IBM platforms (power and s/390) after JDK-8223837 In-Reply-To: <874l3ntst9.fsf@oldenburg2.str.redhat.com> References: <874l3ntst9.fsf@oldenburg2.str.redhat.com> Message-ID: Hi Florian, I agree, the comment is incorrect. Best regards, Martin > -----Original Message----- > From: Florian Weimer > Sent: Montag, 15. Juli 2019 13:56 > To: Doerr, Martin > Cc: hotspot-gc-dev at openjdk.java.net > Subject: Re: RFR(XS): 8226302: Test failures on IBM platforms (power and > s/390) after JDK-8223837 > > * Martin Doerr: > > > I'd like to fix the heap size related issues described in > > https://bugs.openjdk.java.net/browse/JDK-8226302 > > by adjusting the sizes. I prefer this simple approach for jdk13. > > 8226302?focusedCommentId=14272033&page=com.atlassian.jira.plugin.syst > em.issuetabpanels%3Acomment-tabpanel#comment-14272033> > > Where do you see 16 KiB pages on s390? I think s390 is thoroughly 4 KiB > only. Unlike POWER, where some distributions boot with 4 KiB pages, and > others with 64 KiB pages (likewise for AArch64). > > Thanks, > Florian From rkennke at redhat.com Mon Jul 15 13:12:48 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 15 Jul 2019 15:12:48 +0200 Subject: RFR 8225483: Shenandoah: Enhance native access barrier In-Reply-To: <9fd9acf2-446e-0f0f-bccf-e1a21ee88c0b@redhat.com> References: <9d9f5bb2-eff2-e117-d1a4-49339a2cd124@redhat.com> <8a2d332d-6005-64ed-9fbc-7acf388f95b4@redhat.com> <5605fa8e-4e19-5a7a-04cd-e480fd35fa5f@redhat.com> <606DB543-B668-4530-BD8F-5B26BC9AFFE3@redhat.com> <9ccaef06-f6b1-7e8c-8f7f-f23063354290@redhat.com> <13F7044D-4E98-4959-9F2E-8D7ED2D453EA@redhat.com> <9fd9acf2-446e-0f0f-bccf-e1a21ee88c0b@redhat.com> Message-ID: <07c2912f-849f-b99f-7df2-05f9a16a2dc7@redhat.com> >>> Am 11. Juni 2019 16:25:21 MESZ schrieb Zhengyu Gu : >>> ???? Updated: http://cr.openjdk.java.net/~zgu/JDK-8225483/webrev.02/ >>> ???? Okay now? >> >> I still do not like ASSERT around the declaration/definition. What >> prevents us from making that >> method available in both release/debug? That would certainly simplify >> debugging: we would execute >> the same code in both configurations. > > Rebased on Roman's JDK-8226695 fix and made > SHBS::oop_store_not_in_heap() available for both configurations, as > suggested by Aleksey. > > http://cr.openjdk.java.net/~zgu/JDK-8225483/webrev.03/ > > Reran hotspot_gc_shenandoah (fastdebug and release) > Looks good to me. Thanks, Roman From rkennke at redhat.com Mon Jul 15 13:13:42 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 15 Jul 2019 15:13:42 +0200 Subject: RFR: 8227676: Shenandoah: More consistent naming of LRB entry points Message-ID: <1cd8b8aa-c8c4-e296-9855-683b874025a0@redhat.com> Current naming of LRB runtime entry points is a bit weird. We have: - load_reference_barrier_JRT (where _JRT doesn't serve any purpose) - oop_load_from_native_barrier (which is really like LRB except loading from native memory) I propose to rename to: - load_reference_barrier - load_reference_barrier_native Options Bug: https://bugs.openjdk.java.net/browse/JDK-8227676 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8227676/webrev.00/ Testing: hotspot_gc_shenandoah Ok? Roman From shade at redhat.com Mon Jul 15 13:15:53 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 15 Jul 2019 15:15:53 +0200 Subject: RFR: 8227676: Shenandoah: More consistent naming of LRB entry points In-Reply-To: <1cd8b8aa-c8c4-e296-9855-683b874025a0@redhat.com> References: <1cd8b8aa-c8c4-e296-9855-683b874025a0@redhat.com> Message-ID: <08da80b6-6c2d-ccbf-9da4-cbbcd3a1ae9e@redhat.com> On 7/15/19 3:13 PM, Roman Kennke wrote: > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8227676/webrev.00/ Looks good. -- Thanks, -Aleksey From rkennke at redhat.com Mon Jul 15 13:25:39 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 15 Jul 2019 15:25:39 +0200 Subject: RFR: 8227677: Shenandoah: C2: Make in-native LRB special case of normal LRB Message-ID: <3f9ac9c7-c8c3-582c-0825-d774a79c51ad@redhat.com> Currently, IN_NATIVE LRB generates a runtime call during parsing in C2. This may break optimizations. It is known to break the mirror->Klass*->mirror / Klass*->mirror->Klass* optimizations, and possibly some others too (e.g. wholesale elimination of the barrier on new/constant objects). It seems better to emit the normal LoadReferenceBarrier, with a special property 'native' instead, and only generate different runtime-call-addresses. The patch reverts the other C2 parts that I pushed for IN_NATIVE support. Bug: https://bugs.openjdk.java.net/browse/JDK-8227677 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8227677/webrev.00/ Testing: hotspot_gc_shenandoah, ctw-tests Ok? Roman From thomas.schatzl at oracle.com Mon Jul 15 15:38:46 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 15 Jul 2019 17:38:46 +0200 Subject: RFR (S): 8227671: G1: assert_used_and_recalculate_used_equal performs work in product builds Message-ID: <414453dd91d559661f44ad2200db4042245de339.camel@oracle.com> Hi all, can I have reviews for this small change that avoids doing unneccessary work in product builds? In particular, the assert_used_and_recalculate_used_equal macro executes code (the recalculate_used() call) that is potentially lengthy. I measured a few ms on a large heap (50GB) with (forced) 50k regions. I am not sure why this call is so expensive in my tests, but it is completely unnecessary. CR: https://bugs.openjdk.java.net/browse/JDK-8227671 Webrev: http://cr.openjdk.java.net/~tschatzl/8227671/webrev/ Testing: Currently running hs-tier1-3, local gc/g1 jtreg run, local benchmarks Thanks, Thomas From sangheon.kim at oracle.com Mon Jul 15 17:57:53 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Mon, 15 Jul 2019 10:57:53 -0700 Subject: RFR (S): 8227671: G1: assert_used_and_recalculate_used_equal performs work in product builds In-Reply-To: <414453dd91d559661f44ad2200db4042245de339.camel@oracle.com> References: <414453dd91d559661f44ad2200db4042245de339.camel@oracle.com> Message-ID: Hi Thomas, On 7/15/19 8:38 AM, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this small change that avoids doing > unneccessary work in product builds? > > In particular, the assert_used_and_recalculate_used_equal macro > executes code (the recalculate_used() call) that is potentially > lengthy. > > I measured a few ms on a large heap (50GB) with (forced) 50k regions. I > am not sure why this call is so expensive in my tests, but it is > completely unnecessary. It is expensive because recalculate_used() iterates all heap regions? > > CR: > https://bugs.openjdk.java.net/browse/JDK-8227671 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8227671/webrev/ Looks good to me and thanks for fixing this! Thanks, Sangheon > Testing: > Currently running hs-tier1-3, local gc/g1 jtreg run, local benchmarks > > Thanks, > Thomas > > From thomas.schatzl at oracle.com Mon Jul 15 18:22:30 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 15 Jul 2019 20:22:30 +0200 Subject: RFR (S): 8227671: G1: assert_used_and_recalculate_used_equal performs work in product builds In-Reply-To: References: <414453dd91d559661f44ad2200db4042245de339.camel@oracle.com> Message-ID: <7eb2e22d167f6271349f8f1e58a83b71bc36c02a.camel@oracle.com> Hi Sangheon, thanks for your review. On Mon, 2019-07-15 at 10:57 -0700, sangheon.kim at oracle.com wrote: > Hi Thomas, > > On 7/15/19 8:38 AM, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this small change that avoids doing > > unneccessary work in product builds? > > > > In particular, the assert_used_and_recalculate_used_equal macro > > executes code (the recalculate_used() call) that is potentially > > lengthy. > > > > I measured a few ms on a large heap (50GB) with (forced) 50k > > regions. I am not sure why this call is so expensive in my tests, > > but it is completely unnecessary. > > It is expensive because recalculate_used() iterates all heap regions? That's true, however the time it takes is still imho unreasonably high (but it should be fixed in any case). > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8227671 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8227671/webrev/ > > Looks good to me and thanks for fixing this! Thanks for looking into this so quickly! > > > Testing: > > Currently running hs-tier1-3, local gc/g1 jtreg run, local > > benchmarks > > Testing almost done, with no issues. Thanks, Thomas From kim.barrett at oracle.com Mon Jul 15 19:03:05 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 15 Jul 2019 15:03:05 -0400 Subject: RFR (S): 8227671: G1: assert_used_and_recalculate_used_equal performs work in product builds In-Reply-To: <414453dd91d559661f44ad2200db4042245de339.camel@oracle.com> References: <414453dd91d559661f44ad2200db4042245de339.camel@oracle.com> Message-ID: <39FD638F-AE96-4C76-A490-3811208123BE@oracle.com> > On Jul 15, 2019, at 11:38 AM, Thomas Schatzl wrote: > > Hi all, > > can I have reviews for this small change that avoids doing > unneccessary work in product builds? > > In particular, the assert_used_and_recalculate_used_equal macro > executes code (the recalculate_used() call) that is potentially > lengthy. > > I measured a few ms on a large heap (50GB) with (forced) 50k regions. I > am not sure why this call is so expensive in my tests, but it is > completely unnecessary. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8227671 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8227671/webrev/ > Testing: > Currently running hs-tier1-3, local gc/g1 jtreg run, local benchmarks > > Thanks, > Thomas Looks good. One minor nit issue, for which I don't need a new webrev. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1CollectedHeap.hpp 366 #define assert_used_and_recalculate_used_equal(g1h) while(0) That should probably be "do {} while (0)". It probably doesn't matter too much, since the case that can go wrong in a non-debug build will probably not compile in a debug build. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1CollectedHeap.hpp 356 #ifdef ASSERT 357 #define assert_used_and_recalculate_used_equal(g1h) I wonder why this is a macro and not a NOT_DEBUG_RETURN function. The same goes for some of the others nearby. I think macros were used because we were getting poor stacktrace info in hs_err files, but I thought that was pretty much fixed. But this is all a separate issue from the performance bug being addressed by this change. ------------------------------------------------------------------------------ From sangheon.kim at oracle.com Mon Jul 15 21:12:14 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Mon, 15 Jul 2019 14:12:14 -0700 Subject: RFR (S): 8227084: Add timing information for merge heap root preparation In-Reply-To: References: Message-ID: Hi Thomas, On 7/10/19 8:36 AM, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that adds timing information for > parts of the Merge Heap Roots phase as preparation for JDK-8226913. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8227084 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8227084 Looks good. Just minor knit: 419 debug_time("Prepare Merge Heap Roots", _cur_optional_prepare_merge_heap_roots_time_ms); 434 debug_time("Prepare Merge Heap Roots", _cur_prepare_merge_heap_roots_time_ms); - Line 419 seems to include 'optional'? I don't need extra webrev for this. Thanks, Sangheon > Testing: > hs-tier1-5 > > Thanks, > Thomas > > From sangheon.kim at oracle.com Mon Jul 15 21:16:35 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Mon, 15 Jul 2019 14:16:35 -0700 Subject: RFR (S): 8227089: Add timing information for merging humongous remembered sets In-Reply-To: <7eafbdb5dc7c9906021c1fbc43f5912d8ddc267f.camel@oracle.com> References: <7eafbdb5dc7c9906021c1fbc43f5912d8ddc267f.camel@oracle.com> Message-ID: <0d990540-aacf-694f-97a4-3fc053f76900@oracle.com> Hi Thomas, On 7/10/19 8:40 AM, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that adds timing information for > the part of the merge heap roots phase that dumps eager reclaim > candidates onto the card table? > > I am aware that the eager reclaim phase is actually single threaded - > there is an extra CR for fixing this (JDK-8141637). At least with > recent change that effort is hidden by the other work in this phase; > but until now we did not know that there may be an issue. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8227089 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8227089/webrev Looks good. Thanks, Sangheon > Testing: > hs-tier1-5 > > Thanks, > Thomas > > From sangheon.kim at oracle.com Mon Jul 15 21:29:19 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Mon, 15 Jul 2019 14:29:19 -0700 Subject: RFR (XXS): 8227090: G1 does not account the OptMergeRS in cost per card calculation In-Reply-To: <80247f44da3ad38ed6c66d13a89ff52b17e69e34.camel@oracle.com> References: <80247f44da3ad38ed6c66d13a89ff52b17e69e34.camel@oracle.com> Message-ID: <862dafba-9f39-ffd6-2326-9bcfa1518375@oracle.com> Hi Thomas, On 7/10/19 8:42 AM, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this (almost-)one-liner that makes sure that > G1 accounts the OptMergeRS time in the cost per card calculation? > > CR: > https://bugs.openjdk.java.net/browse/JDK-8227090 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8227090/webrev Looks good. Thanks, Sangheon > Testing: > hs-tier1-5 > > Thanks, > Thomas > From kim.barrett at oracle.com Mon Jul 15 21:37:36 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 15 Jul 2019 17:37:36 -0400 Subject: RFR: 8162929: Enqueuing dirty cards into a single DCQS during GC does not scale In-Reply-To: References: Message-ID: > On Jul 12, 2019, at 8:36 AM, Thomas Schatzl wrote: > On Fri, 2019-07-05 at 00:50 -0400, Kim Barrett wrote: >> Please review this refactoring of G1DirtyCardQueue[Set], splitting >> into two sets of classes that each better support the purpose they >> are used for. [?] >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8162929 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8162929/open.00/ >> > > some comments: > > - pre-existing: I would prefer if PtrQueue::~PtrQueue would be virtual > to avoid the risk of object slicing. There is no problem right now in > the code afaics though, but we've been bitten by object slicing issues > before, and it's imho okay to use a virtual destructor here. There is no risk of object slicing. ~PtrQueue (and ~PtrQueueSet and ~SATBMarkQueueSet) are, by design, protected. The "usual" rule for base class destructors is public and virtual (if support for deleting through a base class pointer is desired), or protected and non-virtual. (See, for example, http://www.gotw.ca/publications/mill18.htm, Guideline #4.) Also, making these destructors virtual would require PtrQueue and PtrQueueSet to have an accessible operator delete. The presence of a virtual destructor induces the creation of a "deleting destructor", which will call the operator delete found by lookup in the class. That probably means deriving PtrQueue and PtrQueueSet from CHeapObj, even though we don't actually want to heap allocate any of these objects. (In the original version of this change I made PtrQueueSet derive from CHeapObj, and forgot to back that out before starting the review. What I was doing there is really more properly part of JDK-8221361, and is probably the wrong approach anyway. The redirty card queue set should be local to a collection, perhaps part of the evacuation info.) The problem of unnecessarily inducing deleting destructors was also discussed here: http://mail.openjdk.java.net/pipermail/hotspot-dev/2012-December/007566.html RFR (S): 8004845: Catch incorrect usage of new and delete during compile time for value objects and stack objects AbstractGangTask was given a protected non-virtual destructor as part of that change. Unfortunately, that destructor was removed by JDK-8087323, leaving it with the default (public) destructor. Regretably, what seems like it should be the most relevant gcc warning option (-Wnon-virtual-dtor) is overenthusiastic; -Wnon-virtual-dtor warns about a class with virtual functions and an accessible non-virtual destructor. That's broken. (Not just my opinion; I've seen multiple discussions of this in various places.) That warns about a leaf derived class with a (necessarily) accessible non-virtual destructor, because the compiler doesn't know it's a leaf class. Warning if a class has a problematic base class would be fine. But warning that some hypothetical (but nonexistent) derived class could be sliced, because the compiler doesn't know that the leaf derived class is actually a leaf, is just wrong. (This might be improved using the C++11 final class specifier to let the compiler recognize a leaf class. But apparently not yet: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58140) Unfortunately, even if -Wnon-virtual-dtor did the right thing, we'd still get warnings from it because PtrQueue befriends VMStructs, making its destructor accessible (though unused) from a non-derived class. Drat! The other relevant warning option (-Wdelete-non-virtual-dtor) has some similar problems IMO (warning about hypothetical derived classes that don't actually exist). While I was looking at that, I noticed that while PtrQueue is non-copyable, someone (likely me) forgot to do the same for PtrQueueSet. So I've made the following changes: * Reverted PtrQueueSet derivation from CHeapObj. * Made PtrQueueSet non-copyable. but did not make ~PtrQueue or ~PtrQueueSet virtual. > - I would prefer to add an "is_" prefix to the > G1RedirtyCardsQueueSet::empty() member as this reads better. empty() was only being used for assertions. I've replaced it with verify_empty(). > - maybe rename the local "QSet" to "LocalPtrQueueSet" to make it more > self-describing. At least something a bit more descriptive if possible. I renamed QSet to LocalQSet. > - it's a bit unfortunate that on the way from the redirty qset to the > global qsets (in G1DirtyCardQueueSet::merge_bufferlists) we can not > easily keep a "tail" pointer and count, and then when merging the > redirty qset with the dirty qset have to iterate over the BufferNodes > to determine it. The tail pointer and count are now carried all the way through to the merge into the dirty card queue set, eliminating the iteration down the (possibly very long) list in that merge. > This might have some minor (seemingly unnecessary) impact on huge loads > like we discussed internally when doing perf testing. Probably not > worth the effort dealing with here, given that you may simply increase > the buffer size there. > I will do some cursory tests, with probably no detrimental outcome > though. > > Unfortunately there is no good way to just move the data between them > other than the one actually implemented. I think I found a good way to do that, enabled by the phased usage pattern. > - in G1DirtyCardQueueSet::merge_bufferlists() I am not sure whether > that loop used to determine the tail and the count is much favorable > (e.g. easiert to read) than this due to endless loop + break > "somewhere" within the loop body: That loop is gone now. New webrevs: full: http://cr.openjdk.java.net/~kbarrett/8162929/open.01/ incr: http://cr.openjdk.java.net/~kbarrett/8162929/open.01.inc/ Testing: mach5 tier1-5 From kim.barrett at oracle.com Mon Jul 15 21:39:43 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 15 Jul 2019 17:39:43 -0400 Subject: RFR: 8162929: Enqueuing dirty cards into a single DCQS during GC does not scale In-Reply-To: <65125e989f9746cc17f26b6ab8c419e3911469ad.camel@oracle.com> References: <65125e989f9746cc17f26b6ab8c419e3911469ad.camel@oracle.com> Message-ID: > On Jul 12, 2019, at 9:52 AM, Thomas Schatzl wrote: > On Fri, 2019-07-12 at 14:36 +0200, Thomas Schatzl wrote: >> - it's a bit unfortunate that on the way from the redirty qset to the >> global qsets (in G1DirtyCardQueueSet::merge_bufferlists) we can not >> easily keep a "tail" pointer and count, and then when merging the >> redirty qset with the dirty qset have to iterate over the BufferNodes >> to determine it. >> >> This might have some minor (seemingly unnecessary) impact on huge >> loads like we discussed internally when doing perf testing. Probably >> not worth the effort dealing with here, given that you may simply >> increase the buffer size there. >> I will do some cursory tests, with probably no detrimental outcome >> though. > > Merging the bufferlists (adding a GCtraceTime scoped object before the > call) adds up to ~2.5ms, that results in up to 0.5% of total pause > time, on BigRAMTester 50G with 1M regions (a somewhat contrived example > in many ways). > It's not much better with 32M regions either unfortunately, as GC > itself is faster there (around ~0.3% of total pause time, little bit > less contrived). > > Is there a way to improve that? Should be fixed now (in open.01 webrev). > (The previous merge code took in the single digit us range) > > Thanks, > Thomas From thomas.schatzl at oracle.com Tue Jul 16 08:31:45 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 16 Jul 2019 10:31:45 +0200 Subject: RFR (S): 8227084: Add timing information for merge heap root preparation In-Reply-To: References: Message-ID: <55eb112c82d1ae6064967ce9979bd2be6ec386ac.camel@oracle.com> Hi Sangheon, On Mon, 2019-07-15 at 14:12 -0700, sangheon.kim at oracle.com wrote: > Hi Thomas, > > On 7/10/19 8:36 AM, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this change that adds timing information > > for > > parts of the Merge Heap Roots phase as preparation for JDK-8226913. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8227084 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8227084 > Looks good. > > Just minor knit: > > 419 debug_time("Prepare Merge Heap Roots", > _cur_optional_prepare_merge_heap_roots_time_ms); > 434 debug_time("Prepare Merge Heap Roots", > _cur_prepare_merge_heap_roots_time_ms); > - Line 419 seems to include 'optional'? I don't need extra webrev for > this. > this is correct as far as I can see. Line 419 adds the timing for the "optional" output, while 434 for the regular output. They do have the same name though, so maybe that is irritating you? So you suggest that 419 should be "Prepare Optional Merge Heap Roots"? I regenerated the webrev in place, see http://cr.openjdk.java.net/~tschatzl/8227084/webrev/src/hotspot/share/gc/g1/g1GCPhaseTimes.cpp.frames.html line 419. Thanks for your review, Thomas From thomas.schatzl at oracle.com Tue Jul 16 08:33:02 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 16 Jul 2019 10:33:02 +0200 Subject: RFR (S): 8227089: Add timing information for merging humongous remembered sets In-Reply-To: <0d990540-aacf-694f-97a4-3fc053f76900@oracle.com> References: <7eafbdb5dc7c9906021c1fbc43f5912d8ddc267f.camel@oracle.com> <0d990540-aacf-694f-97a4-3fc053f76900@oracle.com> Message-ID: <5b8d139db136478a0f834715b1322e691b787a11.camel@oracle.com> Hi, On Mon, 2019-07-15 at 14:16 -0700, sangheon.kim at oracle.com wrote: > Hi Thomas, > > On 7/10/19 8:40 AM, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this change that adds timing information > > for the part of the merge heap roots phase that dumps eager reclaim > > candidates onto the card table? > > > > I am aware that the eager reclaim phase is actually single threaded > > - there is an extra CR for fixing this (JDK-8141637). At least with > > recent change that effort is hidden by the other work in this > > phase; but until now we did not know that there may be an issue. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8227089 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8227089/webrev > > Looks good. Thanks for your review. Thomas From thomas.schatzl at oracle.com Tue Jul 16 08:33:48 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 16 Jul 2019 10:33:48 +0200 Subject: RFR (XXS): 8227090: G1 does not account the OptMergeRS in cost per card calculation In-Reply-To: <862dafba-9f39-ffd6-2326-9bcfa1518375@oracle.com> References: <80247f44da3ad38ed6c66d13a89ff52b17e69e34.camel@oracle.com> <862dafba-9f39-ffd6-2326-9bcfa1518375@oracle.com> Message-ID: Hi, On Mon, 2019-07-15 at 14:29 -0700, sangheon.kim at oracle.com wrote: > Hi Thomas, > > On 7/10/19 8:42 AM, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this (almost-)one-liner that makes sure > > that G1 accounts the OptMergeRS time in the cost per card > > calculation? > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8227090 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8227090/webrev > > Looks good. > > thanks for your review. Thomas From thomas.schatzl at oracle.com Tue Jul 16 08:49:10 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 16 Jul 2019 10:49:10 +0200 Subject: RFR (M): 8226913: Scale cards per chunk used during heap root scanning with region In-Reply-To: References: <4ec44e39449e8aba219a491afbfc5871e109faee.camel@oracle.com> Message-ID: Hi Kim, On Sat, 2019-07-13 at 19:59 -0400, Kim Barrett wrote: > > On Jul 10, 2019, at 11:48 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > Hi all, > > > > can I have reviews for this change that scales the amount of cards > > processed per chunk depending on heap/heap region size to decrease > > some unnecessary overhead when clearing it on larger heaps. > > > > It is basically the answer to this comment in the code: > > 96 // Random power of two number of cards we want to claim per > > thread. This corresponds > > 97 // to a 64k of memory work chunk area for every thread. > > 98 // We use the same claim size as Parallel GC. No particular > > measurements have been > > 99 // performed to determine an optimal number. > > > > I.e. the chosen number simply seemed suboptimal, and should > > actually be determined adaptively. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8226913 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8226913/webrev > > Testing: > > hs-tier1-5, manually checking that correct chunk sizes are returned > > for all possible region sizes > > > > Thanks, > > Thomas > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1RemSet.cpp > 98 // size we limit the total number of chunks to limit memory > usage and maintenance > > I think there should be a sentence break in "size we". Added a comma. > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1RemSet.cpp > 100 // Testing showed that 8 for 1M/2M region, 16 for 4M/8M > regions, 32 for 16/32M regions > 101 // seems to be such a good trade-off. > ... > 105 // The original formula is y = 0.4 * x - 5. The below > formula is tweaked so > > I'm not sure how that formula was derived (some linear fit?), but > > 1u << ((log_region_size / 2) - 7) > > seems to give equivalent results for the given range and seems (to > me) more "obviously" based on the given numbers. Linear fit. I will use your formula though since it is much more obvious. > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1RemSet.cpp > 103 assert(log_region_size >= 20 && log_region_size <= 25, > 104 "expected value in [20,25], but got %u", > log_region_size); > > This is hard-coded to the current region size range limits, and > doesn't seem to be related to any requirements of the calcuation > being done. Any future expansion of the permitted region size range > would fail here for no immediately obvious reason. The only reason to > choose those bounds here seems to be that those are what's actually > been tested, so as to not make any assumptions about what might be > good outside that range. That is the reason - if you do not mind I would prefer the code to fail and the person changing this think about this code once more (and do at least cursory testing). I added a comment indicating that. If you feel strongly about this, I can remove the assert completely - maybe at least give a "reasonable" lower bound like 2^16 or so (so that the calculation does not give ridiculously low values or even underflow). > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1RemSet.cpp > 370 return result * (HeapRegion::CardsPerRegion / > get_chunks_per_region(HeapRegion::LogOfHRGrainBytes)); > > Instead of get_chunks_per_region(...), why not > _scan_chunks_per_region? > > ------------------------------------------------------------------- > ----------- Done. New webrevs: http://cr.openjdk.java.net/~tschatzl/8226913/webrev.0_to_1 http://cr.openjdk.java.net/~tschatzl/8226913/webrev.1 Thanks a lot, Thomas From rkennke at redhat.com Tue Jul 16 08:50:38 2019 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 16 Jul 2019 10:50:38 +0200 Subject: RFR: 8227677: Shenandoah: C2: Make in-native LRB special case of normal LRB In-Reply-To: <3f9ac9c7-c8c3-582c-0825-d774a79c51ad@redhat.com> References: <3f9ac9c7-c8c3-582c-0825-d774a79c51ad@redhat.com> Message-ID: <7793a4a4-4152-0588-c631-e67c9efc7fe4@redhat.com> I spotted a leftover string rename: Incremental: http://cr.openjdk.java.net/~rkennke/JDK-8227677/webrev.01.diff/ Full: http://cr.openjdk.java.net/~rkennke/JDK-8227677/webrev.01/ Roman > Currently, IN_NATIVE LRB generates a runtime call during parsing in C2. > This may break optimizations. It is known to break the > mirror->Klass*->mirror / Klass*->mirror->Klass* optimizations, and > possibly some others too (e.g. wholesale elimination of the barrier on > new/constant objects). > > It seems better to emit the normal LoadReferenceBarrier, with a special > property 'native' instead, and only generate different > runtime-call-addresses. > > The patch reverts the other C2 parts that I pushed for IN_NATIVE support. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8227677 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8227677/webrev.00/ > > Testing: hotspot_gc_shenandoah, ctw-tests > > Ok? > > Roman > From thomas.schatzl at oracle.com Tue Jul 16 08:58:34 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 16 Jul 2019 10:58:34 +0200 Subject: RFR (S): 8227671: G1: assert_used_and_recalculate_used_equal performs work in product builds In-Reply-To: <39FD638F-AE96-4C76-A490-3811208123BE@oracle.com> References: <414453dd91d559661f44ad2200db4042245de339.camel@oracle.com> <39FD638F-AE96-4C76-A490-3811208123BE@oracle.com> Message-ID: <3347c765161e2fcf4c500f975f8ede13c96bdfcb.camel@oracle.com> Hi, On Mon, 2019-07-15 at 15:03 -0400, Kim Barrett wrote: > > On Jul 15, 2019, at 11:38 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > Hi all, > > > > can I have reviews for this small change that avoids doing > > unneccessary work in product builds? > > > > In particular, the assert_used_and_recalculate_used_equal macro > > executes code (the recalculate_used() call) that is potentially > > lengthy. > > > > I measured a few ms on a large heap (50GB) with (forced) 50k > > regions. I am not sure why this call is so expensive in my tests, > > but it is completely unnecessary. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8227671 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8227671/webrev/ > > Testing: > > Currently running hs-tier1-3, local gc/g1 jtreg run, local > > benchmarks > > > > Thanks, > > Thomas > > Looks good. > > One minor nit issue, for which I don't need a new webrev. > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1CollectedHeap.hpp > 366 #define assert_used_and_recalculate_used_equal(g1h) while(0) > > That should probably be "do {} while (0)". It probably doesn't > matter too much, since the case that can go wrong in a non-debug > build will probably not compile in a debug build. Fixed and regenerated webrev. > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1CollectedHeap.hpp > 356 #ifdef ASSERT > 357 #define assert_used_and_recalculate_used_equal(g1h) > > I wonder why this is a macro and not a NOT_DEBUG_RETURN function. > > The same goes for some of the others nearby. I think macros were > used because we were getting poor stacktrace info in hs_err files, > but I thought that was pretty much fixed. > > But this is all a separate issue from the performance bug being > addressed by this change. I filed JDK-8227713 for that. Thanks, Thomas From thomas.schatzl at oracle.com Tue Jul 16 09:06:12 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 16 Jul 2019 11:06:12 +0200 Subject: RFR: 8162929: Enqueuing dirty cards into a single DCQS during GC does not scale In-Reply-To: References: Message-ID: Hi Kim, On Mon, 2019-07-15 at 17:37 -0400, Kim Barrett wrote: > > On Jul 12, 2019, at 8:36 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > On Fri, 2019-07-05 at 00:50 -0400, Kim Barrett wrote: > > > Please review this refactoring of G1DirtyCardQueue[Set], > > > splitting > > > into two sets of classes that each better support the purpose > > > they > > > are used for. [?] > > > > > > CR: > > > https://bugs.openjdk.java.net/browse/JDK-8162929 > > > > > > Webrev: > > > http://cr.openjdk.java.net/~kbarrett/8162929/open.00/ > > > > > > > some comments: > > > > - pre-existing: I would prefer if PtrQueue::~PtrQueue would be > > virtual to avoid the risk of object slicing. There is no problem > > right now in the code afaics though, but we've been bitten by > > object slicing issues before, and it's imho okay to use a virtual > > destructor here. > > There is no risk of object slicing. ~PtrQueue (and ~PtrQueueSet and > ~SATBMarkQueueSet) are, by design, protected. The "usual" rule for > base class destructors is public and virtual (if support for deleting > through a base class pointer is desired), or protected and > non-virtual. (See, for example, > http://www.gotw.ca/publications/mill18.htm, > Guideline #4.) I fully agree with that. Thanks. > Also, making these destructors virtual would require PtrQueue and > PtrQueueSet to have an accessible operator delete. The presence of a > virtual destructor induces the creation of a "deleting destructor", > which will call the operator delete found by lookup in the class. > That probably means deriving PtrQueue and PtrQueueSet from CHeapObj, > even though we don't actually want to heap allocate any of these > objects. (In the original version of this change I made PtrQueueSet > derive from CHeapObj, and forgot to back that out before > starting the review. What I was doing there is really more properly > part of JDK-8221361, and is probably the wrong approach anyway. The > redirty card queue set should be local to a collection, perhaps part > of the evacuation info.) > > The problem of unnecessarily inducing deleting destructors was also > discussed here: > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2012-December/007566.html > RFR (S): 8004845: Catch incorrect usage of new and delete during > compile time for value objects and stack objects > > AbstractGangTask was given a protected non-virtual destructor as part > of that change. Unfortunately, that destructor was removed by > JDK-8087323, leaving it with the default (public) destructor. > > Regretably, what seems like it should be the most relevant gcc > warning option (-Wnon-virtual-dtor) is overenthusiastic; -Wnon- > virtual-dtor warns about a class with virtual functions and an > accessible non-virtual destructor. That's broken. (Not just my > opinion; I've seen multiple discussions of this in various > places.) That warns about a leaf derived class with a (necessarily) > accessible non-virtual destructor, because the compiler doesn't know > it's a leaf class. Warning if a class has a problematic base class > would be fine. But warning that some hypothetical (but nonexistent) > derived class could be sliced, because the compiler doesn't know that > the leaf derived class is actually a leaf, is just wrong. (This might > be improved using the C++11 final class specifier to let the compiler > recognize a leaf class. But apparently not yet: > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58140) > > Unfortunately, even if -Wnon-virtual-dtor did the right thing, we'd > still get warnings from it because PtrQueue befriends VMStructs, > making its destructor accessible (though unused) from a non-derived > class. Drat! > > The other relevant warning option (-Wdelete-non-virtual-dtor) has > some similar problems IMO (warning about hypothetical derived classes > that don't actually exist). > > While I was looking at that, I noticed that while PtrQueue is > non-copyable, someone (likely me) forgot to do the same for > PtrQueueSet. > > So I've made the following changes: > * Reverted PtrQueueSet derivation from CHeapObj. > * Made PtrQueueSet non-copyable. > > but did not make ~PtrQueue or ~PtrQueueSet virtual. Thanks a lot for this extensive explanation. > > > - I would prefer to add an "is_" prefix to the > > G1RedirtyCardsQueueSet::empty() member as this reads better. > > empty() was only being used for assertions. I've replaced it with > verify_empty(). Okay. > > > - maybe rename the local "QSet" to "LocalPtrQueueSet" to make it > > more self-describing. At least something a bit more descriptive if > > possible. > > I renamed QSet to LocalQSet. Thanks. > > > - it's a bit unfortunate that on the way from the redirty qset to > > the global qsets (in G1DirtyCardQueueSet::merge_bufferlists) we can > > not easily keep a "tail" pointer and count, and then when merging > > the redirty qset with the dirty qset have to iterate over the > > BufferNodes to determine it. > > The tail pointer and count are now carried all the way through to the > merge into the dirty card queue set, eliminating the iteration down > the (possibly very long) list in that merge. > > > This might have some minor (seemingly unnecessary) impact on huge > > loads like we discussed internally when doing perf testing. > > Probably not worth the effort dealing with here, given that you may > > simply increase the buffer size there. > > I will do some cursory tests, with probably no detrimental outcome > > though. > > > > Unfortunately there is no good way to just move the data between > > them other than the one actually implemented. > > I think I found a good way to do that, enabled by the phased usage > pattern. > > > - in G1DirtyCardQueueSet::merge_bufferlists() I am not sure whether > > that loop used to determine the tail and the count is much > > favorable > > (e.g. easiert to read) than this due to endless loop + break > > "somewhere" within the loop body: > > That loop is gone now. > > New webrevs: > full: http://cr.openjdk.java.net/~kbarrett/8162929/open.01/ > incr: http://cr.openjdk.java.net/~kbarrett/8162929/open.01.inc/ > > Testing: > mach5 tier1-5 > looks good to me. The additional delay is also gone now in my testing, thanks a lot. Thanks, Thomas From thomas.schatzl at oracle.com Tue Jul 16 09:26:48 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 16 Jul 2019 11:26:48 +0200 Subject: RFR (S): 8226232: Move merge heap roots code out from G1RemSetScanState In-Reply-To: <5b6ee803-05e3-9272-4b30-98f8cd5ba689@oracle.com> References: <5b6ee803-05e3-9272-4b30-98f8cd5ba689@oracle.com> Message-ID: Hi, On Fri, 2019-07-12 at 13:07 -0700, sangheon.kim at oracle.com wrote: > Hi Thomas, > > On 7/10/19 8:30 AM, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this small change that moves code related > > to the merge heap roots phase out of G1RemSetScanState? This imho > > significantly increases readability of G1RemSetScanState. > > > > More minor cleanups to improve the code are planned, but I think it > > is good to take it in steps. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8226232 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8226232/webrev/ > > Looks good. thanks for your review. Thomas From zgu at redhat.com Tue Jul 16 15:04:27 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 16 Jul 2019 11:04:27 -0400 Subject: RFR 8227635: Shenandoah: SHBSA::load_at() needs to deal IN_NATIVE load In-Reply-To: <87v9w34vhe.fsf@redhat.com> References: <263efc0d-3a10-bcbc-8ff1-065a6d161947@redhat.com> <87v9w34vhe.fsf@redhat.com> Message-ID: <3e99813b-32b1-bce2-14e0-a89366ed50e3@redhat.com> Please review this updated webrev: http://cr.openjdk.java.net/~zgu/JDK-8227635/webrev.01/ 1) As Roland suggested, don't save vector registers. 2) Renamed oop_load_natiev_barrier() to load_reference_barrier_native() to be consistent with current Roman's naming changes. 3) Use is_traversal_mode() instead of is_concurrent_traversal_in_progress(), due to recent traversal GC changes. 4) Restore old traversal behavior, disregard NO_KEEPALIVE decorator. Test: hotspot_gc_shenandoah (fastdebug and release) on Linux x86_64 and aarch64. -Zhengyu On 7/15/19 3:17 AM, Roland Westrelin wrote: > >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8227635/webrev.00/index.html > > Do you really need to save vector registers? Isn't this called from the > interpreter only? > > Roland. > From rkennke at redhat.com Tue Jul 16 15:27:51 2019 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 16 Jul 2019 17:27:51 +0200 Subject: RFR 8227635: Shenandoah: SHBSA::load_at() needs to deal IN_NATIVE load In-Reply-To: <3e99813b-32b1-bce2-14e0-a89366ed50e3@redhat.com> References: <263efc0d-3a10-bcbc-8ff1-065a6d161947@redhat.com> <87v9w34vhe.fsf@redhat.com> <3e99813b-32b1-bce2-14e0-a89366ed50e3@redhat.com> Message-ID: <15942bb2-eef7-277d-e8dc-dbad31dd08c9@redhat.com> There seem to be gratituous whitespace changes in : src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.hpp and src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.hpp Other than that, looks good. I will look at the traversal issue, I am not quite sure how this needs to be treated. Roman > Please review this updated webrev: > > http://cr.openjdk.java.net/~zgu/JDK-8227635/webrev.01/ > > > 1) As Roland suggested, don't save vector registers. > > 2) Renamed oop_load_natiev_barrier() to load_reference_barrier_native() > to be consistent with current Roman's naming changes. > > 3) Use is_traversal_mode() instead of > is_concurrent_traversal_in_progress(), due to recent traversal GC changes. > > 4) Restore old traversal behavior, disregard NO_KEEPALIVE decorator. > > Test: > > ? hotspot_gc_shenandoah (fastdebug and release) on Linux x86_64 and > aarch64. > > > -Zhengyu > > > > On 7/15/19 3:17 AM, Roland Westrelin wrote: >> >>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8227635/webrev.00/index.html >> >> Do you really need to save vector registers? Isn't this called from the >> interpreter only? >> >> Roland. >> From zgu at redhat.com Tue Jul 16 15:42:53 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 16 Jul 2019 11:42:53 -0400 Subject: RFR 8227635: Shenandoah: SHBSA::load_at() needs to deal IN_NATIVE load In-Reply-To: <15942bb2-eef7-277d-e8dc-dbad31dd08c9@redhat.com> References: <263efc0d-3a10-bcbc-8ff1-065a6d161947@redhat.com> <87v9w34vhe.fsf@redhat.com> <3e99813b-32b1-bce2-14e0-a89366ed50e3@redhat.com> <15942bb2-eef7-277d-e8dc-dbad31dd08c9@redhat.com> Message-ID: <9bfd9ab8-3e7d-3f33-998f-9ad03d0e094e@redhat.com> On 7/16/19 11:27 AM, Roman Kennke wrote: > There seem to be gratituous whitespace changes in : > > src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.hpp > and > src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.hpp > Thanks. Will fix before push. -Zhengyu > Other than that, looks good. I will look at the traversal issue, I am > not quite sure how this needs to be treated. > > Roman > > >> Please review this updated webrev: >> >> http://cr.openjdk.java.net/~zgu/JDK-8227635/webrev.01/ >> >> >> 1) As Roland suggested, don't save vector registers. >> >> 2) Renamed oop_load_natiev_barrier() to load_reference_barrier_native() >> to be consistent with current Roman's naming changes. >> >> 3) Use is_traversal_mode() instead of >> is_concurrent_traversal_in_progress(), due to recent traversal GC changes. >> >> 4) Restore old traversal behavior, disregard NO_KEEPALIVE decorator. >> >> Test: >> >> ? hotspot_gc_shenandoah (fastdebug and release) on Linux x86_64 and >> aarch64. >> >> >> -Zhengyu >> >> >> >> On 7/15/19 3:17 AM, Roland Westrelin wrote: >>> >>>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8227635/webrev.00/index.html >>> >>> Do you really need to save vector registers? Isn't this called from the >>> interpreter only? >>> >>> Roland. >>> > From sangheon.kim at oracle.com Tue Jul 16 16:18:43 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Tue, 16 Jul 2019 09:18:43 -0700 Subject: RFR (S): 8227084: Add timing information for merge heap root preparation In-Reply-To: <55eb112c82d1ae6064967ce9979bd2be6ec386ac.camel@oracle.com> References: <55eb112c82d1ae6064967ce9979bd2be6ec386ac.camel@oracle.com> Message-ID: <05f46f8b-8705-943c-243f-273636cefd4a@oracle.com> Hi Thomas, On 7/16/19 1:31 AM, Thomas Schatzl wrote: > Hi Sangheon, > > On Mon, 2019-07-15 at 14:12 -0700, sangheon.kim at oracle.com wrote: >> Hi Thomas, >> >> On 7/10/19 8:36 AM, Thomas Schatzl wrote: >>> Hi all, >>> >>> can I have reviews for this change that adds timing information >>> for >>> parts of the Merge Heap Roots phase as preparation for JDK-8226913. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8227084 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8227084 >> Looks good. >> >> Just minor knit: >> >> 419 debug_time("Prepare Merge Heap Roots", >> _cur_optional_prepare_merge_heap_roots_time_ms); >> 434 debug_time("Prepare Merge Heap Roots", >> _cur_prepare_merge_heap_roots_time_ms); >> - Line 419 seems to include 'optional'? I don't need extra webrev for >> this. >> > this is correct as far as I can see. Line 419 adds the timing for the > "optional" output, while 434 for the regular output. > > They do have the same name though, so maybe that is irritating you? So > you suggest that 419 should be "Prepare Optional Merge Heap Roots"? Yes, this is what I wanted to spot. > > I regenerated the webrev in place, see > http://cr.openjdk.java.net/~tschatzl/8227084/webrev/src/hotspot/share/gc/g1/g1GCPhaseTimes.cpp.frames.html > line 419. Looks good, thanks for fixing this too. Sangheon > > Thanks for your review, > Thomas > From kim.barrett at oracle.com Tue Jul 16 16:31:27 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 16 Jul 2019 12:31:27 -0400 Subject: RFR (S): 8227671: G1: assert_used_and_recalculate_used_equal performs work in product builds In-Reply-To: <3347c765161e2fcf4c500f975f8ede13c96bdfcb.camel@oracle.com> References: <414453dd91d559661f44ad2200db4042245de339.camel@oracle.com> <39FD638F-AE96-4C76-A490-3811208123BE@oracle.com> <3347c765161e2fcf4c500f975f8ede13c96bdfcb.camel@oracle.com> Message-ID: <4999665A-B9B6-447A-AF1F-3D7E41262B55@oracle.com> > On Jul 16, 2019, at 4:58 AM, Thomas Schatzl wrote: > [?] > Fixed and regenerated webrev. > > [?] > > I filed JDK-8227713 for that. > > Thanks, > Thomas Looks good. From kim.barrett at oracle.com Tue Jul 16 16:35:54 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 16 Jul 2019 12:35:54 -0400 Subject: RFR (M): 8226913: Scale cards per chunk used during heap root scanning with region In-Reply-To: References: <4ec44e39449e8aba219a491afbfc5871e109faee.camel@oracle.com> Message-ID: > On Jul 16, 2019, at 4:49 AM, Thomas Schatzl wrote: > On Sat, 2019-07-13 at 19:59 -0400, Kim Barrett wrote: >> src/hotspot/share/gc/g1/g1RemSet.cpp >> 103 assert(log_region_size >= 20 && log_region_size <= 25, >> 104 "expected value in [20,25], but got %u", >> log_region_size); >> >> This is hard-coded to the current region size range limits, and >> doesn't seem to be related to any requirements of the calcuation >> being done. Any future expansion of the permitted region size range >> would fail here for no immediately obvious reason. The only reason to >> choose those bounds here seems to be that those are what's actually >> been tested, so as to not make any assumptions about what might be >> good outside that range. > > That is the reason - if you do not mind I would prefer the code to fail > and the person changing this think about this code once more (and do at > least cursory testing). > I added a comment indicating that. The comment is good and sufficient. > If you feel strongly about this, I can remove the assert completely - > maybe at least give a "reasonable" lower bound like 2^16 or so (so that > the calculation does not give ridiculously low values or even > underflow). > > [?] > > New webrevs: > http://cr.openjdk.java.net/~tschatzl/8226913/webrev.0_to_1 > > http://cr.openjdk.java.net/~tschatzl/8226913/webrev.1 > > Thanks a lot, > Thomas Looks good. From zgu at redhat.com Wed Jul 17 00:08:06 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 16 Jul 2019 20:08:06 -0400 Subject: RFR(S) 8227755: Need more than 2 distinct CodeCache unloading cycles Message-ID: <916fe9c1-b043-ce2a-4b9d-987d79de08d6@redhat.com> Shenandoah may run out of memory during concurrent nmethod evacuation and upgrade to full GC. Therefore, it needs at least 3 distinct CodeCache unloading cycle numbers. Bug: https://bugs.openjdk.java.net/browse/JDK-8227755 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8227755/webrev.00/ Test: hotspot_gc (fastdebug and release) on Linux x86_64 Submit tests. Thanks, -Zhengyu From rwestrel at redhat.com Wed Jul 17 06:53:13 2019 From: rwestrel at redhat.com (Roland Westrelin) Date: Wed, 17 Jul 2019 08:53:13 +0200 Subject: RFR: 8227677: Shenandoah: C2: Make in-native LRB special case of normal LRB In-Reply-To: <7793a4a4-4152-0588-c631-e67c9efc7fe4@redhat.com> References: <3f9ac9c7-c8c3-582c-0825-d774a79c51ad@redhat.com> <7793a4a4-4152-0588-c631-e67c9efc7fe4@redhat.com> Message-ID: <87r26p40fa.fsf@redhat.com> > http://cr.openjdk.java.net/~rkennke/JDK-8227677/webrev.01/ Good. Roland. From erik.osterlund at oracle.com Wed Jul 17 10:53:00 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 17 Jul 2019 12:53:00 +0200 Subject: RFR(S) 8227755: Need more than 2 distinct CodeCache unloading cycles In-Reply-To: <916fe9c1-b043-ce2a-4b9d-987d79de08d6@redhat.com> References: <916fe9c1-b043-ce2a-4b9d-987d79de08d6@redhat.com> Message-ID: Hi Zhengyu, Having 3 distinct states seems fine. But this patch has 4 states {0, 1, 2, 3}, including the illegal state 0, which is a bug. Here is why: The code cache cycle should never be zero, because that is the cycle that newly created nmethods get assigned, forcing evaluation of oops at the next call to is_unloading(). Imagine the following series of events: 1) Nmethod A is created with state 0 (cycle == 0, is_unloading == false) 2) A GC happens, bumping the current unloading cycle from 3 to 0, nmethod A now has dead oops in it and needs to be unloaded 3) ...but callers to A->is_unloading() will get false, even though it's totally toast. 4) Nmethod A sticks around and callers blow up. So yeah, gotta make sure you don't set the current cycle to zero. Thanks, /Erik On 2019-07-17 02:08, Zhengyu Gu wrote: > Shenandoah may run out of memory during concurrent nmethod evacuation > and upgrade to full GC. Therefore, it needs at least 3 distinct > CodeCache unloading cycle numbers. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8227755 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8227755/webrev.00/ > > Test: > ? hotspot_gc (fastdebug and release) on Linux x86_64 > ? Submit tests. > > Thanks, > > -Zhengyu From rwestrel at redhat.com Wed Jul 17 11:42:12 2019 From: rwestrel at redhat.com (Roland Westrelin) Date: Wed, 17 Jul 2019 13:42:12 +0200 Subject: RFR 8227635: Shenandoah: SHBSA::load_at() needs to deal IN_NATIVE load In-Reply-To: <3e99813b-32b1-bce2-14e0-a89366ed50e3@redhat.com> References: <263efc0d-3a10-bcbc-8ff1-065a6d161947@redhat.com> <87v9w34vhe.fsf@redhat.com> <3e99813b-32b1-bce2-14e0-a89366ed50e3@redhat.com> Message-ID: <87lfww51m3.fsf@redhat.com> > http://cr.openjdk.java.net/~zgu/JDK-8227635/webrev.01/ Looks ok to me. Roland. From zgu at redhat.com Wed Jul 17 17:41:14 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 17 Jul 2019 13:41:14 -0400 Subject: RFR 8227866: Shenandoah: Split weak root processing and class unloading in parallel cleaning task Message-ID: Currently, Shenandoah uses shared ParallelCleaningTask to perform STW cleaning task, the cleaning task performs class unloading along with string deduplication cleaning, if it is enabled. This combination is undesirable, cause we want to have an option to not perform class unloading during STW. This refactor splits cleaning task into two tasks, one performs class unloading and the other performs weak root cleaning up, so it is easy to enable/disable STW class unloading in later changes. Bug: https://bugs.openjdk.java.net/browse/JDK-8227866 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8227866/webrev.00/index.html Test: hotspot_gc_shenandoah (fastdebug and release) Thanks, -Zhengyu From thomas.schatzl at oracle.com Wed Jul 17 18:03:17 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 17 Jul 2019 20:03:17 +0200 Subject: RFR 8227866: Shenandoah: Split weak root processing and class unloading in parallel cleaning task In-Reply-To: References: Message-ID: <37d789394b13cb9e38f30182f7a8cb8de1cfe16d.camel@oracle.com> Hi, On Wed, 2019-07-17 at 13:41 -0400, Zhengyu Gu wrote: > Currently, Shenandoah uses shared ParallelCleaningTask to perform > STW cleaning task, the cleaning task performs class unloading along since the ParallelCleaningTask has originally moved from g1 code to shared to allow use with Shenandoah, now that the code is not used by Shenandoah any more, please move ParallelCleaningTask back to G1 specific code. Thanks, Thomas From zgu at redhat.com Wed Jul 17 18:13:10 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 17 Jul 2019 14:13:10 -0400 Subject: RFR 8227866: Shenandoah: Split weak root processing and class unloading in parallel cleaning task In-Reply-To: <37d789394b13cb9e38f30182f7a8cb8de1cfe16d.camel@oracle.com> References: <37d789394b13cb9e38f30182f7a8cb8de1cfe16d.camel@oracle.com> Message-ID: <39bc59e9-5d8a-2685-dafd-ee64d2b93f3a@redhat.com> On 7/17/19 2:03 PM, Thomas Schatzl wrote: > Hi, > > On Wed, 2019-07-17 at 13:41 -0400, Zhengyu Gu wrote: >> Currently, Shenandoah uses shared ParallelCleaningTask to perform >> STW cleaning task, the cleaning task performs class unloading along > > since the ParallelCleaningTask has originally moved from g1 code to > shared to allow use with Shenandoah, now that the code is not used by > Shenandoah any more, please move ParallelCleaningTask back to G1 > specific code. Sure. Filed JDK-8227927. Thanks, -Zhengyu > > Thanks, > Thomas > > From thomas.schatzl at oracle.com Thu Jul 18 10:02:02 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 18 Jul 2019 12:02:02 +0200 Subject: RFR 8227866: Shenandoah: Split weak root processing and class unloading in parallel cleaning task In-Reply-To: <39bc59e9-5d8a-2685-dafd-ee64d2b93f3a@redhat.com> References: <37d789394b13cb9e38f30182f7a8cb8de1cfe16d.camel@oracle.com> <39bc59e9-5d8a-2685-dafd-ee64d2b93f3a@redhat.com> Message-ID: <54d4a24d5e619cfba095352c82ff4c8721080226.camel@oracle.com> Hi, On Wed, 2019-07-17 at 14:13 -0400, Zhengyu Gu wrote: > > On 7/17/19 2:03 PM, Thomas Schatzl wrote: > > Hi, > > > > On Wed, 2019-07-17 at 13:41 -0400, Zhengyu Gu wrote: > > > Currently, Shenandoah uses shared ParallelCleaningTask to perform > > > STW cleaning task, the cleaning task performs class unloading > > > along > > > > since the ParallelCleaningTask has originally moved from g1 code > > to shared to allow use with Shenandoah, now that the code is not > > used by Shenandoah any more, please move ParallelCleaningTask back > > to G1 specific code. > > Sure. Filed JDK-8227927. I would think that it is appropriate to include the move to G1 code part of this change, but I am okay if you want to do it separately. Thanks, Thomas From rkennke at redhat.com Thu Jul 18 11:21:20 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 18 Jul 2019 13:21:20 +0200 Subject: RFR: 8228364: Shenandoah: Remove unused code from ShenandoahBarrierSetC1 Message-ID: <2a06015c-8a82-893c-4ef7-1ca83b5e8c65@redhat.com> In ShBSC1, needs_null_check is not needed because it is always true, and the CodeEmitInfo is never used for anything. Both look like left-overs from WB. This prepares for a larger refactoring of ShBSC1 and related code. Bug: https://bugs.openjdk.java.net/browse/JDK-8228364 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8228364/webrev.00/ Testing: hotspot_gc_shenandoah (x86/aarch64) Ok? Roman From shade at redhat.com Thu Jul 18 11:41:42 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 18 Jul 2019 13:41:42 +0200 Subject: RFR: 8228364: Shenandoah: Remove unused code from ShenandoahBarrierSetC1 In-Reply-To: <2a06015c-8a82-893c-4ef7-1ca83b5e8c65@redhat.com> References: <2a06015c-8a82-893c-4ef7-1ca83b5e8c65@redhat.com> Message-ID: <8a6a92ed-3901-816a-590d-77a3e77f867b@redhat.com> On 7/18/19 1:21 PM, Roman Kennke wrote: > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8228364/webrev.00/ Looks good to me. -- Thanks, -Aleksey From zgu at redhat.com Thu Jul 18 11:50:36 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 18 Jul 2019 07:50:36 -0400 Subject: RFR 8227866: Shenandoah: Split weak root processing and class unloading in parallel cleaning task In-Reply-To: <54d4a24d5e619cfba095352c82ff4c8721080226.camel@oracle.com> References: <37d789394b13cb9e38f30182f7a8cb8de1cfe16d.camel@oracle.com> <39bc59e9-5d8a-2685-dafd-ee64d2b93f3a@redhat.com> <54d4a24d5e619cfba095352c82ff4c8721080226.camel@oracle.com> Message-ID: <062a1d72-db50-fa8d-ef00-4cdc8dcd9591@redhat.com> Hi Thomas, Although, Shenandoah no long uses ParallelCleaningTask, it still uses tasks defined in it, such as StringDedupCleaningTask, CodeCacheUnloadingTask and etc. So, it should still remain in "shared". What you think? Thanks, -Zhengyu On 7/18/19 6:02 AM, Thomas Schatzl wrote: > Hi, > > On Wed, 2019-07-17 at 14:13 -0400, Zhengyu Gu wrote: >> >> On 7/17/19 2:03 PM, Thomas Schatzl wrote: >>> Hi, >>> >>> On Wed, 2019-07-17 at 13:41 -0400, Zhengyu Gu wrote: >>>> Currently, Shenandoah uses shared ParallelCleaningTask to perform >>>> STW cleaning task, the cleaning task performs class unloading >>>> along >>> >>> since the ParallelCleaningTask has originally moved from g1 code >>> to shared to allow use with Shenandoah, now that the code is not >>> used by Shenandoah any more, please move ParallelCleaningTask back >>> to G1 specific code. >> >> Sure. Filed JDK-8227927. > > I would think that it is appropriate to include the move to G1 code > part of this change, but I am okay if you want to do it separately. > > Thanks, > Thomas > > From zgu at redhat.com Thu Jul 18 14:12:37 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 18 Jul 2019 10:12:37 -0400 Subject: RFR(S) 8227755: Need more than 2 distinct CodeCache unloading cycles In-Reply-To: References: <916fe9c1-b043-ce2a-4b9d-987d79de08d6@redhat.com> Message-ID: Hi Erik, Thanks for the explanation! I were not sure about zero value, but all tests seem happy. Update webrev: http://cr.openjdk.java.net/~zgu/JDK-8227755/webrev.01/index.html Submit tests still fine. Thanks again. -Zhengyu On 7/17/19 6:53 AM, Erik ?sterlund wrote: > Hi Zhengyu, > > Having 3 distinct states seems fine. But this patch has 4 states {0, 1, > 2, 3}, including the illegal state 0, which is a bug. Here is why: > > The code cache cycle should never be zero, because that is the cycle > that newly created nmethods get assigned, forcing evaluation of oops at > the next call to is_unloading(). > > Imagine the following series of events: > 1) Nmethod A is created with state 0 (cycle == 0, is_unloading == false) > 2) A GC happens, bumping the current unloading cycle from 3 to 0, > nmethod A now has dead oops in it and needs to be unloaded > 3) ...but callers to A->is_unloading() will get false, even though it's > totally toast. > 4) Nmethod A sticks around and callers blow up. > > So yeah, gotta make sure you don't set the current cycle to zero. > > Thanks, > /Erik > > On 2019-07-17 02:08, Zhengyu Gu wrote: >> Shenandoah may run out of memory during concurrent nmethod evacuation >> and upgrade to full GC. Therefore, it needs at least 3 distinct >> CodeCache unloading cycle numbers. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8227755 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8227755/webrev.00/ >> >> Test: >> ?? hotspot_gc (fastdebug and release) on Linux x86_64 >> ?? Submit tests. >> >> Thanks, >> >> -Zhengyu From erik.osterlund at oracle.com Thu Jul 18 14:17:28 2019 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Thu, 18 Jul 2019 16:17:28 +0200 Subject: RFR(S) 8227755: Need more than 2 distinct CodeCache unloading cycles In-Reply-To: References: <916fe9c1-b043-ce2a-4b9d-987d79de08d6@redhat.com> Message-ID: <9FB3F098-49F2-43A8-BD33-9E4F2C18616B@oracle.com> Hi Zhengyu, Looks good. Thanks, /Erik > On 18 Jul 2019, at 16:12, Zhengyu Gu wrote: > > Hi Erik, > > Thanks for the explanation! > > I were not sure about zero value, but all tests seem happy. > > Update webrev: http://cr.openjdk.java.net/~zgu/JDK-8227755/webrev.01/index.html > > Submit tests still fine. > > Thanks again. > > -Zhengyu > >> On 7/17/19 6:53 AM, Erik ?sterlund wrote: >> Hi Zhengyu, >> Having 3 distinct states seems fine. But this patch has 4 states {0, 1, 2, 3}, including the illegal state 0, which is a bug. Here is why: >> The code cache cycle should never be zero, because that is the cycle that newly created nmethods get assigned, forcing evaluation of oops at the next call to is_unloading(). >> Imagine the following series of events: >> 1) Nmethod A is created with state 0 (cycle == 0, is_unloading == false) >> 2) A GC happens, bumping the current unloading cycle from 3 to 0, nmethod A now has dead oops in it and needs to be unloaded >> 3) ...but callers to A->is_unloading() will get false, even though it's totally toast. >> 4) Nmethod A sticks around and callers blow up. >> So yeah, gotta make sure you don't set the current cycle to zero. >> Thanks, >> /Erik >>> On 2019-07-17 02:08, Zhengyu Gu wrote: >>> Shenandoah may run out of memory during concurrent nmethod evacuation and upgrade to full GC. Therefore, it needs at least 3 distinct CodeCache unloading cycle numbers. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8227755 >>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8227755/webrev.00/ >>> >>> Test: >>> hotspot_gc (fastdebug and release) on Linux x86_64 >>> Submit tests. >>> >>> Thanks, >>> >>> -Zhengyu From thomas.schatzl at oracle.com Thu Jul 18 14:26:19 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 18 Jul 2019 16:26:19 +0200 Subject: RFR (S): 8227719: G1 pending cards estimation too conservative for cost prediction Message-ID: Hi all, can I have reviews for this change that instead of using a very conservative estimate of the number of pending cards in the DCQS, that is assuming that all completed buffers are full, simply calculates it as buffers are completed, i.e. giving the exact value. Having a better input values to the algorithms seems to be desirable :) Anecdotally, in my tests this makes a significant difference in cases where G1 has formerly been unable to utilize the allowed pause time goal fully, i.e. use significantly larger eden, decreasing total pause time and number of GCs. There is also the risk that the results are less conservative now, leading to more overshooting though. There are imho a lot issues with the prediction heuristics remaining, but at least one input value is a bit better now. CR: https://bugs.openjdk.java.net/browse/JDK-8227719 Webrev: http://cr.openjdk.java.net/~tschatzl/8227719/webrev/ Testing: hs-tier1-3, self verification, (manual) perf testing Thanks, Thomas From thomas.schatzl at oracle.com Thu Jul 18 14:28:07 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 18 Jul 2019 16:28:07 +0200 Subject: RFR (S): 8227719: G1 pending cards estimation too conservative for cost prediction In-Reply-To: References: Message-ID: Hi, On Thu, 2019-07-18 at 16:26 +0200, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that instead of using a very > conservative estimate of the number of pending cards in the DCQS, > that is assuming that all completed buffers are full, simply > calculates it as buffers are completed, i.e. giving the exact value. > > Having a better input values to the algorithms seems to be desirable > :) > > Anecdotally, in my tests this makes a significant difference in cases > where G1 has formerly been unable to utilize the allowed pause time > goal fully, i.e. use significantly larger eden, decreasing total > pause time and number of GCs. There is also the risk that the results > are less conservative now, leading to more overshooting though. > > There are imho a lot issues with the prediction heuristics remaining, > but at least one input value is a bit better now. I forgot to mention that this change is based on JDK-8162929 also out for review. Thomas > > CR: > https://bugs.openjdk.java.net/browse/JDK-8227719 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8227719/webrev/ > Testing: > hs-tier1-3, self verification, (manual) perf testing > > Thanks, > Thomas > From zgu at redhat.com Thu Jul 18 14:40:26 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 18 Jul 2019 10:40:26 -0400 Subject: RFR(S) 8227755: Need more than 2 distinct CodeCache unloading cycles In-Reply-To: <9FB3F098-49F2-43A8-BD33-9E4F2C18616B@oracle.com> References: <916fe9c1-b043-ce2a-4b9d-987d79de08d6@redhat.com> <9FB3F098-49F2-43A8-BD33-9E4F2C18616B@oracle.com> Message-ID: Thanks, Erik. -Zhengyu On 7/18/19 10:17 AM, Erik Osterlund wrote: > Hi Zhengyu, > > Looks good. > > Thanks, > /Erik > >> On 18 Jul 2019, at 16:12, Zhengyu Gu wrote: >> >> Hi Erik, >> >> Thanks for the explanation! >> >> I were not sure about zero value, but all tests seem happy. >> >> Update webrev: http://cr.openjdk.java.net/~zgu/JDK-8227755/webrev.01/index.html >> >> Submit tests still fine. >> >> Thanks again. >> >> -Zhengyu >> >>> On 7/17/19 6:53 AM, Erik ?sterlund wrote: >>> Hi Zhengyu, >>> Having 3 distinct states seems fine. But this patch has 4 states {0, 1, 2, 3}, including the illegal state 0, which is a bug. Here is why: >>> The code cache cycle should never be zero, because that is the cycle that newly created nmethods get assigned, forcing evaluation of oops at the next call to is_unloading(). >>> Imagine the following series of events: >>> 1) Nmethod A is created with state 0 (cycle == 0, is_unloading == false) >>> 2) A GC happens, bumping the current unloading cycle from 3 to 0, nmethod A now has dead oops in it and needs to be unloaded >>> 3) ...but callers to A->is_unloading() will get false, even though it's totally toast. >>> 4) Nmethod A sticks around and callers blow up. >>> So yeah, gotta make sure you don't set the current cycle to zero. >>> Thanks, >>> /Erik >>>> On 2019-07-17 02:08, Zhengyu Gu wrote: >>>> Shenandoah may run out of memory during concurrent nmethod evacuation and upgrade to full GC. Therefore, it needs at least 3 distinct CodeCache unloading cycle numbers. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8227755 >>>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8227755/webrev.00/ >>>> >>>> Test: >>>> hotspot_gc (fastdebug and release) on Linux x86_64 >>>> Submit tests. >>>> >>>> Thanks, >>>> >>>> -Zhengyu > From rkennke at redhat.com Thu Jul 18 15:21:14 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 18 Jul 2019 17:21:14 +0200 Subject: RFR(S) 8227755: Need more than 2 distinct CodeCache unloading cycles In-Reply-To: References: <916fe9c1-b043-ce2a-4b9d-987d79de08d6@redhat.com> Message-ID: Looks good to me. Thanks! Roman > Hi Erik, > > Thanks for the explanation! > > I were not sure about zero value, but all tests seem happy. > > Update webrev: > http://cr.openjdk.java.net/~zgu/JDK-8227755/webrev.01/index.html > > Submit tests still fine. > > Thanks again. > > -Zhengyu > > On 7/17/19 6:53 AM, Erik ?sterlund wrote: >> Hi Zhengyu, >> >> Having 3 distinct states seems fine. But this patch has 4 states {0, >> 1, 2, 3}, including the illegal state 0, which is a bug. Here is why: >> >> The code cache cycle should never be zero, because that is the cycle >> that newly created nmethods get assigned, forcing evaluation of oops >> at the next call to is_unloading(). >> >> Imagine the following series of events: >> 1) Nmethod A is created with state 0 (cycle == 0, is_unloading == false) >> 2) A GC happens, bumping the current unloading cycle from 3 to 0, >> nmethod A now has dead oops in it and needs to be unloaded >> 3) ...but callers to A->is_unloading() will get false, even though >> it's totally toast. >> 4) Nmethod A sticks around and callers blow up. >> >> So yeah, gotta make sure you don't set the current cycle to zero. >> >> Thanks, >> /Erik >> >> On 2019-07-17 02:08, Zhengyu Gu wrote: >>> Shenandoah may run out of memory during concurrent nmethod evacuation >>> and upgrade to full GC. Therefore, it needs at least 3 distinct >>> CodeCache unloading cycle numbers. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8227755 >>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8227755/webrev.00/ >>> >>> Test: >>> ?? hotspot_gc (fastdebug and release) on Linux x86_64 >>> ?? Submit tests. >>> >>> Thanks, >>> >>> -Zhengyu From zgu at redhat.com Thu Jul 18 15:54:18 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 18 Jul 2019 11:54:18 -0400 Subject: RFR(S) 8227755: Need more than 2 distinct CodeCache unloading cycles In-Reply-To: References: <916fe9c1-b043-ce2a-4b9d-987d79de08d6@redhat.com> Message-ID: <570a7b12-5023-b719-764b-3d4fc5076a37@redhat.com> Thanks, Roman. -Zhengyu On 7/18/19 11:21 AM, Roman Kennke wrote: > Looks good to me. Thanks! > > Roman > > >> Hi Erik, >> >> Thanks for the explanation! >> >> I were not sure about zero value, but all tests seem happy. >> >> Update webrev: >> http://cr.openjdk.java.net/~zgu/JDK-8227755/webrev.01/index.html >> >> Submit tests still fine. >> >> Thanks again. >> >> -Zhengyu >> >> On 7/17/19 6:53 AM, Erik ?sterlund wrote: >>> Hi Zhengyu, >>> >>> Having 3 distinct states seems fine. But this patch has 4 states {0, >>> 1, 2, 3}, including the illegal state 0, which is a bug. Here is why: >>> >>> The code cache cycle should never be zero, because that is the cycle >>> that newly created nmethods get assigned, forcing evaluation of oops >>> at the next call to is_unloading(). >>> >>> Imagine the following series of events: >>> 1) Nmethod A is created with state 0 (cycle == 0, is_unloading == false) >>> 2) A GC happens, bumping the current unloading cycle from 3 to 0, >>> nmethod A now has dead oops in it and needs to be unloaded >>> 3) ...but callers to A->is_unloading() will get false, even though >>> it's totally toast. >>> 4) Nmethod A sticks around and callers blow up. >>> >>> So yeah, gotta make sure you don't set the current cycle to zero. >>> >>> Thanks, >>> /Erik >>> >>> On 2019-07-17 02:08, Zhengyu Gu wrote: >>>> Shenandoah may run out of memory during concurrent nmethod evacuation >>>> and upgrade to full GC. Therefore, it needs at least 3 distinct >>>> CodeCache unloading cycle numbers. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8227755 >>>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8227755/webrev.00/ >>>> >>>> Test: >>>> ?? hotspot_gc (fastdebug and release) on Linux x86_64 >>>> ?? Submit tests. >>>> >>>> Thanks, >>>> >>>> -Zhengyu > From thomas.schatzl at oracle.com Thu Jul 18 16:27:31 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 18 Jul 2019 18:27:31 +0200 Subject: RFR (S): 8228388: Add information about dirty/skipped card for Merge HCC in G1 log Message-ID: Hi all, can I have reviews for this small change that adds information about how many cards were flushed/used from the HCC during GC? This removes one annoyance that otherwise the amount of scanned cards in the scan heap roots phase looks to be larger than the sum of cards from the remembered sets and log buffers at times. Also I need that value for JDK-8227739 (so far). I added a small test case that checks whether G1 survives a GC without HCC enabled - there is no such test yet, but seemed appropriate. CR: https://bugs.openjdk.java.net/browse/JDK-8228388 Webrev: http://cr.openjdk.java.net/~tschatzl/8228388/webrev/ Testing: hs-tier1, local gc/g1 run, new test, eyeballing log message output Thanks, Thomas From tprintezis at twitter.com Thu Jul 18 19:17:28 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Thu, 18 Jul 2019 12:17:28 -0700 Subject: RFR: 8217170: gc/arguments/TestUseCompressedOopsErgo.java timed out In-Reply-To: <8F2C5A99-A199-47C1-A0C4-1E5A3D59B6AC@oracle.com> References: <235C3194-422B-4028-83C4-44D1E4BC599B@oracle.com> <3A7F4E5C-E4D1-4CEE-B9F2-7C1840A54E39@oracle.com> <8F2C5A99-A199-47C1-A0C4-1E5A3D59B6AC@oracle.com> Message-ID: Hey Kim, First apologies for the late response. I was on vacation for the last week or so. I?ll try to see if I can reproduce it on Linux / Mac. But I don?t have access to any Windows boxes, so I won?t be able to help with that unfortunately (if it mainly manifests itself on Windows). Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On July 9, 2019 at 5:56:26 PM, Kim Barrett (kim.barrett at oracle.com) wrote: > On Jul 9, 2019, at 1:18 PM, Tony Printezis wrote: > > Hi Kim, > > I?ll be happy to try trouble-shooting this. It's clearly of interest to us as we do use CMS and it?d be nice to get the tests to finish quicker. I tried reproducing the 20sec durations for the child processes with CMS but I was not able to. They seem to have more or less the same duration as with the other GCs. Do you remember which tests you saw this issue in? Also, what type of machine did you run on (?small? laptop, ?larger? workstation, etc)? I'm in the middle of a couple of other things right now, so would be really happy to have someone else poke at this for a bit. I filed JDK-8227414 for followup investigation. Markus Gronlund recently attached a hacky patch (windows_process_abort.patch) to 8217170 to help with the investigation. That probably ought to get copied to JDK-8227414. The test I've mostly been looking at is TestUseCompressedOopsErgo, which runs over a dozen subprocesses. There's a fair amount of information in JDK-8217170 comments, but I'll try to summarize some of it here to save you from lots of scrolling. The machines where I've been able to fairly reliably provoke the problem are a couple of Windows VMs in our build&test farm. They are configured with 8 cores and 60G of memory. They are known to be "slow", so we don't use them for builds, only for tests. jtreg test concurrency is 4, I think. Because they are VMs, the hypervisor might be running other VMs that are also running tests. (Indeed, those two machines are on the same hypervisor.) Some of the issue may be related to load on the VM and/or hypervisor; at least, load seems to be useful in provocation. I've seen the problem to a similar or lesser degree on other Windows machines, e.g. the reported waitFor time is unusually long (at least seconds rather than (usually significantly) subsecond). There's a bit of logging spew in the .jtr files to help spot that, added by JDK-8219149. I *think* I've seen the problem to a lesser degree on Linux. I've not looked at Mac or Solaris at all. Something that I find puzzling is that I think in all the runs of TestUseCompressedOopsErgo that I've looked at, either all of the subprocesses were quick, or all were multi-second slow. > BTW, meant to say along with my review: I tried your patch locally and it shaved off around 35% of the elapsed time for the gc/arguments tests on my workstation. So a nice improvement! Thanks! Oh, that?s nice! I hadn?t measured that. From rkennke at redhat.com Thu Jul 18 19:35:03 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 18 Jul 2019 21:35:03 +0200 Subject: RFR: 8228369: Shenandoah: Refactor LRB C1 stubs Message-ID: Currently, interpreter and C1 are calling through a shared stub. This is somewhat rigid, and not really the C1-ish way to do it. In particular, it makes it very difficult to pass an additional argument to the runtime-call, which we're going to need for self-fixing-LRBs. This patch makes the C1 path go through its own stub that is now generated in the same way as other C1 runtime stubs. Notable changes: - the save_vector_registers()/restore_vector_registers() is no longer needed and we can instead re-use the similar methods from StubAssembler. This is also more useful maintenance-wise if the calling convention/registers ever change, if bugs come up in that code, etc. - The null-check in the CodeStub branches directly to the continuation instead of skipping the call. - Register allocation for the result forces rax/r0 (the return register), which means we can avoid some shuffling in the CodeStub. - Calling the runtime-stub now allows to pass parameters, which will be *very* helpful when we want to the sfx-lrbs. - In order to make the interpreter stub and the C1 stub share as much code as possible, I extracted gen_cset_check() and gen_resolved_check() to be used by both. We could in-fact consider to do away with the interpreter stub altogether, inline the whole code, and benefit from the passed-in tmp register, but I took the approach to not touch that code unless needed: it's quite a nightmare to get the register shuffling right with the interpreter (even more so than C1). Bug: https://bugs.openjdk.java.net/browse/JDK-8228369 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8228369/webrev.00/ Testing: hotspot_gc_shenandoah (x86_64, aarch64) Good? Roman From kim.barrett at oracle.com Thu Jul 18 19:43:33 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 18 Jul 2019 15:43:33 -0400 Subject: RFR (S): 8227719: G1 pending cards estimation too conservative for cost prediction In-Reply-To: References: Message-ID: > On Jul 18, 2019, at 10:26 AM, Thomas Schatzl wrote: > > Hi all, > > can I have reviews for this change that instead of using a very > conservative estimate of the number of pending cards in the DCQS, that > is assuming that all completed buffers are full, simply calculates it > as buffers are completed, i.e. giving the exact value. Before I review this, I have a high-level comment that strongly affects this. I suspect that we don't really want the number of buffers at all, but only the number of cards. I think the number of buffers is just being used to estimate the number of cards. Rather than adding a second counter, perhaps consider replacement of one by the other? DirtyCardQueueSet::completed_buffers_num() could be retained (at least for now) as an estimate based on the card count. That would probably simplify the replacement effort. From kim.barrett at oracle.com Thu Jul 18 19:47:28 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 18 Jul 2019 15:47:28 -0400 Subject: RFR (S): 8228388: Add information about dirty/skipped card for Merge HCC in G1 log In-Reply-To: References: Message-ID: > On Jul 18, 2019, at 12:27 PM, Thomas Schatzl wrote: > > Hi all, > > can I have reviews for this small change that adds information about > how many cards were flushed/used from the HCC during GC? > > This removes one annoyance that otherwise the amount of scanned cards > in the scan heap roots phase looks to be larger than the sum of cards > from the remembered sets and log buffers at times. > > Also I need that value for JDK-8227739 (so far). > > I added a small test case that checks whether G1 survives a GC without > HCC enabled - there is no such test yet, but seemed appropriate. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8228388 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8228388/webrev/ > Testing: > hs-tier1, local gc/g1 run, new test, eyeballing log message output > > Thanks, > Thomas Looks good. From thomas.schatzl at oracle.com Thu Jul 18 20:33:05 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 18 Jul 2019 22:33:05 +0200 Subject: RFR (S): 8227719: G1 pending cards estimation too conservative for cost prediction In-Reply-To: References: Message-ID: Hi Kim, On Thu, 2019-07-18 at 15:43 -0400, Kim Barrett wrote: > > On Jul 18, 2019, at 10:26 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > Hi all, > > > > can I have reviews for this change that instead of using a very > > conservative estimate of the number of pending cards in the DCQS, > > that is assuming that all completed buffers are full, simply > > calculates it as buffers are completed, i.e. giving the exact > > value. > > Before I review this, I have a high-level comment that strongly > affects this. > > I suspect that we don't really want the number of buffers at all, but > only the number of cards. I think the number of buffers is just > being used to estimate the number of cards. Rather than adding a > second counter, perhaps consider replacement of one by the other? I did not look at whether there are other interesting uses for the number of buffers, but I will look at this tomorrow. Without looking, I guess that the refinement thread activation needs the amount of buffers too? > DirtyCardQueueSet::completed_buffers_num() could be retained (at > least for now) as an estimate based on the card count. That would > probably simplify the replacement effort. > Sure it does not make sense to carry around a value that is effectively not used anywhere. Do you have an idea how this would affect refinement thread activation? It will likely to be less aggressive then. Could we base refinement thread activation on the entry count instead then? If so, this seems to be a change that is better made separately though. Not against doing this myselves, but it just seems a change big enough that would imho warrant a separate investigation. Thanks, Thomas From kim.barrett at oracle.com Thu Jul 18 23:05:52 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 18 Jul 2019 19:05:52 -0400 Subject: RFR (S): 8227719: G1 pending cards estimation too conservative for cost prediction In-Reply-To: References: Message-ID: <4069BA1F-2D36-4567-A655-EE08EC948934@oracle.com> > On Jul 18, 2019, at 4:33 PM, Thomas Schatzl wrote: > > Hi Kim, > > On Thu, 2019-07-18 at 15:43 -0400, Kim Barrett wrote: >>> On Jul 18, 2019, at 10:26 AM, Thomas Schatzl < >>> thomas.schatzl at oracle.com> wrote: >>> >>> Hi all, >>> >>> can I have reviews for this change that instead of using a very >>> conservative estimate of the number of pending cards in the DCQS, >>> that is assuming that all completed buffers are full, simply >>> calculates it as buffers are completed, i.e. giving the exact >>> value. >> >> Before I review this, I have a high-level comment that strongly >> affects this. >> >> I suspect that we don't really want the number of buffers at all, but >> only the number of cards. I think the number of buffers is just >> being used to estimate the number of cards. Rather than adding a >> second counter, perhaps consider replacement of one by the other? > > I did not look at whether there are other interesting uses for the > number of buffers, but I will look at this tomorrow. > > Without looking, I guess that the refinement thread activation needs > the amount of buffers too? > >> DirtyCardQueueSet::completed_buffers_num() could be retained (at >> least for now) as an estimate based on the card count. That would >> probably simplify the replacement effort. >> > > Sure it does not make sense to carry around a value that is effectively > not used anywhere. > > Do you have an idea how this would affect refinement thread activation? > It will likely to be less aggressive then. > > Could we base refinement thread activation on the entry count instead > then? If so, this seems to be a change that is better made separately > though. Not against doing this myselves, but it just seems a change big > enough that would imho warrant a separate investigation. > > Thanks, > Thomas I think it won't have any significant effect on refinement. Most of the buffers in or going through the DCQS during concurrent refinement are full. And number of buffers is just used as an estimate of work to be done, with number of cards being a better estimate. I think a much larger source of inaccuracy in that estimate is card duplication, though your recent changes have eliminated redirty buffers as a source for that. I suggested keeping completed_buffers_num() for now, using a definition of ceiling(card_count / buffer_size()), to reduce the fanout of the change to internally carry around card counts rather than buffer counts. I can deal with followup on that as part of working on improving DCQS and refinement thread control, which I think I'm finally ready to start working on again, now that my PtrQueue[Set] refactoring work seems to be approaching completion. I would prefer to only be managing one counter rather than two, especially as I start looking at mitigations for the DCQS lock. From rkennke at redhat.com Fri Jul 19 07:48:10 2019 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 19 Jul 2019 09:48:10 +0200 Subject: RFR: 8228369: Shenandoah: Refactor LRB C1 stubs In-Reply-To: References: Message-ID: <4555ab0e-4a8f-034f-73b5-494b6b0ba9fe@redhat.com> Hold off reviewing this, I will make some improvements to the change today. :-) Roman > Currently, interpreter and C1 are calling through a shared stub. This > is somewhat rigid, and not really the C1-ish way to do it. In > particular, it makes it very difficult to pass an additional argument to > the runtime-call, which we're going to need for self-fixing-LRBs. > > This patch makes the C1 path go through its own stub that is now > generated in the same way as other C1 runtime stubs. Notable changes: > > - the save_vector_registers()/restore_vector_registers() is no longer > needed and we can instead re-use the similar methods from StubAssembler. > This is also more useful maintenance-wise if the calling > convention/registers ever change, if bugs come up in that code, etc. > - The null-check in the CodeStub branches directly to the continuation > instead of skipping the call. > - Register allocation for the result forces rax/r0 (the return > register), which means we can avoid some shuffling in the CodeStub. > - Calling the runtime-stub now allows to pass parameters, which will be > *very* helpful when we want to the sfx-lrbs. > - In order to make the interpreter stub and the C1 stub share as much > code as possible, I extracted gen_cset_check() and gen_resolved_check() > to be used by both. We could in-fact consider to do away with the > interpreter stub altogether, inline the whole code, and benefit from the > passed-in tmp register, but I took the approach to not touch that code > unless needed: it's quite a nightmare to get the register shuffling > right with the interpreter (even more so than C1). > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8228369 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8228369/webrev.00/ > > Testing: hotspot_gc_shenandoah (x86_64, aarch64) > > Good? > > Roman > From thomas.schatzl at oracle.com Fri Jul 19 14:15:55 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 19 Jul 2019 16:15:55 +0200 Subject: RFR (S): 8227719: G1 pending cards estimation too conservative for cost prediction In-Reply-To: <4069BA1F-2D36-4567-A655-EE08EC948934@oracle.com> References: <4069BA1F-2D36-4567-A655-EE08EC948934@oracle.com> Message-ID: Hi Kim, thanks for your thoughts on this. On Thu, 2019-07-18 at 19:05 -0400, Kim Barrett wrote: > > On Jul 18, 2019, at 4:33 PM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > [...] > > I suggested keeping completed_buffers_num() for now, using a > definition of ceiling(card_count / buffer_size()), to reduce the > fanout of the change to internally carry around card counts rather > than buffer counts. I can deal with followup on that as part of > working on improving DCQS and refinement thread control, which I > think I'm finally ready to start working on again, now that my > PtrQueue[Set] refactoring work seems to be approaching completion. > > > I would prefer to only be managing one counter rather than two, > especially as I start looking at mitigations for the DCQS lock. > New webrev at http://cr.openjdk.java.net/~tschatzl/8227719/webrev.0_to_1 (diff) http://cr.openjdk.java.net/~tschatzl/8227719/webrev.1 (full) which removes the buffer count, and replaces the "completed_buffers_num" method with what you suggested. I did some very cursory perf testing with no differences. Thanks, Thomas From rkennke at redhat.com Fri Jul 19 14:27:29 2019 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 19 Jul 2019 16:27:29 +0200 Subject: RFR: 8228369: Shenandoah: Refactor LRB C1 stubs In-Reply-To: References: Message-ID: <0788a5a5-3826-6693-608f-b48dd5805ef9@redhat.com> Please review this updated patch: http://cr.openjdk.java.net/~rkennke/JDK-8228369/webrev.01/ It moves the cset-check and forwarded-check out of the runtime-stub, and one level up into the code-stub. This has the advantage that we can utilize the C1 reg-alloc to get tmp-regs and not need to mess with shuffling registers. It also avoids building the call-frame unless we really need to. It also does some short-cuts when not in cset or when forwarded, branching directly to the continuation rather than to some label (and then to the continuation). The runtime call stub now really only does the runtime call. What do you think? Roman > Currently, interpreter and C1 are calling through a shared stub. This > is somewhat rigid, and not really the C1-ish way to do it. In > particular, it makes it very difficult to pass an additional argument to > the runtime-call, which we're going to need for self-fixing-LRBs. > > This patch makes the C1 path go through its own stub that is now > generated in the same way as other C1 runtime stubs. Notable changes: > > - the save_vector_registers()/restore_vector_registers() is no longer > needed and we can instead re-use the similar methods from StubAssembler. > This is also more useful maintenance-wise if the calling > convention/registers ever change, if bugs come up in that code, etc. > - The null-check in the CodeStub branches directly to the continuation > instead of skipping the call. > - Register allocation for the result forces rax/r0 (the return > register), which means we can avoid some shuffling in the CodeStub. > - Calling the runtime-stub now allows to pass parameters, which will be > *very* helpful when we want to the sfx-lrbs. > - In order to make the interpreter stub and the C1 stub share as much > code as possible, I extracted gen_cset_check() and gen_resolved_check() > to be used by both. We could in-fact consider to do away with the > interpreter stub altogether, inline the whole code, and benefit from the > passed-in tmp register, but I took the approach to not touch that code > unless needed: it's quite a nightmare to get the register shuffling > right with the interpreter (even more so than C1). > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8228369 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8228369/webrev.00/ > > Testing: hotspot_gc_shenandoah (x86_64, aarch64) > > Good? > > Roman > From tprintezis at twitter.com Fri Jul 19 14:40:13 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Fri, 19 Jul 2019 07:40:13 -0700 Subject: RFR: 8217170: gc/arguments/TestUseCompressedOopsErgo.java timed out In-Reply-To: References: <235C3194-422B-4028-83C4-44D1E4BC599B@oracle.com> <3A7F4E5C-E4D1-4CEE-B9F2-7C1840A54E39@oracle.com> <8F2C5A99-A199-47C1-A0C4-1E5A3D59B6AC@oracle.com> Message-ID: Hey Kim, I ran TestUseCompressedOopsErgo on my Mac laptop and Linux workstation. The CMS subtest always takes twice as long as the subtests for the other GCs (10-12 sec vs. 5-6 sec). Almost all the extra overhead is caused by the the assert that checks that the marking bitmap is clear: concurrentMarkSweepGeneration.cpp: bool CMSBitMap::allocate(MemRegion mr) { ... assert(isAllClear(), "Expected zero'd memory from ReservedSpace constructor"); ? } If I comment out the assert, the CMS subtest elapsed time is similar to the rest (still a tad slower, but not anywhere near 2x). Interestingly, if I call isAllClear() twice, the second invocation is more than twice as fast as the first one. So I guess the OS lazily creates pages when isAllClear() is iterating over the bitmap for the first time? Is it worth putting the assert on a switch and disable it for the gc/arguments tests? Not sure if it will help with the 20+ sec elapsed times you were seeing though... Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On July 18, 2019 at 3:17:29 PM, Tony Printezis (tprintezis at twitter.com) wrote: Hey Kim, First apologies for the late response. I was on vacation for the last week or so. I?ll try to see if I can reproduce it on Linux / Mac. But I don?t have access to any Windows boxes, so I won?t be able to help with that unfortunately (if it mainly manifests itself on Windows). Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On July 9, 2019 at 5:56:26 PM, Kim Barrett (kim.barrett at oracle.com) wrote: > On Jul 9, 2019, at 1:18 PM, Tony Printezis wrote: > > Hi Kim, > > I?ll be happy to try trouble-shooting this. It's clearly of interest to us as we do use CMS and it?d be nice to get the tests to finish quicker. I tried reproducing the 20sec durations for the child processes with CMS but I was not able to. They seem to have more or less the same duration as with the other GCs. Do you remember which tests you saw this issue in? Also, what type of machine did you run on (?small? laptop, ?larger? workstation, etc)? I'm in the middle of a couple of other things right now, so would be really happy to have someone else poke at this for a bit. I filed JDK-8227414 for followup investigation. Markus Gronlund recently attached a hacky patch (windows_process_abort.patch) to 8217170 to help with the investigation. That probably ought to get copied to JDK-8227414. The test I've mostly been looking at is TestUseCompressedOopsErgo, which runs over a dozen subprocesses. There's a fair amount of information in JDK-8217170 comments, but I'll try to summarize some of it here to save you from lots of scrolling. The machines where I've been able to fairly reliably provoke the problem are a couple of Windows VMs in our build&test farm. They are configured with 8 cores and 60G of memory. They are known to be "slow", so we don't use them for builds, only for tests. jtreg test concurrency is 4, I think. Because they are VMs, the hypervisor might be running other VMs that are also running tests. (Indeed, those two machines are on the same hypervisor.) Some of the issue may be related to load on the VM and/or hypervisor; at least, load seems to be useful in provocation. I've seen the problem to a similar or lesser degree on other Windows machines, e.g. the reported waitFor time is unusually long (at least seconds rather than (usually significantly) subsecond). There's a bit of logging spew in the .jtr files to help spot that, added by JDK-8219149. I *think* I've seen the problem to a lesser degree on Linux. I've not looked at Mac or Solaris at all. Something that I find puzzling is that I think in all the runs of TestUseCompressedOopsErgo that I've looked at, either all of the subprocesses were quick, or all were multi-second slow. > BTW, meant to say along with my review: I tried your patch locally and it shaved off around 35% of the elapsed time for the gc/arguments tests on my workstation. So a nice improvement! Thanks! Oh, that?s nice! I hadn?t measured that. From tprintezis at twitter.com Fri Jul 19 16:40:41 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Fri, 19 Jul 2019 09:40:41 -0700 Subject: RFR (S): 8227179: Test for new gc+metaspace=info output format In-Reply-To: <44B40714-1AE7-47BD-8D53-D5C8CFF267C1@oracle.com> References: <44B40714-1AE7-47BD-8D53-D5C8CFF267C1@oracle.com> Message-ID: Kim and Thomas, Thanks. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On July 11, 2019 at 3:17:37 PM, Kim Barrett (kim.barrett at oracle.com) wrote: > On Jul 8, 2019, at 11:09 AM, Tony Printezis wrote: > > Thanks for pointing that out, Thomas. Updated webrev (the only change is > the @test comments at the top): > > http://cr.openjdk.java.net/~tonyp/8227179/webrev.1/ > > I tested it with a ?normal? build (all GCs built) and one with no CMS and > the test seemed to have done the right thing in both occasions. > > Tony Looks good. From tprintezis at twitter.com Fri Jul 19 17:40:50 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Fri, 19 Jul 2019 10:40:50 -0700 Subject: RFR(S): 8227225: ParallelGC: add subspace transitions for young gen for gc+heap=info log lines In-Reply-To: <877bd17cff3be20bf8acd95d9eddbb5c9cfb7cf5.camel@oracle.com> References: <877bd17cff3be20bf8acd95d9eddbb5c9cfb7cf5.camel@oracle.com> Message-ID: Hey Thomas, Thanks for taking a look and again apologies for the late reply, I was on vacation until earlier this week. Please see inline. ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On July 10, 2019 at 11:19:37 AM, Thomas Schatzl (thomas.schatzl at oracle.com) wrote: On Fri, 2019-07-05 at 11:06 -0700, Tony Printezis wrote: > Hi all, > > This is a follow-up to 8223575 and adds subspace info for eden and > from space to the gc+heap=info output for ParallelGC. I also added > the before capacity, similar to 8223575, as it can change during the > GC (see, in fact, example below). > > Before: > > [11.123s][info][gc,heap ] GC(17) PSYoungGen: 4759616K- > >1664K(4545536K) > [11.123s][info][gc,heap ] GC(17) ParOldGen: 1280K- > >1280K(1376256K) > > After: > > [12.133s][info][gc,heap ] GC(25) PSYoungGen: > 2065952K(2066432K)->1536K(1983488K) Eden: 2064384K(2064384K)- > >0K(1981952K) > From: 1568K(2048K)->1536K(1536K) Note that I think this is confusing, or the "From" label should be renamed. The second part of this heap transition actually prints the "To" space from what I understand. That?s not entirely true, though. The spaces do switch roles at the end of the GC, so the To space does become the From space of the following GC. So using From here is not entirely incorrect (but I do get your point). What does it print on evacuation failure? [55.239s][info][gc,start ] GC(18) Pause Young (Allocation Failure) [57.547s][info][gc,promotion ] GC(18) Promotion failed [57.548s][info][gc,heap ] GC(18) PSYoungGen: 3522720K(4032000K)->3522720K(4032000K) Eden: 3456000K(3456000K)->3456000K(3456000K) From: 66720K(576000K)->66720K(576000K) [57.548s][info][gc,heap ] GC(18) ParOldGen: 502048K(634880K)->634872K(634880K) [57.548s][info][gc,metaspace ] GC(18) Metaspace: 2681K(5504K)->2681K(5504K) NonClass: 2376K(4864K)->2376K(4864K) Class: 305K(640K)->305K(640K) [57.548s][info][gc ] GC(18) Pause Young (Allocation Failure) 3930M->4060M(4557M) 2309.101ms [57.548s][info][gc,cpu ] GC(18) User=7.43s Sys=5.66s Real=2.31s It would be cleaner imho to print From and To space separately. I can definitely add To space. It will be empty most of the time, minus some edge cases (IIRC). > > [12.133s][info][gc,heap ] GC(25) ParOldGen: > 1272K(1376256K)->1272K(1376256K) > > http://cr.openjdk.java.net/~tonyp/8227225/webrev.0/ > > I have a similar change ready to go for GenCollectedHeap (Serial / > CMS). > I?ll see what the feedback is on this before I open that for code > review. > > A couple of notes: > > - I moved the PreGCValues class to gc/shared as I?d like to re-use > it for the GenCollectedHeap changes too. It's a bit awkwards as there are non-generational GCs now, but I do not have a better idea. Maybe somebody else has. I could just rename this to GenPreGCValues so it?s clear it?s only for the generational GCs? Or I could come up with a convoluted class hierarchy that can handle both generational and non-generational GCs. But I?d rather not, given we don?t have any use cases for this for the latter. So, maybe, rename PreGCValues to GenPreGCValues and, if we need to in the future, we can just split the class as needed? > - Is it worth also adding info about To space to the output? I think so. (Note that I did not really review this change yet, just answering questions) #understood (and thanks for the explanation re: the gc-pending-review label) Tony Thanks, Thomas > > Tony > > PS Should I add the gc-pending-review label to any CR I open for code > review? As discussed, feel free to do so, but it may or may not help. Note that there is no need to remove that label manually: if you set your JIRA filter to ignore already fixed/resolved issues they will not be shown automatically (if you filter). Once or twice a year we typically bulk-remove these labels from already resolved/closed issues. From shade at redhat.com Fri Jul 19 18:32:13 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 19 Jul 2019 20:32:13 +0200 Subject: RFR: 8228369: Shenandoah: Refactor LRB C1 stubs In-Reply-To: <0788a5a5-3826-6693-608f-b48dd5805ef9@redhat.com> References: <0788a5a5-3826-6693-608f-b48dd5805ef9@redhat.com> Message-ID: <72b9fa73-9ae4-7322-09a4-8e66058043db@redhat.com> On 7/19/19 4:27 PM, Roman Kennke wrote: > Please review this updated patch: > > http://cr.openjdk.java.net/~rkennke/JDK-8228369/webrev.01/ Brief look: *) The comment here is inconsistent with the implementation: mentions "not_resolved", while argument is "resolved": shenandoahBarrierSetAssembler_aarch64.cpp 485 // Generate check if object is resolved. Branch to not_resolved label, if not. Otherwise return resolved 486 // object in obj register. 487 // obj: object, resolved object on normal return 488 // tmp1, tmp2: temp registers, trashed 489 void ShenandoahBarrierSetAssembler::gen_resolved_check(MacroAssembler* masm, Register obj, Register tmp1, Register tmp2, Label& resolved) { 490 __ mov(tmp2, obj); 491 resolve_forward_pointer_not_null(masm, obj, tmp1); 492 __ cmp(tmp2, obj); 493 __ br(Assembler::EQ, resolved); 494 } *) Is there any actual benefit for separating gen_cset_check and gen_resolved_check? It seems to have only two uses, and copy-paste looks like less evil. *) In x86 code, can't you use resolve_forward_pointer too, instead of doing decoding by hand? -Aleksey From sangheon.kim at oracle.com Fri Jul 19 20:36:57 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Fri, 19 Jul 2019 13:36:57 -0700 Subject: RFR: 8162929: Enqueuing dirty cards into a single DCQS during GC does not scale In-Reply-To: References: Message-ID: Hi Kim, On 7/15/19 2:37 PM, Kim Barrett wrote: >> On Jul 12, 2019, at 8:36 AM, Thomas Schatzl wrote: >> On Fri, 2019-07-05 at 00:50 -0400, Kim Barrett wrote: >>> Please review this refactoring of G1DirtyCardQueue[Set], splitting >>> into two sets of classes that each better support the purpose they >>> are used for. [?] >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8162929 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8162929/open.00/ >>> >> some comments: >> >> - pre-existing: I would prefer if PtrQueue::~PtrQueue would be virtual >> to avoid the risk of object slicing. There is no problem right now in >> the code afaics though, but we've been bitten by object slicing issues >> before, and it's imho okay to use a virtual destructor here. > There is no risk of object slicing. ~PtrQueue (and ~PtrQueueSet and > ~SATBMarkQueueSet) are, by design, protected. The "usual" rule for > base class destructors is public and virtual (if support for deleting > through a base class pointer is desired), or protected and > non-virtual. (See, for example, http://www.gotw.ca/publications/mill18.htm, > Guideline #4.) > > Also, making these destructors virtual would require PtrQueue and > PtrQueueSet to have an accessible operator delete. The presence of a > virtual destructor induces the creation of a "deleting destructor", > which will call the operator delete found by lookup in the class. > That probably means deriving PtrQueue and PtrQueueSet from CHeapObj, > even though we don't actually want to heap allocate any of these > objects. (In the original version of this change I made PtrQueueSet > derive from CHeapObj, and forgot to back that out before > starting the review. What I was doing there is really more properly > part of JDK-8221361, and is probably the wrong approach anyway. The > redirty card queue set should be local to a collection, perhaps part > of the evacuation info.) > > The problem of unnecessarily inducing deleting destructors was also > discussed here: > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2012-December/007566.html > RFR (S): 8004845: Catch incorrect usage of new and delete during compile time for value objects and stack objects > > AbstractGangTask was given a protected non-virtual destructor as part > of that change. Unfortunately, that destructor was removed by > JDK-8087323, leaving it with the default (public) destructor. > > Regretably, what seems like it should be the most relevant gcc warning > option (-Wnon-virtual-dtor) is overenthusiastic; -Wnon-virtual-dtor > warns about a class with virtual functions and an accessible > non-virtual destructor. That's broken. (Not just my opinion; I've seen > multiple discussions of this in various places.) That warns about a > leaf derived class with a (necessarily) accessible non-virtual > destructor, because the compiler doesn't know it's a leaf > class. Warning if a class has a problematic base class would be fine. > But warning that some hypothetical (but nonexistent) derived class > could be sliced, because the compiler doesn't know that the leaf > derived class is actually a leaf, is just wrong. (This might be > improved using the C++11 final class specifier to let the compiler > recognize a leaf class. But apparently not yet: > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58140) > > Unfortunately, even if -Wnon-virtual-dtor did the right thing, we'd > still get warnings from it because PtrQueue befriends VMStructs, > making its destructor accessible (though unused) from a non-derived > class. Drat! > > The other relevant warning option (-Wdelete-non-virtual-dtor) has some > similar problems IMO (warning about hypothetical derived classes that > don't actually exist). > > While I was looking at that, I noticed that while PtrQueue is > non-copyable, someone (likely me) forgot to do the same for > PtrQueueSet. > > So I've made the following changes: > * Reverted PtrQueueSet derivation from CHeapObj. > * Made PtrQueueSet non-copyable. > > but did not make ~PtrQueue or ~PtrQueueSet virtual. > >> - I would prefer to add an "is_" prefix to the >> G1RedirtyCardsQueueSet::empty() member as this reads better. > empty() was only being used for assertions. I've replaced it with > verify_empty(). > >> - maybe rename the local "QSet" to "LocalPtrQueueSet" to make it more >> self-describing. At least something a bit more descriptive if possible. > I renamed QSet to LocalQSet. > >> - it's a bit unfortunate that on the way from the redirty qset to the >> global qsets (in G1DirtyCardQueueSet::merge_bufferlists) we can not >> easily keep a "tail" pointer and count, and then when merging the >> redirty qset with the dirty qset have to iterate over the BufferNodes >> to determine it. > The tail pointer and count are now carried all the way through to the > merge into the dirty card queue set, eliminating the iteration down > the (possibly very long) list in that merge. > >> This might have some minor (seemingly unnecessary) impact on huge loads >> like we discussed internally when doing perf testing. Probably not >> worth the effort dealing with here, given that you may simply increase >> the buffer size there. >> I will do some cursory tests, with probably no detrimental outcome >> though. >> >> Unfortunately there is no good way to just move the data between them >> other than the one actually implemented. > I think I found a good way to do that, enabled by the phased usage pattern. > >> - in G1DirtyCardQueueSet::merge_bufferlists() I am not sure whether >> that loop used to determine the tail and the count is much favorable >> (e.g. easiert to read) than this due to endless loop + break >> "somewhere" within the loop body: > That loop is gone now. > > New webrevs: > full: http://cr.openjdk.java.net/~kbarrett/8162929/open.01/ > incr: http://cr.openjdk.java.net/~kbarrett/8162929/open.01.inc/ open.01 looks nice to me. BTW, I had trouble applying the patch but probably my local problem? Failed when applying g1DirtyCardQueue.hpp. Thanks, Sangheon > > Testing: > mach5 tier1-5 > From kim.barrett at oracle.com Fri Jul 19 22:12:05 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 19 Jul 2019 18:12:05 -0400 Subject: RFR: 8162929: Enqueuing dirty cards into a single DCQS during GC does not scale In-Reply-To: References: Message-ID: > On Jul 16, 2019, at 5:06 AM, Thomas Schatzl wrote: > > Hi Kim, > > On Mon, 2019-07-15 at 17:37 -0400, Kim Barrett wrote: >> [..] >> New webrevs: >> full: http://cr.openjdk.java.net/~kbarrett/8162929/open.01/ >> incr: http://cr.openjdk.java.net/~kbarrett/8162929/open.01.inc/ >> >> Testing: >> mach5 tier1-5 >> > > looks good to me. The additional delay is also gone now in my > testing, thanks a lot. > > Thanks, > Thomas Thanks. From kim.barrett at oracle.com Fri Jul 19 22:13:42 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 19 Jul 2019 18:13:42 -0400 Subject: RFR: 8162929: Enqueuing dirty cards into a single DCQS during GC does not scale In-Reply-To: References: Message-ID: <1F6F7B9F-E741-4A31-A173-C61895BFACED@oracle.com> > On Jul 19, 2019, at 4:36 PM, sangheon.kim at oracle.com wrote: > > Hi Kim, > > On 7/15/19 2:37 PM, Kim Barrett wrote: >> New webrevs: >> full: http://cr.openjdk.java.net/~kbarrett/8162929/open.01/ >> incr: http://cr.openjdk.java.net/~kbarrett/8162929/open.01.inc/ > open.01 looks nice to me. Thanks. > BTW, I had trouble applying the patch but probably my local problem? > Failed when applying g1DirtyCardQueue.hpp. Applied cleanly for me when updating local repo before pushing. Problem in your local repo? From kim.barrett at oracle.com Fri Jul 19 22:42:38 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 19 Jul 2019 18:42:38 -0400 Subject: RFR (S): 8227719: G1 pending cards estimation too conservative for cost prediction In-Reply-To: References: <4069BA1F-2D36-4567-A655-EE08EC948934@oracle.com> Message-ID: <2B5E38CC-EAA4-48D9-8E21-3F17875FC765@oracle.com> > On Jul 19, 2019, at 10:15 AM, Thomas Schatzl wrote: > > Hi Kim, > > thanks for your thoughts on this. > > On Thu, 2019-07-18 at 19:05 -0400, Kim Barrett wrote: >>> On Jul 18, 2019, at 4:33 PM, Thomas Schatzl < >>> thomas.schatzl at oracle.com> wrote: >>> >>> [...] >> >> I suggested keeping completed_buffers_num() for now, using a >> definition of ceiling(card_count / buffer_size()), to reduce the >> fanout of the change to internally carry around card counts rather >> than buffer counts. I can deal with followup on that as part of >> working on improving DCQS and refinement thread control, which I >> think I'm finally ready to start working on again, now that my >> PtrQueue[Set] refactoring work seems to be approaching completion. >> >> >> I would prefer to only be managing one counter rather than two, >> especially as I start looking at mitigations for the DCQS lock. >> > > New webrev at > > http://cr.openjdk.java.net/~tschatzl/8227719/webrev.0_to_1 (diff) > http://cr.openjdk.java.net/~tschatzl/8227719/webrev.1 (full) > > which removes the buffer count, and replaces the > "completed_buffers_num" method with what you suggested. > > I did some very cursory perf testing with no differences. > > Thanks, > Thomas Looks good. I don't think I would have renamed completed_buffers_num at this point, but I can understand why one might find that name annoying... (Someday I will do something about process_completed_buffers.) One minor item, for which I don't need a new webrev. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1RedirtyCardsQueue.cpp 115 void G1RedirtyCardsQueueSet::verify_empty() const { Removed assertion on _count / _entry_count. Should reinstate. ------------------------------------------------------------------------------ From kim.barrett at oracle.com Sat Jul 20 02:12:34 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 19 Jul 2019 22:12:34 -0400 Subject: RFR: 8224663: Parallel GC: Use WorkGang (5: ScavengeRootsTask) In-Reply-To: <3ea47da5-4b1e-0301-461a-ede9047cb638@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <3ea47da5-4b1e-0301-461a-ede9047cb638@oracle.com> Message-ID: <8CA8F38A-1B9F-4B7D-902E-846BF7718D2C@oracle.com> > On May 27, 2019, at 12:30 PM, Leo Korinth wrote: > > Hi, > > Here is the fifth patch in a proposed patch series of eight that > removes gcTaskManager and uses the WorkGang abstraction instead. > > ScavengeRootsTask, ThreadRootsTask and OldToYoungRootsTask is replaced > with ScavengeRootsTask. Code is basically the same, the major > difference is that roots are visited using EnumClaimer and > Threads::possibly_parallel_threads_do. Here we can reuse the RootType > and EnumClaimer from patch number two. > > The reason "case threads:" was removed is that the code is dead. That > part is confusing as the code is done (in parallel from the calling > function). > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8224663 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224663-Parallel-GC-Use-WorkGang-5-ScavengeRootsTask/ > http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ > > Testing (on the whole patch series): > mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC > gc test suite > > Thanks, > Leo 8224663-Parallel-GC-Use-WorkGang-5-ScavengeRootsTask 8224663-Parallel-GC-Use-WorkGang-5-ScavengeRootsTask-fixup-1 8224663-Parallel-GC-Use-WorkGang-5-ScavengeRootsTask-fixup-2 Looks good, other than a couple formatting nits and some stale comments. ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psScavenge.cpp 55 #if INCLUDE_JVMCI 56 #include "jvmci/jvmci.hpp" 57 #endif Conditional includes go at the end, per the style guide. There are probably counter-examples :) ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psScavenge.cpp 337 }; 338 // 339 // OldToYoungRootsTask Add a blank line between the class and the block comment. ------------------------------------------------------------------------------ 393 ScavengeRootsTask( 394 PSOldGen* old_gen, 395 HeapWord* gen_top, 396 uint active_workers, 397 bool is_empty) : It's more usual to line these up with the "(" with the first parameter on the same line as the function name. ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psScavenge.cpp 339 // OldToYoungRootsTask 340 // 341 // This task is used to scan old to young roots in parallel This comment seems like it needs some update, and maybe is misplaced? This seems like it's really a description of scavenge_contents_parallel, in conjunction with the old task creation model. ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psScavenge.cpp 446 // If active_workers can exceed 1, add a StrealTask. 447 // PSPromotionManager::drain_stacks_depth() does not fully drain its 448 // stacks and expects a StealTask to complete the draining if 449 // ParallelGCThreads is > 1. Stale comment now? ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psScavenge.cpp 438 for (Parallel::RootType::Value root_type; _enum_claimer.try_claim(root_type); /* empty */) { 439 scavenge_roots_task(root_type, worker_id); 440 } For the future, maybe serial processing phases should be moved earlier. OK to leave it for now to maintain correlation with the old code. ------------------------------------------------------------------------------ From kim.barrett at oracle.com Sat Jul 20 11:29:58 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 20 Jul 2019 07:29:58 -0400 Subject: RFR: 8224664: Parallel GC: Use WorkGang (6: PSRefProcTaskProxy) In-Reply-To: References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> Message-ID: <611B4251-A92F-40CD-BCC4-AB748A8C82AF@oracle.com> > On May 27, 2019, at 1:34 PM, Leo Korinth wrote: > > Hi, > > Here is the sixth patch in a proposed patch series of eight that > removes gcTaskManager and uses the WorkGang abstraction instead. > > Here, the new PSRefProcTask is composed of PSRefProcTaskProxy and the > old StealTask (that was partially moved before). > > Now both psTasks.* and pcTasks.* are removed. > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8224664 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224664-Parallel-GC-Use-WorkGang-6-PSRefProcTaskProxy/ > http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ > > Testing (on the whole patch series): > mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC > gc test suite > > Thanks, > Leo 8224664-Parallel-GC-Use-WorkGang-6-PSRefProcTaskProxy Looks good. From kim.barrett at oracle.com Sat Jul 20 11:43:03 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 20 Jul 2019 07:43:03 -0400 Subject: RFR: 8224665: Parallel GC: Use WorkGang (7: remove task manager) In-Reply-To: <5bc88bed-7c2c-b4e7-fef2-2d457d7ba4f1@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <5bc88bed-7c2c-b4e7-fef2-2d457d7ba4f1@oracle.com> Message-ID: <75E8E7E0-DDE4-44EC-B561-3AC1DCCE7C86@oracle.com> > On May 27, 2019, at 1:44 PM, Leo Korinth wrote: > > Hi, > > Here is the seventh patch in a proposed patch series of eight that > removes gcTaskManager and uses the WorkGang abstraction instead. > > We try to remove everything task manager and task thread related. > > Some utility methods (gc_threads_do, print_gc_threads_on) are changed > to use the WorkGang versions. Some worker counts are fetched from > WorkGang as well. Most of the change is just code deletion. > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8224665 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224665-Parallel-GC-Use-WorkGang-7-remove-task-manager/ > http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ > > Testing (on the whole patch series): > mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC > gc test suite > > Thanks, > Leo Looks good. From kim.barrett at oracle.com Sat Jul 20 11:55:57 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 20 Jul 2019 07:55:57 -0400 Subject: RFR: 8224666: Parallel GC: Use WorkGang (8: obsolete and remove flags) In-Reply-To: References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> Message-ID: > On May 29, 2019, at 8:55 AM, Leo Korinth wrote: > > Hi, > > Here is the eighth and last patch that removes gcTaskManager and uses > the WorkGang abstraction instead. > > This patch removes and obsoletes two product flags. > > BindGCTaskThreadsToCPUs is used to run threads pinned on specific > CPUs. Only implemented on Solaris and seems mostly unused. > > UseGCTaskAffinity is used to take tasks with a certain thread > affinity; it does not work and should have been obsoleted since (at > least) JDK-8138707. > > The code tries to obsolete BindGCTaskThreadsToCPUs and UseGCTaskAffinity > in JDK 13. Push the removal _before_ next LTS(17); proposed removal is thus 16. > > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8224666 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224666-Parallel-GC-Use-WorkGang-8-obsolete-and-remove-flags/ > http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ > > Testing (on the whole patch series): > mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC > gc test suite > > Thanks, > Leo 8224666-Parallel-GC-Use-WorkGang-8-obsolete-and-remove-flags 8224666-Parallel-GC-Use-WorkGang-8-obsolete-and-remove-flags-fixup-1 Looks good. I see there is a draft CSR for removal of these product flags (JDK-8224668). I think it needs to be pushed along in the state machine so it can be reviewed. I've reviewed the current text and it looks okay to me. From kim.barrett at oracle.com Sun Jul 21 02:03:13 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 20 Jul 2019 22:03:13 -0400 Subject: RFR (S): 8227719: G1 pending cards estimation too conservative for cost prediction In-Reply-To: <2B5E38CC-EAA4-48D9-8E21-3F17875FC765@oracle.com> References: <4069BA1F-2D36-4567-A655-EE08EC948934@oracle.com> <2B5E38CC-EAA4-48D9-8E21-3F17875FC765@oracle.com> Message-ID: > On Jul 19, 2019, at 6:42 PM, Kim Barrett wrote: > >> On Jul 19, 2019, at 10:15 AM, Thomas Schatzl wrote: >> New webrev at >> >> http://cr.openjdk.java.net/~tschatzl/8227719/webrev.0_to_1 (diff) >> http://cr.openjdk.java.net/~tschatzl/8227719/webrev.1 (full) >> >> which removes the buffer count, and replaces the >> "completed_buffers_num" method with what you suggested. >> >> I did some very cursory perf testing with no differences. >> >> Thanks, >> Thomas > > Looks good. Oops, I realized later that I missed something. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1DirtyCardQueue.hpp 156 return align_up(num_entries_in_completed_buffers(), buffer_size()) / buffer_size(); align_up requires the alignment value to be a power of 2, but the buffer size is not required to be a power of 2, so this doesn't work. It just so happens that the default buffer_size *is* a power of 2 (G1UpdateBufferSize defaults to 256), and we probably don't have any tests that use a non-default buffer_size. ------------------------------------------------------------------------------ From thomas.schatzl at oracle.com Mon Jul 22 08:41:35 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 22 Jul 2019 10:41:35 +0200 Subject: RFR (S): 8227719: G1 pending cards estimation too conservative for cost prediction In-Reply-To: References: <4069BA1F-2D36-4567-A655-EE08EC948934@oracle.com> <2B5E38CC-EAA4-48D9-8E21-3F17875FC765@oracle.com> Message-ID: Hi Kim, On Sat, 2019-07-20 at 22:03 -0400, Kim Barrett wrote: > > On Jul 19, 2019, at 6:42 PM, Kim Barrett > > wrote: > > > > > On Jul 19, 2019, at 10:15 AM, Thomas Schatzl < > > > thomas.schatzl at oracle.com> wrote: > > > New webrev at > > > > > > http://cr.openjdk.java.net/~tschatzl/8227719/webrev.0_to_1 (diff) > > > http://cr.openjdk.java.net/~tschatzl/8227719/webrev.1 (full) > > > > > > which removes the buffer count, and replaces the > > > "completed_buffers_num" method with what you suggested. > > > > > > I did some very cursory perf testing with no differences. > > > > > > Thanks, > > > Thomas > > > > Looks good. > > Oops, I realized later that I missed something. > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1DirtyCardQueue.hpp > 156 return align_up(num_entries_in_completed_buffers(), > buffer_size()) / buffer_size(); > > align_up requires the alignment value to be a power of 2, but the > buffer size is not required to be a power of 2, so this doesn't work. > It just so happens that the default buffer_size *is* a power of 2 > (G1UpdateBufferSize defaults to 256), and we probably don't have any > tests that use a non-default buffer_size. > > Oops. http://cr.openjdk.java.net/~tschatzl/8227719/webrev.1_to_2 (diff) http://cr.openjdk.java.net/~tschatzl/8227719/webrev.2 (full) Thanks, Thomas From rkennke at redhat.com Mon Jul 22 11:30:55 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 22 Jul 2019 13:30:55 +0200 Subject: RFR 8227866: Shenandoah: Split weak root processing and class unloading in parallel cleaning task In-Reply-To: <062a1d72-db50-fa8d-ef00-4cdc8dcd9591@redhat.com> References: <37d789394b13cb9e38f30182f7a8cb8de1cfe16d.camel@oracle.com> <39bc59e9-5d8a-2685-dafd-ee64d2b93f3a@redhat.com> <54d4a24d5e619cfba095352c82ff4c8721080226.camel@oracle.com> <062a1d72-db50-fa8d-ef00-4cdc8dcd9591@redhat.com> Message-ID: <2735142e-4073-b1fe-9e25-7996f8bb5d01@redhat.com> Yes, I think it should still remain in gc/shared as long as Shenandoah is using (parts of) it. The change looks good to me. Thanks, Roman > Hi Thomas, > > Although, Shenandoah no long uses ParallelCleaningTask, it still uses > tasks defined in it, such as StringDedupCleaningTask, > CodeCacheUnloadingTask and etc. So, it should still remain in "shared". > > What you think? > > Thanks, > > -Zhengyu > > > > On 7/18/19 6:02 AM, Thomas Schatzl wrote: >> Hi, >> >> On Wed, 2019-07-17 at 14:13 -0400, Zhengyu Gu wrote: >>> >>> On 7/17/19 2:03 PM, Thomas Schatzl wrote: >>>> Hi, >>>> >>>> On Wed, 2019-07-17 at 13:41 -0400, Zhengyu Gu wrote: >>>>> Currently, Shenandoah uses shared ParallelCleaningTask to perform >>>>> STW cleaning task, the cleaning task performs class unloading >>>>> along >>>> >>>> ??? since the ParallelCleaningTask has originally moved from g1 code >>>> to shared to allow use with Shenandoah, now that the code is not >>>> used by Shenandoah any more, please move ParallelCleaningTask back >>>> to G1 specific code. >>> >>> Sure. Filed JDK-8227927. >> >> I would think that it is appropriate to include the move to G1 code >> part of this change, but I am okay if you want to do it separately. >> >> Thanks, >> ?? Thomas >> >> From zgu at redhat.com Mon Jul 22 11:50:09 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 22 Jul 2019 07:50:09 -0400 Subject: RFR 8227866: Shenandoah: Split weak root processing and class unloading in parallel cleaning task In-Reply-To: <2735142e-4073-b1fe-9e25-7996f8bb5d01@redhat.com> References: <37d789394b13cb9e38f30182f7a8cb8de1cfe16d.camel@oracle.com> <39bc59e9-5d8a-2685-dafd-ee64d2b93f3a@redhat.com> <54d4a24d5e619cfba095352c82ff4c8721080226.camel@oracle.com> <062a1d72-db50-fa8d-ef00-4cdc8dcd9591@redhat.com> <2735142e-4073-b1fe-9e25-7996f8bb5d01@redhat.com> Message-ID: <2f4bb88a-879a-9fa5-fdb4-924438b855f1@redhat.com> On 7/22/19 7:30 AM, Roman Kennke wrote: > Yes, I think it should still remain in gc/shared as long as Shenandoah > is using (parts of) it. Thanks for reviewing, Roman. I think we will just move "ParallelCleaningTask" back to G1 and keep subtasks in shared. Thanks, -Zhengyu > > The change looks good to me. > > Thanks, > Roman > > >> Hi Thomas, >> >> Although, Shenandoah no long uses ParallelCleaningTask, it still uses >> tasks defined in it, such as StringDedupCleaningTask, >> CodeCacheUnloadingTask and etc. So, it should still remain in "shared". >> >> What you think? >> >> Thanks, >> >> -Zhengyu >> >> >> >> On 7/18/19 6:02 AM, Thomas Schatzl wrote: >>> Hi, >>> >>> On Wed, 2019-07-17 at 14:13 -0400, Zhengyu Gu wrote: >>>> >>>> On 7/17/19 2:03 PM, Thomas Schatzl wrote: >>>>> Hi, >>>>> >>>>> On Wed, 2019-07-17 at 13:41 -0400, Zhengyu Gu wrote: >>>>>> Currently, Shenandoah uses shared ParallelCleaningTask to perform >>>>>> STW cleaning task, the cleaning task performs class unloading >>>>>> along >>>>> >>>>> ??? since the ParallelCleaningTask has originally moved from g1 code >>>>> to shared to allow use with Shenandoah, now that the code is not >>>>> used by Shenandoah any more, please move ParallelCleaningTask back >>>>> to G1 specific code. >>>> >>>> Sure. Filed JDK-8227927. >>> >>> I would think that it is appropriate to include the move to G1 code >>> part of this change, but I am okay if you want to do it separately. >>> >>> Thanks, >>> ?? Thomas >>> >>> > From thomas.schatzl at oracle.com Mon Jul 22 12:41:39 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 22 Jul 2019 14:41:39 +0200 Subject: RFR 8227866: Shenandoah: Split weak root processing and class unloading in parallel cleaning task In-Reply-To: <2f4bb88a-879a-9fa5-fdb4-924438b855f1@redhat.com> References: <37d789394b13cb9e38f30182f7a8cb8de1cfe16d.camel@oracle.com> <39bc59e9-5d8a-2685-dafd-ee64d2b93f3a@redhat.com> <54d4a24d5e619cfba095352c82ff4c8721080226.camel@oracle.com> <062a1d72-db50-fa8d-ef00-4cdc8dcd9591@redhat.com> <2735142e-4073-b1fe-9e25-7996f8bb5d01@redhat.com> <2f4bb88a-879a-9fa5-fdb4-924438b855f1@redhat.com> Message-ID: Hi Zhengyu, On Mon, 2019-07-22 at 07:50 -0400, Zhengyu Gu wrote: > > On 7/22/19 7:30 AM, Roman Kennke wrote: > > Yes, I think it should still remain in gc/shared as long as > > Shenandoah is using (parts of) it. > > Thanks for reviewing, Roman. > > I think we will just move "ParallelCleaningTask" back to G1 and keep > subtasks in shared. > Makes sense. Thank you. Thanks, Thomas From kim.barrett at oracle.com Mon Jul 22 17:08:38 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 22 Jul 2019 13:08:38 -0400 Subject: RFR (S): 8227719: G1 pending cards estimation too conservative for cost prediction In-Reply-To: References: <4069BA1F-2D36-4567-A655-EE08EC948934@oracle.com> <2B5E38CC-EAA4-48D9-8E21-3F17875FC765@oracle.com> Message-ID: > On Jul 22, 2019, at 4:41 AM, Thomas Schatzl wrote: > http://cr.openjdk.java.net/~tschatzl/8227719/webrev.1_to_2 (diff) > http://cr.openjdk.java.net/~tschatzl/8227719/webrev.2 (full) > > Thanks, > Thomas Looks good. From thomas.schatzl at oracle.com Mon Jul 22 17:27:13 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 22 Jul 2019 19:27:13 +0200 Subject: RFR (S): 8227719: G1 pending cards estimation too conservative for cost prediction In-Reply-To: References: <4069BA1F-2D36-4567-A655-EE08EC948934@oracle.com> <2B5E38CC-EAA4-48D9-8E21-3F17875FC765@oracle.com> Message-ID: <401c031da52e0099878341632e96ae0f3c517fdd.camel@oracle.com> Hi, On Mon, 2019-07-22 at 13:08 -0400, Kim Barrett wrote: > > On Jul 22, 2019, at 4:41 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > http://cr.openjdk.java.net/~tschatzl/8227719/webrev.1_to_2 (diff) > > http://cr.openjdk.java.net/~tschatzl/8227719/webrev.2 (full) > > > > Thanks, > > Thomas > > Looks good. > thanks for your review. Thomas From yasuenag at gmail.com Tue Jul 23 07:24:14 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Tue, 23 Jul 2019 16:24:14 +0900 Subject: ResourceExhausted JVMTI event and -XX:*OnOutOfMemoryError Message-ID: Hi all, I tried to use JVMTI agent which hooks ResourceExhausted event and -XX:ExitOnOutOfMemoryError together. The process was exited when OutOfMemoryError was occurred, but ResourceExhausted event was not fired. I checked HotSpot implementation, -XX:*OutOfMemoryError is processed before JVMTI event processing. http://hg.openjdk.java.net/jdk/jdk/file/d999a1a11485/src/hotspot/share/gc/shared/memAllocator.cpp#l125 IMHO ResourceExhausted should be called before -XX:*OutOfMemoryError. If it is a bug, I will file it to JBS and fix it. Thanks, Yasumasa From david.holmes at oracle.com Tue Jul 23 07:43:14 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 23 Jul 2019 17:43:14 +1000 Subject: ResourceExhausted JVMTI event and -XX:*OnOutOfMemoryError In-Reply-To: References: Message-ID: <2bba059e-bac8-7ec8-7e14-e142324cbb64@oracle.com> Hi Yasumasa, On 23/07/2019 5:24 pm, Yasumasa Suenaga wrote: > Hi all, > > I tried to use JVMTI agent which hooks ResourceExhausted event and > -XX:ExitOnOutOfMemoryError together. > The process was exited when OutOfMemoryError was occurred, but > ResourceExhausted event was not fired. > > I checked HotSpot implementation, -XX:*OutOfMemoryError is processed > before JVMTI event processing. > > http://hg.openjdk.java.net/jdk/jdk/file/d999a1a11485/src/hotspot/share/gc/shared/memAllocator.cpp#l125 > > > IMHO ResourceExhausted should be called before -XX:*OutOfMemoryError. > If it is a bug, I will file it to JBS and fix it. I don't think it is a bug. First, there's no specification for how these two things interact - -XX:*OnOutOfMemoryError is just a VM specific debugging/diagnostic hook. Second, if we're out of memory then we don't want to be doing other things which may need to also acquire memory. Third, no matter which way you code this someone may in some case want it the other way - and it's not worth coding to allow such flexibility IMO. Just my $2. :) Cheers, David > > Thanks, > > Yasumasa From yasuenag at gmail.com Tue Jul 23 08:13:18 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Tue, 23 Jul 2019 17:13:18 +0900 Subject: ResourceExhausted JVMTI event and -XX:*OnOutOfMemoryError In-Reply-To: <2bba059e-bac8-7ec8-7e14-e142324cbb64@oracle.com> References: <2bba059e-bac8-7ec8-7e14-e142324cbb64@oracle.com> Message-ID: Hi David, There are not documented the relation between -XX:*OnOutOfMemoryError and ResourceExhausted, but JVMTI spec says ResourceExhausted is sent when VM resource are exhausted. https://docs.oracle.com/en/java/javase/12/docs/specs/jvmti.html#ResourceExhausted As you said, we should not do other things when OutOfMemoryError is occurred. But I think it should be compliant when we will use Java related code in the event handler. If we do with C/C++ only (e.g. call printf()), I think we can do that. If all codes should not be run when OutOfMemoryError is occurred, ResourceExhausted should be obsoleted, or we need to add the note for JVMTI document. Thanks, Yasumasa On 2019/07/23 16:43, David Holmes wrote: > Hi Yasumasa, > > On 23/07/2019 5:24 pm, Yasumasa Suenaga wrote: >> Hi all, >> >> I tried to use JVMTI agent which hooks ResourceExhausted event and -XX:ExitOnOutOfMemoryError together. >> The process was exited when OutOfMemoryError was occurred, but ResourceExhausted event was not fired. >> >> I checked HotSpot implementation, -XX:*OutOfMemoryError is processed before JVMTI event processing. >> >> http://hg.openjdk.java.net/jdk/jdk/file/d999a1a11485/src/hotspot/share/gc/shared/memAllocator.cpp#l125 >> >> IMHO ResourceExhausted should be called before -XX:*OutOfMemoryError. >> If it is a bug, I will file it to JBS and fix it. > > I don't think it is a bug. First, there's no specification for how these two things interact - -XX:*OnOutOfMemoryError is just a VM specific debugging/diagnostic hook. Second, if we're out of memory then we don't want to be doing other things which may need to also acquire memory. Third, no matter which way you code this someone may in some case want it the other way - and it's not worth coding to allow such flexibility IMO. > > Just my $2. :) > > Cheers, > David > >> >> Thanks, >> >> Yasumasa From yasuenag at gmail.com Tue Jul 23 08:16:10 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Tue, 23 Jul 2019 17:16:10 +0900 Subject: ResourceExhausted JVMTI event and -XX:*OnOutOfMemoryError In-Reply-To: <2bba059e-bac8-7ec8-7e14-e142324cbb64@oracle.com> References: <2bba059e-bac8-7ec8-7e14-e142324cbb64@oracle.com> Message-ID: <1f58ace5-305a-e587-3e9b-9c65e0531620@gmail.com> Sorry, the format is broken... I re-sent the email: There are not documented the relation between -XX:*OnOutOfMemoryError and ResourceExhausted, but JVMTI spec says ResourceExhausted is sent when VM resource are exhausted. https://docs.oracle.com/en/java/javase/12/docs/specs/jvmti.html#ResourceExhausted As you said, we should not do other things when OutOfMemoryError is occurred. But I think it should be compliant when we will use Java related code in the event handler. If we do with C/C++ only (e.g. call printf()), I think we can do that. If all codes should not be run when OutOfMemoryError is occurred, ResourceExhausted should be obsoleted, or we need to add the note for JVMTI document. Thanks, Yasumasa On 2019/07/23 16:43, David Holmes wrote: > Hi Yasumasa, > > On 23/07/2019 5:24 pm, Yasumasa Suenaga wrote: >> Hi all, >> >> I tried to use JVMTI agent which hooks ResourceExhausted event and -XX:ExitOnOutOfMemoryError together. >> The process was exited when OutOfMemoryError was occurred, but ResourceExhausted event was not fired. >> >> I checked HotSpot implementation, -XX:*OutOfMemoryError is processed before JVMTI event processing. >> >> http://hg.openjdk.java.net/jdk/jdk/file/d999a1a11485/src/hotspot/share/gc/shared/memAllocator.cpp#l125 >> >> IMHO ResourceExhausted should be called before -XX:*OutOfMemoryError. >> If it is a bug, I will file it to JBS and fix it. > > I don't think it is a bug. First, there's no specification for how these two things interact - -XX:*OnOutOfMemoryError is just a VM specific debugging/diagnostic hook. Second, if we're out of memory then we don't want to be doing other things which may need to also acquire memory. Third, no matter which way you code this someone may in some case want it the other way - and it's not worth coding to allow such flexibility IMO. > > Just my $2. :) > > Cheers, > David > >> >> Thanks, >> >> Yasumasa From david.holmes at oracle.com Tue Jul 23 11:08:51 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 23 Jul 2019 21:08:51 +1000 Subject: ResourceExhausted JVMTI event and -XX:*OnOutOfMemoryError In-Reply-To: <1f58ace5-305a-e587-3e9b-9c65e0531620@gmail.com> References: <2bba059e-bac8-7ec8-7e14-e142324cbb64@oracle.com> <1f58ace5-305a-e587-3e9b-9c65e0531620@gmail.com> Message-ID: <74df0ed5-a953-a2bb-cb23-688f758b1c3e@oracle.com> On 23/07/2019 6:16 pm, Yasumasa Suenaga wrote: > Sorry, the format is broken... I re-sent the email: > > > There are not documented the relation between -XX:*OnOutOfMemoryError > and ResourceExhausted, > but JVMTI spec says ResourceExhausted is sent when VM resource are > exhausted. > > https://docs.oracle.com/en/java/javase/12/docs/specs/jvmti.html#ResourceExhausted The JVMTI spec can't say anything about -XX:*OnOutOfMemoryError as they are a hotspot implementation detail. If you use the -XX flag you get whatever the hotspot implementation does. That may override normal specified behaviour. For example, if you use -XX:+ExitOnOutOfMemoryError you can't then complain that the VM never executes a catch block for OutOfMemoryError because it exited first! It's the same for the event processing - we're exiting before we reach that code. > As you said, we should not do other things when OutOfMemoryError is > occurred. > But I think it should be compliant when we will use Java related code in > the event handler. > If we do with C/C++ only (e.g. call printf()), I think we can do that. But we don't know what the handler will do. Nor do we know why the -XX flag was used, or even what it may do in some cases. > If all codes should not be run when OutOfMemoryError is occurred, > ResourceExhausted should be obsoleted, or we need to add the note for > JVMTI document. There is no logical argument there for obsoleting the ResourceExhausted event. I understand you want the event to be posted before the flag is processed, but somebody else may want it exactly as it is - e.g. dumping the heap before their event handler does some cleanup/recovery work. And as I said the JVM TI spec knows nothing of this XX flag so cannot say anything about it. David ----- > > Thanks, > > Yasumasa > > > On 2019/07/23 16:43, David Holmes wrote: >> Hi Yasumasa, >> >> On 23/07/2019 5:24 pm, Yasumasa Suenaga wrote: >>> Hi all, >>> >>> I tried to use JVMTI agent which hooks ResourceExhausted event and >>> -XX:ExitOnOutOfMemoryError together. >>> The process was exited when OutOfMemoryError was occurred, but >>> ResourceExhausted event was not fired. >>> >>> I checked HotSpot implementation, -XX:*OutOfMemoryError is processed >>> before JVMTI event processing. >>> >>> http://hg.openjdk.java.net/jdk/jdk/file/d999a1a11485/src/hotspot/share/gc/shared/memAllocator.cpp#l125 >>> >>> >>> IMHO ResourceExhausted should be called before -XX:*OutOfMemoryError. >>> If it is a bug, I will file it to JBS and fix it. >> >> I don't think it is a bug. First, there's no specification for how >> these two things interact - -XX:*OnOutOfMemoryError is just a VM >> specific debugging/diagnostic hook. Second, if we're out of memory >> then we don't want to be doing other things which may need to also >> acquire memory. Third, no matter which way you code this someone may >> in some case want it the other way - and it's not worth coding to >> allow such flexibility IMO. >> >> Just my $2. :) >> >> Cheers, >> David >> >>> >>> Thanks, >>> >>> Yasumasa From daniel.daugherty at oracle.com Tue Jul 23 13:57:21 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 23 Jul 2019 09:57:21 -0400 Subject: ResourceExhausted JVMTI event and -XX:*OnOutOfMemoryError In-Reply-To: <74df0ed5-a953-a2bb-cb23-688f758b1c3e@oracle.com> References: <2bba059e-bac8-7ec8-7e14-e142324cbb64@oracle.com> <1f58ace5-305a-e587-3e9b-9c65e0531620@gmail.com> <74df0ed5-a953-a2bb-cb23-688f758b1c3e@oracle.com> Message-ID: <5c58faf4-989e-97a7-db1b-fbc05ae57d6e@oracle.com> I agree with David's opinion here. Please do not make this change. Dan On 7/23/19 7:08 AM, David Holmes wrote: > On 23/07/2019 6:16 pm, Yasumasa Suenaga wrote: >> Sorry, the format is broken... I re-sent the email: >> >> >> There are not documented the relation between -XX:*OnOutOfMemoryError >> and ResourceExhausted, >> but JVMTI spec says ResourceExhausted is sent when VM resource are >> exhausted. >> >> https://docs.oracle.com/en/java/javase/12/docs/specs/jvmti.html#ResourceExhausted > > > The JVMTI spec can't say anything about -XX:*OnOutOfMemoryError as > they are a hotspot implementation detail. If you use the -XX flag you > get whatever the hotspot implementation does. That may override normal > specified behaviour. For example, if you use > -XX:+ExitOnOutOfMemoryError you can't then complain that the VM never > executes a catch block for OutOfMemoryError because it exited first! > It's the same for the event processing - we're exiting before we reach > that code. > >> As you said, we should not do other things when OutOfMemoryError is >> occurred. >> But I think it should be compliant when we will use Java related code >> in the event handler. >> If we do with C/C++ only (e.g. call printf()), I think we can do that. > > But we don't know what the handler will do. Nor do we know why the -XX > flag was used, or even what it may do in some cases. > >> If all codes should not be run when OutOfMemoryError is occurred, >> ResourceExhausted should be obsoleted, or we need to add the note for >> JVMTI document. > > There is no logical argument there for obsoleting the > ResourceExhausted event. > > I understand you want the event to be posted before the flag is > processed, but somebody else may want it exactly as it is - e.g. > dumping the heap before their event handler does some cleanup/recovery > work. > > And as I said the JVM TI spec knows nothing of this XX flag so cannot > say anything about it. > > David > ----- > >> >> Thanks, >> >> Yasumasa >> >> >> On 2019/07/23 16:43, David Holmes wrote: >>> Hi Yasumasa, >>> >>> On 23/07/2019 5:24 pm, Yasumasa Suenaga wrote: >>>> Hi all, >>>> >>>> I tried to use JVMTI agent which hooks ResourceExhausted event and >>>> -XX:ExitOnOutOfMemoryError together. >>>> The process was exited when OutOfMemoryError was occurred, but >>>> ResourceExhausted event was not fired. >>>> >>>> I checked HotSpot implementation, -XX:*OutOfMemoryError is >>>> processed before JVMTI event processing. >>>> >>>> http://hg.openjdk.java.net/jdk/jdk/file/d999a1a11485/src/hotspot/share/gc/shared/memAllocator.cpp#l125 >>>> >>>> >>>> IMHO ResourceExhausted should be called before -XX:*OutOfMemoryError. >>>> If it is a bug, I will file it to JBS and fix it. >>> >>> I don't think it is a bug. First, there's no specification for how >>> these two things interact - -XX:*OnOutOfMemoryError is just a VM >>> specific debugging/diagnostic hook. Second, if we're out of memory >>> then we don't want to be doing other things which may need to also >>> acquire memory. Third, no matter which way you code this someone may >>> in some case want it the other way - and it's not worth coding to >>> allow such flexibility IMO. >>> >>> Just my $2. :) >>> >>> Cheers, >>> David >>> >>>> >>>> Thanks, >>>> >>>> Yasumasa From sangheon.kim at oracle.com Tue Jul 23 17:19:49 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Tue, 23 Jul 2019 10:19:49 -0700 Subject: RFR (S): 8228388: Add information about dirty/skipped card for Merge HCC in G1 log In-Reply-To: References: Message-ID: Hi Thomas, On 7/18/19 9:27 AM, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this small change that adds information about > how many cards were flushed/used from the HCC during GC? > > This removes one annoyance that otherwise the amount of scanned cards > in the scan heap roots phase looks to be larger than the sum of cards > from the remembered sets and log buffers at times. > > Also I need that value for JDK-8227739 (so far). > > I added a small test case that checks whether G1 survives a GC without > HCC enabled - there is no such test yet, but seemed appropriate. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8228388 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8228388/webrev/ Looks good. Thanks, Sangheon > Testing: > hs-tier1, local gc/g1 run, new test, eyeballing log message output > > Thanks, > Thomas > > From zgu at redhat.com Tue Jul 23 19:04:05 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 23 Jul 2019 15:04:05 -0400 Subject: RFR(S) 8227927: Move ParallelCleaningTask back to G1 Message-ID: <2e778671-c082-fece-8269-f621be1708e5@redhat.com> ParallelCleaningTask was moved from G1 to shared directory, so Shenandoah can use it. Now, Shenandoah no longer uses it, it can return to G1. However, Shenandoah still uses some of sub-tasks defined in shared/parallelCleaning.hpp/cpp, therefore, only ParallelCleaningTask and JVMCICleaningTask are moved to G1. Bug: https://bugs.openjdk.java.net/browse/JDK-8227927 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8227927/webrev.00/index.html Test: hotspot_gc (fastdebug and release) on Linux x86_64. Submit tests in progress Thanks, -Zhengyu From zgu at redhat.com Tue Jul 23 19:42:21 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 23 Jul 2019 15:42:21 -0400 Subject: RFR 8228532: Shenandoah: Implement SBSA::try_resolve_jobject_in_native() Message-ID: Shenandoah needs to take slow path when resolving jobject during evacuation phase, to ensure it resolves to-space oops. Bug: https://bugs.openjdk.java.net/browse/JDK-8228532 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8228532/webrev.00/index.html Test: hotspot_gc_shenandoah (fastdebug and release) on Linux x86_64 and aarch64. Thanks, -Zhengyu From sangheon.kim at oracle.com Tue Jul 23 21:15:10 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Tue, 23 Jul 2019 14:15:10 -0700 Subject: RFR (S): 8227719: G1 pending cards estimation too conservative for cost prediction In-Reply-To: References: <4069BA1F-2D36-4567-A655-EE08EC948934@oracle.com> <2B5E38CC-EAA4-48D9-8E21-3F17875FC765@oracle.com> Message-ID: <70372af9-85ee-f67b-d79c-551bbb14d403@oracle.com> Hi Thomas, On 7/22/19 1:41 AM, Thomas Schatzl wrote: > Hi Kim, > > On Sat, 2019-07-20 at 22:03 -0400, Kim Barrett wrote: >>> On Jul 19, 2019, at 6:42 PM, Kim Barrett >>> wrote: >>> >>>> On Jul 19, 2019, at 10:15 AM, Thomas Schatzl < >>>> thomas.schatzl at oracle.com> wrote: >>>> New webrev at >>>> >>>> http://cr.openjdk.java.net/~tschatzl/8227719/webrev.0_to_1 (diff) >>>> http://cr.openjdk.java.net/~tschatzl/8227719/webrev.1 (full) >>>> >>>> which removes the buffer count, and replaces the >>>> "completed_buffers_num" method with what you suggested. >>>> >>>> I did some very cursory perf testing with no differences. >>>> >>>> Thanks, >>>> Thomas >>> Looks good. >> Oops, I realized later that I missed something. >> >> ------------------------------------------------------------------- >> ----------- >> src/hotspot/share/gc/g1/g1DirtyCardQueue.hpp >> 156 return align_up(num_entries_in_completed_buffers(), >> buffer_size()) / buffer_size(); >> >> align_up requires the alignment value to be a power of 2, but the >> buffer size is not required to be a power of 2, so this doesn't work. >> It just so happens that the default buffer_size *is* a power of 2 >> (G1UpdateBufferSize defaults to 256), and we probably don't have any >> tests that use a non-default buffer_size. >> >> > Oops. > > http://cr.openjdk.java.net/~tschatzl/8227719/webrev.1_to_2 (diff) > http://cr.openjdk.java.net/~tschatzl/8227719/webrev.2 (full) webrev.2 looks good. Thanks, Sangheon > > Thanks, > Thomas > > From mikhailo.seledtsov at oracle.com Wed Jul 24 00:01:48 2019 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Tue, 23 Jul 2019 17:01:48 -0700 Subject: RFR(XS) 8228359: [TESTBUG] jdk.jfr.e.g.c.TestGCHeapConfigurationEventWith32BitOops.java does not expect MinHeapSize to be aligned to HeapAlignment In-Reply-To: References: Message-ID: <6d437432-a43c-3e27-5b48-84d5486ea5aa@oracle.com> Looks good from my POV. Adding hotspot-gc-dev at openjdk.java.net since this test concerns a GC event. Thank you, Misha On 7/18/19 6:56 AM, Reingruber, Richard wrote: > Hi, > > could I please get reviews for this small TESTBUG fix? > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/2019/8228359/webrev/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8228359 > > Since JDK-8223837 MinHeapSize is aligned to HeapAlignment and the test fails therefore on linuxppc, because there we have 64k pages and HeapAlignment is proportional to the page size. > The fix changes the test to expect the configured size of 100m to be aligned to HeapAlignment (32m on linuxppc). > > Tested on linuxppcle and on linuxx86_64. > > Thanks, Richard. From yasuenag at gmail.com Wed Jul 24 00:43:34 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Wed, 24 Jul 2019 09:43:34 +0900 Subject: ResourceExhausted JVMTI event and -XX:*OnOutOfMemoryError In-Reply-To: <5c58faf4-989e-97a7-db1b-fbc05ae57d6e@oracle.com> References: <2bba059e-bac8-7ec8-7e14-e142324cbb64@oracle.com> <1f58ace5-305a-e587-3e9b-9c65e0531620@gmail.com> <74df0ed5-a953-a2bb-cb23-688f758b1c3e@oracle.com> <5c58faf4-989e-97a7-db1b-fbc05ae57d6e@oracle.com> Message-ID: <9760936a-1008-1fe5-35bc-72a83591d318@gmail.com> Hi David, Dan, Thank you for explanation. I will not file this to JBS. Yasumasa On 2019/07/23 22:57, Daniel D. Daugherty wrote: > I agree with David's opinion here. Please do not make this change. > > Dan > > > On 7/23/19 7:08 AM, David Holmes wrote: >> On 23/07/2019 6:16 pm, Yasumasa Suenaga wrote: >>> Sorry, the format is broken... I re-sent the email: >>> >>> >>> There are not documented the relation between -XX:*OnOutOfMemoryError and ResourceExhausted, >>> but JVMTI spec says ResourceExhausted is sent when VM resource are exhausted. >>> >>> https://docs.oracle.com/en/java/javase/12/docs/specs/jvmti.html#ResourceExhausted >> >> >> The JVMTI spec can't say anything about -XX:*OnOutOfMemoryError as they are a hotspot implementation detail. If you use the -XX flag you get whatever the hotspot implementation does. That may override normal specified behaviour. For example, if you use -XX:+ExitOnOutOfMemoryError you can't then complain that the VM never executes a catch block for OutOfMemoryError because it exited first! It's the same for the event processing - we're exiting before we reach that code. >> >>> As you said, we should not do other things when OutOfMemoryError is occurred. >>> But I think it should be compliant when we will use Java related code in the event handler. >>> If we do with C/C++ only (e.g. call printf()), I think we can do that. >> >> But we don't know what the handler will do. Nor do we know why the -XX flag was used, or even what it may do in some cases. >> >>> If all codes should not be run when OutOfMemoryError is occurred, >>> ResourceExhausted should be obsoleted, or we need to add the note for JVMTI document. >> >> There is no logical argument there for obsoleting the ResourceExhausted event. >> >> I understand you want the event to be posted before the flag is processed, but somebody else may want it exactly as it is - e.g. dumping the heap before their event handler does some cleanup/recovery work. >> >> And as I said the JVM TI spec knows nothing of this XX flag so cannot say anything about it. >> >> David >> ----- >> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2019/07/23 16:43, David Holmes wrote: >>>> Hi Yasumasa, >>>> >>>> On 23/07/2019 5:24 pm, Yasumasa Suenaga wrote: >>>>> Hi all, >>>>> >>>>> I tried to use JVMTI agent which hooks ResourceExhausted event and -XX:ExitOnOutOfMemoryError together. >>>>> The process was exited when OutOfMemoryError was occurred, but ResourceExhausted event was not fired. >>>>> >>>>> I checked HotSpot implementation, -XX:*OutOfMemoryError is processed before JVMTI event processing. >>>>> >>>>> http://hg.openjdk.java.net/jdk/jdk/file/d999a1a11485/src/hotspot/share/gc/shared/memAllocator.cpp#l125 >>>>> >>>>> IMHO ResourceExhausted should be called before -XX:*OutOfMemoryError. >>>>> If it is a bug, I will file it to JBS and fix it. >>>> >>>> I don't think it is a bug. First, there's no specification for how these two things interact - -XX:*OnOutOfMemoryError is just a VM specific debugging/diagnostic hook. Second, if we're out of memory then we don't want to be doing other things which may need to also acquire memory. Third, no matter which way you code this someone may in some case want it the other way - and it's not worth coding to allow such flexibility IMO. >>>> >>>> Just my $2. :) >>>> >>>> Cheers, >>>> David >>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa > From kim.barrett at oracle.com Wed Jul 24 00:52:47 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 23 Jul 2019 20:52:47 -0400 Subject: RFR(S) 8227927: Move ParallelCleaningTask back to G1 In-Reply-To: <2e778671-c082-fece-8269-f621be1708e5@redhat.com> References: <2e778671-c082-fece-8269-f621be1708e5@redhat.com> Message-ID: > On Jul 23, 2019, at 3:04 PM, Zhengyu Gu wrote: > > ParallelCleaningTask was moved from G1 to shared directory, so Shenandoah can use it. Now, Shenandoah no longer uses it, it can return to G1. > > However, Shenandoah still uses some of sub-tasks defined in shared/parallelCleaning.hpp/cpp, therefore, only ParallelCleaningTask and JVMCICleaningTask are moved to G1. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8227927 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8227927/webrev.00/index.html > > Test: > hotspot_gc (fastdebug and release) on Linux x86_64. > Submit tests in progress > > > Thanks, > > -Zhengyu Looks good. From thomas.schatzl at oracle.com Wed Jul 24 08:03:25 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 24 Jul 2019 10:03:25 +0200 Subject: RFR(S) 8227927: Move ParallelCleaningTask back to G1 In-Reply-To: <2e778671-c082-fece-8269-f621be1708e5@redhat.com> References: <2e778671-c082-fece-8269-f621be1708e5@redhat.com> Message-ID: <2891e13afc05e268c83142bae87be93b3d94f9ef.camel@oracle.com> Hi Zhengyu, On Tue, 2019-07-23 at 15:04 -0400, Zhengyu Gu wrote: > ParallelCleaningTask was moved from G1 to shared directory, so > Shenandoah can use it. Now, Shenandoah no longer uses it, it can > return to G1. > > However, Shenandoah still uses some of sub-tasks defined in > shared/parallelCleaning.hpp/cpp, therefore, only > ParallelCleaningTask and JVMCICleaningTask are moved to G1. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8227927 > Webrev: > http://cr.openjdk.java.net/~zgu/JDK-8227927/webrev.00/index.html > > Test: > hotspot_gc (fastdebug and release) on Linux x86_64. > Submit tests in progress > looks good, thanks a lot. Thomas From thomas.schatzl at oracle.com Wed Jul 24 08:04:22 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 24 Jul 2019 10:04:22 +0200 Subject: RFR (S): 8227719: G1 pending cards estimation too conservative for cost prediction In-Reply-To: <70372af9-85ee-f67b-d79c-551bbb14d403@oracle.com> References: <4069BA1F-2D36-4567-A655-EE08EC948934@oracle.com> <2B5E38CC-EAA4-48D9-8E21-3F17875FC765@oracle.com> <70372af9-85ee-f67b-d79c-551bbb14d403@oracle.com> Message-ID: Hi, On Tue, 2019-07-23 at 14:15 -0700, sangheon.kim at oracle.com wrote: > Hi Thomas, > > On 7/22/19 1:41 AM, Thomas Schatzl wrote: > > Hi Kim, > > > > On Sat, 2019-07-20 at 22:03 -0400, Kim Barrett wrote: > > > > On Jul 19, 2019, at 6:42 PM, Kim Barrett < > > > > kim.barrett at oracle.com> > > > > wrote: > > > > [...] > > > Oops, I realized later that I missed something. > > > > > > --------------------------------------------------------------- > > > ---- > > > ----------- > > > src/hotspot/share/gc/g1/g1DirtyCardQueue.hpp > > > 156 return align_up(num_entries_in_completed_buffers(), > > > buffer_size()) / buffer_size(); > > > > > > align_up requires the alignment value to be a power of 2, but the > > > buffer size is not required to be a power of 2, so this doesn't > > > work. It just so happens that the default buffer_size *is* a > > > power of 2 (G1UpdateBufferSize defaults to 256), and we probably > > > don't have any tests that use a non-default buffer_size. > > > > > > > > > > Oops. > > > > http://cr.openjdk.java.net/~tschatzl/8227719/webrev.1_to_2 (diff) > > http://cr.openjdk.java.net/~tschatzl/8227719/webrev.2 (full) > > webrev.2 looks good. > > Thanks, > Sangheon thanks for your review. Thomas From thomas.schatzl at oracle.com Wed Jul 24 08:06:16 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 24 Jul 2019 10:06:16 +0200 Subject: RFR (S): 8228388: Add information about dirty/skipped card for Merge HCC in G1 log In-Reply-To: References: Message-ID: <1e569cf7b49a820405f0d24e9b9f8ec01e6934ec.camel@oracle.com> Hi Kim, On Thu, 2019-07-18 at 15:47 -0400, Kim Barrett wrote: > > On Jul 18, 2019, at 12:27 PM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > Hi all, > > > > can I have reviews for this small change that adds information > > about how many cards were flushed/used from the HCC during GC? > > > > [...] > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8228388 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8228388/webrev/ > > Testing: > > hs-tier1, local gc/g1 run, new test, eyeballing log message output > > > > Thanks, > > Thomas > > Looks good. > thanks for your review. Thomas From thomas.schatzl at oracle.com Wed Jul 24 08:06:46 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 24 Jul 2019 10:06:46 +0200 Subject: RFR (S): 8228388: Add information about dirty/skipped card for Merge HCC in G1 log In-Reply-To: References: Message-ID: <8f213aa7630a36192331e749f2d12a5782548a4d.camel@oracle.com> Hi Sangheon, On Tue, 2019-07-23 at 10:19 -0700, sangheon.kim at oracle.com wrote: > Hi Thomas, > > On 7/18/19 9:27 AM, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this small change that adds information > > about how many cards were flushed/used from the HCC during GC? [...] > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8228388 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8228388/webrev/ > > Looks good. > thanks for your review. Thomas From thomas.schatzl at oracle.com Wed Jul 24 09:56:34 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 24 Jul 2019 11:56:34 +0200 Subject: RFR(S): 8227225: ParallelGC: add subspace transitions for young gen for gc+heap=info log lines In-Reply-To: References: <877bd17cff3be20bf8acd95d9eddbb5c9cfb7cf5.camel@oracle.com> Message-ID: <8e10c88f3881c78ac779b8a3b520b333ecf2c48f.camel@oracle.com> Hi, On Fri, 2019-07-19 at 10:40 -0700, Tony Printezis wrote: > Hey Thomas, > > Thanks for taking a look and again apologies for the late reply, I > was on vacation until earlier this week. Please see inline. > > [...] > > > I have a similar change ready to go for GenCollectedHeap (Serial > > > / CMS). > > > I?ll see what the feedback is on this before I open that for > > > code review. > > > > > > A couple of notes: > > > > > > - I moved the PreGCValues class to gc/shared as I?d like to re- > > > use it for the GenCollectedHeap changes too. > > > > It's a bit awkwards as there are non-generational GCs now, but I do > > not have a better idea. Maybe somebody else has. > > I could just rename this to GenPreGCValues so it?s clear it?s only > for the generational GCs? Or I could come up with a convoluted class > hierarchy that can handle both generational and non-generational GCs. > But I?d rather not, given we don?t have any use cases for this for > the latter. So, maybe, rename PreGCValues to GenPreGCValues and, if > we need to in the future, we can just split the class as needed? I agree with your plan. Don't do too much potentially unnecessary work :) Thanks, Thomas From fairoz.matte at oracle.com Wed Jul 24 10:01:06 2019 From: fairoz.matte at oracle.com (Fairoz Matte) Date: Wed, 24 Jul 2019 03:01:06 -0700 (PDT) Subject: RFR: 8048556: Unnecessary GCLocker-initiated young GCs Message-ID: <6a3227ce-3cf0-46a3-9eb7-2fc7c3f9d675@default> Hi, Please review the change to avoid Unnecessary GCLocker Initiated GC's. JBS: https://bugs.openjdk.java.net/browse/JDK-8048556 Webrev: http://cr.openjdk.java.net/~fmatte/8048556/webrev.00/ Background: This fix is proposed by "Tony Printezis" in the comments section. GC locker-initiated young GC happens immediately after another young GC. The fix is to send the right gc counts and fixing the vm gc op not to signal "full" Reading right GC count from "GC_locker::jni_unlock(JavaThread* thread)" method and provide implementations for GCH, PS, and G1 to get that count in Collect() method. I hope I have done justification to Tony's approach. I have modified a regression test (again from Tony's work) that verifies when "GCLocker Initiated GC" occurs, It runs gc only if young generation occupancy is > 75%. Thanks, Fairoz From thomas.schatzl at oracle.com Wed Jul 24 10:05:24 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 24 Jul 2019 12:05:24 +0200 Subject: RFR(S): 8227225: ParallelGC: add subspace transitions for young gen for gc+heap=info log lines In-Reply-To: References: <877bd17cff3be20bf8acd95d9eddbb5c9cfb7cf5.camel@oracle.com> Message-ID: Hi again, On Fri, 2019-07-19 at 10:40 -0700, Tony Printezis wrote: > Hey Thomas, > > Thanks for taking a look and again apologies for the late reply, I > was on vacation until earlier this week. Please see inline. > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > On July 10, 2019 at 11:19:37 AM, Thomas Schatzl ( > thomas.schatzl at oracle.com) wrote: > > On Fri, 2019-07-05 at 11:06 -0700, Tony Printezis wrote: > > > Hi all, > > > > > > This is a follow-up to 8223575 and adds subspace info for eden > > > and from space to the gc+heap=info output for ParallelGC. I also > > > added the before capacity, similar to 8223575, as it can change > > > during the GC (see, in fact, example below). > > > > > > Before: > > > > > > [11.123s][info][gc,heap ] GC(17) PSYoungGen: 4759616K- > > > >1664K(4545536K) > > > [11.123s][info][gc,heap ] GC(17) ParOldGen: 1280K- > > > >1280K(1376256K) > > > > > > After: > > > > > > [12.133s][info][gc,heap ] GC(25) PSYoungGen: > > > 2065952K(2066432K)->1536K(1983488K) Eden: 2064384K(2064384K)- > > > >0K(1981952K) > > > From: 1568K(2048K)->1536K(1536K) > > > > Note that I think this is confusing, or the "From" label should be > > renamed. The second part of this heap transition actually prints > > the "To" space from what I understand. > > That?s not entirely true, though. The spaces do switch roles at the > end of the GC, so the To space does become the From space of the > following GC. So using From here is not entirely incorrect (but I do > get your point). Wait - on evac failure the generational collectors just do not switch from and to-space, don't they? I can't think of any other situation where To-space contains something for these collectors. Then indeed printing To-space is useless as it is always empty. I can't think of a situation where this would be the case. Thanks, Thomas From zgu at redhat.com Wed Jul 24 11:14:23 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 24 Jul 2019 07:14:23 -0400 Subject: RFR(S) 8227927: Move ParallelCleaningTask back to G1 In-Reply-To: References: <2e778671-c082-fece-8269-f621be1708e5@redhat.com> Message-ID: <576611b4-5f04-f0ba-048b-694383c7c10c@redhat.com> Thanks, Kim. -Zhengyu On 7/23/19 8:52 PM, Kim Barrett wrote: >> On Jul 23, 2019, at 3:04 PM, Zhengyu Gu wrote: >> >> ParallelCleaningTask was moved from G1 to shared directory, so Shenandoah can use it. Now, Shenandoah no longer uses it, it can return to G1. >> >> However, Shenandoah still uses some of sub-tasks defined in shared/parallelCleaning.hpp/cpp, therefore, only ParallelCleaningTask and JVMCICleaningTask are moved to G1. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8227927 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8227927/webrev.00/index.html >> >> Test: >> hotspot_gc (fastdebug and release) on Linux x86_64. >> Submit tests in progress >> >> >> Thanks, >> >> -Zhengyu > > Looks good. > From zgu at redhat.com Wed Jul 24 11:14:44 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 24 Jul 2019 07:14:44 -0400 Subject: RFR(S) 8227927: Move ParallelCleaningTask back to G1 In-Reply-To: <2891e13afc05e268c83142bae87be93b3d94f9ef.camel@oracle.com> References: <2e778671-c082-fece-8269-f621be1708e5@redhat.com> <2891e13afc05e268c83142bae87be93b3d94f9ef.camel@oracle.com> Message-ID: <036f18dd-7ce2-196a-110d-159cec1fe937@redhat.com> Thanks, Thomas. -Zhengyu On 7/24/19 4:03 AM, Thomas Schatzl wrote: > Hi Zhengyu, > > On Tue, 2019-07-23 at 15:04 -0400, Zhengyu Gu wrote: >> ParallelCleaningTask was moved from G1 to shared directory, so >> Shenandoah can use it. Now, Shenandoah no longer uses it, it can >> return to G1. >> >> However, Shenandoah still uses some of sub-tasks defined in >> shared/parallelCleaning.hpp/cpp, therefore, only >> ParallelCleaningTask and JVMCICleaningTask are moved to G1. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8227927 >> Webrev: >> http://cr.openjdk.java.net/~zgu/JDK-8227927/webrev.00/index.html >> >> Test: >> hotspot_gc (fastdebug and release) on Linux x86_64. >> Submit tests in progress >> > > looks good, thanks a lot. > > Thomas > > From sci at amazon.com Wed Jul 24 14:37:07 2019 From: sci at amazon.com (Sciampacone, Ryan) Date: Wed, 24 Jul 2019 14:37:07 +0000 Subject: RFR: 8227226: Segmented array clearing for ZGC In-Reply-To: <625131FC-8B02-4BEC-80B5-F1757232277A@amazon.com> References: <5809FFE3-ED37-429B-9189-49D8FD14D092@amazon.com> <625131FC-8B02-4BEC-80B5-F1757232277A@amazon.com> Message-ID: <7C95DE35-C9F8-4F11-8D07-9511748128C4@amazon.com> Somehow I lost the RFR off the front and started a new thread. Now that we're both off vacation I'd like to revisit this. Can you take a look? ?On 7/8/19, 10:40 AM, "hotspot-gc-dev on behalf of Sciampacone, Ryan" wrote: http://cr.openjdk.java.net/~phh/8227226/webrev.01/ This shifts away from abusing the constructor do_zero magic in exchange for virtualizing mem_clear() and specializing for the Z version. It does create a change in mem_clear in that it returns an updated version of mem. It does create change outside of the Z code however it does feel cleaner. I didn't make a change to PinAllocating - looking at it, the safety of keeping it constructor / destructor based still seemed appropriate to me. If the objection is to using the sequence numbers to pin (and instead using handles to update) - this to me seems less error prone. I had originally discussed handles with Stefan but the proposal came down to this which looks much cleaner. On 7/8/19, 6:36 AM, "hotspot-gc-dev on behalf of Sciampacone, Ryan" wrote: 1) Yes this was a conscious decision. There was discussion on determining the optimal point for breakup but given the existing sizes this seemed sufficient. This doesn't preclude the ability to go down that path if its deemed absolutely necessary. The path for more complex decisions is now available. 2) Agree 3) I'm not clear here. Do you mean effectively going direct to ZHeap and bypassing the single function PinAllocating? Agree. Otherwise I'll ask you to be a bit clearer. 4) Agree 5) I initially had the exact same reaction but I played around with a few other versions (including breaking up initialization points between header and body to get the desired results) and this ended up looking correct. I'll try mixing in the mem clearer function again (fresh start) to see if it looks any better. On 7/8/19, 5:49 AM, "Per Liden" wrote: Hi Ryan, A few general comments: 1) It looks like this still only work for large pages? 2) The log_info stuff should be removed. 3) I'm not a huge fan of single-use utilities like PinAllocating, at least not when, IMO, the alternative is more straight forward and less code. 4) Please make locals const when possible. 5) Duplicating _do_zero looks odd. Injecting a "mem clearer", similar to what Stefans original patch did, seems worth exploring. cheers, /Per (Btw, I'm on vacation so I might not be super-responsive to emails) On 2019-07-08 12:42, Erik ?sterlund wrote: > Hi Ryan, > > This looks good in general. Just some stylistic things... > > 1) In the ZGC project we like the letter 'Z' so much that we put it in > front of everything we possibly can, including all class names. > 2) We also explicitly state things are private even though it's > bleedingly obvious. > > So: > > 39 class PinAllocating { > 40 HeapWord* _mem; > 41 public: -> 39 class ZPinAllocating { 40 private: 41 HeapWord* _mem; > 42 > 41 public: I can be your sponsor and push this change for you. I don't > think there is a need for another webrev for my small stylistic remarks, > so I can just fix that before pushing this for you. On that note, I'll > add me and StefanK to the contributed-by section as we all worked out > the right solution to this problem collaboratively. I have run through > mach5 tier1-5, and found no issues with this patch. Thanks, /Erik > > On 2019-07-05 17:18, Sciampacone, Ryan wrote: >> http://cr.openjdk.java.net/~phh/8227226/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8227226 >> >> This patch introduces safe point checks into array clearing during >> allocation for ZGC. The patch isolates the changes to ZGC as (in >> particular with the more modern collectors) the approach to >> incrementalizing or respecting safe point checks is going to be >> different. >> >> The approach is to keep the region holding the array in the allocating >> state (pin logic) while updating the color to the array after checks. >> >> Can I get a review? Thanks. >> >> Ryan > From zgu at redhat.com Wed Jul 24 15:46:04 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 24 Jul 2019 11:46:04 -0400 Subject: RFR 8228490: Shenandoah: Shenandoah concurrent root evacuation may race against OopStorage::release() Message-ID: <968310d7-8303-5c86-3804-b095b42eb20c@redhat.com> Concurrent evacuating OopStorage backed roots, may race against OopStorage::release() calls. Therefore, it needs to update the roots with Atomic::cmpxchg(), to avoid overwriting the roots if they are released. Bug: https://bugs.openjdk.java.net/browse/JDK-8228490 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8228490/webrev.00/index.html Test: hotspot_gc_shenandoah (fastdebug and release) jcstress fastdebug nightly tests Thanks, -Zhengyu From rkennke at redhat.com Wed Jul 24 15:53:25 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 24 Jul 2019 17:53:25 +0200 Subject: RFR 8228490: Shenandoah: Shenandoah concurrent root evacuation may race against OopStorage::release() In-Reply-To: <968310d7-8303-5c86-3804-b095b42eb20c@redhat.com> References: <968310d7-8303-5c86-3804-b095b42eb20c@redhat.com> Message-ID: <7b13f03f-ffb6-4248-1563-5cd58f8c9ea0@redhat.com> We have the fix in shenandoah/jdk already, and it seems to resolve the trouble. Patch looks good! Thanks, Roman > Concurrent evacuating OopStorage backed roots, may race against > OopStorage::release() calls. Therefore, it needs to update the roots > with Atomic::cmpxchg(), to avoid overwriting the roots if they are > released. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8228490 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8228490/webrev.00/index.html > > Test: > ? hotspot_gc_shenandoah (fastdebug and release) > ? jcstress fastdebug > ? nightly tests > > Thanks, > > -Zhengyu From kim.barrett at oracle.com Wed Jul 24 23:12:17 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 24 Jul 2019 19:12:17 -0400 Subject: RFR: 8048556: Unnecessary GCLocker-initiated young GCs In-Reply-To: <6a3227ce-3cf0-46a3-9eb7-2fc7c3f9d675@default> References: <6a3227ce-3cf0-46a3-9eb7-2fc7c3f9d675@default> Message-ID: <7A07B132-2ED7-4992-90E4-F8E57B731A75@oracle.com> > On Jul 24, 2019, at 6:01 AM, Fairoz Matte wrote: > > Hi, > > Please review the change to avoid Unnecessary GCLocker Initiated GC's. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8048556 > Webrev: http://cr.openjdk.java.net/~fmatte/8048556/webrev.00/ > > Background: This fix is proposed by "Tony Printezis" in the comments section. > GC locker-initiated young GC happens immediately after another young GC. > > The fix is to send the right gc counts and fixing the vm gc op not to signal "full" > Reading right GC count from "GC_locker::jni_unlock(JavaThread* thread)" method and > provide implementations for GCH, PS, and G1 to get that count in Collect() method. > I hope I have done justification to Tony's approach. > > I have modified a regression test (again from Tony's work) that verifies when "GCLocker Initiated GC" > occurs, It runs gc only if young generation occupancy is > 75%. > > Thanks, > Fairoz I had a number of other comments on or issues with this change, but I'm not going to bring them up just now because I think they end up not mattering due to what follows here. I think G1 doesn't need this change. From comments in the CR it seems the problem hasn't been reproduced with G1. I think I can explain that. When G1 has an allocation failure that might require a GC, and it can't satisfy the allocation in some other way (such as by allocating more young regions), then if GCLocker::needs_gc() is true it will stall and wait for the gc-locker gc request to be processed, and then retry the allocation. ParallelGC only stalls and retries if GCLocker::is_active_and_needs_gc(). This opens the window for the race condition Tony pointed out in the first comment in the CR, because is_active may no longer true, even though there may be a gc-locker gc request on the way. GenCollectedHeap (used by SerialGC and CMS) is similar to ParallelGC. This suggests an entirely different approach should be taken, directly addressing ParallelGC and GenCollectedHeap and making them similarly wait for a pending gc-locker gc request if there is one. That would eliminate the whole "capture and indirectly pass along the proper counters" mechanism discussed in the CR and proposed by the current webrev, which I have a number of issues with. Note that this has nothing to do with the second (GCH-specific?) cause for the issue, discussed in Tony's 3rd comment in the CR. That part of the change seems okay. From kim.barrett at oracle.com Wed Jul 24 23:35:44 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 24 Jul 2019 19:35:44 -0400 Subject: RFR: 8048556: Unnecessary GCLocker-initiated young GCs In-Reply-To: <7A07B132-2ED7-4992-90E4-F8E57B731A75@oracle.com> References: <6a3227ce-3cf0-46a3-9eb7-2fc7c3f9d675@default> <7A07B132-2ED7-4992-90E4-F8E57B731A75@oracle.com> Message-ID: > On Jul 24, 2019, at 7:12 PM, Kim Barrett wrote: > When G1 has an allocation failure that might require a GC, and it > can't satisfy the allocation in some other way (such as by allocating > more young regions), then if GCLocker::needs_gc() is true it will > stall and wait for the gc-locker gc request to be processed, and then > retry the allocation. Looking at the history of the G1 code, the present behavior of deferring to a pending gc-locker gc request was introduced by 7143858: G1: Back to back young GCs with the second GC having a minimally sized eden Sound familiar? There are even comments in that CR discussing exactly the race described by Tony in JDK-8048556. Pity other GCs weren't examined for the same problem and similarly modified at the same time. From fairoz.matte at oracle.com Thu Jul 25 12:32:38 2019 From: fairoz.matte at oracle.com (Fairoz Matte) Date: Thu, 25 Jul 2019 05:32:38 -0700 (PDT) Subject: RFR: 8048556: Unnecessary GCLocker-initiated young GCs In-Reply-To: References: <6a3227ce-3cf0-46a3-9eb7-2fc7c3f9d675@default> <7A07B132-2ED7-4992-90E4-F8E57B731A75@oracle.com> Message-ID: <9c40acac-e85e-4186-93ee-715d3a909549@default> Hi Kim, Thanks for looking into this. What you have suggested is totally different approach. I will investigate more. Thanks, Fairoz > -----Original Message----- > From: Kim Barrett > Sent: Thursday, July 25, 2019 5:06 AM > To: Fairoz Matte > Cc: Hotspot-Gc-Dev > Subject: Re: RFR: 8048556: Unnecessary GCLocker-initiated young GCs > > > On Jul 24, 2019, at 7:12 PM, Kim Barrett wrote: > > When G1 has an allocation failure that might require a GC, and it > > can't satisfy the allocation in some other way (such as by allocating > > more young regions), then if GCLocker::needs_gc() is true it will > > stall and wait for the gc-locker gc request to be processed, and then > > retry the allocation. > > Looking at the history of the G1 code, the present behavior of deferring to a > pending gc-locker gc request was introduced by > > 7143858: G1: Back to back young GCs with the second GC having a minimally > sized eden > > Sound familiar? There are even comments in that CR discussing exactly the > race described by Tony in JDK-8048556. Pity other GCs weren't examined for > the same problem and similarly modified at the same time. > From zgu at redhat.com Thu Jul 25 13:01:07 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 25 Jul 2019 09:01:07 -0400 Subject: RFR 8228532: Shenandoah: Implement SBSA::try_resolve_jobject_in_native() In-Reply-To: References: Message-ID: <5a2ee72a-ea25-669e-226f-7eb62084068a@redhat.com> Further tests discovered some issues with this patch, I would like to withdraw this RFR for now. Thanks, -Zhengyu On 7/23/19 3:42 PM, Zhengyu Gu wrote: > Shenandoah needs to take slow path when resolving jobject during > evacuation phase, to ensure it resolves to-space oops. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8228532 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8228532/webrev.00/index.html > > Test: > ? hotspot_gc_shenandoah (fastdebug and release) on Linux x86_64 and > aarch64. > > > Thanks, > > -Zhengyu From thomas.schatzl at oracle.com Thu Jul 25 13:55:47 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 25 Jul 2019 15:55:47 +0200 Subject: RFR (): JDK-8228503: Rename "rs_lengths" to "rs_length" in ergo code Message-ID: Hi, can I have reviews for this change that (mechanically) changes all prediction related identifiers with the "rs_lengths" suffix to "rs_length"? The reason is that all these values are not vectors like the plural-s suggests (and I think we use this naming style in other places), but contain a scalar (sum) value. Reading through the prediction code with this style is really annoying, given that the code later performs standard mathematical ops with single values on that: E.g. double time_spent = predicted_rs_lengths * predict_cost_per_card(); Note that I understand that there is some reasoning for that as these variables contain the sum of differences, or the sum of remembered set "lengths" (for which we typically use "num_*_entries"/"num_*_cards" in other places); however due to way these are used (e.g. to predict the singular remembered set "length" of the entire young gen) I would prefer to remove the "s". I am okay with keeping the "rs_length" as it is to me by far not as annoying, and I did so. Alternatively I would suggest to add at least "sum" postfix to the names. The change also fixes multiple similar pluralization like "_inc_recorded_rs_lengths_diffs". There is some particular misnomer ("max_rs_length") that is going to be fixed in the next CR (JDK-8228504). CR: https://bugs.openjdk.java.net/browse/JDK-8228503 Webrev: http://cr.openjdk.java.net/~tschatzl/8228503/webrev/ Testing: hs-tier1-3 Thanks, Thomas From thomas.schatzl at oracle.com Thu Jul 25 14:01:56 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 25 Jul 2019 16:01:56 +0200 Subject: RFR (S): JDK-8228504: Rename max_rs_length to young_rs_length Message-ID: <70fbbf7294eb9a985b3ad9e245d41dc6dcd9577c.camel@oracle.com> Hi all, can I have reviews for this change that fixes the identifier "max_rs_length" across G1 code? The problem with it is that it is no maximum of a single remembered set length, but a sum, and to clarify that it is supposed to be the sum of young gen remembered set lengths, this change calls it "young_rs_length". With that change the issue in JDK-8228505, that we add old remembered set lengths to it during mixed gc, immediately stands out. Fixed in that change. Based on JDK-8228503. CR: https://bugs.openjdk.java.net/browse/JDK-8228504 Webrev: http://cr.openjdk.java.net/~tschatzl/8228504/webrev/ Testing: hs-tier1-3 Thanks, Thomas From thomas.schatzl at oracle.com Thu Jul 25 14:16:14 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 25 Jul 2019 16:16:14 +0200 Subject: RFR (S): JDK-8228504: Rename max_rs_length to young_rs_length In-Reply-To: <70fbbf7294eb9a985b3ad9e245d41dc6dcd9577c.camel@oracle.com> References: <70fbbf7294eb9a985b3ad9e245d41dc6dcd9577c.camel@oracle.com> Message-ID: Hi all, let me retract this change. Further looking at it showed that what the prediction actually needs seems to be both young and total lengths, or needs to be put much later in my current chain of changes. Thanks, Thomas On Thu, 2019-07-25 at 16:01 +0200, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that fixes the identifier > "max_rs_length" across G1 code? > > The problem with it is that it is no maximum of a single remembered > set length, but a sum, and to clarify that it is supposed to be the > sum of young gen remembered set lengths, this change calls it > "young_rs_length". > > With that change the issue in JDK-8228505, that we add old remembered > set lengths to it during mixed gc, immediately stands out. Fixed in > that change. > > Based on JDK-8228503. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8228504 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8228504/webrev/ > Testing: > hs-tier1-3 > > Thanks, > Thomas > > From kim.barrett at oracle.com Thu Jul 25 22:19:39 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 25 Jul 2019 18:19:39 -0400 Subject: RFR (): JDK-8228503: Rename "rs_lengths" to "rs_length" in ergo code In-Reply-To: References: Message-ID: <4651266F-65C5-45EA-9D25-20AA45E692E6@oracle.com> > On Jul 25, 2019, at 9:55 AM, Thomas Schatzl wrote: > > [?] > CR: > https://bugs.openjdk.java.net/browse/JDK-8228503 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8228503/webrev/ > Testing: > hs-tier1-3 > > Thanks, > Thomas Looks good. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1Policy.cpp 304 G1Policy::calculate_young_list_target_length(size_t rs_length, 305 uint base_min_length, ... [pre-existing] Unusual indentation of parameters. ------------------------------------------------------------------------------ From tprintezis at twitter.com Thu Jul 25 23:02:55 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Thu, 25 Jul 2019 16:02:55 -0700 Subject: RFR(S): 8227225: ParallelGC: add subspace transitions for young gen for gc+heap=info log lines In-Reply-To: References: <877bd17cff3be20bf8acd95d9eddbb5c9cfb7cf5.camel@oracle.com> Message-ID: Thomas, We *think* there is an edge case with ParallelGC that might leave both survivors non-empty after a compaction (but we haven?t convinced ourselves 100% that this is the case). We?re also pretty sure that this cannot happen in CMS (and also Serial). So maybe I can skip the To-Space for now. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On July 24, 2019 at 3:07:35 AM, Thomas Schatzl (thomas.schatzl at oracle.com) wrote: Hi again, On Fri, 2019-07-19 at 10:40 -0700, Tony Printezis wrote: > Hey Thomas, > > Thanks for taking a look and again apologies for the late reply, I > was on vacation until earlier this week. Please see inline. > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > On July 10, 2019 at 11:19:37 AM, Thomas Schatzl ( > thomas.schatzl at oracle.com) wrote: > > On Fri, 2019-07-05 at 11:06 -0700, Tony Printezis wrote: > > > Hi all, > > > > > > This is a follow-up to 8223575 and adds subspace info for eden > > > and from space to the gc+heap=info output for ParallelGC. I also > > > added the before capacity, similar to 8223575, as it can change > > > during the GC (see, in fact, example below). > > > > > > Before: > > > > > > [11.123s][info][gc,heap ] GC(17) PSYoungGen: 4759616K- > > > >1664K(4545536K) > > > [11.123s][info][gc,heap ] GC(17) ParOldGen: 1280K- > > > >1280K(1376256K) > > > > > > After: > > > > > > [12.133s][info][gc,heap ] GC(25) PSYoungGen: > > > 2065952K(2066432K)->1536K(1983488K) Eden: 2064384K(2064384K)- > > > >0K(1981952K) > > > From: 1568K(2048K)->1536K(1536K) > > > > Note that I think this is confusing, or the "From" label should be > > renamed. The second part of this heap transition actually prints > > the "To" space from what I understand. > > That?s not entirely true, though. The spaces do switch roles at the > end of the GC, so the To space does become the From space of the > following GC. So using From here is not entirely incorrect (but I do > get your point). Wait - on evac failure the generational collectors just do not switch from and to-space, don't they? I can't think of any other situation where To-space contains something for these collectors. Then indeed printing To-space is useless as it is always empty. I can't think of a situation where this would be the case. Thanks, Thomas From kim.barrett at oracle.com Thu Jul 25 23:41:44 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 25 Jul 2019 19:41:44 -0400 Subject: RFR(S): 8228631: Fix inconsistent OopStorage::Block owner usage Message-ID: <52C8E9D9-AEC2-47AA-8F9F-8BB4703A5AEE@oracle.com> Please review this cleanup of the use of the _owner member of OopStorage blocks. It's now just the intptr_t address of the owning storage object, and no longer sometimes used as the actual owning object. CR: https://bugs.openjdk.java.net/browse/JDK-8228631 Webrev: http://cr.openjdk.java.net/~kbarrett/8228631/open.00/ Testing: mach5 tier1 From kim.barrett at oracle.com Fri Jul 26 00:55:03 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 25 Jul 2019 20:55:03 -0400 Subject: RFR(S): 8226796: Reduce debug level logging for oopstorage+blocks Message-ID: <59022DD2-ADB4-482D-9A18-A881555020DA@oracle.com> Please review this adjustment to some of the logging levels in the OopStorage implementation, under the (oopstorage, blocks) tag sequence. High volume logging statements have been reduced to trace level, and memory allocation failure has been raised to info. CR: https://bugs.openjdk.java.net/browse/JDK-8226796 Webrev: http://cr.openjdk.java.net/~kbarrett/8226796/open.00/ Note: This webrev is baselined from the JDK-8228631 change that is also out for review. Testing: mach5 tier1 Local testing with various logging options and looked at output. From zgu at redhat.com Fri Jul 26 01:18:22 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 25 Jul 2019 21:18:22 -0400 Subject: RFR 8228532: Shenandoah: Implement SBSA::try_resolve_jobject_in_native() In-Reply-To: <5a2ee72a-ea25-669e-226f-7eb62084068a@redhat.com> References: <5a2ee72a-ea25-669e-226f-7eb62084068a@redhat.com> Message-ID: <2b7ec01b-085b-6e33-2946-2ad570d89ec6@redhat.com> Updated Webrev: http://cr.openjdk.java.net/~zgu/JDK-8228532/webrev.01/ On X86 platforms, r15 does not have valid thread value, instead, it should be derived from jni_env argument. Test: hotspot_gc_shenandoah (fastdebug and release) on Linux x86_64, x86_32 Windows x86_64. Thanks, -Zhengyu On 7/25/19 9:01 AM, Zhengyu Gu wrote: > Further tests discovered some issues with this patch, I would like to > withdraw this RFR for now. > > Thanks, > > -Zhengyu > > On 7/23/19 3:42 PM, Zhengyu Gu wrote: >> Shenandoah needs to take slow path when resolving jobject during >> evacuation phase, to ensure it resolves to-space oops. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8228532 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8228532/webrev.00/index.html >> >> Test: >> ?? hotspot_gc_shenandoah (fastdebug and release) on Linux x86_64 and >> aarch64. >> >> >> Thanks, >> >> -Zhengyu From rkennke at redhat.com Fri Jul 26 09:46:22 2019 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 26 Jul 2019 11:46:22 +0200 Subject: RFR 8228532: Shenandoah: Implement SBSA::try_resolve_jobject_in_native() In-Reply-To: <2b7ec01b-085b-6e33-2946-2ad570d89ec6@redhat.com> References: <5a2ee72a-ea25-669e-226f-7eb62084068a@redhat.com> <2b7ec01b-085b-6e33-2946-2ad570d89ec6@redhat.com> Message-ID: <8241fd2e-4873-399d-73fd-df861f1a5592@redhat.com> Looks good, thanks! Roman > Updated Webrev: http://cr.openjdk.java.net/~zgu/JDK-8228532/webrev.01/ > > On X86 platforms, r15 does not have valid thread value, instead, it > should be derived from jni_env argument. > > Test: > ? hotspot_gc_shenandoah (fastdebug and release) on > ? Linux x86_64, x86_32 > ? Windows x86_64. > > Thanks, > > -Zhengyu > > > On 7/25/19 9:01 AM, Zhengyu Gu wrote: >> Further tests discovered some issues with this patch, I would like to >> withdraw this RFR for now. >> >> Thanks, >> >> -Zhengyu >> >> On 7/23/19 3:42 PM, Zhengyu Gu wrote: >>> Shenandoah needs to take slow path when resolving jobject during >>> evacuation phase, to ensure it resolves to-space oops. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8228532 >>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8228532/webrev.00/index.html >>> >>> Test: >>> ?? hotspot_gc_shenandoah (fastdebug and release) on Linux x86_64 and >>> aarch64. >>> >>> >>> Thanks, >>> >>> -Zhengyu From thomas.schatzl at oracle.com Fri Jul 26 10:03:52 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 26 Jul 2019 12:03:52 +0200 Subject: RFR(S): 8228631: Fix inconsistent OopStorage::Block owner usage In-Reply-To: <52C8E9D9-AEC2-47AA-8F9F-8BB4703A5AEE@oracle.com> References: <52C8E9D9-AEC2-47AA-8F9F-8BB4703A5AEE@oracle.com> Message-ID: <9238852623d76c7aaf644093c134d2b8cc75c0e1.camel@oracle.com> Hi, On Thu, 2019-07-25 at 19:41 -0400, Kim Barrett wrote: > Please review this cleanup of the use of the _owner member of > OopStorage blocks. It's now just the intptr_t address of the owning > storage object, and no longer sometimes used as the actual owning > object. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8228631 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8228631/open.00/ > looks good. Thomas From thomas.schatzl at oracle.com Fri Jul 26 10:06:37 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 26 Jul 2019 12:06:37 +0200 Subject: RFR(S): 8226796: Reduce debug level logging for oopstorage+blocks In-Reply-To: <59022DD2-ADB4-482D-9A18-A881555020DA@oracle.com> References: <59022DD2-ADB4-482D-9A18-A881555020DA@oracle.com> Message-ID: <04f62248c03a2b9100d5aa65652126994454ae00.camel@oracle.com> Hi, On Thu, 2019-07-25 at 20:55 -0400, Kim Barrett wrote: > Please review this adjustment to some of the logging levels in the > OopStorage implementation, under the (oopstorage, blocks) tag > sequence. High volume logging statements have been reduced to trace > level, and memory allocation failure has been raised to info. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8226796 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8226796/open.00/ > Note: This webrev is baselined from the JDK-8228631 change that is > also out for review. > > Testing: > mach5 tier1 > Local testing with various logging options and looked at output. > looks good. Note that I did not check whether the log level changes make sense as I have no good grasp of what is important and what not in this area, but the change itself seems good. Thanks, Thomas From per.liden at oracle.com Fri Jul 26 12:27:11 2019 From: per.liden at oracle.com (Per Liden) Date: Fri, 26 Jul 2019 14:27:11 +0200 Subject: RFR: 8227226: Segmented array clearing for ZGC In-Reply-To: <7C95DE35-C9F8-4F11-8D07-9511748128C4@amazon.com> References: <5809FFE3-ED37-429B-9189-49D8FD14D092@amazon.com> <625131FC-8B02-4BEC-80B5-F1757232277A@amazon.com> <7C95DE35-C9F8-4F11-8D07-9511748128C4@amazon.com> Message-ID: Hi Ryan & Erik, I had a look at this and started exploring a slightly different approach. Instead doing segmented clearing in the allocation path, we can have the concurrent GC thread clear pages when they are reclaimed and not do any clearing in the allocation path at all. That would look like this: http://cr.openjdk.java.net/~pliden/8227226/webrev.0-base (I've had to temporarily comment out three lines of assert/debug code to make this work) The relocation set selection phase will now be tasked with some potentially expensive clearing work, so we'll want to make that part parallel also. http://cr.openjdk.java.net/~pliden/8227226/webrev.0-parallel Moving this work from Java threads onto the concurrent GC threads means we will potentially prolong the RelocationSetSelection and Relocation phases. That might be a trade-off worth doing. In return, we get: * Faster array allocations, as there's now less work done in the allocation path. * This benefits all arrays, not just those allocated in large pages. * No need to consider/tune a "chunk size". * I also tend think we'll end up with slightly less complex code, that is a bit easier to reason about. Can be debated of course. This approach might also "survive" longer, because the YC scheme we've been loosely thinking about currently requires newly allocated pages to be cleared anyway. It's of course too early to tell if that requirement will stand in the end, but it's possible anyway. I'll need to do some more testing and benchmarking to make sure there's no regression or bugs here. The commented out debug code also needs to be addressed of course. Comments? Other ideas? cheers, Per On 7/24/19 4:37 PM, Sciampacone, Ryan wrote: > > Somehow I lost the RFR off the front and started a new thread. > > Now that we're both off vacation I'd like to revisit this. Can you take a look? > > ?On 7/8/19, 10:40 AM, "hotspot-gc-dev on behalf of Sciampacone, Ryan" wrote: > > > http://cr.openjdk.java.net/~phh/8227226/webrev.01/ > > This shifts away from abusing the constructor do_zero magic in exchange for virtualizing mem_clear() and specializing for the Z version. It does create a change in mem_clear in that it returns an updated version of mem. It does create change outside of the Z code however it does feel cleaner. > > I didn't make a change to PinAllocating - looking at it, the safety of keeping it constructor / destructor based still seemed appropriate to me. If the objection is to using the sequence numbers to pin (and instead using handles to update) - this to me seems less error prone. I had originally discussed handles with Stefan but the proposal came down to this which looks much cleaner. > > On 7/8/19, 6:36 AM, "hotspot-gc-dev on behalf of Sciampacone, Ryan" wrote: > > > 1) Yes this was a conscious decision. There was discussion on determining the optimal point for breakup but given the existing sizes this seemed sufficient. This doesn't preclude the ability to go down that path if its deemed absolutely necessary. The path for more complex decisions is now available. > 2) Agree > 3) I'm not clear here. Do you mean effectively going direct to ZHeap and bypassing the single function PinAllocating? Agree. Otherwise I'll ask you to be a bit clearer. > 4) Agree > 5) I initially had the exact same reaction but I played around with a few other versions (including breaking up initialization points between header and body to get the desired results) and this ended up looking correct. I'll try mixing in the mem clearer function again (fresh start) to see if it looks any better. > > On 7/8/19, 5:49 AM, "Per Liden" wrote: > > Hi Ryan, > > A few general comments: > > 1) It looks like this still only work for large pages? > 2) The log_info stuff should be removed. > 3) I'm not a huge fan of single-use utilities like PinAllocating, at > least not when, IMO, the alternative is more straight forward and less code. > 4) Please make locals const when possible. > 5) Duplicating _do_zero looks odd. Injecting a "mem clearer", similar to > what Stefans original patch did, seems worth exploring. > > cheers, > /Per > > (Btw, I'm on vacation so I might not be super-responsive to emails) > > On 2019-07-08 12:42, Erik ?sterlund wrote: > > Hi Ryan, > > > > This looks good in general. Just some stylistic things... > > > > 1) In the ZGC project we like the letter 'Z' so much that we put it in > > front of everything we possibly can, including all class names. > > 2) We also explicitly state things are private even though it's > > bleedingly obvious. > > > > So: > > > > 39 class PinAllocating { > > 40 HeapWord* _mem; > > 41 public: -> 39 class ZPinAllocating { 40 private: 41 HeapWord* _mem; > > 42 > > 41 public: I can be your sponsor and push this change for you. I don't > > think there is a need for another webrev for my small stylistic remarks, > > so I can just fix that before pushing this for you. On that note, I'll > > add me and StefanK to the contributed-by section as we all worked out > > the right solution to this problem collaboratively. I have run through > > mach5 tier1-5, and found no issues with this patch. Thanks, /Erik > > > > On 2019-07-05 17:18, Sciampacone, Ryan wrote: > >> http://cr.openjdk.java.net/~phh/8227226/webrev.00/ > >> https://bugs.openjdk.java.net/browse/JDK-8227226 > >> > >> This patch introduces safe point checks into array clearing during > >> allocation for ZGC. The patch isolates the changes to ZGC as (in > >> particular with the more modern collectors) the approach to > >> incrementalizing or respecting safe point checks is going to be > >> different. > >> > >> The approach is to keep the region holding the array in the allocating > >> state (pin logic) while updating the color to the array after checks. > >> > >> Can I get a review? Thanks. > >> > >> Ryan > > > > > > > > From per.liden at oracle.com Fri Jul 26 12:36:15 2019 From: per.liden at oracle.com (Per Liden) Date: Fri, 26 Jul 2019 14:36:15 +0200 Subject: RFR(S): 8228631: Fix inconsistent OopStorage::Block owner usage In-Reply-To: <52C8E9D9-AEC2-47AA-8F9F-8BB4703A5AEE@oracle.com> References: <52C8E9D9-AEC2-47AA-8F9F-8BB4703A5AEE@oracle.com> Message-ID: <2b082554-6804-e286-ee78-9de791fa5d4c@oracle.com> Looks good! /Per On 7/26/19 1:41 AM, Kim Barrett wrote: > Please review this cleanup of the use of the _owner member of > OopStorage blocks. It's now just the intptr_t address of the owning > storage object, and no longer sometimes used as the actual owning > object. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8228631 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8228631/open.00/ > > Testing: > mach5 tier1 > From per.liden at oracle.com Fri Jul 26 12:38:20 2019 From: per.liden at oracle.com (Per Liden) Date: Fri, 26 Jul 2019 14:38:20 +0200 Subject: RFR(S): 8226796: Reduce debug level logging for oopstorage+blocks In-Reply-To: <59022DD2-ADB4-482D-9A18-A881555020DA@oracle.com> References: <59022DD2-ADB4-482D-9A18-A881555020DA@oracle.com> Message-ID: Looks good! /Per On 7/26/19 2:55 AM, Kim Barrett wrote: > Please review this adjustment to some of the logging levels in the > OopStorage implementation, under the (oopstorage, blocks) tag > sequence. High volume logging statements have been reduced to trace > level, and memory allocation failure has been raised to info. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8226796 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8226796/open.00/ > Note: This webrev is baselined from the JDK-8228631 change that is > also out for review. > > Testing: > mach5 tier1 > Local testing with various logging options and looked at output. > From per.liden at oracle.com Fri Jul 26 12:41:32 2019 From: per.liden at oracle.com (Per Liden) Date: Fri, 26 Jul 2019 14:41:32 +0200 Subject: RFR (): JDK-8228503: Rename "rs_lengths" to "rs_length" in ergo code In-Reply-To: References: Message-ID: <32b26945-ac15-0d47-c6e6-379003f1bca5@oracle.com> Looks good! /Per On 7/25/19 3:55 PM, Thomas Schatzl wrote: > Hi, > > can I have reviews for this change that (mechanically) changes all > prediction related identifiers with the "rs_lengths" suffix to > "rs_length"? > > The reason is that all these values are not vectors like the plural-s > suggests (and I think we use this naming style in other places), but > contain a scalar (sum) value. Reading through the prediction code with > this style is really annoying, given that the code later performs > standard mathematical ops with single values on that: > > E.g. > double time_spent = predicted_rs_lengths * predict_cost_per_card(); > > Note that I understand that there is some reasoning for that as these > variables contain the sum of differences, or the sum of remembered set > "lengths" (for which we typically use "num_*_entries"/"num_*_cards" in > other places); however due to way these are used (e.g. to predict the > singular remembered set "length" of the entire young gen) I would > prefer to remove the "s". > > I am okay with keeping the "rs_length" as it is to me by far not as > annoying, and I did so. > > Alternatively I would suggest to add at least "sum" postfix to the > names. > > The change also fixes multiple similar pluralization like > "_inc_recorded_rs_lengths_diffs". > > There is some particular misnomer ("max_rs_length") that is going to be > fixed in the next CR (JDK-8228504). > > CR: > https://bugs.openjdk.java.net/browse/JDK-8228503 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8228503/webrev/ > Testing: > hs-tier1-3 > > Thanks, > Thomas > From per.liden at oracle.com Fri Jul 26 13:11:46 2019 From: per.liden at oracle.com (Per Liden) Date: Fri, 26 Jul 2019 15:11:46 +0200 Subject: RFR: 8228657: ZGC: ZObjectAllocator::used() should take undone allocations into account Message-ID: <1d7717b8-0d4e-a022-31c2-728c0e89bf95@oracle.com> In ZObjectAllocator, when we allocate a page we increment _used. However, we might later undo that page allocation, but we then leave _used as is. This results in a sometimes slightly incorrect value being reported by ZCollectedHeap::tlab_used(), which is fed into the TLAB heuristics. This incorrectness is not that big of a deal, but we should still fix this. We can't safely decrement _used (because we might be executing on a different CPU now), but we can track the amount of undos we've done, and later subtract that from _used. Bug: https://bugs.openjdk.java.net/browse/JDK-8228657 Webrev: http://cr.openjdk.java.net/~pliden/8228657/webrev.0 /Per From fairoz.matte at oracle.com Fri Jul 26 15:02:07 2019 From: fairoz.matte at oracle.com (Fairoz Matte) Date: Fri, 26 Jul 2019 08:02:07 -0700 (PDT) Subject: RFR: 8048556: Unnecessary GCLocker-initiated young GCs In-Reply-To: <9c40acac-e85e-4186-93ee-715d3a909549@default> References: <6a3227ce-3cf0-46a3-9eb7-2fc7c3f9d675@default> <7A07B132-2ED7-4992-90E4-F8E57B731A75@oracle.com> <9c40acac-e85e-4186-93ee-715d3a909549@default> Message-ID: Hi Kim, Below is the updated webrev specific to GenCollectedHeap and ParalleGC http://cr.openjdk.java.net/~fmatte/8048556/webrev.01/ Thanks, Fairoz > -----Original Message----- > From: Fairoz Matte > Sent: Thursday, July 25, 2019 6:03 PM > To: Kim Barrett > Cc: Hotspot-Gc-Dev > Subject: RE: RFR: 8048556: Unnecessary GCLocker-initiated young GCs > > Hi Kim, > > Thanks for looking into this. > What you have suggested is totally different approach. > I will investigate more. > > Thanks, > Fairoz > > > -----Original Message----- > > From: Kim Barrett > > Sent: Thursday, July 25, 2019 5:06 AM > > To: Fairoz Matte > > Cc: Hotspot-Gc-Dev > > Subject: Re: RFR: 8048556: Unnecessary GCLocker-initiated young GCs > > > > > On Jul 24, 2019, at 7:12 PM, Kim Barrett wrote: > > > When G1 has an allocation failure that might require a GC, and it > > > can't satisfy the allocation in some other way (such as by > > > allocating more young regions), then if GCLocker::needs_gc() is true > > > it will stall and wait for the gc-locker gc request to be processed, > > > and then retry the allocation. > > > > Looking at the history of the G1 code, the present behavior of > > deferring to a pending gc-locker gc request was introduced by > > > > 7143858: G1: Back to back young GCs with the second GC having a > > minimally sized eden > > > > Sound familiar? There are even comments in that CR discussing exactly > > the race described by Tony in JDK-8048556. Pity other GCs weren't > > examined for the same problem and similarly modified at the same time. > > From erik.osterlund at oracle.com Fri Jul 26 17:56:57 2019 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Fri, 26 Jul 2019 19:56:57 +0200 Subject: RFR: 8227226: Segmented array clearing for ZGC In-Reply-To: References: <5809FFE3-ED37-429B-9189-49D8FD14D092@amazon.com> <625131FC-8B02-4BEC-80B5-F1757232277A@amazon.com> <7C95DE35-C9F8-4F11-8D07-9511748128C4@amazon.com> Message-ID: <46C0D0B2-5C33-442A-B4B6-2DC2358F75FC@oracle.com> Hi Per, I like the direction of this, as I think it will help us down the road. Note though that if you go down this path, you might want to make sure zero initializing stores are not emitted by the JITs. These places can be found if you search for ZeroTLAB. Otherwise we lose out on some key synergy effects with that approach (clearing lots of memory twice) And if we go down that route, we might want to interface this better between the GC and compiler; find a new property name that the compiler checks for, like BarrierSetC2::zero_initializing_stores() or something, which may or may not always correspond to ZeroTLAB. Might also want to consider NUMA effects when clearing the whole relocation set. But that could be an optional step 2. /Erik > On 26 Jul 2019, at 14:27, Per Liden wrote: > > Hi Ryan & Erik, > > I had a look at this and started exploring a slightly different approach. Instead doing segmented clearing in the allocation path, we can have the concurrent GC thread clear pages when they are reclaimed and not do any clearing in the allocation path at all. > > That would look like this: > > http://cr.openjdk.java.net/~pliden/8227226/webrev.0-base > > (I've had to temporarily comment out three lines of assert/debug code to make this work) > > The relocation set selection phase will now be tasked with some potentially expensive clearing work, so we'll want to make that part parallel also. > > http://cr.openjdk.java.net/~pliden/8227226/webrev.0-parallel > > Moving this work from Java threads onto the concurrent GC threads means we will potentially prolong the RelocationSetSelection and Relocation phases. That might be a trade-off worth doing. In return, we get: > > * Faster array allocations, as there's now less work done in the allocation path. > * This benefits all arrays, not just those allocated in large pages. > * No need to consider/tune a "chunk size". > * I also tend think we'll end up with slightly less complex code, that is a bit easier to reason about. Can be debated of course. > > This approach might also "survive" longer, because the YC scheme we've been loosely thinking about currently requires newly allocated pages to be cleared anyway. It's of course too early to tell if that requirement will stand in the end, but it's possible anyway. > > I'll need to do some more testing and benchmarking to make sure there's no regression or bugs here. The commented out debug code also needs to be addressed of course. > > Comments? Other ideas? > > cheers, > Per > >> On 7/24/19 4:37 PM, Sciampacone, Ryan wrote: >> Somehow I lost the RFR off the front and started a new thread. >> Now that we're both off vacation I'd like to revisit this. Can you take a look? >> ?On 7/8/19, 10:40 AM, "hotspot-gc-dev on behalf of Sciampacone, Ryan" wrote: >> http://cr.openjdk.java.net/~phh/8227226/webrev.01/ >> This shifts away from abusing the constructor do_zero magic in exchange for virtualizing mem_clear() and specializing for the Z version. It does create a change in mem_clear in that it returns an updated version of mem. It does create change outside of the Z code however it does feel cleaner. >> I didn't make a change to PinAllocating - looking at it, the safety of keeping it constructor / destructor based still seemed appropriate to me. If the objection is to using the sequence numbers to pin (and instead using handles to update) - this to me seems less error prone. I had originally discussed handles with Stefan but the proposal came down to this which looks much cleaner. >> On 7/8/19, 6:36 AM, "hotspot-gc-dev on behalf of Sciampacone, Ryan" wrote: >> 1) Yes this was a conscious decision. There was discussion on determining the optimal point for breakup but given the existing sizes this seemed sufficient. This doesn't preclude the ability to go down that path if its deemed absolutely necessary. The path for more complex decisions is now available. >> 2) Agree >> 3) I'm not clear here. Do you mean effectively going direct to ZHeap and bypassing the single function PinAllocating? Agree. Otherwise I'll ask you to be a bit clearer. >> 4) Agree >> 5) I initially had the exact same reaction but I played around with a few other versions (including breaking up initialization points between header and body to get the desired results) and this ended up looking correct. I'll try mixing in the mem clearer function again (fresh start) to see if it looks any better. >> On 7/8/19, 5:49 AM, "Per Liden" wrote: >> Hi Ryan, >> A few general comments: >> 1) It looks like this still only work for large pages? >> 2) The log_info stuff should be removed. >> 3) I'm not a huge fan of single-use utilities like PinAllocating, at >> least not when, IMO, the alternative is more straight forward and less code. >> 4) Please make locals const when possible. >> 5) Duplicating _do_zero looks odd. Injecting a "mem clearer", similar to >> what Stefans original patch did, seems worth exploring. >> cheers, >> /Per >> (Btw, I'm on vacation so I might not be super-responsive to emails) >> On 2019-07-08 12:42, Erik ?sterlund wrote: >> > Hi Ryan, >> > >> > This looks good in general. Just some stylistic things... >> > >> > 1) In the ZGC project we like the letter 'Z' so much that we put it in >> > front of everything we possibly can, including all class names. >> > 2) We also explicitly state things are private even though it's >> > bleedingly obvious. >> > >> > So: >> > >> > 39 class PinAllocating { >> > 40 HeapWord* _mem; >> > 41 public: -> 39 class ZPinAllocating { 40 private: 41 HeapWord* _mem; >> > 42 >> > 41 public: I can be your sponsor and push this change for you. I don't >> > think there is a need for another webrev for my small stylistic remarks, >> > so I can just fix that before pushing this for you. On that note, I'll >> > add me and StefanK to the contributed-by section as we all worked out >> > the right solution to this problem collaboratively. I have run through >> > mach5 tier1-5, and found no issues with this patch. Thanks, /Erik >> > >> > On 2019-07-05 17:18, Sciampacone, Ryan wrote: >> >> http://cr.openjdk.java.net/~phh/8227226/webrev.00/ >> >> https://bugs.openjdk.java.net/browse/JDK-8227226 >> >> >> >> This patch introduces safe point checks into array clearing during >> >> allocation for ZGC. The patch isolates the changes to ZGC as (in >> >> particular with the more modern collectors) the approach to >> >> incrementalizing or respecting safe point checks is going to be >> >> different. >> >> >> >> The approach is to keep the region holding the array in the allocating >> >> state (pin logic) while updating the color to the array after checks. >> >> >> >> Can I get a review? Thanks. >> >> >> >> Ryan >> > >> From aph at redhat.com Fri Jul 26 23:57:10 2019 From: aph at redhat.com (Andrew Haley) Date: Sat, 27 Jul 2019 00:57:10 +0100 Subject: RFR: 8227226: Segmented array clearing for ZGC In-Reply-To: <46C0D0B2-5C33-442A-B4B6-2DC2358F75FC@oracle.com> References: <5809FFE3-ED37-429B-9189-49D8FD14D092@amazon.com> <625131FC-8B02-4BEC-80B5-F1757232277A@amazon.com> <7C95DE35-C9F8-4F11-8D07-9511748128C4@amazon.com> <46C0D0B2-5C33-442A-B4B6-2DC2358F75FC@oracle.com> Message-ID: <8ed46fe6-1b58-54a3-a0bf-746736d7c29e@redhat.com> On 7/26/19 6:56 PM, Erik Osterlund wrote: > I like the direction of [clearing pages when they are reclaimed], as > I think it will help us down the road. Are you sure this will help? If the cache lines for a page are already in exclusive state (i.e. they've been written) when reclaimed this will be fine, otherwise you're generating extra cache coherence traffic. You're also evicting the cache of the GC thread by filling it with zeroed lines. I guess some careful benchmarking will tell us. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From kim.barrett at oracle.com Sat Jul 27 00:09:39 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 26 Jul 2019 20:09:39 -0400 Subject: RFR(S): 8228631: Fix inconsistent OopStorage::Block owner usage In-Reply-To: <9238852623d76c7aaf644093c134d2b8cc75c0e1.camel@oracle.com> References: <52C8E9D9-AEC2-47AA-8F9F-8BB4703A5AEE@oracle.com> <9238852623d76c7aaf644093c134d2b8cc75c0e1.camel@oracle.com> Message-ID: <66ACA8A0-3B33-406C-9A21-D329C2F4852F@oracle.com> > On Jul 26, 2019, at 6:03 AM, Thomas Schatzl wrote: > > Hi, > > On Thu, 2019-07-25 at 19:41 -0400, Kim Barrett wrote: >> Please review this cleanup of the use of the _owner member of >> OopStorage blocks. It's now just the intptr_t address of the owning >> storage object, and no longer sometimes used as the actual owning >> object. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8228631 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8228631/open.00/ >> > > looks good. > > Thomas thanks. From kim.barrett at oracle.com Sat Jul 27 00:09:55 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 26 Jul 2019 20:09:55 -0400 Subject: RFR(S): 8228631: Fix inconsistent OopStorage::Block owner usage In-Reply-To: <2b082554-6804-e286-ee78-9de791fa5d4c@oracle.com> References: <52C8E9D9-AEC2-47AA-8F9F-8BB4703A5AEE@oracle.com> <2b082554-6804-e286-ee78-9de791fa5d4c@oracle.com> Message-ID: > On Jul 26, 2019, at 8:36 AM, Per Liden wrote: > > Looks good! thanks. > > /Per > > On 7/26/19 1:41 AM, Kim Barrett wrote: >> Please review this cleanup of the use of the _owner member of >> OopStorage blocks. It's now just the intptr_t address of the owning >> storage object, and no longer sometimes used as the actual owning >> object. >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8228631 >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8228631/open.00/ >> Testing: >> mach5 tier1 From kim.barrett at oracle.com Sat Jul 27 03:36:12 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 26 Jul 2019 23:36:12 -0400 Subject: RFR(S): 8226796: Reduce debug level logging for oopstorage+blocks In-Reply-To: <04f62248c03a2b9100d5aa65652126994454ae00.camel@oracle.com> References: <59022DD2-ADB4-482D-9A18-A881555020DA@oracle.com> <04f62248c03a2b9100d5aa65652126994454ae00.camel@oracle.com> Message-ID: > On Jul 26, 2019, at 6:06 AM, Thomas Schatzl wrote: > looks good. Note that I did not check whether the log level changes > make sense as I have no good grasp of what is important and what not in > this area, but the change itself seems good. > > Thanks, > Thomas Thanks. I based the log levels on likely frequency, e.g. the things at trace level would generate a lot of output when enabled. From kim.barrett at oracle.com Sat Jul 27 03:36:22 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 26 Jul 2019 23:36:22 -0400 Subject: RFR(S): 8226796: Reduce debug level logging for oopstorage+blocks In-Reply-To: References: <59022DD2-ADB4-482D-9A18-A881555020DA@oracle.com> Message-ID: <23AD8A52-2474-4165-A815-F83300014F4B@oracle.com> > On Jul 26, 2019, at 8:38 AM, Per Liden wrote: > > Looks good! Thanks. > > /Per > > On 7/26/19 2:55 AM, Kim Barrett wrote: >> Please review this adjustment to some of the logging levels in the >> OopStorage implementation, under the (oopstorage, blocks) tag >> sequence. High volume logging statements have been reduced to trace >> level, and memory allocation failure has been raised to info. >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8226796 >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8226796/open.00/ >> Note: This webrev is baselined from the JDK-8228631 change that is >> also out for review. >> Testing: >> mach5 tier1 >> Local testing with various logging options and looked at output. From kim.barrett at oracle.com Sat Jul 27 04:17:47 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 27 Jul 2019 00:17:47 -0400 Subject: RFR: 8048556: Unnecessary GCLocker-initiated young GCs In-Reply-To: References: <6a3227ce-3cf0-46a3-9eb7-2fc7c3f9d675@default> <7A07B132-2ED7-4992-90E4-F8E57B731A75@oracle.com> <9c40acac-e85e-4186-93ee-715d3a909549@default> Message-ID: > On Jul 26, 2019, at 11:02 AM, Fairoz Matte wrote: > > Hi Kim, > > Below is the updated webrev specific to GenCollectedHeap and ParalleGC > http://cr.openjdk.java.net/~fmatte/8048556/webrev.01/ ------------------------------------------------------------------------------ src/hotspot/share/gc/shared/genCollectedHeap.cpp 373 if(!GCLocker::needs_gc()) { 374 gc_count_before = total_collections(); This leaves gc_count_before uninitialized if GCLocker::needs_gc() is true, leading to undefined behavior when it is referenced. It probably passes testing because the uninitialized value that gets used _probably_ differs from the current total_collections() value, leading the VMOpt to report failure, which is what we want in this case. But it's still UB. ------------------------------------------------------------------------------ src/hotspot/share/gc/shared/genCollectedHeap.cpp 373 if(!GCLocker::needs_gc()) { 374 gc_count_before = total_collections(); Assuming the above UB problem is fixed, this thread could just spin a bunch of times, logging a bunch of excessive retry messages, until the GCLocker's GC request finally runs. G1 puts GCLocker::stall_until_clear() on its version of this path. I think something more like what G1 does should be done here too. ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/parallelScavengeHeap.cpp 319 // The GCLocker may not be active but the GCLocker initiated 320 // GC may not yet have been performed (GCLocker::needs_gc() 321 // returns true). In this case we do not try this GC and 322 // wait until the GCLocker initiated GC is performed, and 323 // then retry the allocation. 324 } else if(GCLocker::needs_gc()) { 325 should_try_gc = false; 326 } else { 327 should_try_gc = true; 328 } I think the comment ought to be inside the relevant clause. I also think a better name than should_try_gc should be used; something relevant to GCLocker. ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/parallelScavengeHeap.cpp 319 // The GCLocker may not be active but the GCLocker initiated 320 // GC may not yet have been performed (GCLocker::needs_gc() 321 // returns true). In this case we do not try this GC and 322 // wait until the GCLocker initiated GC is performed, and 323 // then retry the allocation. 324 } else if(GCLocker::needs_gc()) { 325 should_try_gc = false; 326 } else { 327 should_try_gc = true; 328 } [Similar to the above mentioned problem in the genCollectedHeap change.] If GCLocker::needs_gc() then this thread could just spin a bunch of times, logging a bunch of excessive retry messages, until the GCLocker's GC request finally runs. G1 puts GCLocker::stall_until_clear() on its version of this path. I think something more like what G1 does should be done here too. ------------------------------------------------------------------------------ From thomas.schatzl at oracle.com Sat Jul 27 08:05:34 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Sat, 27 Jul 2019 01:05:34 -0700 (PDT) Subject: RFR (): JDK-8228503: Rename "rs_lengths" to "rs_length" in ergo code Message-ID: Hi Kim, Per, thanks for your reviews. Thomas ----- Original Message ----- From: per.liden at oracle.com To: thomas.schatzl at oracle.com, hotspot-gc-dev at openjdk.java.net Sent: Friday, July 26, 2019 2:41:34 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: Re: RFR (): JDK-8228503: Rename "rs_lengths" to "rs_length" in ergo code Looks good! /Per On 7/25/19 3:55 PM, Thomas Schatzl wrote: > Hi, > > can I have reviews for this change that (mechanically) changes all > prediction related identifiers with the "rs_lengths" suffix to > "rs_length"? [...] From thomas.schatzl at oracle.com Sun Jul 28 17:22:50 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Sun, 28 Jul 2019 10:22:50 -0700 (PDT) Subject: RFR: 8224660: Parallel GC: Use WorkGang (2: MarksFromRootsTask) Message-ID: <25ea3609-09ae-4601-95c9-f7a6a2d50c6c@default> Hi Leo, http://cr.openjdk.java.net/~lkorinth/workgang/2/_8224660-Parallel-GC-Use-WorkGang-2-MarksFromRootsTask-fixup-2/ looks good. Thanks for the renaming. Thomas ----- Original Message ----- From: leo.korinth at oracle.com To: thomas.schatzl at oracle.com, kim.barrett at oracle.com Cc: hotspot-gc-dev at openjdk.java.net Sent: Wednesday, July 3, 2019 2:49:12 AM GMT -08:00 US/Canada Pacific Subject: Re: RFR: 8224660: Parallel GC: Use WorkGang (2: MarksFromRootsTask) On 03/07/2019 11:35, Thomas Schatzl wrote: > Hi, > > On Tue, 2019-07-02 at 17:39 -0400, Kim Barrett wrote: >> Leo and I discussed my issues with the task claiming in the first >> version. We agreed to use SequentialSubTasksDone in this WorkGang >> conversion project, rather than gating that project on fixing various >> issues we have with the existing subtask claiming mechanisms. We'll >> just squint and ignore those issues for this project, and file some >> RFEs. >> >> > http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224660-Parallel-GC-Use-WorkGang-2-MarksFromRootsTask/ >> > http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224660-Parallel-GC-Use-WorkGang-2-MarksFromRootsTask-fixup-1/ >> >> These look good. >> >> Just one minor naming change requested: >> >> ------------------------------------------------------------------- >> ----------- >> src/hotspot/share/gc/parallel/psParallelCompact.cpp >> 2114 static void mark_from_roots_task(ParallelRootType::Value >> root_type, uint which) { >> >> Instead of "which", please stay with "worker_id" or "worker_i", as >> used in the caller and more or less everywhere else. >> > > Please use "worker_id". Almost everything is using "worker_id" by now, > so let's avoid "worker_i" creeping back in. > > Looks good. No need to re-review for this renaming. > > Thanks, > Thomas > I will fix to using worker_id in all places. My reason for using "which" was not to change the old code, but I agree that it ought to be fixed. Thanks, Leo From thomas.schatzl at oracle.com Sun Jul 28 17:29:55 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Sun, 28 Jul 2019 10:29:55 -0700 (PDT) Subject: RFR: 8224661: Parallel GC: Use WorkGang (3: UpdateDensePrefixAndCompactionTask) Message-ID: <0a0a1c53-a879-492c-811f-4bd7ffe92d67@default> Hi, http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224661-Parallel-GC-Use-WorkGang-3-UpdateDensePrefixAndCompactionTask-fixup-2 looks good. Thanks, Thomas ----- Original Message ----- From: leo.korinth at oracle.com To: kim.barrett at oracle.com Cc: hotspot-gc-dev at openjdk.java.net Sent: Wednesday, July 3, 2019 4:47:21 AM GMT -08:00 US/Canada Pacific Subject: Re: RFR: 8224661: Parallel GC: Use WorkGang (3: UpdateDensePrefixAndCompactionTask) On 03/07/2019 01:55, Kim Barrett wrote: >> On May 27, 2019, at 4:21 AM, Leo Korinth wrote: >> [?] >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8224661 >> >> Webrev: >> http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224661-Parallel-GC-Use-WorkGang-3-UpdateDensePrefixAndCompactionTask/ >> http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ >> >> Testing (on the whole patch series): >> mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC >> gc test suite >> >> Thanks, >> Leo > > http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224661-Parallel-GC-Use-WorkGang-3-UpdateDensePrefixAndCompactionTask/ > http://cr.openjdk.java.net/~lkorinth/workgang/1/_8224661-Parallel-GC-Use-WorkGang-3-UpdateDensePrefixAndCompactionTask-fixup-1/ > > Looks good. > > A few minor comments. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psParallelCompact.hpp > 33 #include "gc/shared/taskqueue.hpp" > > I don't see any changes to this file that need this #include. > Presumably it's needed by some .cpp files, but they should be doing > the #include. Old artifact from older version where I used the shared task queue, thanks for finding this, I will remove it as it is not needed. > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psParallelCompact.cpp > 2467 guarantee(_backing_array != NULL, "alloc failure"); > > Not needed; NEW_C_HEAP_ARRAY uses EXIT_OOM for the allocation failure > mode. Good catch, copied from other code that did it, will fix. I have (also) created https://bugs.openjdk.java.net/browse/JDK-8227168 to remove the source from where I got it. And fix some related problems. > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psParallelCompact.cpp > 2618 static void compaction_with_stealing_task(ParallelTaskTerminator* terminator, uint which) { > > s/which/worker_id/ will fix. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psParallelCompact.cpp > 2684 TaskQueue task_queue(last_space_id*(active_gc_threads * PAR_OLD_DENSE_PREFIX_OVER_PARTITIONING + 1)); > > Some breadcrumbs to explain that calculation would be really helpful. > Maybe comments, or splitting up the calculation with helpful variable > names, or something. will add some comment. > > Anyway, I think it's correct, but harder to verify than it could be. > > ------------------------------------------------------------------------------ > Thanks, Leo From thomas.schatzl at oracle.com Sun Jul 28 17:43:28 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Sun, 28 Jul 2019 10:43:28 -0700 Subject: RFR: 8224662: Parallel GC: Use WorkGang (4: SharedRestorePreservedMarksTaskExecutor) In-Reply-To: References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <226f085b-6767-5ca8-a32e-76c0eea2de63@oracle.com> Message-ID: Hi, On 02.07.19 18:06, Kim Barrett wrote: >> On May 27, 2019, at 4:56 AM, Leo Korinth wrote: >> [?] >> >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8224662 >> >> Webrev: >> http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224662-Parallel-GC-Use-WorkGang-4-SharedRestorePreservedMarksTaskExecutor/ >> http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ >> >> Testing (on the whole patch series): >> mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC >> gc test suite >> >> Thanks, >> Leo > > Looks good. > looks good. Thanks, Thomas From thomas.schatzl at oracle.com Sun Jul 28 17:47:55 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Sun, 28 Jul 2019 10:47:55 -0700 Subject: RFR: 8224663: Parallel GC: Use WorkGang (5: ScavengeRootsTask) In-Reply-To: <8CA8F38A-1B9F-4B7D-902E-846BF7718D2C@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <3ea47da5-4b1e-0301-461a-ede9047cb638@oracle.com> <8CA8F38A-1B9F-4B7D-902E-846BF7718D2C@oracle.com> Message-ID: <4d7cfde5-c0bb-fe90-7df2-88407b26a0d8@oracle.com> Hi, On 19.07.19 19:12, Kim Barrett wrote: >> On May 27, 2019, at 12:30 PM, Leo Korinth wrote: >> >> Hi, >> >> Here is the fifth patch in a proposed patch series of eight that >> removes gcTaskManager and uses the WorkGang abstraction instead. >> >> ScavengeRootsTask, ThreadRootsTask and OldToYoungRootsTask is replaced >> with ScavengeRootsTask. Code is basically the same, the major >> difference is that roots are visited using EnumClaimer and >> Threads::possibly_parallel_threads_do. Here we can reuse the RootType >> and EnumClaimer from patch number two. >> >> The reason "case threads:" was removed is that the code is dead. That >> part is confusing as the code is done (in parallel from the calling >> function). >> >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8224663 >> >> Webrev: >> http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224663-Parallel-GC-Use-WorkGang-5-ScavengeRootsTask/ >> http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ >> >> Testing (on the whole patch series): >> mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC >> gc test suite >> >> Thanks, >> Leo > > 8224663-Parallel-GC-Use-WorkGang-5-ScavengeRootsTask > 8224663-Parallel-GC-Use-WorkGang-5-ScavengeRootsTask-fixup-1 > 8224663-Parallel-GC-Use-WorkGang-5-ScavengeRootsTask-fixup-2 looks good sans Kim's comments. Thanks, Thomas From thomas.schatzl at oracle.com Sun Jul 28 17:54:28 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Sun, 28 Jul 2019 10:54:28 -0700 Subject: RFR: 8224664: Parallel GC: Use WorkGang (6: PSRefProcTaskProxy) In-Reply-To: References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> Message-ID: <40c31548-5e28-9f03-0bff-5afabd140f74@oracle.com> Hi, On 27.05.19 10:34, Leo Korinth wrote: > Hi, > > Here is the sixth patch in a proposed patch series of eight that > removes gcTaskManager and uses the WorkGang abstraction instead. > > Here, the new PSRefProcTask is composed of PSRefProcTaskProxy and the > old StealTask (that was partially moved before). > > Now both psTasks.* and pcTasks.* are removed. > > Enhancement: > ? https://bugs.openjdk.java.net/browse/JDK-8224664 > > Webrev: > > http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224664-Parallel-GC-Use-WorkGang-6-PSRefProcTaskProxy/ > > ? http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ > > Testing (on the whole patch series): > ? mach5 remote-build-and-test --build-profiles > linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test > open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC > ? gc test suite http://cr.openjdk.java.net/~lkorinth/workgang/2/_8224664-Parallel-GC-Use-WorkGang-6-PSRefProcTaskProxy Potentially psCardTable.cpp could get a copyright update, it's up to you. No need for an extra webrev for this change. Thanks, Thomas From thomas.schatzl at oracle.com Sun Jul 28 17:55:59 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Sun, 28 Jul 2019 10:55:59 -0700 Subject: RFR: 8224665: Parallel GC: Use WorkGang (7: remove task manager) In-Reply-To: <75E8E7E0-DDE4-44EC-B561-3AC1DCCE7C86@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <5bc88bed-7c2c-b4e7-fef2-2d457d7ba4f1@oracle.com> <75E8E7E0-DDE4-44EC-B561-3AC1DCCE7C86@oracle.com> Message-ID: <3a695282-8b5a-54b0-8f52-e3509ae572c2@oracle.com> Hi, On 20.07.19 04:43, Kim Barrett wrote: >> On May 27, 2019, at 1:44 PM, Leo Korinth wrote: >> >> Hi, >> >> Here is the seventh patch in a proposed patch series of eight that >> removes gcTaskManager and uses the WorkGang abstraction instead. >> >> We try to remove everything task manager and task thread related. >> >> Some utility methods (gc_threads_do, print_gc_threads_on) are changed >> to use the WorkGang versions. Some worker counts are fetched from >> WorkGang as well. Most of the change is just code deletion. >> >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8224665 >> >> Webrev: >> http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224665-Parallel-GC-Use-WorkGang-7-remove-task-manager/ >> http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ >> >> Testing (on the whole patch series): >> mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC >> gc test suite >> >> Thanks, >> Leo > > Looks good. > looks good. Thomas From thomas.schatzl at oracle.com Sun Jul 28 18:19:39 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Sun, 28 Jul 2019 11:19:39 -0700 Subject: RFR: 8224666: Parallel GC: Use WorkGang (8: obsolete and remove flags) In-Reply-To: References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> Message-ID: <86684073-36cc-5c86-7f0c-313c1bb129d8@oracle.com> Hi, On 20.07.19 04:55, Kim Barrett wrote: >> On May 29, 2019, at 8:55 AM, Leo Korinth wrote: >> >> Hi, >> >> Here is the eighth and last patch that removes gcTaskManager and uses >> the WorkGang abstraction instead. [...] >> >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8224666 >> >> Webrev: >> http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224666-Parallel-GC-Use-WorkGang-8-obsolete-and-remove-flags/ >> http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ >> >> Testing (on the whole patch series): >> mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC >> gc test suite >> >> Thanks, >> Leo > > 8224666-Parallel-GC-Use-WorkGang-8-obsolete-and-remove-flags > 8224666-Parallel-GC-Use-WorkGang-8-obsolete-and-remove-flags-fixup-1 > > Looks good. > > I see there is a draft CSR for removal of these product flags > (JDK-8224668). I think it needs to be pushed along in the state > machine so it can be reviewed. I've reviewed the current text and it > looks okay to me. > I udpated the CSR a little too and signed it. Not sure if we should keep the name since it is not descriptive enough. We should ask if it is possible to have different names for CSR and bugfix. Thanks, Thomas From erik.osterlund at oracle.com Mon Jul 29 12:28:33 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 29 Jul 2019 14:28:33 +0200 Subject: RFR: 8228657: ZGC: ZObjectAllocator::used() should take undone allocations into account In-Reply-To: <1d7717b8-0d4e-a022-31c2-728c0e89bf95@oracle.com> References: <1d7717b8-0d4e-a022-31c2-728c0e89bf95@oracle.com> Message-ID: Hi Per, Looks good. Thanks, /Erik On 2019-07-26 15:11, Per Liden wrote: > In ZObjectAllocator, when we allocate a page we increment _used. > However, we might later undo that page allocation, but we then leave > _used as is. This results in a sometimes slightly incorrect value > being reported by ZCollectedHeap::tlab_used(), which is fed into the > TLAB heuristics. This incorrectness is not that big of a deal, but we > should still fix this. > > We can't safely decrement _used (because we might be executing on a > different CPU now), but we can track the amount of undos we've done, > and later subtract that from _used. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8228657 > Webrev: http://cr.openjdk.java.net/~pliden/8228657/webrev.0 > > /Per From tprintezis at twitter.com Mon Jul 29 14:41:09 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Mon, 29 Jul 2019 07:41:09 -0700 Subject: RFR(S): 8227225: ParallelGC: add subspace transitions for young gen for gc+heap=info log lines In-Reply-To: References: <877bd17cff3be20bf8acd95d9eddbb5c9cfb7cf5.camel@oracle.com> Message-ID: Hi Thomas, Latest webrev here: http://cr.openjdk.java.net/~tonyp/8227225/webrev.1/ Main change: I renamed the PreGCValues class to PreGenGCValues so that it?s clear it?s mainly for generational GCs. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On July 25, 2019 at 7:02:55 PM, Tony Printezis (tprintezis at twitter.com) wrote: Thomas, We *think* there is an edge case with ParallelGC that might leave both survivors non-empty after a compaction (but we haven?t convinced ourselves 100% that this is the case). We?re also pretty sure that this cannot happen in CMS (and also Serial). So maybe I can skip the To-Space for now. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On July 24, 2019 at 3:07:35 AM, Thomas Schatzl (thomas.schatzl at oracle.com) wrote: Hi again, On Fri, 2019-07-19 at 10:40 -0700, Tony Printezis wrote: > Hey Thomas, > > Thanks for taking a look and again apologies for the late reply, I > was on vacation until earlier this week. Please see inline. > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > On July 10, 2019 at 11:19:37 AM, Thomas Schatzl ( > thomas.schatzl at oracle.com) wrote: > > On Fri, 2019-07-05 at 11:06 -0700, Tony Printezis wrote: > > > Hi all, > > > > > > This is a follow-up to 8223575 and adds subspace info for eden > > > and from space to the gc+heap=info output for ParallelGC. I also > > > added the before capacity, similar to 8223575, as it can change > > > during the GC (see, in fact, example below). > > > > > > Before: > > > > > > [11.123s][info][gc,heap ] GC(17) PSYoungGen: 4759616K- > > > >1664K(4545536K) > > > [11.123s][info][gc,heap ] GC(17) ParOldGen: 1280K- > > > >1280K(1376256K) > > > > > > After: > > > > > > [12.133s][info][gc,heap ] GC(25) PSYoungGen: > > > 2065952K(2066432K)->1536K(1983488K) Eden: 2064384K(2064384K)- > > > >0K(1981952K) > > > From: 1568K(2048K)->1536K(1536K) > > > > Note that I think this is confusing, or the "From" label should be > > renamed. The second part of this heap transition actually prints > > the "To" space from what I understand. > > That?s not entirely true, though. The spaces do switch roles at the > end of the GC, so the To space does become the From space of the > following GC. So using From here is not entirely incorrect (but I do > get your point). Wait - on evac failure the generational collectors just do not switch from and to-space, don't they? I can't think of any other situation where To-space contains something for these collectors. Then indeed printing To-space is useless as it is always empty. I can't think of a situation where this would be the case. Thanks, Thomas From hohensee at amazon.com Mon Jul 29 15:07:11 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 29 Jul 2019 15:07:11 +0000 Subject: 8227226: Segmented array clearing for ZGC Message-ID: <96BD470F-7152-42C6-A642-430A12C27273@amazon.com> +1. Pretty much all the time the GC thread doesn't own the lines, so getting them into its cache(s) can produce coherence traffic, though that's usually through the LLC because there aren't many multi-socket systems in use. The main effect we've always seen when we turn on ZeroTLAB is LLC eviction of lines being actively used by other threads. Later when an app thread wants the lines, more coherence traffic might happen. ZeroTLAB hasn't been removed because every once in awhile someone enables it to see if current hardware has done something new to fix the problem. C2 goes to some trouble to eliminate zeroing by noticing when zero-stores are dead because of initializing stores in a constructor. For scalar and reference fields the zero-store usually goes away, but arrays are more difficult because their content is usually built up during app execution rather than in the constructor. Parenthetically, on the allocation side we'd love to have an machine instruction that allocates a cache line and zeros it rather than go to memory for it. Such an instruction could replace the current allocation prefetch scheme and significantly reduce off-chip memory bandwidth utilization. Thanks, Paul ?On 7/26/19, 4:58 PM, "hotspot-gc-dev on behalf of Andrew Haley" wrote: On 7/26/19 6:56 PM, Erik Osterlund wrote: > I like the direction of [clearing pages when they are reclaimed], as > I think it will help us down the road. Are you sure this will help? If the cache lines for a page are already in exclusive state (i.e. they've been written) when reclaimed this will be fine, otherwise you're generating extra cache coherence traffic. You're also evicting the cache of the GC thread by filling it with zeroed lines. I guess some careful benchmarking will tell us. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From shade at redhat.com Mon Jul 29 20:20:13 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 29 Jul 2019 22:20:13 +0200 Subject: RFR (S) 8228672: [TESTBUG] gc/metaspace/TestSizeTransitions.java fails on 32-bit platforms Message-ID: Bug: https://bugs.openjdk.java.net/browse/JDK-8228672 Fix: http://cr.openjdk.java.net/~shade/8228672/webrev.02/ Instead of duplicating @run sections even more, decided to test for Platform.is64bit, like other tests in vicinity do. Testing: affected test {x86_64, x86_32}, jdk-submit (running) -- Thanks, -Aleksey From rkennke at redhat.com Tue Jul 30 13:14:50 2019 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 30 Jul 2019 15:14:50 +0200 Subject: RFR: JDK-8228775: Shenandoah: Remove useless null-input-verification in Shenandoah/C2 verifier Message-ID: <721cf09e-8499-fb37-6a31-a94ff884ef9f@redhat.com> Post-LRB it doesn't make a difference whether or not inputs to intrinsics have non-null-types or not because no barriers are feeding directly into intrinsics. We can remove the whole check. In sh/jdk11, nightlies are failing because of that, in jdk14 this is only a bunch of useless code. Bug: https://bugs.openjdk.java.net/browse/JDK-8228775 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8228775/webrev.00/ Testing: hotspot_gc_shenandoah Good? Roman From per.liden at oracle.com Tue Jul 30 14:38:19 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 30 Jul 2019 16:38:19 +0200 Subject: RFR: 8228657: ZGC: ZObjectAllocator::used() should take undone allocations into account In-Reply-To: References: <1d7717b8-0d4e-a022-31c2-728c0e89bf95@oracle.com> Message-ID: Thanks Erik! /Per On 7/29/19 2:28 PM, Erik ?sterlund wrote: > Hi Per, > > Looks good. > > Thanks, > /Erik > > On 2019-07-26 15:11, Per Liden wrote: >> In ZObjectAllocator, when we allocate a page we increment _used. >> However, we might later undo that page allocation, but we then leave >> _used as is. This results in a sometimes slightly incorrect value >> being reported by ZCollectedHeap::tlab_used(), which is fed into the >> TLAB heuristics. This incorrectness is not that big of a deal, but we >> should still fix this. >> >> We can't safely decrement _used (because we might be executing on a >> different CPU now), but we can track the amount of undos we've done, >> and later subtract that from _used. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8228657 >> Webrev: http://cr.openjdk.java.net/~pliden/8228657/webrev.0 >> >> /Per > From zgu at redhat.com Tue Jul 30 14:56:50 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 30 Jul 2019 10:56:50 -0400 Subject: RFR(S) 228777: Shenandoah: Cleanup STW weak root processing Message-ID: Shenandoah pre-evacuates or updates weak roots at STW pauses. Therefore, weak roots should not have forwarded references during mark phase. Current has_forwarded_objects() branch does not break things, but does unncessary works, let's prune it. Bug: https://bugs.openjdk.java.net/browse/JDK-8228777 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8228777/webrev.00/ Test: hotspot_gc_shenandoah (fastdebug and release) Thanks, -Zhengyu From shade at redhat.com Tue Jul 30 16:20:52 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 30 Jul 2019 18:20:52 +0200 Subject: RFR (S) 8228672: [TESTBUG] gc/metaspace/TestSizeTransitions.java fails on 32-bit platforms In-Reply-To: References: Message-ID: <549fbf47-cacf-97ee-9bba-3e66dca1af00@redhat.com> On 7/29/19 10:20 PM, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8228672 > > Fix: > http://cr.openjdk.java.net/~shade/8228672/webrev.02/ Update: http://cr.openjdk.java.net/~shade/8228672/webrev.03/ Tony suggested offline to cut the test short when compressed oops are not available, instead of doing essentially the same test twice. Still passes x86_32, x86_64 tests. -- Thanks, -Aleksey From tprintezis at twitter.com Tue Jul 30 16:36:14 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Tue, 30 Jul 2019 16:36:14 +0000 Subject: RFR (S) 8228672: [TESTBUG] gc/metaspace/TestSizeTransitions.java fails on 32-bit platforms In-Reply-To: References: Message-ID: (trying again as I didn?t do Reply-All in the first attempt) Thanks for looking into this. Looks good. One suggestion: on a 32-bit platform when args[0] == true, maybe make the test a nop? As is, you?ll basically run the same test twice on 32-bit platforms. And webrev.3 looks good. Only suggestion is to print ?skipping test? or something like that when you return. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On July 29, 2019 at 4:31:00 PM, Aleksey Shipilev (shade at redhat.com) wrote: Thanks for looking into this. Looks good. One suggestion: on a 32-bit platform when args[0] == true, maybe make the test a nop? As is, you?ll basically run the same test twice on 32-bit platforms. Tony From rkennke at redhat.com Tue Jul 30 16:50:58 2019 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 30 Jul 2019 18:50:58 +0200 Subject: RFR(S) 228777: Shenandoah: Cleanup STW weak root processing In-Reply-To: References: Message-ID: I am not sure about this. If final-mark could not finish to pre-evacuate all roots we might have from-space refs there. Or does the OOM-protocol ensure that any ref should be updated correctly? We need to be careful here. Better safe than sorry. Roman > Shenandoah pre-evacuates or updates weak roots at STW pauses. Therefore, > weak roots should not have forwarded references during mark phase. > > Current has_forwarded_objects() branch does not break things, but does > unncessary works, let's prune it. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8228777 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8228777/webrev.00/ > > Test: > ? hotspot_gc_shenandoah (fastdebug and release) > > Thanks, > > -Zhengyu From shade at redhat.com Tue Jul 30 16:51:50 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 30 Jul 2019 18:51:50 +0200 Subject: RFR (S) 8228672: [TESTBUG] gc/metaspace/TestSizeTransitions.java fails on 32-bit platforms In-Reply-To: References: Message-ID: <059d1931-f384-3bc3-755d-f340f1a23e5d@redhat.com> On 7/30/19 6:36 PM, Tony Printezis wrote: > Looks good. One suggestion: on a 32-bit platform when args[0] == true, maybe make the test a nop? As > is, you?ll basically run the same test twice on 32-bit platforms. > > And webrev.3 looks good. Only suggestion is to print ?skipping test? or something like that when you > return. Right. Updated webrev.03 in place: http://cr.openjdk.java.net/~shade/8228672/webrev.03/ -- Thanks, -Aleksey From tprintezis at twitter.com Tue Jul 30 16:52:22 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Tue, 30 Jul 2019 16:52:22 +0000 Subject: RFR (S) 8228672: [TESTBUG] gc/metaspace/TestSizeTransitions.java fails on 32-bit platforms In-Reply-To: <059d1931-f384-3bc3-755d-f340f1a23e5d@redhat.com> References: <059d1931-f384-3bc3-755d-f340f1a23e5d@redhat.com> Message-ID: Looks good to me, thanks! ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On July 30, 2019 at 12:51:52 PM, Aleksey Shipilev (shade at redhat.com) wrote: On 7/30/19 6:36 PM, Tony Printezis wrote: > Looks good. One suggestion: on a 32-bit platform when args[0] == true, maybe make the test a nop? As > is, you?ll basically run the same test twice on 32-bit platforms. > > And webrev.3 looks good. Only suggestion is to print ?skipping test? or something like that when you > return. Right. Updated webrev.03 in place: http://cr.openjdk.java.net/~shade/8228672/webrev.03/ -- Thanks, -Aleksey From zgu at redhat.com Tue Jul 30 17:26:47 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 30 Jul 2019 13:26:47 -0400 Subject: RFR(S) 228777: Shenandoah: Cleanup STW weak root processing In-Reply-To: References: Message-ID: On 7/30/19 12:50 PM, Roman Kennke wrote: > I am not sure about this. If final-mark could not finish to pre-evacuate > all roots we might have from-space refs there. Or does the OOM-protocol > ensure that any ref should be updated correctly? We need to be careful > here. Better safe than sorry. If pre-evacuation fails, it runs degenerated GC cycle and roots are updated in final_updaterefs(). We do have no-forwarded assertion in place, should catch the problem if otherwise. Thanks, -Zhengyu > > Roman > > >> Shenandoah pre-evacuates or updates weak roots at STW pauses. Therefore, >> weak roots should not have forwarded references during mark phase. >> >> Current has_forwarded_objects() branch does not break things, but does >> unncessary works, let's prune it. >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8228777 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8228777/webrev.00/ >> >> Test: >> ? hotspot_gc_shenandoah (fastdebug and release) >> >> Thanks, >> >> -Zhengyu > From kim.barrett at oracle.com Tue Jul 30 17:59:19 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 30 Jul 2019 13:59:19 -0400 Subject: RFR: 8048556: Unnecessary GCLocker-initiated young GCs In-Reply-To: References: <6a3227ce-3cf0-46a3-9eb7-2fc7c3f9d675@default> <7A07B132-2ED7-4992-90E4-F8E57B731A75@oracle.com> <9c40acac-e85e-4186-93ee-715d3a909549@default> Message-ID: <4DBC652F-3559-4CCF-A3CF-86D6375B7844@oracle.com> RFR: 8048556: Unnecessary GCLocker-initiated young GCs [Fairoz and I have agreed that I should take over this bug, in order to take things in a somewhat different direction.] Please review this change to the handling of GCLocker collections, to prevent unnecessary back-to-back GCs with the second having a nearly empty younggen to work with. There are two parts to this change. Part 1: When unlocking the last JNI critical section, we record the current number of collections for use when the associated _gc_locker collection is considered for VMOp enqueuing. Each of the STW collectors handles a _gc_locker cause specially, checking that the current number of collections is the same as when the JNI critical section ended. This allows us to discard the _gc_locker request if there has been some other GC since the critical section ended, eliminating a race that could otherwise result in the _gc_locker collection immediately following some other collection. If that test passes, it's still possible for the _gc_locker VMOp to be discarded because of another GC VMOp being ahead of it; that's as intended. This change is addressing the first problem discussed in Tony's analysis, the race after the GCLocker becomes inactive. This change is sufficient to eliminate unnecessary _gc_locker collections immediately following another collection for G1. It reduces, but does not eliminate the problem for the other STW collectors. Note that the (G1-specific) fix for JDK-7143858 is probably wrong. The GCLocker stalling should probably only be done if GCLocker::is_active_and_needs_gc() (like the other STW collectors), rather than just GCLocker::needs_gc(). What was done there is, I think, sort of an incomplete version of one of Tony's "Fix 1", and has some similar problems, as well as being insufficient on its own, and no longer needed with the changes here. Reconsidering JDK-7143858 can be addressed later. Part 2: Fix confusion over full/nonfull collections in the non-G1 STW collectors. (G1 doesn't have this problem.) ParallelGC and the GCH collectors were initializing VM_GC_Operation::_full to true, always, and then have some kludgy workaround attempts for some GCCauses. Instead, _full should be set appropriately for the cause (making skip_operation work properly), and used appropriately elsewhere. There remains some confusion about "full" collections around GTH::do_collection, but it doesn't seem to lead to unnecessary _gc_locker collections or other problems. That can be addressed later. This change addresses the second problem discussed in Tony's analysis, the treatment of _gc_locker collections as sort of full and sort of not. This change reduced but did not eliminate unnecessary _gc_locker collections for ParallelGC and GCH collectors. However, used in conjunction with the Part 1 changes, they no longer seem to occur. Running the attached stress test for 10 minutes produced none, where the baseline without these changes might produce several hundred per minute for the GCH collectors. There are no changes to ZGC needed here; ZGC handles _gc_locker in an entirely different fashion that doesn't have the described problem. There are no changes to Shenandoah here; Shenandoah doesn't use the GCLocker mechanism, instead using object pinning. There are no changes to Epsilon here; Epsilon just ignores _gc_locker GC requests. The attached stress test is based on JNICriticalStressTest2.java from the bug report, converted into a jtreg test. The functional test is run as a subprocess inheriting framework supplied options, with the associated GC log checked for GCLocker collections with small from space sizes; the test fails if there are any. The test relies on GC option rotation by the testing framework to provide coverage of the various collectors. CR: https://bugs.openjdk.java.net/browse/JDK-8048556 Webrev: http://cr.openjdk.java.net/~kbarrett/8048556/open.00/ Testing: mach5 tier1-5 locally ran new test for 30 minutes with each STW collector. From shade at redhat.com Tue Jul 30 19:30:03 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 30 Jul 2019 21:30:03 +0200 Subject: RFR: JDK-8228775: Shenandoah: Remove useless null-input-verification in Shenandoah/C2 verifier In-Reply-To: <721cf09e-8499-fb37-6a31-a94ff884ef9f@redhat.com> References: <721cf09e-8499-fb37-6a31-a94ff884ef9f@redhat.com> Message-ID: <2affbf58-373d-7504-5d01-c135aa8d32e5@redhat.com> On 7/30/19 3:14 PM, Roman Kennke wrote: > Post-LRB it doesn't make a difference whether or not inputs to > intrinsics have non-null-types or not because no barriers are feeding > directly into intrinsics. We can remove the whole check. > > In sh/jdk11, nightlies are failing because of that, in jdk14 this is > only a bunch of useless code. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8228775 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8228775/webrev.00/ I do not mind removing this. Looks good. -- Thanks, -Aleksey From zgu at redhat.com Tue Jul 30 20:31:58 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 30 Jul 2019 16:31:58 -0400 Subject: RFR(S) 228777: Shenandoah: Cleanup STW weak root processing In-Reply-To: References: Message-ID: On 7/30/19 1:26 PM, Zhengyu Gu wrote: > > > On 7/30/19 12:50 PM, Roman Kennke wrote: >> I am not sure about this. If final-mark could not finish to pre-evacuate >> all roots we might have from-space refs there. Or does the OOM-protocol >> ensure that any ref should be updated correctly? We need to be careful >> here. Better safe than sorry. Actually, you are right. It does not work with traversal. I wonder how the tests passed early. I would like to withdraw this RFR. Thanks, -Zhengyu > > If pre-evacuation fails, it runs degenerated GC cycle and roots are > updated in final_updaterefs(). > > We do have no-forwarded assertion in place, should catch the problem if > otherwise. > > Thanks, > > -Zhengyu > >> >> Roman >> >> >>> Shenandoah pre-evacuates or updates weak roots at STW pauses. Therefore, >>> weak roots should not have forwarded references during mark phase. >>> >>> Current has_forwarded_objects() branch does not break things, but does >>> unncessary works, let's prune it. >>> >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8228777 >>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8228777/webrev.00/ >>> >>> Test: >>> ?? hotspot_gc_shenandoah (fastdebug and release) >>> >>> Thanks, >>> >>> -Zhengyu >> From per.liden at oracle.com Wed Jul 31 09:44:01 2019 From: per.liden at oracle.com (Per Liden) Date: Wed, 31 Jul 2019 11:44:01 +0200 Subject: RFR: 8048556: Unnecessary GCLocker-initiated young GCs In-Reply-To: <4DBC652F-3559-4CCF-A3CF-86D6375B7844@oracle.com> References: <6a3227ce-3cf0-46a3-9eb7-2fc7c3f9d675@default> <7A07B132-2ED7-4992-90E4-F8E57B731A75@oracle.com> <9c40acac-e85e-4186-93ee-715d3a909549@default> <4DBC652F-3559-4CCF-A3CF-86D6375B7844@oracle.com> Message-ID: <8df28fbd-bbfb-1988-e057-0af9cac606d2@oracle.com> Hi Kim, Change looks good. /Per On 7/30/19 7:59 PM, Kim Barrett wrote: > RFR: 8048556: Unnecessary GCLocker-initiated young GCs > > [Fairoz and I have agreed that I should take over this bug, in order > to take things in a somewhat different direction.] > > Please review this change to the handling of GCLocker collections, to > prevent unnecessary back-to-back GCs with the second having a nearly > empty younggen to work with. > > There are two parts to this change. > > Part 1: > > When unlocking the last JNI critical section, we record the current > number of collections for use when the associated _gc_locker > collection is considered for VMOp enqueuing. > > Each of the STW collectors handles a _gc_locker cause specially, > checking that the current number of collections is the same as when > the JNI critical section ended. This allows us to discard the > _gc_locker request if there has been some other GC since the critical > section ended, eliminating a race that could otherwise result in the > _gc_locker collection immediately following some other collection. If > that test passes, it's still possible for the _gc_locker VMOp to be > discarded because of another GC VMOp being ahead of it; that's as > intended. > > This change is addressing the first problem discussed in Tony's > analysis, the race after the GCLocker becomes inactive. This change > is sufficient to eliminate unnecessary _gc_locker collections > immediately following another collection for G1. It reduces, but does > not eliminate the problem for the other STW collectors. > > Note that the (G1-specific) fix for JDK-7143858 is probably wrong. > The GCLocker stalling should probably only be done if > GCLocker::is_active_and_needs_gc() (like the other STW collectors), > rather than just GCLocker::needs_gc(). What was done there is, I > think, sort of an incomplete version of one of Tony's "Fix 1", and has > some similar problems, as well as being insufficient on its own, and > no longer needed with the changes here. Reconsidering JDK-7143858 can > be addressed later. > > Part 2: > > Fix confusion over full/nonfull collections in the non-G1 STW > collectors. (G1 doesn't have this problem.) ParallelGC and the GCH > collectors were initializing VM_GC_Operation::_full to true, always, > and then have some kludgy workaround attempts for some GCCauses. > Instead, _full should be set appropriately for the cause (making > skip_operation work properly), and used appropriately elsewhere. > > There remains some confusion about "full" collections around > GTH::do_collection, but it doesn't seem to lead to unnecessary > _gc_locker collections or other problems. That can be addressed > later. > > This change addresses the second problem discussed in Tony's analysis, > the treatment of _gc_locker collections as sort of full and sort of > not. This change reduced but did not eliminate unnecessary _gc_locker > collections for ParallelGC and GCH collectors. However, used in > conjunction with the Part 1 changes, they no longer seem to occur. > Running the attached stress test for 10 minutes produced none, where > the baseline without these changes might produce several hundred per > minute for the GCH collectors. > > There are no changes to ZGC needed here; ZGC handles _gc_locker in an > entirely different fashion that doesn't have the described problem. > > There are no changes to Shenandoah here; Shenandoah doesn't use the > GCLocker mechanism, instead using object pinning. > > There are no changes to Epsilon here; Epsilon just ignores _gc_locker > GC requests. > > The attached stress test is based on JNICriticalStressTest2.java from > the bug report, converted into a jtreg test. The functional test is > run as a subprocess inheriting framework supplied options, with the > associated GC log checked for GCLocker collections with small from > space sizes; the test fails if there are any. The test relies on GC > option rotation by the testing framework to provide coverage of the > various collectors. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8048556 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8048556/open.00/ > > Testing: > mach5 tier1-5 > locally ran new test for 30 minutes with each STW collector. > From chengjingwei1 at huawei.com Wed Jul 31 09:47:46 2019 From: chengjingwei1 at huawei.com (chengjingwei) Date: Wed, 31 Jul 2019 17:47:46 +0800 Subject: [8u] Request for backporting JDK-8131600 Message-ID: Hi, all Would someone help to approve the backport of JDK-8131600 to 8u? Bug: https://bugs.openjdk.java.net/browse/JDK-8131600 Here is the discussion on this issue: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2015-July/014176.html This issue decreases the stability of our applications. It's always reproducible with jdk built from the latest jdk8u master repo. From shade at redhat.com Wed Jul 31 10:01:40 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 31 Jul 2019 12:01:40 +0200 Subject: [8u] Request for backporting JDK-8131600 In-Reply-To: References: Message-ID: On 7/31/19 11:47 AM, chengjingwei wrote: > Would someone help to approve the backport of JDK-8131600 to 8u? Same here. Please follow the process outlined here: https://wiki.openjdk.java.net/display/JDKUpdates/How+to+contribute+a+fix -- Thanks, -Aleksey From zgu at redhat.com Wed Jul 31 13:32:48 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 31 Jul 2019 09:32:48 -0400 Subject: RFR(S) 228777: Shenandoah: Cleanup STW weak root processing In-Reply-To: References: Message-ID: <1ad6e05d-e0a3-7497-4c28-653cb356eb96@redhat.com> Hi Roman, On 7/30/19 4:31 PM, Zhengyu Gu wrote: > > > On 7/30/19 1:26 PM, Zhengyu Gu wrote: >> >> >> On 7/30/19 12:50 PM, Roman Kennke wrote: >>> I am not sure about this. If final-mark could not finish to pre-evacuate >>> all roots we might have from-space refs there. Or does the OOM-protocol >>> ensure that any ref should be updated correctly? We need to be careful >>> here. Better safe than sorry. > > Actually, you are right. It does not work with traversal. I wonder how > the tests passed early. > > I would like to withdraw this RFR. It is not true. Traversal fixes forwarding before processes weak roots, so this patch should work. The failed tests were due to broken builds, they are passed on clean builds, also specjvm tests (normal and traversal mode). Therefore, I would like re-open this RFR. This invariant is important for implementing concurrent weak root processing (JDK-8228818). If you still have concerns, let's push it to shenandoah/jdk, Okay? Thanks, -Zhengyu > > Thanks, > > -Zhengyu > > > >> >> If pre-evacuation fails, it runs degenerated GC cycle and roots are >> updated in final_updaterefs(). >> >> We do have no-forwarded assertion in place, should catch the problem >> if otherwise. >> >> Thanks, >> >> -Zhengyu >> >>> >>> Roman >>> >>> >>>> Shenandoah pre-evacuates or updates weak roots at STW pauses. >>>> Therefore, >>>> weak roots should not have forwarded references during mark phase. >>>> >>>> Current has_forwarded_objects() branch does not break things, but does >>>> unncessary works, let's prune it. >>>> >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8228777 >>>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8228777/webrev.00/ >>>> >>>> Test: >>>> ?? hotspot_gc_shenandoah (fastdebug and release) >>>> >>>> Thanks, >>>> >>>> -Zhengyu >>> From rkennke at redhat.com Wed Jul 31 13:43:40 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 31 Jul 2019 15:43:40 +0200 Subject: RFR(S) 228777: Shenandoah: Cleanup STW weak root processing In-Reply-To: <1ad6e05d-e0a3-7497-4c28-653cb356eb96@redhat.com> References: <1ad6e05d-e0a3-7497-4c28-653cb356eb96@redhat.com> Message-ID: <52614d0c-4dc7-7176-2f1e-6769db141792@redhat.com> >>>> I am not sure about this. If final-mark could not finish to >>>> pre-evacuate >>>> all roots we might have from-space refs there. Or does the OOM-protocol >>>> ensure that any ref should be updated correctly? We need to be careful >>>> here. Better safe than sorry. >> >> Actually, you are right. It does not work with traversal. I wonder how >> the tests passed early. >> >> I would like to withdraw this RFR. > > It is not true. Traversal fixes forwarding before processes weak roots, > so this patch should work. > > The failed tests were due to broken builds, they are passed on clean > builds, also specjvm tests (normal and traversal mode). Therefore, I > would like re-open this RFR. This invariant is important for > implementing concurrent weak root processing (JDK-8228818). > > If you still have concerns, let's push it to shenandoah/jdk, Okay? Yeah, let's push it to sh/jdk and if it's ok, take it upstream to jdk/jdk. Thanks, Roman > Thanks, > > -Zhengyu > >> >> Thanks, >> >> -Zhengyu >> >> >> >>> >>> If pre-evacuation fails, it runs degenerated GC cycle and roots are >>> updated in final_updaterefs(). >>> >>> We do have no-forwarded assertion in place, should catch the problem >>> if otherwise. >>> >>> Thanks, >>> >>> -Zhengyu >>> >>>> >>>> Roman >>>> >>>> >>>>> Shenandoah pre-evacuates or updates weak roots at STW pauses. >>>>> Therefore, >>>>> weak roots should not have forwarded references during mark phase. >>>>> >>>>> Current has_forwarded_objects() branch does not break things, but does >>>>> unncessary works, let's prune it. >>>>> >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8228777 >>>>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8228777/webrev.00/ >>>>> >>>>> Test: >>>>> ?? hotspot_gc_shenandoah (fastdebug and release) >>>>> >>>>> Thanks, >>>>> >>>>> -Zhengyu >>>> From kim.barrett at oracle.com Wed Jul 31 14:51:42 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 31 Jul 2019 10:51:42 -0400 Subject: RFR: 8048556: Unnecessary GCLocker-initiated young GCs In-Reply-To: <8df28fbd-bbfb-1988-e057-0af9cac606d2@oracle.com> References: <6a3227ce-3cf0-46a3-9eb7-2fc7c3f9d675@default> <7A07B132-2ED7-4992-90E4-F8E57B731A75@oracle.com> <9c40acac-e85e-4186-93ee-715d3a909549@default> <4DBC652F-3559-4CCF-A3CF-86D6375B7844@oracle.com> <8df28fbd-bbfb-1988-e057-0af9cac606d2@oracle.com> Message-ID: > On Jul 31, 2019, at 5:44 AM, Per Liden wrote: > > Hi Kim, > > Change looks good. Thanks. > > /Per > > On 7/30/19 7:59 PM, Kim Barrett wrote: >> RFR: 8048556: Unnecessary GCLocker-initiated young GCs >> [?] >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8048556 >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8048556/open.00/ >> Testing: >> mach5 tier1-5 >> locally ran new test for 30 minutes with each STW collector. From thomas.schatzl at oracle.com Wed Jul 31 15:57:51 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 31 Jul 2019 08:57:51 -0700 Subject: RFR: 8048556: Unnecessary GCLocker-initiated young GCs In-Reply-To: <8df28fbd-bbfb-1988-e057-0af9cac606d2@oracle.com> References: <6a3227ce-3cf0-46a3-9eb7-2fc7c3f9d675@default> <7A07B132-2ED7-4992-90E4-F8E57B731A75@oracle.com> <9c40acac-e85e-4186-93ee-715d3a909549@default> <4DBC652F-3559-4CCF-A3CF-86D6375B7844@oracle.com> <8df28fbd-bbfb-1988-e057-0af9cac606d2@oracle.com> Message-ID: <2bf350cc-f209-cf56-d598-6f8a42139824@oracle.com> Hi Kim, On 31.07.19 02:44, Per Liden wrote: > Hi Kim, > > Change looks good. +1. Thanks, Thomas From thomas.schatzl at oracle.com Wed Jul 31 16:44:19 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 31 Jul 2019 09:44:19 -0700 Subject: RFR(S): 8227225: ParallelGC: add subspace transitions for young gen for gc+heap=info log lines In-Reply-To: References: <877bd17cff3be20bf8acd95d9eddbb5c9cfb7cf5.camel@oracle.com> Message-ID: <7df6ab28-aa69-c3c7-d2f8-18a34c0304c5@oracle.com> Hi, On 29.07.19 07:41, Tony Printezis wrote: > Hi Thomas, > > Latest webrev here: > > http://cr.openjdk.java.net/~tonyp/8227225/webrev.1/ > > Main change: I renamed the PreGCValues class to PreGenGCValues so that > it?s clear it?s mainly for generational GCs. > please also update the file name containing only PreGenGCValues. It would also be useful to provide a diff webrev from the last version; using Mercurial mq and webrev -r XX may help with maintaining all changes that could be folded before pushing. Looks good otherwise. Thanks, Thomas From per.liden at oracle.com Wed Jul 31 17:19:28 2019 From: per.liden at oracle.com (Per Liden) Date: Wed, 31 Jul 2019 19:19:28 +0200 Subject: RFR: 8227226: Segmented array clearing for ZGC In-Reply-To: References: <5809FFE3-ED37-429B-9189-49D8FD14D092@amazon.com> <625131FC-8B02-4BEC-80B5-F1757232277A@amazon.com> <7C95DE35-C9F8-4F11-8D07-9511748128C4@amazon.com> Message-ID: <62b08b98-f54c-5359-4cc0-be36d43febdd@oracle.com> Hi, I found some time to benchmark the "GC clears pages"-approach, and it's fairly clear that it's not paying off. So ditching that idea. However, I'm still looking for something that would not just do segmented clearing of arrays in large zpages. Letting oop arrays temporarily be typed arrays while it's being cleared could be an option. I did a prototype for that, which looks like this: http://cr.openjdk.java.net/~pliden/8227226/webrev.1 There's at least one issue here, the code doing allocation sampling will see that we allocated long arrays instead of oop arrays, so the reporting there will be skewed. That can be addressed if we go down this path. The code is otherwise fairly simple and contained. Feel free to spot any issues. cheers, Per On 7/26/19 2:27 PM, Per Liden wrote: > Hi Ryan & Erik, > > I had a look at this and started exploring a slightly different > approach. Instead doing segmented clearing in the allocation path, we > can have the concurrent GC thread clear pages when they are reclaimed > and not do any clearing in the allocation path at all. > > That would look like this: > > http://cr.openjdk.java.net/~pliden/8227226/webrev.0-base > > (I've had to temporarily comment out three lines of assert/debug code to > make this work) > > The relocation set selection phase will now be tasked with some > potentially expensive clearing work, so we'll want to make that part > parallel also. > > http://cr.openjdk.java.net/~pliden/8227226/webrev.0-parallel > > Moving this work from Java threads onto the concurrent GC threads means > we will potentially prolong the RelocationSetSelection and Relocation > phases. That might be a trade-off worth doing. In return, we get: > > * Faster array allocations, as there's now less work done in the > allocation path. > * This benefits all arrays, not just those allocated in large pages. > * No need to consider/tune a "chunk size". > * I also tend think we'll end up with slightly less complex code, that > is a bit easier to reason about. Can be debated of course. > > This approach might also "survive" longer, because the YC scheme we've > been loosely thinking about currently requires newly allocated pages to > be cleared anyway. It's of course too early to tell if that requirement > will stand in the end, but it's possible anyway. > > I'll need to do some more testing and benchmarking to make sure there's > no regression or bugs here. The commented out debug code also needs to > be addressed of course. > > Comments? Other ideas? > > cheers, > Per > > On 7/24/19 4:37 PM, Sciampacone, Ryan wrote: >> >> Somehow I lost the RFR off the front and started a new thread. >> Now that we're both off vacation I'd like to revisit this.? Can you >> take a look? >> >> ?On 7/8/19, 10:40 AM, "hotspot-gc-dev on behalf of Sciampacone, Ryan" >> >> wrote: >> >> ???? http://cr.openjdk.java.net/~phh/8227226/webrev.01/ >> ???? This shifts away from abusing the constructor do_zero magic in >> exchange for virtualizing mem_clear() and specializing for the Z >> version.? It does create a change in mem_clear in that it returns an >> updated version of mem.? It does create change outside of the Z code >> however it does feel cleaner. >> ???? I didn't make a change to PinAllocating - looking at it, the >> safety of keeping it constructor / destructor based still seemed >> appropriate to me.? If the objection is to using the sequence numbers >> to pin (and instead using handles to update) - this to me seems less >> error prone.? I had originally discussed handles with Stefan but the >> proposal came down to this which looks much cleaner. >> ???? On 7/8/19, 6:36 AM, "hotspot-gc-dev on behalf of Sciampacone, >> Ryan" > sci at amazon.com> wrote: >> ???????? 1) Yes this was a conscious decision.? There was discussion >> on determining the optimal point for breakup but given the existing >> sizes this seemed sufficient.? This doesn't preclude the ability to go >> down that path if its deemed absolutely necessary.? The path for more >> complex decisions is now available. >> ???????? 2) Agree >> ???????? 3) I'm not clear here.? Do you mean effectively going direct >> to ZHeap and bypassing the single function PinAllocating?? Agree. >> Otherwise I'll ask you to be a bit clearer. >> ???????? 4) Agree >> ???????? 5) I initially had the exact same reaction but I played >> around with a few other versions (including breaking up initialization >> points between header and body to get the desired results) and this >> ended up looking correct.? I'll try mixing in the mem clearer function >> again (fresh start) to see if it looks any better. >> ???????? On 7/8/19, 5:49 AM, "Per Liden" wrote: >> ???????????? Hi Ryan, >> ???????????? A few general comments: >> ???????????? 1) It looks like this still only work for large pages? >> ???????????? 2) The log_info stuff should be removed. >> ???????????? 3) I'm not a huge fan of single-use utilities like >> PinAllocating, at >> ???????????? least not when, IMO, the alternative is more straight >> forward and less code. >> ???????????? 4) Please make locals const when possible. >> ???????????? 5) Duplicating _do_zero looks odd. Injecting a "mem >> clearer", similar to >> ???????????? what Stefans original patch did, seems worth exploring. >> ???????????? cheers, >> ???????????? /Per >> ???????????? (Btw, I'm on vacation so I might not be super-responsive >> to emails) >> ???????????? On 2019-07-08 12:42, Erik ?sterlund wrote: >> ???????????? > Hi Ryan, >> ???????????? > >> ???????????? > This looks good in general. Just some stylistic things... >> ???????????? > >> ???????????? > 1) In the ZGC project we like the letter 'Z' so much >> that we put it in >> ???????????? > front of everything we possibly can, including all >> class names. >> ???????????? > 2) We also explicitly state things are private even >> though it's >> ???????????? > bleedingly obvious. >> ???????????? > >> ???????????? > So: >> ???????????? > >> ???????????? > 39 class PinAllocating { >> ???????????? > 40 HeapWord* _mem; >> ???????????? > 41 public: -> 39 class ZPinAllocating { 40 private: >> 41 HeapWord* _mem; >> ???????????? >??? 42 >> ???????????? >?? 41 public: I can be your sponsor and push this change >> for you. I don't >> ???????????? > think there is a need for another webrev for my small >> stylistic remarks, >> ???????????? > so I can just fix that before pushing this for you. On >> that note, I'll >> ???????????? > add me and StefanK to the contributed-by section as we >> all worked out >> ???????????? > the right solution to this problem collaboratively. I >> have run through >> ???????????? > mach5 tier1-5, and found no issues with this patch. >> Thanks, /Erik >> ???????????? > >> ???????????? > On 2019-07-05 17:18, Sciampacone, Ryan wrote: >> ???????????? >> http://cr.openjdk.java.net/~phh/8227226/webrev.00/ >> ???????????? >> https://bugs.openjdk.java.net/browse/JDK-8227226 >> ???????????? >> >> ???????????? >> This patch introduces safe point checks into array >> clearing during >> ???????????? >> allocation for ZGC.? The patch isolates the changes to >> ZGC as (in >> ???????????? >> particular with the more modern collectors) the >> approach to >> ???????????? >> incrementalizing or respecting safe point checks is >> going to be >> ???????????? >> different. >> ???????????? >> >> ???????????? >> The approach is to keep the region holding the array >> in the allocating >> ???????????? >> state (pin logic) while updating the color to the >> array after checks. >> ???????????? >> >> ???????????? >> Can I get a review?? Thanks. >> ???????????? >> >> ???????????? >> Ryan >> ???????????? > >> From kim.barrett at oracle.com Wed Jul 31 17:46:56 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 31 Jul 2019 13:46:56 -0400 Subject: RFR: 8048556: Unnecessary GCLocker-initiated young GCs In-Reply-To: <2bf350cc-f209-cf56-d598-6f8a42139824@oracle.com> References: <6a3227ce-3cf0-46a3-9eb7-2fc7c3f9d675@default> <7A07B132-2ED7-4992-90E4-F8E57B731A75@oracle.com> <9c40acac-e85e-4186-93ee-715d3a909549@default> <4DBC652F-3559-4CCF-A3CF-86D6375B7844@oracle.com> <8df28fbd-bbfb-1988-e057-0af9cac606d2@oracle.com> <2bf350cc-f209-cf56-d598-6f8a42139824@oracle.com> Message-ID: > On Jul 31, 2019, at 11:57 AM, Thomas Schatzl wrote: > > Hi Kim, > > On 31.07.19 02:44, Per Liden wrote: >> Hi Kim, >> Change looks good. > > +1. > > Thanks, > Thomas Thanks. From thomas.schatzl at oracle.com Wed Jul 31 17:59:42 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 31 Jul 2019 10:59:42 -0700 Subject: RFR: 8227226: Segmented array clearing for ZGC In-Reply-To: <62b08b98-f54c-5359-4cc0-be36d43febdd@oracle.com> References: <5809FFE3-ED37-429B-9189-49D8FD14D092@amazon.com> <625131FC-8B02-4BEC-80B5-F1757232277A@amazon.com> <7C95DE35-C9F8-4F11-8D07-9511748128C4@amazon.com> <62b08b98-f54c-5359-4cc0-be36d43febdd@oracle.com> Message-ID: <2a64dceb-6188-f740-708e-983a5f6f681e@oracle.com> Hi, On 31.07.19 10:19, Per Liden wrote: > Hi, > > I found some time to benchmark the "GC clears pages"-approach, and it's > fairly clear that it's not paying off. So ditching that idea. > > However, I'm still looking for something that would not just do > segmented clearing of arrays in large zpages. Letting oop arrays > temporarily be typed arrays while it's being cleared could be an option. > I did a prototype for that, which looks like this: > > http://cr.openjdk.java.net/~pliden/8227226/webrev.1 > > There's at least one issue here, the code doing allocation sampling will > see that we allocated long arrays instead of oop arrays, so the > reporting there will be skewed. That can be addressed if we go down this > path. The code is otherwise fairly simple and contained. Feel free to > spot any issues. that looks like a really neat way of doing this. Looking over this there does not seem to be any real dependency on ZGC code, so if you went this way, would it be possible to provide this solution for all collectors? For other collectors slightly larger segment sizes might be sufficient too to slightly favor performance. Did you measure the impact on zeroing throughput of this? Thanks, Thomas From kim.barrett at oracle.com Wed Jul 31 18:45:25 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 31 Jul 2019 14:45:25 -0400 Subject: RFR: 8048556: Unnecessary GCLocker-initiated young GCs In-Reply-To: References: <6a3227ce-3cf0-46a3-9eb7-2fc7c3f9d675@default> <7A07B132-2ED7-4992-90E4-F8E57B731A75@oracle.com> <9c40acac-e85e-4186-93ee-715d3a909549@default> <4DBC652F-3559-4CCF-A3CF-86D6375B7844@oracle.com> <8df28fbd-bbfb-1988-e057-0af9cac606d2@oracle.com> <2bf350cc-f209-cf56-d598-6f8a42139824@oracle.com> Message-ID: > On Jul 31, 2019, at 1:46 PM, Kim Barrett wrote: > >> On Jul 31, 2019, at 11:57 AM, Thomas Schatzl wrote: >> >> Hi Kim, >> >> On 31.07.19 02:44, Per Liden wrote: >>> Hi Kim, >>> Change looks good. >> >> +1. >> >> Thanks, >> Thomas > > Thanks. For the record, during an offline discussion with Fairoz, we noticed the no-arg printlin() in the test is unused, so I removed that before pushing.