From shade at redhat.com Wed May 1 10:19:37 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 1 May 2019 12:19:37 +0200 Subject: RFR(M) 8222992: Shenandoah: Pre-evacuate all roots In-Reply-To: <22cabc5f-5116-07da-30f7-854248e1b932@redhat.com> References: <2f6eeb40-ccf5-9dd8-7ccc-181e993dedf0@redhat.com> <3cf86adc-6865-0b8a-23bd-c93884d6d456@redhat.com> <0f6499ee-7b2e-818d-96f8-2d8787b7a10f@redhat.com> <22cabc5f-5116-07da-30f7-854248e1b932@redhat.com> Message-ID: <7166dbbb-71f3-2673-3afd-bc67efe572bc@redhat.com> On 4/30/19 9:51 PM, Zhengyu Gu wrote: > Updated webrev: http://cr.openjdk.java.net/~zgu/JDK-8222992/webrev.01/index.html Something is wrong with the webrev. The patch has: +void ShenandoahVerifier::verify_roots_no_forwarded() { + verify_at_safepoint( + "Roots Have Not Forwarded References", + _verify_forwarded_none, // no forwarded references + _verify_marked_disable, // walk only roots + _verify_cset_disable, // no cset verification + _verify_liveness_disable, // no reliable liveness data anymore + _verify_regions_notrash, // trash regions have been recycled already + _verify_gcstate_disable // no gc state check + ); +} ...yet webrevs has: 943 void ShenandoahVerifier::verify_roots_no_forwarded() { 944 guarantee(ShenandoahSafepoint::is_at_shenandoah_safepoint(), "only when nothing else happens"); 945 ShenandoahRootProcessor rp(_heap, 1, ShenandoahPhaseTimings::_num_phases); // no need for stats 946 ShenandoahVerifyNoForwared cl; 947 rp.process_all_roots_slow(&cl); 948 } I think missing roots SRP::process_all_roots_slow require a separate changeset, as it should be backported ahead of this change to other releases; -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Wed May 1 12:32:25 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 1 May 2019 08:32:25 -0400 Subject: RFR(XXS) 8223184: Shenandoah: Missing roots in SRP::process_all_roots_slow Message-ID: <89fccd7c-f931-3496-2d2c-7a1f49161a73@redhat.com> Added missing roots in SRP::process_all_root_slow(). Bug: https://bugs.openjdk.java.net/browse/JDK-8223184 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8223184/webrev.00/ Test: hotspot_gc_shenandoah with -XX:+ShenandoahVerify (fastdebug and release) Thanks, -Zhengyu From shade at redhat.com Wed May 1 12:34:37 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 1 May 2019 14:34:37 +0200 Subject: RFR(XXS) 8223184: Shenandoah: Missing roots in SRP::process_all_roots_slow In-Reply-To: <89fccd7c-f931-3496-2d2c-7a1f49161a73@redhat.com> References: <89fccd7c-f931-3496-2d2c-7a1f49161a73@redhat.com> Message-ID: <541dafa2-0977-d588-2946-c373f6a6f82a@redhat.com> On 5/1/19 2:32 PM, Zhengyu Gu wrote: > Added missing roots in SRP::process_all_root_slow(). > > Bug: https://bugs.openjdk.java.net/browse/JDK-8223184 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8223184/webrev.00/ Looks good. We have JvmtiExport::oops_do call in the same method, it does not catch what JvmtiExport::weak_oops_do does? -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Wed May 1 13:18:34 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 1 May 2019 09:18:34 -0400 Subject: RFR(XXS) 8223184: Shenandoah: Missing roots in SRP::process_all_roots_slow In-Reply-To: <541dafa2-0977-d588-2946-c373f6a6f82a@redhat.com> References: <89fccd7c-f931-3496-2d2c-7a1f49161a73@redhat.com> <541dafa2-0977-d588-2946-c373f6a6f82a@redhat.com> Message-ID: <75e6ef85-53c4-7e3d-7f13-ebc300ad34c3@redhat.com> On 5/1/19 8:34 AM, Aleksey Shipilev wrote: > On 5/1/19 2:32 PM, Zhengyu Gu wrote: >> Added missing roots in SRP::process_all_root_slow(). >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8223184 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8223184/webrev.00/ > > Looks good. Thanks for reviewing. > > We have JvmtiExport::oops_do call in the same method, it does not catch what > JvmtiExport::weak_oops_do does? No, they scan different things. void JvmtiExport::oops_do(OopClosure* f) { JvmtiCurrentBreakpoints::oops_do(f); JvmtiObjectAllocEventCollector::oops_do_for_all_threads(f); } void JvmtiExport::weak_oops_do(BoolObjectClosure* is_alive, OopClosure* f) { JvmtiTagMap::weak_oops_do(is_alive, f); } -Zhengyu > > -Aleksey > From shade at redhat.com Wed May 1 13:49:54 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 1 May 2019 15:49:54 +0200 Subject: RFR(XXS) 8223184: Shenandoah: Missing roots in SRP::process_all_roots_slow In-Reply-To: <75e6ef85-53c4-7e3d-7f13-ebc300ad34c3@redhat.com> References: <89fccd7c-f931-3496-2d2c-7a1f49161a73@redhat.com> <541dafa2-0977-d588-2946-c373f6a6f82a@redhat.com> <75e6ef85-53c4-7e3d-7f13-ebc300ad34c3@redhat.com> Message-ID: <67c0d871-fa19-5072-8af5-0c17978c7d43@redhat.com> On 5/1/19 3:18 PM, Zhengyu Gu wrote: >> We have JvmtiExport::oops_do call in the same method, it does not catch what >> JvmtiExport::weak_oops_do does? > > No, they scan different things. > > void JvmtiExport::oops_do(OopClosure* f) { > ? JvmtiCurrentBreakpoints::oops_do(f); > ? JvmtiObjectAllocEventCollector::oops_do_for_all_threads(f); > } > > void JvmtiExport::weak_oops_do(BoolObjectClosure* is_alive, OopClosure* f) { > ? JvmtiTagMap::weak_oops_do(is_alive, f); > } Ah. Okay then! -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Wed May 1 14:22:16 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 1 May 2019 10:22:16 -0400 Subject: RFR(M) 8222992: Shenandoah: Pre-evacuate all roots In-Reply-To: <7166dbbb-71f3-2673-3afd-bc67efe572bc@redhat.com> References: <2f6eeb40-ccf5-9dd8-7ccc-181e993dedf0@redhat.com> <3cf86adc-6865-0b8a-23bd-c93884d6d456@redhat.com> <0f6499ee-7b2e-818d-96f8-2d8787b7a10f@redhat.com> <22cabc5f-5116-07da-30f7-854248e1b932@redhat.com> <7166dbbb-71f3-2673-3afd-bc67efe572bc@redhat.com> Message-ID: <49aab5a0-8f04-8f52-9b7e-c489cb3dbd93@redhat.com> On 5/1/19 6:19 AM, Aleksey Shipilev wrote: > On 4/30/19 9:51 PM, Zhengyu Gu wrote: >> Updated webrev: http://cr.openjdk.java.net/~zgu/JDK-8222992/webrev.01/index.html > > Something is wrong with the webrev. The patch has: > > +void ShenandoahVerifier::verify_roots_no_forwarded() { > + verify_at_safepoint( > + "Roots Have Not Forwarded References", > + _verify_forwarded_none, // no forwarded references > + _verify_marked_disable, // walk only roots > + _verify_cset_disable, // no cset verification > + _verify_liveness_disable, // no reliable liveness data anymore > + _verify_regions_notrash, // trash regions have been recycled already > + _verify_gcstate_disable // no gc state check > + ); > +} > > ...yet webrevs has: > > 943 void ShenandoahVerifier::verify_roots_no_forwarded() { > 944 guarantee(ShenandoahSafepoint::is_at_shenandoah_safepoint(), "only when nothing else happens"); > 945 ShenandoahRootProcessor rp(_heap, 1, ShenandoahPhaseTimings::_num_phases); // no need for stats > 946 ShenandoahVerifyNoForwared cl; > 947 rp.process_all_roots_slow(&cl); > 948 } > > > I think missing roots SRP::process_all_roots_slow require a separate changeset, as it should be > backported ahead of this change to other releases; > Fixed SRP::process_all_roots_slow() in JDK-8223184 and rebased this patch: Updated webrev: http://cr.openjdk.java.net/~zgu/JDK-8222992/webrev.02/index.html Reran hotspot_gc_shenandoah tests with -XX:+ShenandoahVerify Thanks, -Zhengyu > -Aleksey > From rkennke at redhat.com Wed May 1 18:18:49 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 1 May 2019 20:18:49 +0200 Subject: RFR(XXS) 8223184: Shenandoah: Missing roots in SRP::process_all_roots_slow In-Reply-To: <89fccd7c-f931-3496-2d2c-7a1f49161a73@redhat.com> References: <89fccd7c-f931-3496-2d2c-7a1f49161a73@redhat.com> Message-ID: Isn't JvmtiExport::weak_oops_do() covered by WeakProcessor already? Roman > Added missing roots in SRP::process_all_root_slow(). > > Bug: https://bugs.openjdk.java.net/browse/JDK-8223184 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8223184/webrev.00/ > > Test: > ? hotspot_gc_shenandoah with -XX:+ShenandoahVerify (fastdebug and release) > > Thanks, > > -Zhengyu > From zgu at redhat.com Wed May 1 18:57:03 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 1 May 2019 14:57:03 -0400 Subject: RFR(XXS) 8223184: Shenandoah: Missing roots in SRP::process_all_roots_slow In-Reply-To: References: <89fccd7c-f931-3496-2d2c-7a1f49161a73@redhat.com> Message-ID: <386e73ab-53f4-5e46-e651-d73ac014d05c@redhat.com> On 5/1/19 2:18 PM, Roman Kennke wrote: > Isn't JvmtiExport::weak_oops_do() covered by WeakProcessor already? You are right. Unfortunately, I pushed. Will file RFE to remove it. -Zhengyu > > Roman > >> Added missing roots in SRP::process_all_root_slow(). >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8223184 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8223184/webrev.00/ >> >> Test: >> ?? hotspot_gc_shenandoah with -XX:+ShenandoahVerify (fastdebug and >> release) >> >> Thanks, >> >> -Zhengyu >> From per.liden at oracle.com Thu May 2 08:26:18 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 2 May 2019 10:26:18 +0200 Subject: RFR: 8222480: Implementation: JEP 351: ZGC: Uncommit Unused Memory In-Reply-To: <75e1632e-0885-3454-239f-2df161db3b9a@oracle.com> References: <5960ec85-c6bc-3860-e74d-de48bace9665@oracle.com> <625b7e87-3f57-2893-0f0b-45b98b698e57@oracle.com> <75e1632e-0885-3454-239f-2df161db3b9a@oracle.com> Message-ID: Hi, Some additional refinements and rebased on the latest jdk/jdk. I'm hoping this is the last revision. Testing: Passed tier 1-6 Diff: http://cr.openjdk.java.net/~pliden/8222480/webrev.3-diff Full: http://cr.openjdk.java.net/~pliden/8222480/webrev.3 /Per On 4/25/19 4:07 PM, Per Liden wrote: > Hi, > > Additional performance testing uncovered a regression in the allocation > path, due to the removal of the ZPreMapped class. This was fixed, by > priming the page cache with the initial heap size, and allowing the page > cache to split pages. This also helps avoid a certain class of page > cache evictions, where we can simply split a page instead. > > Diff: http://cr.openjdk.java.net/~pliden/8222480/webrev.2-diff > Full: http://cr.openjdk.java.net/~pliden/8222480/webrev.2 > > cheers, > Per > > On 4/23/19 3:26 PM, Per Liden wrote: >> Hi, >> >> I sat down with Stefan to review this. A few adjustments where made. >> >> Updated webrev: >> >> Diff: http://cr.openjdk.java.net/~pliden/8222480/webrev.1-diff >> Full: http://cr.openjdk.java.net/~pliden/8222480/webrev.1 >> >> /Per >> >> On 4/15/19 11:34 AM, Per Liden wrote: >>> Please review the implementation for JEP 351: ZGC: Uncommit Unused >>> Memory. >>> >>> Once the review has been completed, I intend to move the JEP to >>> Proposed To Target for JDK 13. >>> >>> The JEP has been updated to reflect the latest state of the >>> implementation. >>> >>> Testing: Passes tier1-3 on all Oracle platformsm and tier1-7 on >>> Linux/x86_64. >>> >>> JEP: https://bugs.openjdk.java.net/browse/JDK-8220347 >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8222480 >>> Webrev: http://cr.openjdk.java.net/~pliden/8222480/webrev.0 >>> >>> /Per From shade at redhat.com Thu May 2 11:33:45 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 2 May 2019 13:33:45 +0200 Subject: RFR(M) 8222992: Shenandoah: Pre-evacuate all roots In-Reply-To: <49aab5a0-8f04-8f52-9b7e-c489cb3dbd93@redhat.com> References: <2f6eeb40-ccf5-9dd8-7ccc-181e993dedf0@redhat.com> <3cf86adc-6865-0b8a-23bd-c93884d6d456@redhat.com> <0f6499ee-7b2e-818d-96f8-2d8787b7a10f@redhat.com> <22cabc5f-5116-07da-30f7-854248e1b932@redhat.com> <7166dbbb-71f3-2673-3afd-bc67efe572bc@redhat.com> <49aab5a0-8f04-8f52-9b7e-c489cb3dbd93@redhat.com> Message-ID: On 5/1/19 4:22 PM, Zhengyu Gu wrote: > Updated webrev: http://cr.openjdk.java.net/~zgu/JDK-8222992/webrev.02/index.html This looks okay to me. Not a fan of the separate ShenandoahVerifier::verify_roots_no_forwarded, since we could "just" fold stronger (non-forwarded) root verification to the generic verification code (e.g. in ShenandoahVerifierReachableTask). That, however, might end up with some failures in Traversal and such. We can do that as follow-up. -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From stefan.karlsson at oracle.com Thu May 2 12:34:56 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 2 May 2019 14:34:56 +0200 Subject: RFR: 8222480: Implementation: JEP 351: ZGC: Uncommit Unused Memory In-Reply-To: References: <5960ec85-c6bc-3860-e74d-de48bace9665@oracle.com> <625b7e87-3f57-2893-0f0b-45b98b698e57@oracle.com> <75e1632e-0885-3454-239f-2df161db3b9a@oracle.com> Message-ID: Looks good. StefanK On 2019-05-02 10:26, Per Liden wrote: > Hi, > > Some additional refinements and rebased on the latest jdk/jdk. I'm > hoping this is the last revision. > > Testing: Passed tier 1-6 > > Diff: http://cr.openjdk.java.net/~pliden/8222480/webrev.3-diff > Full: http://cr.openjdk.java.net/~pliden/8222480/webrev.3 > > /Per > > On 4/25/19 4:07 PM, Per Liden wrote: >> Hi, >> >> Additional performance testing uncovered a regression in the >> allocation path, due to the removal of the ZPreMapped class. This was >> fixed, by priming the page cache with the initial heap size, and >> allowing the page cache to split pages. This also helps avoid a >> certain class of page cache evictions, where we can simply split a >> page instead. >> >> Diff: http://cr.openjdk.java.net/~pliden/8222480/webrev.2-diff >> Full: http://cr.openjdk.java.net/~pliden/8222480/webrev.2 >> >> cheers, >> Per >> >> On 4/23/19 3:26 PM, Per Liden wrote: >>> Hi, >>> >>> I sat down with Stefan to review this. A few adjustments where made. >>> >>> Updated webrev: >>> >>> Diff: http://cr.openjdk.java.net/~pliden/8222480/webrev.1-diff >>> Full: http://cr.openjdk.java.net/~pliden/8222480/webrev.1 >>> >>> /Per >>> >>> On 4/15/19 11:34 AM, Per Liden wrote: >>>> Please review the implementation for JEP 351: ZGC: Uncommit Unused >>>> Memory. >>>> >>>> Once the review has been completed, I intend to move the JEP to >>>> Proposed To Target for JDK 13. >>>> >>>> The JEP has been updated to reflect the latest state of the >>>> implementation. >>>> >>>> Testing: Passes tier1-3 on all Oracle platformsm and tier1-7 on >>>> Linux/x86_64. >>>> >>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8220347 >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8222480 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8222480/webrev.0 >>>> >>>> /Per From zgu at redhat.com Thu May 2 12:39:41 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 2 May 2019 08:39:41 -0400 Subject: RFR(M) 8222992: Shenandoah: Pre-evacuate all roots In-Reply-To: References: <2f6eeb40-ccf5-9dd8-7ccc-181e993dedf0@redhat.com> <3cf86adc-6865-0b8a-23bd-c93884d6d456@redhat.com> <0f6499ee-7b2e-818d-96f8-2d8787b7a10f@redhat.com> <22cabc5f-5116-07da-30f7-854248e1b932@redhat.com> <7166dbbb-71f3-2673-3afd-bc67efe572bc@redhat.com> <49aab5a0-8f04-8f52-9b7e-c489cb3dbd93@redhat.com> Message-ID: <413753d3-3ebf-8164-97ed-93c50ff9d478@redhat.com> On 5/2/19 7:33 AM, Aleksey Shipilev wrote: > On 5/1/19 4:22 PM, Zhengyu Gu wrote: >> Updated webrev: http://cr.openjdk.java.net/~zgu/JDK-8222992/webrev.02/index.html > > This looks okay to me. > > Not a fan of the separate ShenandoahVerifier::verify_roots_no_forwarded, since we could "just" fold > stronger (non-forwarded) root verification to the generic verification code (e.g. in > ShenandoahVerifierReachableTask). Me neither. The early version I folded verify_roots_no_forwarded to use generic verification code, but that version always verifies marking, which could be long cycle. I intent to verify roots quite often in concurrent class unloading/concurrent roots evacuation work, and prefer shorter verification cycles. Also, I also want to be able to verify subset of roots (https://bugs.openjdk.java.net/browse/JDK-8223215), which could complicate generic verification code quite a bit. That, however, might end up with some failures in Traversal and > such. We can do that as follow-up. > Yes, we could do a follow-up RFE if needed. Thanks, -Zhengyu From per.liden at oracle.com Thu May 2 14:55:05 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 2 May 2019 16:55:05 +0200 Subject: RFR: 8222480: Implementation: JEP 351: ZGC: Uncommit Unused Memory In-Reply-To: References: <5960ec85-c6bc-3860-e74d-de48bace9665@oracle.com> <625b7e87-3f57-2893-0f0b-45b98b698e57@oracle.com> <75e1632e-0885-3454-239f-2df161db3b9a@oracle.com> Message-ID: Thanks Stefan! /Per On 05/02/2019 02:34 PM, Stefan Karlsson wrote: > Looks good. > > StefanK > > On 2019-05-02 10:26, Per Liden wrote: >> Hi, >> >> Some additional refinements and rebased on the latest jdk/jdk. I'm >> hoping this is the last revision. >> >> Testing: Passed tier 1-6 >> >> Diff: http://cr.openjdk.java.net/~pliden/8222480/webrev.3-diff >> Full: http://cr.openjdk.java.net/~pliden/8222480/webrev.3 >> >> /Per >> >> On 4/25/19 4:07 PM, Per Liden wrote: >>> Hi, >>> >>> Additional performance testing uncovered a regression in the >>> allocation path, due to the removal of the ZPreMapped class. This was >>> fixed, by priming the page cache with the initial heap size, and >>> allowing the page cache to split pages. This also helps avoid a >>> certain class of page cache evictions, where we can simply split a >>> page instead. >>> >>> Diff: http://cr.openjdk.java.net/~pliden/8222480/webrev.2-diff >>> Full: http://cr.openjdk.java.net/~pliden/8222480/webrev.2 >>> >>> cheers, >>> Per >>> >>> On 4/23/19 3:26 PM, Per Liden wrote: >>>> Hi, >>>> >>>> I sat down with Stefan to review this. A few adjustments where made. >>>> >>>> Updated webrev: >>>> >>>> Diff: http://cr.openjdk.java.net/~pliden/8222480/webrev.1-diff >>>> Full: http://cr.openjdk.java.net/~pliden/8222480/webrev.1 >>>> >>>> /Per >>>> >>>> On 4/15/19 11:34 AM, Per Liden wrote: >>>>> Please review the implementation for JEP 351: ZGC: Uncommit Unused >>>>> Memory. >>>>> >>>>> Once the review has been completed, I intend to move the JEP to >>>>> Proposed To Target for JDK 13. >>>>> >>>>> The JEP has been updated to reflect the latest state of the >>>>> implementation. >>>>> >>>>> Testing: Passes tier1-3 on all Oracle platformsm and tier1-7 on >>>>> Linux/x86_64. >>>>> >>>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8220347 >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8222480 >>>>> Webrev: http://cr.openjdk.java.net/~pliden/8222480/webrev.0 >>>>> >>>>> /Per From rkennke at redhat.com Thu May 2 15:19:30 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 2 May 2019 17:19:30 +0200 Subject: RFR: JDK-8223244: Fix usage of ARRAYCOPY_DISJOINT decorator Message-ID: <4e0bbe62-0f1b-8ec7-d768-41ac60f08124@redhat.com> In stubGenerator_x86_64.cpp, the ARRAYCOPY_DISJOINT decorator is missing in one place, and wrongly added in another. This should be fixed, because Shenandoah will rely on correct DISJOINT decorator soon, when we implement JDK-8222859. Specifically, the conjoint routine pretends to be disjoint. The checkcast routine omits the disjoint decorator, even though it is always disjoint (otherwise it wouldn't need checkcast). Unfortunately, the fix is less trivial than it sounds. ModRefBarrierSetAssembler's prologue implicitely assumes that checkcast->!disjoint and gets the register shuffling wrong. Same for ShenandoahBarrierSetAssembler. Hence the re-shuffling of that code. It should be clearer/more explicit now, and equivalent to what we had before. Bug: https://bugs.openjdk.java.net/browse/JDK-8223244 Webrev: http://cr.openjdk.java.net/~rkennke/8223244/webrev.00/ Testing: hotspot/tier1, hotspot_gc_shenandoah both x86 and aarch64. Intend to do jdk-submit before push. Can I please get a review? Thanks, Roman From erik.osterlund at oracle.com Thu May 2 16:18:03 2019 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Thu, 2 May 2019 18:18:03 +0200 Subject: RFR: 8222480: Implementation: JEP 351: ZGC: Uncommit Unused Memory In-Reply-To: References: <5960ec85-c6bc-3860-e74d-de48bace9665@oracle.com> <625b7e87-3f57-2893-0f0b-45b98b698e57@oracle.com> <75e1632e-0885-3454-239f-2df161db3b9a@oracle.com> Message-ID: Hi Per, Looks good. Thanks, /Erik > On 2 May 2019, at 10:26, Per Liden wrote: > > Hi, > > Some additional refinements and rebased on the latest jdk/jdk. I'm hoping this is the last revision. > > Testing: Passed tier 1-6 > > Diff: http://cr.openjdk.java.net/~pliden/8222480/webrev.3-diff > Full: http://cr.openjdk.java.net/~pliden/8222480/webrev.3 > > /Per > >> On 4/25/19 4:07 PM, Per Liden wrote: >> Hi, >> Additional performance testing uncovered a regression in the allocation path, due to the removal of the ZPreMapped class. This was fixed, by priming the page cache with the initial heap size, and allowing the page cache to split pages. This also helps avoid a certain class of page cache evictions, where we can simply split a page instead. >> Diff: http://cr.openjdk.java.net/~pliden/8222480/webrev.2-diff >> Full: http://cr.openjdk.java.net/~pliden/8222480/webrev.2 >> cheers, >> Per >>> On 4/23/19 3:26 PM, Per Liden wrote: >>> Hi, >>> >>> I sat down with Stefan to review this. A few adjustments where made. >>> >>> Updated webrev: >>> >>> Diff: http://cr.openjdk.java.net/~pliden/8222480/webrev.1-diff >>> Full: http://cr.openjdk.java.net/~pliden/8222480/webrev.1 >>> >>> /Per >>> >>>> On 4/15/19 11:34 AM, Per Liden wrote: >>>> Please review the implementation for JEP 351: ZGC: Uncommit Unused Memory. >>>> >>>> Once the review has been completed, I intend to move the JEP to Proposed To Target for JDK 13. >>>> >>>> The JEP has been updated to reflect the latest state of the implementation. >>>> >>>> Testing: Passes tier1-3 on all Oracle platformsm and tier1-7 on Linux/x86_64. >>>> >>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8220347 >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8222480 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8222480/webrev.0 >>>> >>>> /Per From zgu at redhat.com Thu May 2 18:02:17 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 2 May 2019 14:02:17 -0400 Subject: RFR(T) 8223258: Shenandoah: SRP::process_all_roots_slow processes JvmtiExport weak oops twice Message-ID: <0b2066ab-a94c-f3c6-c001-1c3cc1896960@redhat.com> WeakProcessor processes JvmtiExport weak oops. Bug: https://bugs.openjdk.java.net/browse/JDK-8223258 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8223258/webrev.00/ Test: hotspot_gc_shenandoah with -XX:+ShenandoahVerify (fastdebug and release) Thanks, -Zhengyu From rkennke at redhat.com Thu May 2 18:32:20 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 2 May 2019 20:32:20 +0200 Subject: RFR(T) 8223258: Shenandoah: SRP::process_all_roots_slow processes JvmtiExport weak oops twice In-Reply-To: <0b2066ab-a94c-f3c6-c001-1c3cc1896960@redhat.com> References: <0b2066ab-a94c-f3c6-c001-1c3cc1896960@redhat.com> Message-ID: Ok! Thanks! Roman > WeakProcessor processes JvmtiExport weak oops. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8223258 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8223258/webrev.00/ > > Test: > ? hotspot_gc_shenandoah with -XX:+ShenandoahVerify (fastdebug and release) > > Thanks, > > -Zhengyu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From sangheon.kim at oracle.com Thu May 2 19:15:45 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Thu, 2 May 2019 12:15:45 -0700 Subject: RFR: 8221359: Eliminate SATBMarkQueueSet::filter_thread_buffers In-Reply-To: References: Message-ID: <89caf137-f6ee-fdae-d15e-ef3879975610@oracle.com> Hi Kim, On 4/1/19 12:40 AM, Kim Barrett wrote: > Please review this removal of the function used by G1 to clean up the > per-thread SATB queues when a young-only collection will have moved > young object references present in those queues. This cleanup isn't > needed, since the same cleanup may be done before enqueuing a > completed queue buffer, and will also be done during the eventual > processing of the buffer contents. Having this function called during > a pause just moves some work from the concurrent phase into a pause, > which isn't a good tradeoff. > > Also renamed G1GCPhaseTimes::ExtRootScanSubPhases(Start|End) => > ExtRootScanSubPhases(First|Last), to remove the ambiguity over whether > "end" is inclusive or exclusive. (It was inclusive here.) > > CR: > https://bugs.openjdk.java.net/browse/JDK-8221359 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8221359/open.00/ Looks good. But on the latest source code, there's a conflict at g1RootProcessor.hpp. Thanks, Sangheon > > Testing: > mach5 tier1-3 > From rkennke at redhat.com Thu May 2 21:42:59 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 2 May 2019 23:42:59 +0200 Subject: RFR: JDK-8223244: Extend arraycopy stub GC interface to allow GC replacement of the whole loop Message-ID: Currently, when doing oop arraycopy from C2 and C1, we call into runtime for pre-barrier, then do the arraycopy as fast-blt-loop, then call into runtime again for post-barrier. In Shenandoah we would like to call into runtime once, and do the arraycopy in a single-loop, when GC is active, and only do the fast-blt-loop outside of GC. This requires some GC interface changes to allow skipping the fast-loop when it's not needed, and to also pass the element type for doing the arraycopy loop and checkcasts in runtime. For the most part, the change is to add the klass Register, and the L_exit label to jump to. In aarch64, it also involves to actually add+bind those labels, plus a very minor shuffling of code, in order to preserve original element count before diving into the prologue. Bug: https://bugs.openjdk.java.net/browse/JDK-8223240 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8223244/webrev.00/ Testing: hotspot_gc_shenandoah, hotspot/tier1 (x86_64 and aarch64) will jdk-submit before pushing. Also, tried with the prototype for JDK-8222859 both x86_64 and aarch64. Can I please get reviews? Thanks, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From manc at google.com Thu May 2 22:49:23 2019 From: manc at google.com (Man Cao) Date: Thu, 2 May 2019 15:49:23 -0700 Subject: RFR (XS): 8223267: Fix incorrect usage of GCTraceTime in g1FullCollector and g1CollectedHeap Message-ID: Hi all, Can I have reviews for this small bug fix for timer values in the log? Webrev: https://cr.openjdk.java.net/~manc/8223267/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8223267 -Man -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Fri May 3 00:31:03 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 2 May 2019 20:31:03 -0400 Subject: RFR: 8221359: Eliminate SATBMarkQueueSet::filter_thread_buffers In-Reply-To: <89caf137-f6ee-fdae-d15e-ef3879975610@oracle.com> References: <89caf137-f6ee-fdae-d15e-ef3879975610@oracle.com> Message-ID: <036A076D-4AAE-4F1D-BFED-E2C0169F413E@oracle.com> > On May 2, 2019, at 3:15 PM, sangheon.kim at oracle.com wrote: > > Hi Kim, > > On 4/1/19 12:40 AM, Kim Barrett wrote: >> Please review this removal of the function used by G1 to clean up the >> per-thread SATB queues when a young-only collection will have moved >> young object references present in those queues. This cleanup isn't >> needed, since the same cleanup may be done before enqueuing a >> completed queue buffer, and will also be done during the eventual >> processing of the buffer contents. Having this function called during >> a pause just moves some work from the concurrent phase into a pause, >> which isn't a good tradeoff. >> >> Also renamed G1GCPhaseTimes::ExtRootScanSubPhases(Start|End) => >> ExtRootScanSubPhases(First|Last), to remove the ambiguity over whether >> "end" is inclusive or exclusive. (It was inclusive here.) >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8221359 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8221359/open.00/ > Looks good. > But on the latest source code, there's a conflict at g1RootProcessor.hpp. Thanks, and thanks for the warning about the conflict. It looks easy to resolve. > > Thanks, > Sangheon > >> >> Testing: >> mach5 tier1-3 From david.holmes at oracle.com Fri May 3 03:04:06 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 3 May 2019 13:04:06 +1000 Subject: RFR: JDK-8223244: Fix usage of ARRAYCOPY_DISJOINT decorator In-Reply-To: <4e0bbe62-0f1b-8ec7-d768-41ac60f08124@redhat.com> References: <4e0bbe62-0f1b-8ec7-d768-41ac60f08124@redhat.com> Message-ID: <52b79466-e553-8f7a-2106-68ab58710c3f@oracle.com> Hi Roman, Not a review as I'm not familiar with the code, but I did take a look and have one comment ... On 3/05/2019 1:19 am, Roman Kennke wrote: > ?In stubGenerator_x86_64.cpp, the ARRAYCOPY_DISJOINT decorator is > missing in one place, and wrongly added in another. This should be > fixed, because Shenandoah will rely on correct DISJOINT decorator soon, > when we implement JDK-8222859. > > Specifically, the conjoint routine pretends to be disjoint. The > checkcast routine omits the disjoint decorator, even though it is always > disjoint (otherwise it wouldn't need checkcast). Can you add a comment somewhere documenting the fact that the checkcast_arraycopy is always disjoint, as in other cases "disjoint" forms part of the name. Thanks, David ----- > Unfortunately, the fix is less trivial than it sounds. > ModRefBarrierSetAssembler's prologue implicitely assumes that > checkcast->!disjoint and gets the register shuffling wrong. Same for > ShenandoahBarrierSetAssembler. Hence the re-shuffling of that code. It > should be clearer/more explicit now, and equivalent to what we had before. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8223244 > Webrev: > http://cr.openjdk.java.net/~rkennke/8223244/webrev.00/ > > Testing: hotspot/tier1, hotspot_gc_shenandoah both x86 and aarch64. > Intend to do jdk-submit before push. > > Can I please get a review? > > Thanks, > Roman From per.liden at oracle.com Fri May 3 08:33:26 2019 From: per.liden at oracle.com (Per Liden) Date: Fri, 3 May 2019 10:33:26 +0200 Subject: RFR: 8222480: Implementation: JEP 351: ZGC: Uncommit Unused Memory In-Reply-To: References: <5960ec85-c6bc-3860-e74d-de48bace9665@oracle.com> <625b7e87-3f57-2893-0f0b-45b98b698e57@oracle.com> <75e1632e-0885-3454-239f-2df161db3b9a@oracle.com> Message-ID: <04e2c5ec-e607-1fd4-07e5-a4f26a5ccaae@oracle.com> Thanks for reviewing, Erik! /Per On 5/2/19 6:18 PM, Erik Osterlund wrote: > Hi Per, > > Looks good. > > Thanks, > /Erik > >> On 2 May 2019, at 10:26, Per Liden wrote: >> >> Hi, >> >> Some additional refinements and rebased on the latest jdk/jdk. I'm hoping this is the last revision. >> >> Testing: Passed tier 1-6 >> >> Diff: http://cr.openjdk.java.net/~pliden/8222480/webrev.3-diff >> Full: http://cr.openjdk.java.net/~pliden/8222480/webrev.3 >> >> /Per >> >>> On 4/25/19 4:07 PM, Per Liden wrote: >>> Hi, >>> Additional performance testing uncovered a regression in the allocation path, due to the removal of the ZPreMapped class. This was fixed, by priming the page cache with the initial heap size, and allowing the page cache to split pages. This also helps avoid a certain class of page cache evictions, where we can simply split a page instead. >>> Diff: http://cr.openjdk.java.net/~pliden/8222480/webrev.2-diff >>> Full: http://cr.openjdk.java.net/~pliden/8222480/webrev.2 >>> cheers, >>> Per >>>> On 4/23/19 3:26 PM, Per Liden wrote: >>>> Hi, >>>> >>>> I sat down with Stefan to review this. A few adjustments where made. >>>> >>>> Updated webrev: >>>> >>>> Diff: http://cr.openjdk.java.net/~pliden/8222480/webrev.1-diff >>>> Full: http://cr.openjdk.java.net/~pliden/8222480/webrev.1 >>>> >>>> /Per >>>> >>>>> On 4/15/19 11:34 AM, Per Liden wrote: >>>>> Please review the implementation for JEP 351: ZGC: Uncommit Unused Memory. >>>>> >>>>> Once the review has been completed, I intend to move the JEP to Proposed To Target for JDK 13. >>>>> >>>>> The JEP has been updated to reflect the latest state of the implementation. >>>>> >>>>> Testing: Passes tier1-3 on all Oracle platformsm and tier1-7 on Linux/x86_64. >>>>> >>>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8220347 >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8222480 >>>>> Webrev: http://cr.openjdk.java.net/~pliden/8222480/webrev.0 >>>>> >>>>> /Per > From rkennke at redhat.com Fri May 3 10:46:15 2019 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 3 May 2019 12:46:15 +0200 Subject: RFR: JDK-8223244: Fix usage of ARRAYCOPY_DISJOINT decorator In-Reply-To: <52b79466-e553-8f7a-2106-68ab58710c3f@oracle.com> References: <4e0bbe62-0f1b-8ec7-d768-41ac60f08124@redhat.com> <52b79466-e553-8f7a-2106-68ab58710c3f@oracle.com> Message-ID: <7905a8b3-9884-76a2-9f61-9c18ed2c1e5d@redhat.com> Hi David, > Not a review as I'm not familiar with the code, but I did take a look > and have one comment ... > > On 3/05/2019 1:19 am, Roman Kennke wrote: >> ??In stubGenerator_x86_64.cpp, the ARRAYCOPY_DISJOINT decorator is >> missing in one place, and wrongly added in another. This should be >> fixed, because Shenandoah will rely on correct DISJOINT decorator >> soon, when we implement JDK-8222859. >> >> Specifically, the conjoint routine pretends to be disjoint. The >> checkcast routine omits the disjoint decorator, even though it is >> always disjoint (otherwise it wouldn't need checkcast). > > Can you add a comment somewhere documenting the fact that the > checkcast_arraycopy is always disjoint, as in other cases "disjoint" > forms part of the name. Thanks. Done: Incremental: http://cr.openjdk.java.net/~rkennke/JDK-8223244/webrev.01.diff/ Full: http://cr.openjdk.java.net/~rkennke/JDK-8223244/webrev.01/ Roman > Thanks, > David > ----- > >> Unfortunately, the fix is less trivial than it sounds. >> ModRefBarrierSetAssembler's prologue implicitely assumes that >> checkcast->!disjoint and gets the register shuffling wrong. Same for >> ShenandoahBarrierSetAssembler. Hence the re-shuffling of that code. It >> should be clearer/more explicit now, and equivalent to what we had >> before. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8223244 >> Webrev: >> http://cr.openjdk.java.net/~rkennke/8223244/webrev.00/ >> >> Testing: hotspot/tier1, hotspot_gc_shenandoah both x86 and aarch64. >> Intend to do jdk-submit before push. >> >> Can I please get a review? >> >> Thanks, >> Roman From david.holmes at oracle.com Fri May 3 11:03:36 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 3 May 2019 21:03:36 +1000 Subject: RFR: JDK-8223244: Fix usage of ARRAYCOPY_DISJOINT decorator In-Reply-To: <7905a8b3-9884-76a2-9f61-9c18ed2c1e5d@redhat.com> References: <4e0bbe62-0f1b-8ec7-d768-41ac60f08124@redhat.com> <52b79466-e553-8f7a-2106-68ab58710c3f@oracle.com> <7905a8b3-9884-76a2-9f61-9c18ed2c1e5d@redhat.com> Message-ID: <47b2a2b5-5328-3097-df1c-82a56d6243af@oracle.com> On 3/05/2019 8:46 pm, Roman Kennke wrote: > Hi David, > >> Not a review as I'm not familiar with the code, but I did take a look >> and have one comment ... >> >> On 3/05/2019 1:19 am, Roman Kennke wrote: >>> ??In stubGenerator_x86_64.cpp, the ARRAYCOPY_DISJOINT decorator is >>> missing in one place, and wrongly added in another. This should be >>> fixed, because Shenandoah will rely on correct DISJOINT decorator >>> soon, when we implement JDK-8222859. >>> >>> Specifically, the conjoint routine pretends to be disjoint. The >>> checkcast routine omits the disjoint decorator, even though it is >>> always disjoint (otherwise it wouldn't need checkcast). >> >> Can you add a comment somewhere documenting the fact that the >> checkcast_arraycopy is always disjoint, as in other cases "disjoint" >> forms part of the name. > > Thanks. Done: > > Incremental: > http://cr.openjdk.java.net/~rkennke/JDK-8223244/webrev.01.diff/ > Full: > http://cr.openjdk.java.net/~rkennke/JDK-8223244/webrev.01/ Something in ./share/runtime/stubRoutines.hpp would be nice too :) Thanks, David ------ > Roman > > >> Thanks, >> David >> ----- >> >>> Unfortunately, the fix is less trivial than it sounds. >>> ModRefBarrierSetAssembler's prologue implicitely assumes that >>> checkcast->!disjoint and gets the register shuffling wrong. Same for >>> ShenandoahBarrierSetAssembler. Hence the re-shuffling of that code. >>> It should be clearer/more explicit now, and equivalent to what we had >>> before. >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8223244 >>> Webrev: >>> http://cr.openjdk.java.net/~rkennke/8223244/webrev.00/ >>> >>> Testing: hotspot/tier1, hotspot_gc_shenandoah both x86 and aarch64. >>> Intend to do jdk-submit before push. >>> >>> Can I please get a review? >>> >>> Thanks, >>> Roman From per.liden at oracle.com Fri May 3 12:13:41 2019 From: per.liden at oracle.com (Per Liden) Date: Fri, 3 May 2019 14:13:41 +0200 Subject: RFR: 8223308: ZGC: Remove ZGranuleMap::size() Message-ID: <2af0f71d-a997-2340-c5f4-db8964ebd105@oracle.com> ZAddressOffsetMax is no longer a constant, so ZGranuleMap::size() will not be as nicely folded as it used to be. I suggest we instead calculate the size once and store it in a variable. This is not critical for performance, but I still think it's nice to do. Bug: https://bugs.openjdk.java.net/browse/JDK-8223308 Webrev: http://cr.openjdk.java.net/~pliden/8223308/webrev.0 /Per From stefan.karlsson at oracle.com Fri May 3 12:45:48 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 3 May 2019 14:45:48 +0200 Subject: RFR: 8223308: ZGC: Remove ZGranuleMap::size() In-Reply-To: <2af0f71d-a997-2340-c5f4-db8964ebd105@oracle.com> References: <2af0f71d-a997-2340-c5f4-db8964ebd105@oracle.com> Message-ID: <4717e3cd-a39c-5474-b98d-b375c8c6dc3d@oracle.com> Looks good. StefanK On 2019-05-03 14:13, Per Liden wrote: > ZAddressOffsetMax is no longer a constant, so ZGranuleMap::size() will > not be as nicely folded as it used to be. I suggest we instead calculate > the size once and store it in a variable. This is not critical for > performance, but I still think it's nice to do. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8223308 > Webrev: http://cr.openjdk.java.net/~pliden/8223308/webrev.0 > > /Per From per.liden at oracle.com Fri May 3 14:16:54 2019 From: per.liden at oracle.com (Per Liden) Date: Fri, 3 May 2019 16:16:54 +0200 Subject: RFR: 8223308: ZGC: Remove ZGranuleMap::size() In-Reply-To: <4717e3cd-a39c-5474-b98d-b375c8c6dc3d@oracle.com> References: <2af0f71d-a997-2340-c5f4-db8964ebd105@oracle.com> <4717e3cd-a39c-5474-b98d-b375c8c6dc3d@oracle.com> Message-ID: <1608c9a5-e5f0-c2d1-a2c1-788bc30c917b@oracle.com> Thanks Stefan! /Per On 05/03/2019 02:45 PM, Stefan Karlsson wrote: > Looks good. > > StefanK > > On 2019-05-03 14:13, Per Liden wrote: >> ZAddressOffsetMax is no longer a constant, so ZGranuleMap::size() will >> not be as nicely folded as it used to be. I suggest we instead >> calculate the size once and store it in a variable. This is not >> critical for performance, but I still think it's nice to do. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8223308 >> Webrev: http://cr.openjdk.java.net/~pliden/8223308/webrev.0 >> >> /Per From rkennke at redhat.com Fri May 3 14:40:14 2019 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 3 May 2019 16:40:14 +0200 Subject: RFR: JDK-8223244: Fix usage of ARRAYCOPY_DISJOINT decorator In-Reply-To: <47b2a2b5-5328-3097-df1c-82a56d6243af@oracle.com> References: <4e0bbe62-0f1b-8ec7-d768-41ac60f08124@redhat.com> <52b79466-e553-8f7a-2106-68ab58710c3f@oracle.com> <7905a8b3-9884-76a2-9f61-9c18ed2c1e5d@redhat.com> <47b2a2b5-5328-3097-df1c-82a56d6243af@oracle.com> Message-ID: <163d08ac-daaf-7979-87d1-37f86bf2d4cd@redhat.com> >>> Not a review as I'm not familiar with the code, but I did take a look >>> and have one comment ... >>> >>> On 3/05/2019 1:19 am, Roman Kennke wrote: >>>> ??In stubGenerator_x86_64.cpp, the ARRAYCOPY_DISJOINT decorator is >>>> missing in one place, and wrongly added in another. This should be >>>> fixed, because Shenandoah will rely on correct DISJOINT decorator >>>> soon, when we implement JDK-8222859. >>>> >>>> Specifically, the conjoint routine pretends to be disjoint. The >>>> checkcast routine omits the disjoint decorator, even though it is >>>> always disjoint (otherwise it wouldn't need checkcast). >>> >>> Can you add a comment somewhere documenting the fact that the >>> checkcast_arraycopy is always disjoint, as in other cases "disjoint" >>> forms part of the name. >> >> Thanks. Done: >> >> Incremental: >> http://cr.openjdk.java.net/~rkennke/JDK-8223244/webrev.01.diff/ >> Full: >> http://cr.openjdk.java.net/~rkennke/JDK-8223244/webrev.01/ > > Something in ./share/runtime/stubRoutines.hpp would be nice too :) Right. Added a comment there, and also checked and fixed x86_32: http://cr.openjdk.java.net/~rkennke/JDK-8223244/webrev.02/ Thanks for 'reviewing'! Roman > Thanks, > David > ------ > >> Roman >> >> >>> Thanks, >>> David >>> ----- >>> >>>> Unfortunately, the fix is less trivial than it sounds. >>>> ModRefBarrierSetAssembler's prologue implicitely assumes that >>>> checkcast->!disjoint and gets the register shuffling wrong. Same for >>>> ShenandoahBarrierSetAssembler. Hence the re-shuffling of that code. >>>> It should be clearer/more explicit now, and equivalent to what we >>>> had before. >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8223244 >>>> Webrev: >>>> http://cr.openjdk.java.net/~rkennke/8223244/webrev.00/ >>>> >>>> Testing: hotspot/tier1, hotspot_gc_shenandoah both x86 and aarch64. >>>> Intend to do jdk-submit before push. >>>> >>>> Can I please get a review? >>>> >>>> Thanks, >>>> Roman From sgehwolf at redhat.com Fri May 3 15:56:52 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 03 May 2019 17:56:52 +0200 Subject: RFR 8221507: Implement JFR Events for Shenandoah In-Reply-To: References: Message-ID: <9b790032b583039880db3a7da35f5ebe8ea61e2d.camel@redhat.com> cc'ing hotspot-gc-dev as this also touches GC code (shenandoah). On Fri, 2019-05-03 at 10:44 -0400, Ken Dobson wrote: > Hi all, > > Please review this patch that adds support for two new JFR events > ShenandoahHeapRegionStateChange and ShenandoahHeapRegionInformation. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8221507 > Webrev: http://cr.openjdk.java.net/~kdobson/53476/webrev/ > > The Shenandoah team has also reviewed this patch and approved it from their > end. > > Thanks, > > Ken Dobson From rkennke at redhat.com Fri May 3 16:59:18 2019 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 3 May 2019 18:59:18 +0200 Subject: RFR: JDK-8222537: Avoid fetching _klass twice in TypeArrayOop::size() In-Reply-To: <5bad6271-2fb3-875f-d0d9-71bac167f768@redhat.com> References: <5bad6271-2fb3-875f-d0d9-71bac167f768@redhat.com> Message-ID: <3d6e8ebf-aafc-92d4-4f5e-b4a1bca2a3b4@redhat.com> Ping? Roman > Currently, when calling TypeArrayOop::size(), we end up calling klass() > twice: once before calling into size_given_klass() and then again before > calling TypeArrayOop::object_size(). > > This is currently only a minor performance nuisance. > > With upcoming Shenandoah's elimination of forwarding pointer, loading > klass like this is not safe anymore, and therefore we only call > size_given_klass(), and must avoid calling naked klass() altogether. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8222537 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8222537/webrev.00/ > Testing: > hotspot_gc_shenandoah with and without the prototype, hotspot/tier1 > > Can I please get reviews? > > Thanks, > Roman > From manc at google.com Fri May 3 20:07:24 2019 From: manc at google.com (Man Cao) Date: Fri, 3 May 2019 13:07:24 -0700 Subject: RFR (T): 8223018: Rename G1RemSet::*oops_into_collection_set_do methods In-Reply-To: References: Message-ID: Looks good. -Man *From: * *Date: *Tue, Apr 30, 2019 at 1:41 PM *To: *Thomas Schatzl *Cc: * Hi Thomas, > > On 4/26/19 1:29 AM, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this change that fixes the > > G1RemSet::*oops_into_collection_set_do methods to correspond to the > > naming of the main method scan_rem_set again. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8223018 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8223018/webrev > Looks good. > > Thanks, > Sangheon > > > > Testing: > > local compilation > > > > Thanks, > > Thomas > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From manc at google.com Fri May 3 20:18:56 2019 From: manc at google.com (Man Cao) Date: Fri, 3 May 2019 13:18:56 -0700 Subject: RFR (xs): 8221509: Refactor assert( G1CollectedHeap::used() == recalculate_used() ) with better message In-Reply-To: References: Message-ID: Hi Sangheon, The change looks good to me. I'm not an official "Reviewer" though. -Man *From: * *Date: *Tue, Apr 30, 2019 at 1:54 PM *To: *hotspot-gc-dev Hi all, > > Can I have some reviews that replace assert(used() == > recalculated_used()) with better message? > Those assert messages are not much helpful as it doesn't include any > actual values on it so I'm suggesting to include actual values of each > and then replace as a macro. > > CR: https://bugs.openjdk.java.net/browse/JDK-8221509 > webrev: http://cr.openjdk.java.net/~sangheki/8221509/webrev.0/ > testing: local build > > Thanks, > Sangheon > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Sat May 4 22:10:49 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 4 May 2019 18:10:49 -0400 Subject: RFR(trivial): 8223359: JDK-8221359 breaks TestG1ParallelPhases.java Message-ID: Please review this trivial update to the jfr test TestG1ParallelPhases.java, to account for the recent removal of the SATB filtering phase by JDK-8221359. The test should no longer expect an event for that phase. CR: https://bugs.openjdk.java.net/browse/JDK-8223359 Webrev: http://cr.openjdk.java.net/~kbarrett/8223359/open.00/ Testing: Locally (linux-x64) reproduced the failure, and verified the change eliminated the failure. From thomas.schatzl at oracle.com Sun May 5 11:30:56 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Sun, 05 May 2019 13:30:56 +0200 Subject: RFR(trivial): 8223359: JDK-8221359 breaks TestG1ParallelPhases.java In-Reply-To: References: Message-ID: <102759cfa4608fc4ade415ff16579d5814ea377e.camel@oracle.com> Hi, On Sat, 2019-05-04 at 18:10 -0400, Kim Barrett wrote: > Please review this trivial update to the jfr test > TestG1ParallelPhases.java, to account for the recent removal of the > SATB filtering phase by JDK-8221359. The test should no longer > expect an event for that phase. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8223359 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8223359/open.00/ > > Testing: > Locally (linux-x64) reproduced the failure, and verified the change > eliminated the failure. > looks good and trivial. Thomas From kim.barrett at oracle.com Sun May 5 13:06:35 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 5 May 2019 09:06:35 -0400 Subject: RFR(trivial): 8223359: JDK-8221359 breaks TestG1ParallelPhases.java In-Reply-To: <102759cfa4608fc4ade415ff16579d5814ea377e.camel@oracle.com> References: <102759cfa4608fc4ade415ff16579d5814ea377e.camel@oracle.com> Message-ID: <76A9DB95-5787-4FE7-AE13-521E5278F20C@oracle.com> > On May 5, 2019, at 7:30 AM, Thomas Schatzl wrote: > > Hi, > > On Sat, 2019-05-04 at 18:10 -0400, Kim Barrett wrote: >> Please review this trivial update to the jfr test >> TestG1ParallelPhases.java, to account for the recent removal of the >> SATB filtering phase by JDK-8221359. The test should no longer >> expect an event for that phase. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8223359 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8223359/open.00/ >> >> Testing: >> Locally (linux-x64) reproduced the failure, and verified the change >> eliminated the failure. >> > > looks good and trivial. > > Thomas Thanks. From kim.barrett at oracle.com Sun May 5 14:46:24 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 5 May 2019 10:46:24 -0400 Subject: RFR (XS): 8223267: Fix incorrect usage of GCTraceTime in g1FullCollector and g1CollectedHeap In-Reply-To: References: Message-ID: <43D58506-3CF5-4053-9657-0FDBD8085344@oracle.com> > On May 2, 2019, at 6:49 PM, Man Cao wrote: > > Hi all, > > Can I have reviews for this small bug fix for timer values in the log? > Webrev: https://cr.openjdk.java.net/~manc/8223267/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8223267 > > -Man Looks good, and trivial. From thomas.schatzl at oracle.com Sun May 5 14:56:51 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Sun, 05 May 2019 16:56:51 +0200 Subject: RFR (XS): 8223267: Fix incorrect usage of GCTraceTime in g1FullCollector and g1CollectedHeap In-Reply-To: References: Message-ID: Hi, On Thu, 2019-05-02 at 15:49 -0700, Man Cao wrote: > Hi all, > > Can I have reviews for this small bug fix for timer values in the > log? > Webrev: https://cr.openjdk.java.net/~manc/8223267/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8223267 > ship it. Thomas From erik.osterlund at oracle.com Sun May 5 20:32:18 2019 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Sun, 5 May 2019 22:32:18 +0200 Subject: RFR: 8223308: ZGC: Remove ZGranuleMap::size() In-Reply-To: <2af0f71d-a997-2340-c5f4-db8964ebd105@oracle.com> References: <2af0f71d-a997-2340-c5f4-db8964ebd105@oracle.com> Message-ID: Hi Per, Looks good. Thanks, /Erik > On 3 May 2019, at 14:13, Per Liden wrote: > > ZAddressOffsetMax is no longer a constant, so ZGranuleMap::size() will not be as nicely folded as it used to be. I suggest we instead calculate the size once and store it in a variable. This is not critical for performance, but I still think it's nice to do. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8223308 > Webrev: http://cr.openjdk.java.net/~pliden/8223308/webrev.0 > > /Per From thomas.schatzl at oracle.com Mon May 6 09:51:42 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 06 May 2019 11:51:42 +0200 Subject: RFR (M): 8200545: Improve filter for enqueued deferred cards In-Reply-To: <6ca33ce2-28cc-f736-676e-67f16c8f2dc2@oracle.com> References: <81ba1000d2c8eff1b4a467cd7ed75a372857244f.camel@oracle.com> <6ca33ce2-28cc-f736-676e-67f16c8f2dc2@oracle.com> Message-ID: <5b600e3a6116009c4e326bbee277622fc3650471.camel@oracle.com> Hi Sangheon, thanks for your review. On Tue, 2019-04-30 at 13:38 -0700, sangheon.kim at oracle.com wrote: > Hi Thomas, > > On 4/25/19 12:33 AM, Thomas Schatzl wrote: > > Hi all, > > > > ErikD had some improvement suggestions during an internal > > review: > > > > > > [...] > > http://cr.openjdk.java.net/~tschatzl/8200545/webrev.0_to_1/ > > http://cr.openjdk.java.net/~tschatzl/8200545/webrev.1/ > > Looks good. > I have some minor nits below: > > ------------------------------------------------------------------- > ---- > src/hotspot/share/gc/g1/g1CollectedHeap.cpp > 3330 "Only allowed InCSet state is IsHumongous, but is > %d", region_attr.type()); > - Should be G1HeapRegionAttr not InCSet. Fixed. > ------------------------------------------------------------------- > ---- > src/hotspot/share/gc/g1/g1CollectedHeap.hpp > 569 inline void register_humongous_region_with_region_attr(uint > index); > 578 inline void register_region_with_region_attr(HeapRegion* r); > 579 inline void register_old_region_with_region_attr(HeapRegion* > r); > 580 inline void > register_optional_region_with_region_attr(HeapRegion* r); > - (Just a suggestion) Only 569 has index parameter but looking at > the caller sites of other 3 methods, > only register_optional_region_with_region_attr() doesn't use > hrm_index at the caller site. So we can consider using hrm index as a > parameter. It was not that clear cut to me to use one or the other, and still is not. I will defer that issue for a different CR. > ------------------------------------------------------------------- > ---- > src/hotspot/share/gc/g1/g1CollectedHeap.inline.hpp > 175 } > 176 void > G1CollectedHeap::register_old_region_with_region_attr(HeapRegion* r) > { > - Need new line between 175 and 176. > > ------------------------------------------------------------------- > ---- > src/hotspot/share/gc/g1/g1OopClosures.hpp > 65 G1ScanCardClosure(G1CollectedHeap* g1h, > 66 G1ParScanThreadState* pss) : > - Wrong alignment between 65 and 66. > > ------------------------------------------------------------------- > ---- > src/hotspot/share/gc/g1/g1ParScanThreadState.hpp > 55 AgeTable _age_table; > 56 G1HeapRegionAttr _dest[G1HeapRegionAttr::NumGenInCSet]; > - Wrong alignment at line 56. Fixed. For the last one, I removed the alignment that was not completely applied anyway completely. http://cr.openjdk.java.net/~tschatzl/8200545/webrev.1_to_2/ http://cr.openjdk.java.net/~tschatzl/8200545/webrev.2/ (Test: local compilation) Thanks, Thomas From thomas.schatzl at oracle.com Mon May 6 10:08:53 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 06 May 2019 12:08:53 +0200 Subject: RFR (T): 8223018: Rename G1RemSet::*oops_into_collection_set_do methods In-Reply-To: References: Message-ID: Hi Man, Sangheon, On Fri, 2019-05-03 at 13:07 -0700, Man Cao wrote: > Looks good. > > -Man > > > From: > Date: Tue, Apr 30, 2019 at 1:41 PM > To: Thomas Schatzl > Cc: > > > Hi Thomas, > > > > On 4/26/19 1:29 AM, Thomas Schatzl wrote: > > > Hi all, > > > > > > can I have reviews for this change that fixes the > > > G1RemSet::*oops_into_collection_set_do methods to correspond to > > > the naming of the main method scan_rem_set again. > > > > > > CR: > > > https://bugs.openjdk.java.net/browse/JDK-8223018 > > > Webrev: > > > http://cr.openjdk.java.net/~tschatzl/8223018/webrev > > Looks good. > > thanks for your reviews. Thomas From thomas.schatzl at oracle.com Mon May 6 10:12:15 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 06 May 2019 12:12:15 +0200 Subject: RFR (xs): 8221509: Refactor assert( G1CollectedHeap::used() == recalculate_used() ) with better message In-Reply-To: References: Message-ID: <6b28c93cdc395768f15667a93915b62e3d0b94d9.camel@oracle.com> Hi, On Tue, 2019-04-30 at 13:54 -0700, sangheon.kim at oracle.com wrote: > Hi all, > > Can I have some reviews that replace assert(used() == > recalculated_used()) with better message? > Those assert messages are not much helpful as it doesn't include any > actual values on it so I'm suggesting to include actual values of > each and then replace as a macro. > > CR: https://bugs.openjdk.java.net/browse/JDK-8221509 > webrev: http://cr.openjdk.java.net/~sangheki/8221509/webrev.0/ > testing: local build thanks for fixing this :) The only nit I have is maybe the assert could include what is actually checked, ie. something like assert_used_and_recalculate_used_equal() instead of just assert_used_and_recalculate_used(). I would not need a re-review for the renaming. Thanks, Thomas From erik.osterlund at oracle.com Mon May 6 11:41:10 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 6 May 2019 13:41:10 +0200 Subject: RFR: JDK-8223244: Fix usage of ARRAYCOPY_DISJOINT decorator In-Reply-To: <4e0bbe62-0f1b-8ec7-d768-41ac60f08124@redhat.com> References: <4e0bbe62-0f1b-8ec7-d768-41ac60f08124@redhat.com> Message-ID: <8079a97d-0b89-7d9b-8b6d-0d269cb8a039@oracle.com> Hi Roman, This looks good to me. /Erik On 2019-05-02 17:19, Roman Kennke wrote: > ?In stubGenerator_x86_64.cpp, the ARRAYCOPY_DISJOINT decorator is > missing in one place, and wrongly added in another. This should be > fixed, because Shenandoah will rely on correct DISJOINT decorator > soon, when we implement JDK-8222859. > > Specifically, the conjoint routine pretends to be disjoint. The > checkcast routine omits the disjoint decorator, even though it is > always disjoint (otherwise it wouldn't need checkcast). > > Unfortunately, the fix is less trivial than it sounds. > ModRefBarrierSetAssembler's prologue implicitely assumes that > checkcast->!disjoint and gets the register shuffling wrong. Same for > ShenandoahBarrierSetAssembler. Hence the re-shuffling of that code. It > should be clearer/more explicit now, and equivalent to what we had > before. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8223244 > Webrev: > http://cr.openjdk.java.net/~rkennke/8223244/webrev.00/ > > Testing: hotspot/tier1, hotspot_gc_shenandoah both x86 and aarch64. > Intend to do jdk-submit before push. > > Can I please get a review? > > Thanks, > Roman From sangheon.kim at oracle.com Mon May 6 17:05:37 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Mon, 6 May 2019 10:05:37 -0700 Subject: RFR (xs): 8221509: Refactor assert( G1CollectedHeap::used() == recalculate_used() ) with better message In-Reply-To: References: Message-ID: Hi Man, On 5/3/19 1:18 PM, Man Cao wrote: > Hi Sangheon, > > The change looks good?to me. I'm not an official "Reviewer" though. Thanks for your review. Thomas S. suggested to rename to 'assert_used_and_recalculate_used_equal()' before pushing. No more webrev. :) Hope you are okay with that. Thanks, Sangheon > > -Man > > > *From: *> > *Date: *Tue, Apr 30, 2019 at 1:54 PM > *To: *hotspot-gc-dev > > Hi all, > > Can I have some reviews that replace assert(used() == > recalculated_used()) with better message? > Those assert messages are not much helpful as it doesn't include any > actual values on it so I'm suggesting to include actual values of > each > and then replace as a macro. > > CR: https://bugs.openjdk.java.net/browse/JDK-8221509 > webrev: http://cr.openjdk.java.net/~sangheki/8221509/webrev.0/ > > testing: local build > > Thanks, > Sangheon > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sangheon.kim at oracle.com Mon May 6 17:05:56 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Mon, 6 May 2019 10:05:56 -0700 Subject: RFR (xs): 8221509: Refactor assert( G1CollectedHeap::used() == recalculate_used() ) with better message In-Reply-To: <6b28c93cdc395768f15667a93915b62e3d0b94d9.camel@oracle.com> References: <6b28c93cdc395768f15667a93915b62e3d0b94d9.camel@oracle.com> Message-ID: <5b87ebb4-34de-6954-7a01-13a4c4d90f8b@oracle.com> Hi Thomas, On 5/6/19 3:12 AM, Thomas Schatzl wrote: > Hi, > > On Tue, 2019-04-30 at 13:54 -0700, sangheon.kim at oracle.com wrote: >> Hi all, >> >> Can I have some reviews that replace assert(used() == >> recalculated_used()) with better message? >> Those assert messages are not much helpful as it doesn't include any >> actual values on it so I'm suggesting to include actual values of >> each and then replace as a macro. >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8221509 >> webrev: http://cr.openjdk.java.net/~sangheki/8221509/webrev.0/ >> testing: local build > thanks for fixing this :) > > The only nit I have is maybe the assert could include what is actually > checked, ie. something like assert_used_and_recalculate_used_equal() > instead of just assert_used_and_recalculate_used(). > > I would not need a re-review for the renaming. Thanks for the review and the naming suggestion. I will rename for 'assert_used_and_recalculate_used_equal()' before the push. Thanks, Sangheon > > Thanks, > Thomas > From tprintezis at twitter.com Mon May 6 17:27:37 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Mon, 6 May 2019 13:27:37 -0400 Subject: RFR(XXS): 8223397: gc+promotion log lines are missing the GC id Message-ID: Trivial one-line change: http://cr.openjdk.java.net/~tonyp/8223397/webrev.0/ Was there a reason why gc+promotion was not added to LOG_PREFIX_LIST in the first place? Before: [59.899s][info ][gc,task ] GC(17) Using 18 workers of 18 for evacuation [61.934s][info ][gc,promotion ] Promotion failed [61.934s][info ][gc ] GC(17) Pause Young (Allocation Failure) 3948M->4555M(4557M) 2035.171ms After: [55.339s][info ][gc,task ] GC(17) Using 18 workers of 18 for evacuation [57.383s][info ][gc,promotion ] GC(17) Promotion failed [57.383s][info ][gc ] GC(17) Pause Young (Allocation Failure) 3944M->4556M(4557M) 2044.354ms Thanks, Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From manc at google.com Mon May 6 17:36:16 2019 From: manc at google.com (Man Cao) Date: Mon, 6 May 2019 10:36:16 -0700 Subject: RFR (xs): 8221509: Refactor assert( G1CollectedHeap::used() == recalculate_used() ) with better message In-Reply-To: <5b87ebb4-34de-6954-7a01-13a4c4d90f8b@oracle.com> References: <6b28c93cdc395768f15667a93915b62e3d0b94d9.camel@oracle.com> <5b87ebb4-34de-6954-7a01-13a4c4d90f8b@oracle.com> Message-ID: Renaming sounds good to me too, no need for another webrev. -Man *From: * *Date: *Mon, May 6, 2019 at 10:08 AM *To: *Thomas Schatzl *Cc: *hotspot-gc-dev Hi Thomas, > > On 5/6/19 3:12 AM, Thomas Schatzl wrote: > > Hi, > > > > On Tue, 2019-04-30 at 13:54 -0700, sangheon.kim at oracle.com wrote: > >> Hi all, > >> > >> Can I have some reviews that replace assert(used() == > >> recalculated_used()) with better message? > >> Those assert messages are not much helpful as it doesn't include any > >> actual values on it so I'm suggesting to include actual values of > >> each and then replace as a macro. > >> > >> CR: https://bugs.openjdk.java.net/browse/JDK-8221509 > >> webrev: http://cr.openjdk.java.net/~sangheki/8221509/webrev.0/ > >> testing: local build > > thanks for fixing this :) > > > > The only nit I have is maybe the assert could include what is actually > > checked, ie. something like assert_used_and_recalculate_used_equal() > > instead of just assert_used_and_recalculate_used(). > > > > I would not need a re-review for the renaming. > Thanks for the review and the naming suggestion. > I will rename for 'assert_used_and_recalculate_used_equal()' before the > push. > > Thanks, > Sangheon > > > > > > Thanks, > > Thomas > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From manc at google.com Mon May 6 17:37:34 2019 From: manc at google.com (Man Cao) Date: Mon, 6 May 2019 10:37:34 -0700 Subject: RFR (XS): 8223267: Fix incorrect usage of GCTraceTime in g1FullCollector and g1CollectedHeap In-Reply-To: References: Message-ID: Thanks for the reviews. -Man *From: *Thomas Schatzl *Date: *Sun, May 5, 2019 at 7:56 AM *To: *Man Cao, hotspot-gc-dev Hi, > > On Thu, 2019-05-02 at 15:49 -0700, Man Cao wrote: > > Hi all, > > > > Can I have reviews for this small bug fix for timer values in the > > log? > > Webrev: https://cr.openjdk.java.net/~manc/8223267/webrev.00/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8223267 > > > > ship it. > > Thomas > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Mon May 6 19:13:29 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 06 May 2019 21:13:29 +0200 Subject: RFR(XXS): 8223397: gc+promotion log lines are missing the GC id In-Reply-To: References: Message-ID: <9e179c509f281bfe4617dd176210cda20fd5aaea.camel@oracle.com> Hi, On Mon, 2019-05-06 at 13:27 -0400, Tony Printezis wrote: > Trivial one-line change: > > http://cr.openjdk.java.net/~tonyp/8223397/webrev.0/ > > Was there a reason why gc+promotion was not added to LOG_PREFIX_LIST > in the first place? > > > Before: > [...] > After: > [...] > looks good to me. Thomas From per.liden at oracle.com Mon May 6 21:08:54 2019 From: per.liden at oracle.com (Per Liden) Date: Mon, 6 May 2019 23:08:54 +0200 Subject: RFR: 8223308: ZGC: Remove ZGranuleMap::size() In-Reply-To: References: <2af0f71d-a997-2340-c5f4-db8964ebd105@oracle.com> Message-ID: <0c0038aa-6004-8602-c68f-f340b8f6588b@oracle.com> Thanks Erik! /Per On 05/05/2019 10:32 PM, Erik Osterlund wrote: > Hi Per, > > Looks good. > > Thanks, > /Erik > >> On 3 May 2019, at 14:13, Per Liden wrote: >> >> ZAddressOffsetMax is no longer a constant, so ZGranuleMap::size() will not be as nicely folded as it used to be. I suggest we instead calculate the size once and store it in a variable. This is not critical for performance, but I still think it's nice to do. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8223308 >> Webrev: http://cr.openjdk.java.net/~pliden/8223308/webrev.0 >> >> /Per > From per.liden at oracle.com Tue May 7 06:20:19 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 7 May 2019 08:20:19 +0200 Subject: RFR(XXS): 8223397: gc+promotion log lines are missing the GC id In-Reply-To: References: Message-ID: <2f9a87d3-599f-b286-5bfe-340c86687a79@oracle.com> Hi Tony, On 5/6/19 7:27 PM, Tony Printezis wrote: > Trivial one-line change: > > http://cr.openjdk.java.net/~tonyp/8223397/webrev.0/ > > Was there a reason why gc+promotion was not added to LOG_PREFIX_LIST in > the first place? I'm pretty sure that was just a mistake. Patch looks good. cheers, Per > > > Before: > > [59.899s][info ][gc,task? ? ? ? ? ] GC(17) Using 18 workers of 18 for > evacuation > > [61.934s][info ][gc,promotion ? ? ] Promotion failed > > [61.934s][info ][gc ? ? ? ? ? ? ? ] GC(17) Pause Young (Allocation > Failure) 3948M->4555M(4557M) 2035.171ms > > > > After: > > [55.339s][info ][gc,task? ? ? ? ? ] GC(17) Using 18 workers of 18 for > evacuation > > [57.383s][info ][gc,promotion ? ? ] GC(17) Promotion failed > > [57.383s][info ][gc ? ? ? ? ? ? ? ] GC(17) Pause Young (Allocation > Failure) 3944M->4556M(4557M) 2044.354ms > > > > Thanks, > > Tony > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > From per.liden at oracle.com Tue May 7 07:10:15 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 7 May 2019 09:10:15 +0200 Subject: RFR: JDK-8222537: Avoid fetching _klass twice in TypeArrayOop::size() In-Reply-To: <3d6e8ebf-aafc-92d4-4f5e-b4a1bca2a3b4@redhat.com> References: <5bad6271-2fb3-875f-d0d9-71bac167f768@redhat.com> <3d6e8ebf-aafc-92d4-4f5e-b4a1bca2a3b4@redhat.com> Message-ID: Hi Roman, I see you're pining, but I still haven't seen an answer to my initial question on this patch. cheers, Per On 5/3/19 6:59 PM, Roman Kennke wrote: > Ping? > > Roman > >> Currently, when calling TypeArrayOop::size(), we end up calling klass() >> twice: once before calling into size_given_klass() and then again before >> calling TypeArrayOop::object_size(). >> >> This is currently only a minor performance nuisance. >> >> With upcoming Shenandoah's elimination of forwarding pointer, loading >> klass like this is not safe anymore, and therefore we only call >> size_given_klass(), and must avoid calling naked klass() altogether. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8222537 >> Webrev: >> http://cr.openjdk.java.net/~rkennke/JDK-8222537/webrev.00/ >> Testing: >> hotspot_gc_shenandoah with and without the prototype, hotspot/tier1 >> >> Can I please get reviews? >> >> Thanks, >> Roman >> From shade at redhat.com Tue May 7 10:22:08 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 7 May 2019 12:22:08 +0200 Subject: RFR (XS) 8223448: Shenandoah disabled barriers blocks omit LRB Message-ID: Bug: https://bugs.openjdk.java.net/browse/JDK-8223448 This looks like a trivial omission, but it breaks x86_32 (when specifically enabled), which does not have all the barriers implemented yet. Fix: diff -r 3a75a903a488 src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp --- a/src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp Mon May 06 19:20:26 2019 +0200 +++ b/src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp Mon May 06 19:25:02 2019 +0200 @@ -45,6 +45,7 @@ FLAG_SET_DEFAULT(ShenandoahGCHeuristics, "passive"); FLAG_SET_DEFAULT(ShenandoahSATBBarrier, false); + FLAG_SET_DEFAULT(ShenandoahLoadRefBarrier, false); FLAG_SET_DEFAULT(ShenandoahKeepAliveBarrier, false); FLAG_SET_DEFAULT(ShenandoahStoreValEnqueueBarrier, false); FLAG_SET_DEFAULT(ShenandoahCASBarrier, false); @@ -106,6 +107,7 @@ // C2 barrier verification is only reliable when all default barriers are enabled if (ShenandoahVerifyOptoBarriers && (!FLAG_IS_DEFAULT(ShenandoahSATBBarrier) || + !FLAG_IS_DEFAULT(ShenandoahLoadRefBarrier) || !FLAG_IS_DEFAULT(ShenandoahKeepAliveBarrier) || !FLAG_IS_DEFAULT(ShenandoahStoreValEnqueueBarrier) || !FLAG_IS_DEFAULT(ShenandoahCASBarrier) || Testing: hotspot_gc_shenandoah -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Tue May 7 10:32:20 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 7 May 2019 12:32:20 +0200 Subject: RFR (XS) 8223449: Unprotected UseCompressedOops block in gc/shenandoah/shenandoahBarrierSetC1_x86.cpp Message-ID: Bug: https://bugs.openjdk.java.net/browse/JDK-8223449 x86_32 fails to build because of this (when specifically enabled). This is a simple overlook during recent refactoring of CAS barriers. Fix; diff -r edafae6da842 src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetC1_x86.cpp --- a/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetC1_x86.cpp Tue May 07 12:23:13 2019 +0200 +++ b/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetC1_x86.cpp Tue May 07 12:28:00 2019 +0200 @@ -44,16 +44,18 @@ // Apply storeval barrier to newval. ShenandoahBarrierSet::assembler()->storeval_barrier(masm->masm(), newval, tmp1); +#ifdef _LP64 if (UseCompressedOops) { __ encode_heap_oop(cmpval); __ mov(rscratch1, newval); __ encode_heap_oop(rscratch1); newval = rscratch1; } +#endif Testing: hotspot_gc_shenandoah -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Tue May 7 10:35:14 2019 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 7 May 2019 12:35:14 +0200 Subject: RFR (XS) 8223449: Unprotected UseCompressedOops block in gc/shenandoah/shenandoahBarrierSetC1_x86.cpp In-Reply-To: References: Message-ID: <2de42f45-dc99-a66c-6fd7-981f6c40229d@redhat.com> Yep, go! Roman > Bug: > https://bugs.openjdk.java.net/browse/JDK-8223449 > > x86_32 fails to build because of this (when specifically enabled). This is a simple overlook during > recent refactoring of CAS barriers. > > Fix; > > diff -r edafae6da842 src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetC1_x86.cpp > --- a/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetC1_x86.cpp Tue May 07 12:23:13 2019 +0200 > +++ b/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetC1_x86.cpp Tue May 07 12:28:00 2019 +0200 > @@ -44,16 +44,18 @@ > // Apply storeval barrier to newval. > ShenandoahBarrierSet::assembler()->storeval_barrier(masm->masm(), newval, tmp1); > > +#ifdef _LP64 > if (UseCompressedOops) { > __ encode_heap_oop(cmpval); > __ mov(rscratch1, newval); > __ encode_heap_oop(rscratch1); > newval = rscratch1; > } > +#endif > > Testing: hotspot_gc_shenandoah > From shade at redhat.com Tue May 7 10:44:14 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 7 May 2019 12:44:14 +0200 Subject: RFR (S) 8223447: Stabilize gc/shenandoah/TestStringDedupStress test Message-ID: Test bug: https://bugs.openjdk.java.net/browse/JDK-8223447 I find that test failing with OOME almost reliably on x86_32. There are several things that are done to stabilize the test: increased heap size, made sure the termination is time-based rather than overwrite-iterations-based (which unties test configuration from heap size), pre-sized the hashmap, added GC logging to simplify future investigations. Webrev: http://cr.openjdk.java.net/~shade/8223447/webrev.01/ Testing: hotspot_gc_shenandoah, eyeballing test logs to see that dedup happened in most modes -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Tue May 7 10:48:41 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 7 May 2019 12:48:41 +0200 Subject: RFR (S) 8223446: Shenandoah breaks alignment with some HumongousThreshold values Message-ID: <6d858dad-d65e-0487-5cc8-94ba150d7947@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8223446 HumongousThreshold is not aligned to MinObjAlignment. Since HumongousThreshold drives the MaxTLABSize, issuing such max-sized TLAB would leave allocation top unaligned. This readily manifests on x86_32. x86_64 is more or less safe as default alignment matches the single HeapWord. Fix: http://cr.openjdk.java.net/~shade/8223446/webrev.01/ This aligns all affected values, adds asserts, makes some computations use multiplication rather than division to avoid awkward precision issues, and extends the test to capture more problems. Testing: hotspot_gc_shenandoah (x86_32, x86_64) -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Tue May 7 12:24:17 2019 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 7 May 2019 14:24:17 +0200 Subject: RFR (S) 8223446: Shenandoah breaks alignment with some HumongousThreshold values In-Reply-To: <6d858dad-d65e-0487-5cc8-94ba150d7947@redhat.com> References: <6d858dad-d65e-0487-5cc8-94ba150d7947@redhat.com> Message-ID: Looks good, thank you! Roman > Bug: > https://bugs.openjdk.java.net/browse/JDK-8223446 > > HumongousThreshold is not aligned to MinObjAlignment. Since HumongousThreshold drives the > MaxTLABSize, issuing such max-sized TLAB would leave allocation top unaligned. This readily > manifests on x86_32. x86_64 is more or less safe as default alignment matches the single HeapWord. > > Fix: > http://cr.openjdk.java.net/~shade/8223446/webrev.01/ > > This aligns all affected values, adds asserts, makes some computations use multiplication rather > than division to avoid awkward precision issues, and extends the test to capture more problems. > > Testing: hotspot_gc_shenandoah (x86_32, x86_64) > From shade at redhat.com Tue May 7 13:59:39 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 7 May 2019 15:59:39 +0200 Subject: RFR: JDK-8223244: Fix usage of ARRAYCOPY_DISJOINT decorator In-Reply-To: <163d08ac-daaf-7979-87d1-37f86bf2d4cd@redhat.com> References: <4e0bbe62-0f1b-8ec7-d768-41ac60f08124@redhat.com> <52b79466-e553-8f7a-2106-68ab58710c3f@oracle.com> <7905a8b3-9884-76a2-9f61-9c18ed2c1e5d@redhat.com> <47b2a2b5-5328-3097-df1c-82a56d6243af@oracle.com> <163d08ac-daaf-7979-87d1-37f86bf2d4cd@redhat.com> Message-ID: On 5/3/19 4:40 PM, Roman Kennke wrote: > Right. Added a comment there, and also checked and fixed x86_32: > http://cr.openjdk.java.net/~rkennke/JDK-8223244/webrev.02/ Um. Old code: 39 if (!checkcast && !obj_int) { 40 // Save count for barrier 41 __ movptr(r11, count); 42 } else if (disjoint && obj_int) { 43 // Save dst in r11 in the disjoint case 44 __ movq(r11, dst); 45 } New code: 39 if (!checkcast) { 40 if (!obj_int) { 41 // Save count for barrier 42 __ movptr(r11, count); 43 } else if (disjoint) { 44 // Save dst in r11 in the disjoint case 45 __ movq(r11, dst); 46 } 47 } (Scribbles down truth tables). With checkcast=T, obj_int=T, disjoint=T old code saves "dst". New code does not save anything. Is that the fix for new Shenandoah code? -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Tue May 7 14:12:36 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 7 May 2019 16:12:36 +0200 Subject: RFR (XS) 8223449: Unprotected UseCompressedOops block in gc/shenandoah/shenandoahBarrierSetC1_x86.cpp In-Reply-To: <2de42f45-dc99-a66c-6fd7-981f6c40229d@redhat.com> References: <2de42f45-dc99-a66c-6fd7-981f6c40229d@redhat.com> Message-ID: Thanks, pushed under triviality rules. -Aleksey On 5/7/19 12:35 PM, Roman Kennke wrote: > Yep, go! > > Roman > > >> Bug: >> ?? https://bugs.openjdk.java.net/browse/JDK-8223449 >> >> x86_32 fails to build because of this (when specifically enabled). This is a simple overlook during >> recent refactoring of CAS barriers. >> >> Fix; >> >> diff -r edafae6da842 src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetC1_x86.cpp >> --- a/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetC1_x86.cpp? Tue May 07 12:23:13 2019 >> +0200 >> +++ b/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetC1_x86.cpp? Tue May 07 12:28:00 2019 >> +0200 >> @@ -44,16 +44,18 @@ >> ??? // Apply storeval barrier to newval. >> ??? ShenandoahBarrierSet::assembler()->storeval_barrier(masm->masm(), newval, tmp1); >> >> +#ifdef _LP64 >> ??? if (UseCompressedOops) { >> ????? __ encode_heap_oop(cmpval); >> ????? __ mov(rscratch1, newval); >> ????? __ encode_heap_oop(rscratch1); >> ????? newval = rscratch1; >> ??? } >> +#endif >> >> Testing: hotspot_gc_shenandoah >> -- Thanks, -Aleksey Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Michael O'Neill, Tom Savage, Eric Shander -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Tue May 7 14:13:20 2019 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 7 May 2019 16:13:20 +0200 Subject: RFR: JDK-8223244: Fix usage of ARRAYCOPY_DISJOINT decorator In-Reply-To: References: <4e0bbe62-0f1b-8ec7-d768-41ac60f08124@redhat.com> <52b79466-e553-8f7a-2106-68ab58710c3f@oracle.com> <7905a8b3-9884-76a2-9f61-9c18ed2c1e5d@redhat.com> <47b2a2b5-5328-3097-df1c-82a56d6243af@oracle.com> <163d08ac-daaf-7979-87d1-37f86bf2d4cd@redhat.com> Message-ID: <0caf9827-8a2d-85fa-1926-cfeca8006723@redhat.com> >> Right. Added a comment there, and also checked and fixed x86_32: >> http://cr.openjdk.java.net/~rkennke/JDK-8223244/webrev.02/ > > Um. Old code: > > 39 if (!checkcast && !obj_int) { > 40 // Save count for barrier > 41 __ movptr(r11, count); > 42 } else if (disjoint && obj_int) { > 43 // Save dst in r11 in the disjoint case > 44 __ movq(r11, dst); > 45 } > > New code: > > 39 if (!checkcast) { > 40 if (!obj_int) { > 41 // Save count for barrier > 42 __ movptr(r11, count); > 43 } else if (disjoint) { > 44 // Save dst in r11 in the disjoint case > 45 __ movq(r11, dst); > 46 } > 47 } > > (Scribbles down truth tables). With checkcast=T, obj_int=T, disjoint=T old code saves "dst". New > code does not save anything. Is that the fix for new Shenandoah code? With old code, checkcast and disjoint are never true at the same time. The checkcast generator doesn't do that, even though the arraycopy is implicitely disjoint (why would we need checkcast otherwise). We are now always asking checkcast=T && disjoint=T, and end up with the same configuration as previously with checkcast=T && disjoint=F. Agree? Roman From shade at redhat.com Tue May 7 14:20:45 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 7 May 2019 16:20:45 +0200 Subject: RFR: JDK-8223244: Fix usage of ARRAYCOPY_DISJOINT decorator In-Reply-To: <0caf9827-8a2d-85fa-1926-cfeca8006723@redhat.com> References: <4e0bbe62-0f1b-8ec7-d768-41ac60f08124@redhat.com> <52b79466-e553-8f7a-2106-68ab58710c3f@oracle.com> <7905a8b3-9884-76a2-9f61-9c18ed2c1e5d@redhat.com> <47b2a2b5-5328-3097-df1c-82a56d6243af@oracle.com> <163d08ac-daaf-7979-87d1-37f86bf2d4cd@redhat.com> <0caf9827-8a2d-85fa-1926-cfeca8006723@redhat.com> Message-ID: <503e61a5-dc56-c2a3-885e-3eda76bd67e5@redhat.com> On 5/7/19 4:13 PM, Roman Kennke wrote: >>> Right. Added a comment there, and also checked and fixed x86_32: >>> http://cr.openjdk.java.net/~rkennke/JDK-8223244/webrev.02/ >> >> Um. Old code: >> >> ?? 39???? if (!checkcast && !obj_int) { >> ?? 40?????? // Save count for barrier >> ?? 41?????? __ movptr(r11, count); >> ?? 42???? } else if (disjoint && obj_int) { >> ?? 43?????? // Save dst in r11 in the disjoint case >> ?? 44?????? __ movq(r11, dst); >> ?? 45???? } >> >> New code: >> >> ?? 39???? if (!checkcast) { >> ?? 40?????? if (!obj_int) { >> ?? 41???????? // Save count for barrier >> ?? 42???????? __ movptr(r11, count); >> ?? 43?????? } else if (disjoint) { >> ?? 44???????? // Save dst in r11 in the disjoint case >> ?? 45???????? __ movq(r11, dst); >> ?? 46?????? } >> ?? 47???? } >> >> (Scribbles down truth tables). With checkcast=T, obj_int=T, disjoint=T old code saves "dst". New >> code does not save anything. Is that the fix for new Shenandoah code? > > With old code, checkcast and disjoint are never true at the same time. The checkcast generator > doesn't do that, even though the arraycopy is implicitely disjoint (why would we need checkcast > otherwise). We are now always asking checkcast=T && disjoint=T, and end up with the same > configuration as previously with checkcast=T && disjoint=F. Agree? Agreed. Looks good then! -Aleksey -- Thanks, -Aleksey Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Michael O'Neill, Tom Savage, Eric Shander -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Tue May 7 18:50:45 2019 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 7 May 2019 20:50:45 +0200 Subject: RFR (S) 8223447: Stabilize gc/shenandoah/TestStringDedupStress test In-Reply-To: References: Message-ID: <66ad845a-b181-6f0a-4c27-3b0606759a6b@redhat.com> It looks good to me. Roman > Test bug: > https://bugs.openjdk.java.net/browse/JDK-8223447 > > I find that test failing with OOME almost reliably on x86_32. There are several things that are done > to stabilize the test: increased heap size, made sure the termination is time-based rather than > overwrite-iterations-based (which unties test configuration from heap size), pre-sized the hashmap, > added GC logging to simplify future investigations. > > Webrev: > http://cr.openjdk.java.net/~shade/8223447/webrev.01/ > > Testing: hotspot_gc_shenandoah, eyeballing test logs to see that dedup happened in most modes > From tprintezis at twitter.com Tue May 7 19:04:39 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Tue, 7 May 2019 12:04:39 -0700 Subject: RFR(XXS): 8223397: gc+promotion log lines are missing the GC id In-Reply-To: <2f9a87d3-599f-b286-5bfe-340c86687a79@oracle.com> References: <2f9a87d3-599f-b286-5bfe-340c86687a79@oracle.com> Message-ID: Thomas and Per, Thanks! I?ll submit it to the submit repo, just in case (and to refresh my memory on how to do that), and I?ll push it after I get the results... Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On May 7, 2019 at 2:20:25 AM, Per Liden (per.liden at oracle.com) wrote: Hi Tony, On 5/6/19 7:27 PM, Tony Printezis wrote: > Trivial one-line change: > > http://cr.openjdk.java.net/~tonyp/8223397/webrev.0/ > > Was there a reason why gc+promotion was not added to LOG_PREFIX_LIST in > the first place? I'm pretty sure that was just a mistake. Patch looks good. cheers, Per > > > Before: > > [59.899s][info ][gc,task ] GC(17) Using 18 workers of 18 for > evacuation > > [61.934s][info ][gc,promotion ] Promotion failed > > [61.934s][info ][gc ] GC(17) Pause Young (Allocation > Failure) 3948M->4555M(4557M) 2035.171ms > > > > After: > > [55.339s][info ][gc,task ] GC(17) Using 18 workers of 18 for > evacuation > > [57.383s][info ][gc,promotion ] GC(17) Promotion failed > > [57.383s][info ][gc ] GC(17) Pause Young (Allocation > Failure) 3944M->4556M(4557M) 2044.354ms > > > > Thanks, > > Tony > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rkennke at redhat.com Tue May 7 21:05:54 2019 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 7 May 2019 23:05:54 +0200 Subject: RFR: JDK-8223244: Fix usage of ARRAYCOPY_DISJOINT decorator In-Reply-To: <8079a97d-0b89-7d9b-8b6d-0d269cb8a039@oracle.com> References: <4e0bbe62-0f1b-8ec7-d768-41ac60f08124@redhat.com> <8079a97d-0b89-7d9b-8b6d-0d269cb8a039@oracle.com> Message-ID: <811b6579-8fbd-f324-62a5-0c514658bad5@redhat.com> Thanks, Erik! Roman > Hi Roman, > > This looks good to me. > > /Erik > > On 2019-05-02 17:19, Roman Kennke wrote: >> ?In stubGenerator_x86_64.cpp, the ARRAYCOPY_DISJOINT decorator is >> missing in one place, and wrongly added in another. This should be >> fixed, because Shenandoah will rely on correct DISJOINT decorator >> soon, when we implement JDK-8222859. >> >> Specifically, the conjoint routine pretends to be disjoint. The >> checkcast routine omits the disjoint decorator, even though it is >> always disjoint (otherwise it wouldn't need checkcast). >> >> Unfortunately, the fix is less trivial than it sounds. >> ModRefBarrierSetAssembler's prologue implicitely assumes that >> checkcast->!disjoint and gets the register shuffling wrong. Same for >> ShenandoahBarrierSetAssembler. Hence the re-shuffling of that code. It >> should be clearer/more explicit now, and equivalent to what we had >> before. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8223244 >> Webrev: >> http://cr.openjdk.java.net/~rkennke/8223244/webrev.00/ >> >> Testing: hotspot/tier1, hotspot_gc_shenandoah both x86 and aarch64. >> Intend to do jdk-submit before push. >> >> Can I please get a review? >> >> Thanks, >> Roman > From rkennke at redhat.com Tue May 7 21:06:23 2019 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 7 May 2019 23:06:23 +0200 Subject: RFR: JDK-8223244: Fix usage of ARRAYCOPY_DISJOINT decorator In-Reply-To: <503e61a5-dc56-c2a3-885e-3eda76bd67e5@redhat.com> References: <4e0bbe62-0f1b-8ec7-d768-41ac60f08124@redhat.com> <52b79466-e553-8f7a-2106-68ab58710c3f@oracle.com> <7905a8b3-9884-76a2-9f61-9c18ed2c1e5d@redhat.com> <47b2a2b5-5328-3097-df1c-82a56d6243af@oracle.com> <163d08ac-daaf-7979-87d1-37f86bf2d4cd@redhat.com> <0caf9827-8a2d-85fa-1926-cfeca8006723@redhat.com> <503e61a5-dc56-c2a3-885e-3eda76bd67e5@redhat.com> Message-ID: <06bc0ed7-4f05-b850-fb8a-066398e60d8b@redhat.com> >>>> Right. Added a comment there, and also checked and fixed x86_32: >>>> http://cr.openjdk.java.net/~rkennke/JDK-8223244/webrev.02/ >>> >>> Um. Old code: >>> >>> ?? 39???? if (!checkcast && !obj_int) { >>> ?? 40?????? // Save count for barrier >>> ?? 41?????? __ movptr(r11, count); >>> ?? 42???? } else if (disjoint && obj_int) { >>> ?? 43?????? // Save dst in r11 in the disjoint case >>> ?? 44?????? __ movq(r11, dst); >>> ?? 45???? } >>> >>> New code: >>> >>> ?? 39???? if (!checkcast) { >>> ?? 40?????? if (!obj_int) { >>> ?? 41???????? // Save count for barrier >>> ?? 42???????? __ movptr(r11, count); >>> ?? 43?????? } else if (disjoint) { >>> ?? 44???????? // Save dst in r11 in the disjoint case >>> ?? 45???????? __ movq(r11, dst); >>> ?? 46?????? } >>> ?? 47???? } >>> >>> (Scribbles down truth tables). With checkcast=T, obj_int=T, disjoint=T old code saves "dst". New >>> code does not save anything. Is that the fix for new Shenandoah code? >> >> With old code, checkcast and disjoint are never true at the same time. The checkcast generator >> doesn't do that, even though the arraycopy is implicitely disjoint (why would we need checkcast >> otherwise). We are now always asking checkcast=T && disjoint=T, and end up with the same >> configuration as previously with checkcast=T && disjoint=F. Agree? > > Agreed. Looks good then! Thanks! Waiting on green light from jdk-submit now before pushing. Roman From aoqi at loongson.cn Wed May 8 08:43:48 2019 From: aoqi at loongson.cn (Ao Qi) Date: Wed, 8 May 2019 16:43:48 +0800 Subject: RFR: JDK-8222926: Unify cpu related part of shenandoahBarrierSetC1 and fix build when compiler1 is disabled In-Reply-To: References: Message-ID: Hi, The patch [1] dose not apply cleanly again because of JDK-8223449. And src/hotspot/share/jvmci/vmStructs_jvmci.cpp needs to be updated. Here is a new version: http://cr.openjdk.java.net/~aoqi/8222926/webrev.02/ Could I please get reviews? (cc'ed shenandoah-dev) Thanks, Ao Qi [1] http://cr.openjdk.java.net/~aoqi/8222926/webrev.01/ On Fri, Apr 26, 2019 at 3:59 PM Ao Qi wrote: > > Hi, > > The patch [1] dose not apply cleanly any more, because of the same > copyright update in JDK-8222811 [2]. I updated a new version to > exclude this copyright modification: > > http://cr.openjdk.java.net/~aoqi/8222926/webrev.01/ > > Could I please get reviews for this? > > Thanks, > Ao Qi > > [1] http://cr.openjdk.java.net/~aoqi/8222926/webrev.00/ > [2] http://hg.openjdk.java.net/jdk/jdk/diff/1126f0607c70/src/hotspot/share/code/codeBlob.cpp#l1.6 > > > On Wed, Apr 24, 2019 at 9:56 PM Ao Qi wrote: > > > > Hi, > > > > When I configure using --with-jvm-features=-compiler1, I found build > > is broken. The main cause is that some shenandoah related part did not > > consider the situation when compiler1 is disabled. I also found that > > src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetC1_aarch64.cpp > > and src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetC1_x86.cpp > > were basically the same, with only small differences. I unified these > > two files and moved the result to a c1 related file. I also fixed one > > "#undef" only when COMPILER1 is true in > > shenandoahBarrierSetAssembler_x86/aarch64.cpp and one missed include > > in codeBlob.cpp. > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8222926/ > > > > Webrev: > > http://cr.openjdk.java.net/~aoqi/8222926/webrev.00/ > > > > Tested: > > - build: linux-x86_64-{server, zero, minimal}-{release, fastdebug}, > > linux-aarch64-server-release > > - jtreg: linux-x86_64-server-release tier1, > > linux-aarch64-server-release tier1_gc_shenandoah > > > > Thanks, > > Ao Qi From shade at redhat.com Wed May 8 08:49:59 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 8 May 2019 10:49:59 +0200 Subject: RFR: JDK-8222926: Unify cpu related part of shenandoahBarrierSetC1 and fix build when compiler1 is disabled In-Reply-To: References: Message-ID: <75d6106a-f162-73b6-c1c3-c622f47329fc@redhat.com> On 5/8/19 10:43 AM, Ao Qi wrote: > The patch [1] dose not apply cleanly again because of JDK-8223449. And > src/hotspot/share/jvmci/vmStructs_jvmci.cpp needs to be updated. Here > is a new version: > > http://cr.openjdk.java.net/~aoqi/8222926/webrev.02/ I understand the need for fixing -compiler1 configuration. I am, however, against cobbling together x86 and aarch64 code with sprinkled #ifdefs. It would be another story if the code was indeed the same, but they are not. What is the fix for -compiler1 build issue? Can it be done on its own? -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From aoqi at loongson.cn Wed May 8 09:26:29 2019 From: aoqi at loongson.cn (Ao Qi) Date: Wed, 8 May 2019 17:26:29 +0800 Subject: RFR: JDK-8222926: Unify cpu related part of shenandoahBarrierSetC1 and fix build when compiler1 is disabled In-Reply-To: <75d6106a-f162-73b6-c1c3-c622f47329fc@redhat.com> References: <75d6106a-f162-73b6-c1c3-c622f47329fc@redhat.com> Message-ID: On Wed, May 8, 2019 at 4:49 PM Aleksey Shipilev wrote: > > On 5/8/19 10:43 AM, Ao Qi wrote: > > The patch [1] dose not apply cleanly again because of JDK-8223449. And > > src/hotspot/share/jvmci/vmStructs_jvmci.cpp needs to be updated. Here > > is a new version: > > > > http://cr.openjdk.java.net/~aoqi/8222926/webrev.02/ > > I understand the need for fixing -compiler1 configuration. I am, however, against cobbling together > x86 and aarch64 code with sprinkled #ifdefs. It would be another story if the code was indeed the > same, but they are not. > > What is the fix for -compiler1 build issue? Can it be done on its own? Thanks for your reply, Aleksey. If we want to fix -compiler1 build without cobbling together x86 and aarch64 code, I think one way is to put all of the code in shenandoahBarrierSetC1_x86.cpp and shenandoahBarrierSetC1_aarch64.cpp into #ifdef COMPILER1 ... #endif. What do you think about this method? > > -Aleksey > From shade at redhat.com Wed May 8 09:40:55 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 8 May 2019 11:40:55 +0200 Subject: RFR: JDK-8222926: Unify cpu related part of shenandoahBarrierSetC1 and fix build when compiler1 is disabled In-Reply-To: References: <75d6106a-f162-73b6-c1c3-c622f47329fc@redhat.com> Message-ID: <61a3211c-21c5-340b-5823-3825cba20243@redhat.com> On 5/8/19 11:26 AM, Ao Qi wrote: > If we want to fix -compiler1 build without cobbling together x86 and > aarch64 code, I think one way is to put all of the code in > shenandoahBarrierSetC1_x86.cpp and shenandoahBarrierSetC1_aarch64.cpp > into #ifdef COMPILER1 ... #endif. What do you think about this method? That would be fine with me! Better yet, there are exclusion lists in JvmFeature.gmk: ifeq ($(call check-jvm-feature, compiler1), true) JVM_CFLAGS_FEATURES += -DCOMPILER1 else JVM_EXCLUDE_PATTERNS += c1_ c1/ endif So the fix might just amount to moving: ./src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetC1_x86.cpp -> ./src/hotspot/cpu/x86/gc/shenandoah/c1/shenandoahBarrierSetC1_x86.cpp ...and ./src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetC1_aarch64.cpp -> ./src/hotspot/cpu/aarch64/gc/shenandoah/c1/shenandoahBarrierSetC1_aarch64.cpp -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed May 8 10:27:39 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 8 May 2019 12:27:39 +0200 Subject: RFR: JDK-8223567: Rename ShenandoahBrooksPointer to ShenandoahForwarding Message-ID: <1f163407-e8e2-14c6-4e28-578f8bdc624f@redhat.com> In preparation of elimination of the brooks pointer slot, I'd like to rename ShenandoahBrooksPointer to ShenandoahForwarding. This name seems more appropriate then. I also renamed some of the accessors to make more sense in the new context. Bug: https://bugs.openjdk.java.net/browse/JDK-8223567 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8223567/webrev.00/ Testing: hotspot_gc_shenandoah (x86,aarch64), also with and without --disable-precompiled-headers Ok? Roman From shade at redhat.com Wed May 8 11:30:49 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 8 May 2019 13:30:49 +0200 Subject: RFR: JDK-8223567: Rename ShenandoahBrooksPointer to ShenandoahForwarding In-Reply-To: <1f163407-e8e2-14c6-4e28-578f8bdc624f@redhat.com> References: <1f163407-e8e2-14c6-4e28-578f8bdc624f@redhat.com> Message-ID: <097da3c1-80a2-4e32-d7b4-49c6132a2988@redhat.com> On 5/8/19 12:27 PM, Roman Kennke wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8223567 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8223567/webrev.00/ This looks good to me. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Wed May 8 11:39:00 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 8 May 2019 13:39:00 +0200 Subject: RFR (XS) 8223570: Shenandoah needs to acquire lock before CLDG::clear_claimed_marks Message-ID: <10143339-41cb-e1ad-4131-a8c2d680aaeb@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8223570 New assert fails and exposes this legit GC bug. Shenandoah does CLDG::clear_claimed_marks in two places, one under the safepoint (which is fine for that assert), and another one in concurrent thread, which fails. Fix: diff -r 845f5a35241b src/hotspot/share/gc/shenandoah/shenandoahTraversalGC.cpp --- a/src/hotspot/share/gc/shenandoah/shenandoahTraversalGC.cpp Wed May 08 06:40:45 2019 -0400 +++ b/src/hotspot/share/gc/shenandoah/shenandoahTraversalGC.cpp Wed May 08 13:36:56 2019 +0200 @@ -552,11 +552,14 @@ } void ShenandoahTraversalGC::concurrent_traversal_collection() { - ClassLoaderDataGraph::clear_claimed_marks(); + { + MutexLocker ml(ClassLoaderDataGraph_lock); + ClassLoaderDataGraph::clear_claimed_marks(); + } ShenandoahGCPhase phase_work(ShenandoahPhaseTimings::conc_traversal); if (!_heap->cancelled_gc()) { Testing: hotspot_gc_shenandoah (running) -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From aoqi at loongson.cn Wed May 8 11:52:08 2019 From: aoqi at loongson.cn (Ao Qi) Date: Wed, 8 May 2019 19:52:08 +0800 Subject: RFR: JDK-8222926: Unify cpu related part of shenandoahBarrierSetC1 and fix build when compiler1 is disabled In-Reply-To: <61a3211c-21c5-340b-5823-3825cba20243@redhat.com> References: <75d6106a-f162-73b6-c1c3-c622f47329fc@redhat.com> <61a3211c-21c5-340b-5823-3825cba20243@redhat.com> Message-ID: On Wed, May 8, 2019 at 5:40 PM Aleksey Shipilev wrote: > > On 5/8/19 11:26 AM, Ao Qi wrote: > > If we want to fix -compiler1 build without cobbling together x86 and > > aarch64 code, I think one way is to put all of the code in > > shenandoahBarrierSetC1_x86.cpp and shenandoahBarrierSetC1_aarch64.cpp > > into #ifdef COMPILER1 ... #endif. What do you think about this method? > > That would be fine with me! > > Better yet, there are exclusion lists in JvmFeature.gmk: > > ifeq ($(call check-jvm-feature, compiler1), true) > JVM_CFLAGS_FEATURES += -DCOMPILER1 > else > JVM_EXCLUDE_PATTERNS += c1_ c1/ > endif Didn't know that. Thanks! > > So the fix might just amount to moving: > ./src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetC1_x86.cpp > -> > ./src/hotspot/cpu/x86/gc/shenandoah/c1/shenandoahBarrierSetC1_x86.cpp > > ...and > ./src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetC1_aarch64.cpp > -> > ./src/hotspot/cpu/aarch64/gc/shenandoah/c1/shenandoahBarrierSetC1_aarch64.cpp Done that. Please see: http://cr.openjdk.java.net/~aoqi/8222926/webrev.03/ I also updated https://bugs.openjdk.java.net/browse/JDK-8222926. Tested: build: linux-x86_64-{server, zero, minimal}-{release, fastdebug}, linux-aarch64-server-release build (-compiler1): linux-x86_64-server-release, linux-aarch64-server-release hotspot:tier1 linux-x86_64-server-{release,fastdebug} Thanks, Ao Qi > > -Aleksey > From thomas.schatzl at oracle.com Wed May 8 12:07:16 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 08 May 2019 14:07:16 +0200 Subject: RFR (XS) 8223570: Shenandoah needs to acquire lock before CLDG::clear_claimed_marks In-Reply-To: <10143339-41cb-e1ad-4131-a8c2d680aaeb@redhat.com> References: <10143339-41cb-e1ad-4131-a8c2d680aaeb@redhat.com> Message-ID: <4e9b02867ca48cafb7ef88ac55e538ee6d774926.camel@oracle.com> Hi, On Wed, 2019-05-08 at 13:39 +0200, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8223570 > > New assert fails and exposes this legit GC bug. Shenandoah does > CLDG::clear_claimed_marks in two places, one under the safepoint > (which is fine for that assert), and another one in concurrent > thread, which fails. > > Fix: > [...] fix looks good. Sorry for not thinking about Shenandoah too when reviewing the G1 fix. Thanks, Thomas From shade at redhat.com Wed May 8 12:28:08 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 8 May 2019 14:28:08 +0200 Subject: RFR (XS) 8223570: Shenandoah needs to acquire lock before CLDG::clear_claimed_marks In-Reply-To: <45573a8b-53ed-2ea7-be15-0626e6ff0057@kennke.org> References: <10143339-41cb-e1ad-4131-a8c2d680aaeb@redhat.com> <45573a8b-53ed-2ea7-be15-0626e6ff0057@kennke.org> Message-ID: On 5/8/19 2:11 PM, Roman Kennke wrote: > (how has that not failed before? Also, we shall revisit this clear_claimed_marks in traversal. It > should not be necessary and only extend the traversal.) Yes. New assert, recently committed. It probably fails silently otherwise. Pushed. > Roman > >> Bug: >> ?? https://bugs.openjdk.java.net/browse/JDK-8223570 >> >> New assert fails and exposes this legit GC bug. Shenandoah does CLDG::clear_claimed_marks in two >> places, one under the safepoint (which is fine for that assert), and another one in concurrent >> thread, which fails. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Wed May 8 12:30:41 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 8 May 2019 14:30:41 +0200 Subject: RFR: JDK-8222926: Unify cpu related part of shenandoahBarrierSetC1 and fix build when compiler1 is disabled In-Reply-To: References: <75d6106a-f162-73b6-c1c3-c622f47329fc@redhat.com> <61a3211c-21c5-340b-5823-3825cba20243@redhat.com> Message-ID: On 5/8/19 1:52 PM, Ao Qi wrote: > http://cr.openjdk.java.net/~aoqi/8222926/webrev.03/ Webrev says the files were deleted, then added. This loses history. I think you need "hg mv -A" them or something. Or, let me move the files myself, and then you would apply the rest of build fixes? -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From aoqi at loongson.cn Wed May 8 12:47:16 2019 From: aoqi at loongson.cn (Ao Qi) Date: Wed, 8 May 2019 20:47:16 +0800 Subject: RFR: JDK-8222926: Unify cpu related part of shenandoahBarrierSetC1 and fix build when compiler1 is disabled In-Reply-To: References: <75d6106a-f162-73b6-c1c3-c622f47329fc@redhat.com> <61a3211c-21c5-340b-5823-3825cba20243@redhat.com> Message-ID: On Wed, May 8, 2019 at 8:30 PM Aleksey Shipilev wrote: > > On 5/8/19 1:52 PM, Ao Qi wrote: > > http://cr.openjdk.java.net/~aoqi/8222926/webrev.03/ > > Webrev says the files were deleted, then added. This loses history. I think you need "hg mv -A" them > or something. Or, let me move the files myself, and then you would apply the rest of build fixes? Sorry for that. Is this one right? http://cr.openjdk.java.net/~aoqi/8222926/webrev.04/ If not, maybe I need your help. Cheers, Ao Qi > > -Aleksey > From shade at redhat.com Wed May 8 12:52:16 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 8 May 2019 14:52:16 +0200 Subject: RFR: JDK-8222926: Unify cpu related part of shenandoahBarrierSetC1 and fix build when compiler1 is disabled In-Reply-To: References: <75d6106a-f162-73b6-c1c3-c622f47329fc@redhat.com> <61a3211c-21c5-340b-5823-3825cba20243@redhat.com> Message-ID: <4cb3d53c-8990-542d-dc7b-2d70979a23b4@redhat.com> On 5/8/19 2:47 PM, Ao Qi wrote: > http://cr.openjdk.java.net/~aoqi/8222926/webrev.04/ Yes, this looks right. To be extra sure, ask Mercurial if it knows the file history: $ hg log -f src/hotspot/cpu/aarch64/gc/shenandoah/c1/shenandoahBarrierSetC1_aarch64.cpp $ hg log src/hotspot/cpu/x86/gc/shenandoah/c1/shenandoahBarrierSetC1_x86.cpp ...it should go back all the way to: changeset: 52925:9c18c9d839d3 user: rkennke date: Mon Dec 10 15:47:44 2018 +0100 summary: 8214259: Implementation: JEP 189: Shenandoah: A Low-Pause-Time Garbage Collector (Experimental) I changed the synopsis for the bug to: "Shenandoah build fails with --with-jvm-features=-compiler1" -- please change it for the changeset you are pushing. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From aoqi at loongson.cn Wed May 8 13:05:00 2019 From: aoqi at loongson.cn (Ao Qi) Date: Wed, 8 May 2019 21:05:00 +0800 Subject: RFR: JDK-8222926: Unify cpu related part of shenandoahBarrierSetC1 and fix build when compiler1 is disabled In-Reply-To: <4cb3d53c-8990-542d-dc7b-2d70979a23b4@redhat.com> References: <75d6106a-f162-73b6-c1c3-c622f47329fc@redhat.com> <61a3211c-21c5-340b-5823-3825cba20243@redhat.com> <4cb3d53c-8990-542d-dc7b-2d70979a23b4@redhat.com> Message-ID: On Wed, May 8, 2019 at 8:52 PM Aleksey Shipilev wrote: > > On 5/8/19 2:47 PM, Ao Qi wrote: > > http://cr.openjdk.java.net/~aoqi/8222926/webrev.04/ > > Yes, this looks right. To be extra sure, ask Mercurial if it knows the file history: > > $ hg log -f src/hotspot/cpu/aarch64/gc/shenandoah/c1/shenandoahBarrierSetC1_aarch64.cpp > $ hg log src/hotspot/cpu/x86/gc/shenandoah/c1/shenandoahBarrierSetC1_x86.cpp > > ...it should go back all the way to: > > changeset: 52925:9c18c9d839d3 > user: rkennke > date: Mon Dec 10 15:47:44 2018 +0100 > summary: 8214259: Implementation: JEP 189: Shenandoah: A Low-Pause-Time Garbage Collector > (Experimental) $ hg log -f src/hotspot/cpu/aarch64/gc/shenandoah/c1/shenandoahBarrierSetC1_aarch64.cpp changeset: 54760:a1a250da545f tag: tip user: aoqi date: Wed May 08 20:57:12 2019 +0800 summary: 8222926: Shenandoah build fails with --with-jvm-features=-compiler1 changeset: 54423:6c0ab8bd8da5 user: rkennke date: Tue Apr 02 23:00:22 2019 +0200 summary: 8221766: Load-reference barriers for Shenandoah changeset: 53519:74a5ef4c81cc user: rkennke date: Sat Jan 26 01:21:33 2019 +0100 summary: 8217016: Shenandoah: Streamline generation of CAS barriers changeset: 52925:9c18c9d839d3 user: rkennke date: Mon Dec 10 15:47:44 2018 +0100 summary: 8214259: Implementation: JEP 189: Shenandoah: A Low-Pause-Time Garbage Collector (Experimental) $ hg log src/hotspot/cpu/x86/gc/shenandoah/c1/shenandoahBarrierSetC1_x86.cpp changeset: 54760:a1a250da545f tag: tip user: aoqi date: Wed May 08 20:57:12 2019 +0800 summary: 8222926: Shenandoah build fails with --with-jvm-features=-compiler1 $ hg log src/hotspot/cpu/aarch64/gc/shenandoah/c1/shenandoahBarrierSetC1_aarch64.cpp changeset: 54760:a1a250da545f tag: tip user: aoqi date: Wed May 08 20:57:12 2019 +0800 summary: 8222926: Shenandoah build fails with --with-jvm-features=-compiler1 $ hg log -f src/hotspot/cpu/aarch64/gc/shenandoah/c1/shenandoahBarrierSetC1_aarch64.cpp changeset: 54760:a1a250da545f tag: tip user: aoqi date: Wed May 08 20:57:12 2019 +0800 summary: 8222926: Shenandoah build fails with --with-jvm-features=-compiler1 changeset: 54423:6c0ab8bd8da5 user: rkennke date: Tue Apr 02 23:00:22 2019 +0200 summary: 8221766: Load-reference barriers for Shenandoah changeset: 53519:74a5ef4c81cc user: rkennke date: Sat Jan 26 01:21:33 2019 +0100 summary: 8217016: Shenandoah: Streamline generation of CAS barriers changeset: 52925:9c18c9d839d3 user: rkennke date: Mon Dec 10 15:47:44 2018 +0100 summary: 8214259: Implementation: JEP 189: Shenandoah: A Low-Pause-Time Garbage Collector (Experimental) $ hg summ parent: 54760:a1a250da545f tip 8222926: Shenandoah build fails with --with-jvm-features=-compiler1 branch: default commit: 350 unknown (clean) update: (current) phases: 1 draft > > I changed the synopsis for the bug to: "Shenandoah build fails with --with-jvm-features=-compiler1" > -- please change it for the changeset you are pushing. Updated: http://cr.openjdk.java.net/~aoqi/8222926/webrev.05/ I am not a Committers. I need a sponsor:( Thanks, Ao Qi > > -Aleksey > From shade at redhat.com Wed May 8 13:32:51 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 8 May 2019 15:32:51 +0200 Subject: RFR: JDK-8222926: Unify cpu related part of shenandoahBarrierSetC1 and fix build when compiler1 is disabled In-Reply-To: References: <75d6106a-f162-73b6-c1c3-c622f47329fc@redhat.com> <61a3211c-21c5-340b-5823-3825cba20243@redhat.com> <4cb3d53c-8990-542d-dc7b-2d70979a23b4@redhat.com> Message-ID: On 5/8/19 3:05 PM, Ao Qi wrote: >> I changed the synopsis for the bug to: "Shenandoah build fails with --with-jvm-features=-compiler1" >> -- please change it for the changeset you are pushing. > > Updated: http://cr.openjdk.java.net/~aoqi/8222926/webrev.05/ > > I am not a Committers. I need a sponsor:( I will sponsor. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From aoqi at loongson.cn Wed May 8 13:39:32 2019 From: aoqi at loongson.cn (Ao Qi) Date: Wed, 8 May 2019 21:39:32 +0800 Subject: RFR: JDK-8222926: Unify cpu related part of shenandoahBarrierSetC1 and fix build when compiler1 is disabled In-Reply-To: References: <75d6106a-f162-73b6-c1c3-c622f47329fc@redhat.com> <61a3211c-21c5-340b-5823-3825cba20243@redhat.com> <4cb3d53c-8990-542d-dc7b-2d70979a23b4@redhat.com> Message-ID: On Wed, May 8, 2019 at 9:32 PM Aleksey Shipilev wrote: > > On 5/8/19 3:05 PM, Ao Qi wrote: > >> I changed the synopsis for the bug to: "Shenandoah build fails with --with-jvm-features=-compiler1" > >> -- please change it for the changeset you are pushing. > > > > Updated: http://cr.openjdk.java.net/~aoqi/8222926/webrev.05/ > > > > I am not a Committers. I need a sponsor:( > > I will sponsor. Thank you very much! > > -Aleksey > From shade at redhat.com Wed May 8 13:49:25 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 8 May 2019 15:49:25 +0200 Subject: RFR: JDK-8222926: Unify cpu related part of shenandoahBarrierSetC1 and fix build when compiler1 is disabled In-Reply-To: References: <75d6106a-f162-73b6-c1c3-c622f47329fc@redhat.com> <61a3211c-21c5-340b-5823-3825cba20243@redhat.com> <4cb3d53c-8990-542d-dc7b-2d70979a23b4@redhat.com> Message-ID: <18f4ac22-6e90-9bcb-5cdf-73c7431afaf3@redhat.com> On 5/8/19 3:39 PM, Ao Qi wrote: > On Wed, May 8, 2019 at 9:32 PM Aleksey Shipilev wrote: >> >> On 5/8/19 3:05 PM, Ao Qi wrote: >>>> I changed the synopsis for the bug to: "Shenandoah build fails with --with-jvm-features=-compiler1" >>>> -- please change it for the changeset you are pushing. >>> >>> Updated: http://cr.openjdk.java.net/~aoqi/8222926/webrev.05/ >>> >>> I am not a Committers. I need a sponsor:( >> >> I will sponsor. The patch does not pass my local builds without precompiled headers and compiler1. This is also needed, agree? diff -r 0e3ee2061ea7 src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.hpp --- a/src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.hpp Wed May 08 20:57:12 2019 +0800 +++ b/src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.hpp Wed May 08 15:46:04 2019 +0200 @@ -29,12 +29,12 @@ #ifdef COMPILER1 class LIR_Assembler; class ShenandoahPreBarrierStub; class ShenandoahLoadReferenceBarrierStub; class StubAssembler; +#endif class StubCodeGenerator; -#endif class ShenandoahBarrierSetAssembler: public BarrierSetAssembler { private: static address _shenandoah_lrb; diff -r 0e3ee2061ea7 src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.hpp --- a/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.hpp Wed May 08 20:57:12 2019 +0800 +++ b/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.hpp Wed May 08 15:46:04 2019 +0200 @@ -29,12 +29,12 @@ #ifdef COMPILER1 class LIR_Assembler; class ShenandoahPreBarrierStub; class ShenandoahLoadReferenceBarrierStub; class StubAssembler; +#endif class StubCodeGenerator; -#endif class ShenandoahBarrierSetAssembler: public BarrierSetAssembler { private: static address _shenandoah_lrb; -- -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed May 8 14:10:16 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 8 May 2019 16:10:16 +0200 Subject: RFR: JDK-8222926: Unify cpu related part of shenandoahBarrierSetC1 and fix build when compiler1 is disabled In-Reply-To: References: <75d6106a-f162-73b6-c1c3-c622f47329fc@redhat.com> <61a3211c-21c5-340b-5823-3825cba20243@redhat.com> <4cb3d53c-8990-542d-dc7b-2d70979a23b4@redhat.com> Message-ID: <3f51025a-fa57-44a4-dab3-cb8cfd48f85e@redhat.com> >> I changed the synopsis for the bug to: "Shenandoah build fails with --with-jvm-features=-compiler1" >> -- please change it for the changeset you are pushing. > > Updated: http://cr.openjdk.java.net/~aoqi/8222926/webrev.05/ This looks good. Thanks! Roman From tprintezis at twitter.com Wed May 8 14:20:18 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Wed, 8 May 2019 07:20:18 -0700 Subject: RFR(S): 8223575: add subspace transitions to gc+metaspace=info log lines Message-ID: Any chance of someone taking a look at this? http://cr.openjdk.java.net/~tonyp/8223575/webrev.0/ As I said on the JIRA, users can be confused when they metaspace capacity shows as over 1G even though they asked for 128M. Breaking down the spaces, when coops are enabled, can address this confusion (and having the actual occupancy of each space separately is also a lot more informative and avoids guesswork). The output looks like this when coops are enabled: [10.090s][info][gc,metaspace ] GC(8) Metaspace: 2130K->2130K(1056768K) ClassMetadata: 1906K->1906K(8192K) ClassSpace: 223K->223K(1048576K) and like this (i.e., what it was before the change) when coops are not enabled: [25.767s][info][gc,metaspace ] GC(28) Metaspace: 8107K->8107K(10240K) A couple of notes on the changes: - I?m open to different suggestions for what to call ClassMetadata and ClassSpace. :-) - I didn?t know where to put the HEAP_CHANGE_FORMAT and HEAP_CHANGE_FORMAT_ARGS macros, so that they can be easily shared, and I ended up putting them in heap.hpp. Let me know if there?s a better place for them. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From aoqi at loongson.cn Wed May 8 15:02:14 2019 From: aoqi at loongson.cn (Ao Qi) Date: Wed, 8 May 2019 23:02:14 +0800 Subject: RFR: JDK-8222926: Unify cpu related part of shenandoahBarrierSetC1 and fix build when compiler1 is disabled In-Reply-To: <18f4ac22-6e90-9bcb-5cdf-73c7431afaf3@redhat.com> References: <75d6106a-f162-73b6-c1c3-c622f47329fc@redhat.com> <61a3211c-21c5-340b-5823-3825cba20243@redhat.com> <4cb3d53c-8990-542d-dc7b-2d70979a23b4@redhat.com> <18f4ac22-6e90-9bcb-5cdf-73c7431afaf3@redhat.com> Message-ID: On Wed, May 8, 2019 at 9:49 PM Aleksey Shipilev wrote: > > On 5/8/19 3:39 PM, Ao Qi wrote: > > On Wed, May 8, 2019 at 9:32 PM Aleksey Shipilev wrote: > >> > >> On 5/8/19 3:05 PM, Ao Qi wrote: > >>>> I changed the synopsis for the bug to: "Shenandoah build fails with --with-jvm-features=-compiler1" > >>>> -- please change it for the changeset you are pushing. > >>> > >>> Updated: http://cr.openjdk.java.net/~aoqi/8222926/webrev.05/ > >>> > >>> I am not a Committers. I need a sponsor:( > >> > >> I will sponsor. > > The patch does not pass my local builds without precompiled headers and compiler1. This is also > needed, agree? Agreed. Sorry for not testing that. Updated: http://cr.openjdk.java.net/~aoqi/8222926/webrev.06/ Thanks, Ao Qi > > diff -r 0e3ee2061ea7 src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.hpp > --- a/src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.hpp Wed May 08 > 20:57:12 2019 +0800 > +++ b/src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.hpp Wed May 08 > 15:46:04 2019 +0200 > @@ -29,12 +29,12 @@ > #ifdef COMPILER1 > class LIR_Assembler; > class ShenandoahPreBarrierStub; > class ShenandoahLoadReferenceBarrierStub; > class StubAssembler; > +#endif > class StubCodeGenerator; > -#endif > > class ShenandoahBarrierSetAssembler: public BarrierSetAssembler { > private: > > static address _shenandoah_lrb; > diff -r 0e3ee2061ea7 src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.hpp > --- a/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.hpp Wed May 08 20:57:12 > 2019 +0800 > +++ b/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.hpp Wed May 08 15:46:04 > 2019 +0200 > @@ -29,12 +29,12 @@ > #ifdef COMPILER1 > class LIR_Assembler; > class ShenandoahPreBarrierStub; > class ShenandoahLoadReferenceBarrierStub; > class StubAssembler; > +#endif > class StubCodeGenerator; > -#endif > > class ShenandoahBarrierSetAssembler: public BarrierSetAssembler { > private: > > static address _shenandoah_lrb; > > -- > -Aleksey > From aoqi at loongson.cn Wed May 8 15:04:52 2019 From: aoqi at loongson.cn (Ao Qi) Date: Wed, 8 May 2019 23:04:52 +0800 Subject: RFR: JDK-8222926: Unify cpu related part of shenandoahBarrierSetC1 and fix build when compiler1 is disabled In-Reply-To: <3f51025a-fa57-44a4-dab3-cb8cfd48f85e@redhat.com> References: <75d6106a-f162-73b6-c1c3-c622f47329fc@redhat.com> <61a3211c-21c5-340b-5823-3825cba20243@redhat.com> <4cb3d53c-8990-542d-dc7b-2d70979a23b4@redhat.com> <3f51025a-fa57-44a4-dab3-cb8cfd48f85e@redhat.com> Message-ID: On Wed, May 8, 2019 at 10:10 PM Roman Kennke wrote: > > >> I changed the synopsis for the bug to: "Shenandoah build fails with --with-jvm-features=-compiler1" > >> -- please change it for the changeset you are pushing. > > > > Updated: http://cr.openjdk.java.net/~aoqi/8222926/webrev.05/ > > This looks good. Thanks! Thanks, Roman. I updated a new version. Do you agree with Aleksey and are you ok with the new version? Ao Qi > Roman From shade at redhat.com Wed May 8 15:10:20 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 8 May 2019 17:10:20 +0200 Subject: RFR: JDK-8222926: Unify cpu related part of shenandoahBarrierSetC1 and fix build when compiler1 is disabled In-Reply-To: References: <75d6106a-f162-73b6-c1c3-c622f47329fc@redhat.com> <61a3211c-21c5-340b-5823-3825cba20243@redhat.com> <4cb3d53c-8990-542d-dc7b-2d70979a23b4@redhat.com> <18f4ac22-6e90-9bcb-5cdf-73c7431afaf3@redhat.com> Message-ID: <06f822d8-f5c9-3184-49cc-f6b159257670@redhat.com> On 5/8/19 5:02 PM, Ao Qi wrote: > Agreed. Sorry for not testing that. Updated: > http://cr.openjdk.java.net/~aoqi/8222926/webrev.06/ Yup, I'll put it through some testing and push on success. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed May 8 15:14:23 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 8 May 2019 17:14:23 +0200 Subject: RFR: JDK-8222926: Unify cpu related part of shenandoahBarrierSetC1 and fix build when compiler1 is disabled In-Reply-To: References: <75d6106a-f162-73b6-c1c3-c622f47329fc@redhat.com> <61a3211c-21c5-340b-5823-3825cba20243@redhat.com> <4cb3d53c-8990-542d-dc7b-2d70979a23b4@redhat.com> <3f51025a-fa57-44a4-dab3-cb8cfd48f85e@redhat.com> Message-ID: <9dcbd794-15c2-dad6-22ce-c21250308e23@redhat.com> >>>> I changed the synopsis for the bug to: "Shenandoah build fails with --with-jvm-features=-compiler1" >>>> -- please change it for the changeset you are pushing. >>> >>> Updated: http://cr.openjdk.java.net/~aoqi/8222926/webrev.05/ >> >> This looks good. Thanks! > > Thanks, Roman. I updated a new version. Do you agree with Aleksey and > are you ok with the new version? Yep, good! Thanks, Roman From tprintezis at twitter.com Wed May 8 16:08:40 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Wed, 8 May 2019 09:08:40 -0700 Subject: RFR(S): 8223575: add subspace transitions to gc+metaspace=info log lines In-Reply-To: References: Message-ID: BTW, I got the results back from jdk/submit and it failed to build on three platforms. Is there a way to find out what the issue is? All I know is ?return value: 2?. Thanks, Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On May 8, 2019 at 10:20:18 AM, Tony Printezis (tprintezis at twitter.com) wrote: Any chance of someone taking a look at this? http://cr.openjdk.java.net/~tonyp/8223575/webrev.0/ As I said on the JIRA, users can be confused when they metaspace capacity shows as over 1G even though they asked for 128M. Breaking down the spaces, when coops are enabled, can address this confusion (and having the actual occupancy of each space separately is also a lot more informative and avoids guesswork). The output looks like this when coops are enabled: [10.090s][info][gc,metaspace ] GC(8) Metaspace: 2130K->2130K(1056768K) ClassMetadata: 1906K->1906K(8192K) ClassSpace: 223K->223K(1048576K) and like this (i.e., what it was before the change) when coops are not enabled: [25.767s][info][gc,metaspace ] GC(28) Metaspace: 8107K->8107K(10240K) A couple of notes on the changes: - I?m open to different suggestions for what to call ClassMetadata and ClassSpace. :-) - I didn?t know where to put the HEAP_CHANGE_FORMAT and HEAP_CHANGE_FORMAT_ARGS macros, so that they can be easily shared, and I ended up putting them in heap.hpp. Let me know if there?s a better place for them. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From shade at redhat.com Wed May 8 16:10:24 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 8 May 2019 18:10:24 +0200 Subject: RFR: JDK-8222926: Unify cpu related part of shenandoahBarrierSetC1 and fix build when compiler1 is disabled In-Reply-To: <06f822d8-f5c9-3184-49cc-f6b159257670@redhat.com> References: <75d6106a-f162-73b6-c1c3-c622f47329fc@redhat.com> <61a3211c-21c5-340b-5823-3825cba20243@redhat.com> <4cb3d53c-8990-542d-dc7b-2d70979a23b4@redhat.com> <18f4ac22-6e90-9bcb-5cdf-73c7431afaf3@redhat.com> <06f822d8-f5c9-3184-49cc-f6b159257670@redhat.com> Message-ID: <7a8892d4-97cb-a8d6-0c1c-05012abf71a9@redhat.com> On 5/8/19 5:10 PM, Aleksey Shipilev wrote: > On 5/8/19 5:02 PM, Ao Qi wrote: >> Agreed. Sorry for not testing that. Updated: >> http://cr.openjdk.java.net/~aoqi/8222926/webrev.06/ > > Yup, I'll put it through some testing and push on success. Tests passed, pushed: http://hg.openjdk.java.net/jdk/jdk/rev/2584e5772546 -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From stefan.karlsson at oracle.com Wed May 8 18:58:54 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 8 May 2019 20:58:54 +0200 Subject: RFR(S): 8223575: add subspace transitions to gc+metaspace=info log lines In-Reply-To: References: Message-ID: Hi Tony, On 2019-05-08 18:08, Tony Printezis wrote: > BTW, I got the results back from jdk/submit and it failed to build on > three platforms. Is there a way to find out what the issue is? All I > know is ?return value: 2?. I don't know of a way for external contributors to see the logs from the submit repo builds. It seems like your failure happened because of a mismatch between the changeset your patch is based on and Oracle's closed build bits. I think a rebase will fix it. StefanK > > Thanks, > > Tony > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > > On May 8, 2019 at 10:20:18 AM, Tony Printezis (tprintezis at twitter.com > ) wrote: > >> Any chance of someone taking a look at this? >> >> http://cr.openjdk.java.net/~tonyp/8223575/webrev.0/ >> >> As I said on the JIRA, users can be confused when they metaspace >> capacity shows as over 1G even though they asked for 128M. Breaking >> down the spaces, when coops are enabled, can address this confusion >> (and having the actual occupancy of each space separately is also a >> lot more informative and avoids guesswork). The output looks like >> this when coops are enabled: >> >> [10.090s][info][gc,metaspace ? ? ] GC(8) Metaspace: >> 2130K->2130K(1056768K) ClassMetadata: 1906K->1906K(8192K) ClassSpace: >> 223K->223K(1048576K) >> >> >> and like this (i.e., what it was before the change) when coops are >> not enabled: >> >> [25.767s][info][gc,metaspace ? ? ] GC(28) Metaspace: 8107K->8107K(10240K) >> >> >> A couple of notes on the changes: >> >> - I?m open to different suggestions for what to call ClassMetadata >> and ClassSpace. :-) >> - I didn?t know where to put the HEAP_CHANGE_FORMAT and >> HEAP_CHANGE_FORMAT_ARGS macros, so that they can be easily shared, >> and I ended up putting them in heap.hpp. Let me know if there?s a >> better place for them. >> >> Tony >> >> >> ????? >> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >> >> From shade at redhat.com Wed May 8 19:30:25 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 8 May 2019 21:30:25 +0200 Subject: [URGENT] RFR (XS) 8223583: Build failure after JDK-8223567 (Rename ShenandoahBrooksPointer to ShenandoahForwarding) Message-ID: <62e24abc-8fb0-b805-323f-94b8cdc34640@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8223583 This is a merge blunder: the code below was added recently, and Roman probably missed this when pushing to jdk/jdk. Fix: diff -r 6ccc7cd7931e src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp --- a/src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp Wed May 08 11:56:19 2019 -0700 +++ b/src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp Wed May 08 21:27:49 2019 +0200 @@ -925,11 +925,11 @@ template void do_oop_work(T* p) { T o = RawAccess<>::oop_load(p); if (!CompressedOops::is_null(o)) { oop obj = CompressedOops::decode_not_null(o); - oop fwd = (oop) ShenandoahBrooksPointer::get_raw_unchecked(obj); + oop fwd = (oop) ShenandoahForwarding::get_forwardee_raw_unchecked(obj); if (!oopDesc::equals_raw(obj, fwd)) { ShenandoahAsserts::print_failure(ShenandoahAsserts::_safe_all, obj, p, NULL, "Verify Roots", "Should not be forwarded", __FILE__, __LINE__); } } Testing: Linux x86_64 build -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed May 8 19:31:56 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 8 May 2019 21:31:56 +0200 Subject: [URGENT] RFR (XS) 8223583: Build failure after JDK-8223567 (Rename ShenandoahBrooksPointer to ShenandoahForwarding) In-Reply-To: <62e24abc-8fb0-b805-323f-94b8cdc34640@redhat.com> References: <62e24abc-8fb0-b805-323f-94b8cdc34640@redhat.com> Message-ID: Ok. Goes under emergency-fix & triviality. :-) Roman > Bug: > https://bugs.openjdk.java.net/browse/JDK-8223583 > > This is a merge blunder: the code below was added recently, and Roman probably missed this when > pushing to jdk/jdk. > > Fix: > > diff -r 6ccc7cd7931e src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp > --- a/src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp Wed May 08 11:56:19 2019 -0700 > +++ b/src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp Wed May 08 21:27:49 2019 +0200 > @@ -925,11 +925,11 @@ > template > void do_oop_work(T* p) { > T o = RawAccess<>::oop_load(p); > if (!CompressedOops::is_null(o)) { > oop obj = CompressedOops::decode_not_null(o); > - oop fwd = (oop) ShenandoahBrooksPointer::get_raw_unchecked(obj); > + oop fwd = (oop) ShenandoahForwarding::get_forwardee_raw_unchecked(obj); > if (!oopDesc::equals_raw(obj, fwd)) { > ShenandoahAsserts::print_failure(ShenandoahAsserts::_safe_all, obj, p, NULL, > "Verify Roots", "Should not be forwarded", __FILE__, > __LINE__); > } > } > > Testing: Linux x86_64 build > From shade at redhat.com Wed May 8 19:33:46 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 8 May 2019 21:33:46 +0200 Subject: [URGENT] RFR (XS) 8223583: Build failure after JDK-8223567 (Rename ShenandoahBrooksPointer to ShenandoahForwarding) In-Reply-To: References: <62e24abc-8fb0-b805-323f-94b8cdc34640@redhat.com> Message-ID: <7c78aedc-f1ff-2b9c-92c3-6e2c28cc879e@redhat.com> Thanks, pushed. -Aleksey On 5/8/19 9:31 PM, Roman Kennke wrote: > Ok. Goes under emergency-fix & triviality. :-) > > Roman > >> Bug: >> ?? https://bugs.openjdk.java.net/browse/JDK-8223583 >> >> This is a merge blunder: the code below was added recently, and Roman probably missed this when >> pushing to jdk/jdk. >> >> Fix: >> >> diff -r 6ccc7cd7931e src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp >> --- a/src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp??? Wed May 08 11:56:19 2019 -0700 >> +++ b/src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp??? Wed May 08 21:27:49 2019 +0200 >> @@ -925,11 +925,11 @@ >> ??? template >> ??? void do_oop_work(T* p) { >> ????? T o = RawAccess<>::oop_load(p); >> ????? if (!CompressedOops::is_null(o)) { >> ??????? oop obj = CompressedOops::decode_not_null(o); >> -????? oop fwd = (oop) ShenandoahBrooksPointer::get_raw_unchecked(obj); >> +????? oop fwd = (oop) ShenandoahForwarding::get_forwardee_raw_unchecked(obj); >> ??????? if (!oopDesc::equals_raw(obj, fwd)) { >> ????????? ShenandoahAsserts::print_failure(ShenandoahAsserts::_safe_all, obj, p, NULL, >> ?????????????????????????????????????????? "Verify Roots", "Should not be forwarded", __FILE__, >> __LINE__); >> ??????? } >> ????? } >> >> Testing: Linux x86_64 build >> -- Thanks, -Aleksey Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Michael O'Neill, Tom Savage, Eric Shander -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From sangheon.kim at oracle.com Wed May 8 19:35:00 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Wed, 8 May 2019 12:35:00 -0700 Subject: RFR (M): 8200545: Improve filter for enqueued deferred cards In-Reply-To: <5b600e3a6116009c4e326bbee277622fc3650471.camel@oracle.com> References: <81ba1000d2c8eff1b4a467cd7ed75a372857244f.camel@oracle.com> <6ca33ce2-28cc-f736-676e-67f16c8f2dc2@oracle.com> <5b600e3a6116009c4e326bbee277622fc3650471.camel@oracle.com> Message-ID: Hi Thomas, On 5/6/19 2:51 AM, Thomas Schatzl wrote: > Hi Sangheon, > > thanks for your review. > > On Tue, 2019-04-30 at 13:38 -0700, sangheon.kim at oracle.com wrote: >> Hi Thomas, >> >> On 4/25/19 12:33 AM, Thomas Schatzl wrote: >>> Hi all, >>> >>> ErikD had some improvement suggestions during an internal >>> review: >>> >>> >>> [...] >>> http://cr.openjdk.java.net/~tschatzl/8200545/webrev.0_to_1/ >>> http://cr.openjdk.java.net/~tschatzl/8200545/webrev.1/ >> Looks good. >> I have some minor nits below: >> >> ------------------------------------------------------------------- >> ---- >> src/hotspot/share/gc/g1/g1CollectedHeap.cpp >> 3330 "Only allowed InCSet state is IsHumongous, but is >> %d", region_attr.type()); >> - Should be G1HeapRegionAttr not InCSet. > Fixed. > >> ------------------------------------------------------------------- >> ---- >> src/hotspot/share/gc/g1/g1CollectedHeap.hpp >> 569 inline void register_humongous_region_with_region_attr(uint >> index); >> 578 inline void register_region_with_region_attr(HeapRegion* r); >> 579 inline void register_old_region_with_region_attr(HeapRegion* >> r); >> 580 inline void >> register_optional_region_with_region_attr(HeapRegion* r); >> - (Just a suggestion) Only 569 has index parameter but looking at >> the caller sites of other 3 methods, >> only register_optional_region_with_region_attr() doesn't use >> hrm_index at the caller site. So we can consider using hrm index as a >> parameter. > It was not that clear cut to me to use one or the other, and still is > not. I will defer that issue for a different CR. Okay. > >> ------------------------------------------------------------------- >> ---- >> src/hotspot/share/gc/g1/g1CollectedHeap.inline.hpp >> 175 } >> 176 void >> G1CollectedHeap::register_old_region_with_region_attr(HeapRegion* r) >> { >> - Need new line between 175 and 176. >> >> ------------------------------------------------------------------- >> ---- >> src/hotspot/share/gc/g1/g1OopClosures.hpp >> 65 G1ScanCardClosure(G1CollectedHeap* g1h, >> 66 G1ParScanThreadState* pss) : >> - Wrong alignment between 65 and 66. >> >> ------------------------------------------------------------------- >> ---- >> src/hotspot/share/gc/g1/g1ParScanThreadState.hpp >> 55 AgeTable _age_table; >> 56 G1HeapRegionAttr _dest[G1HeapRegionAttr::NumGenInCSet]; >> - Wrong alignment at line 56. > Fixed. For the last one, I removed the alignment that was not > completely applied anyway completely. > > http://cr.openjdk.java.net/~tschatzl/8200545/webrev.1_to_2/ > http://cr.openjdk.java.net/~tschatzl/8200545/webrev.2/ Looks good to me. Thanks, Sangheon > (Test: local compilation) > > Thanks, > Thomas > > From tprintezis at twitter.com Wed May 8 20:00:57 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Wed, 8 May 2019 13:00:57 -0700 Subject: RFR(S): 8223575: add subspace transitions to gc+metaspace=info log lines In-Reply-To: References: Message-ID: Thanks for taking a look Stefan! Trying again? Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On May 8, 2019 at 2:59:01 PM, Stefan Karlsson (stefan.karlsson at oracle.com) wrote: Hi Tony, On 2019-05-08 18:08, Tony Printezis wrote: > BTW, I got the results back from jdk/submit and it failed to build on > three platforms. Is there a way to find out what the issue is? All I > know is ?return value: 2?. I don't know of a way for external contributors to see the logs from the submit repo builds. It seems like your failure happened because of a mismatch between the changeset your patch is based on and Oracle's closed build bits. I think a rebase will fix it. StefanK > > Thanks, > > Tony > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > > On May 8, 2019 at 10:20:18 AM, Tony Printezis (tprintezis at twitter.com > ) wrote: > >> Any chance of someone taking a look at this? >> >> http://cr.openjdk.java.net/~tonyp/8223575/webrev.0/ >> >> As I said on the JIRA, users can be confused when they metaspace >> capacity shows as over 1G even though they asked for 128M. Breaking >> down the spaces, when coops are enabled, can address this confusion >> (and having the actual occupancy of each space separately is also a >> lot more informative and avoids guesswork). The output looks like >> this when coops are enabled: >> >> [10.090s][info][gc,metaspace ] GC(8) Metaspace: >> 2130K->2130K(1056768K) ClassMetadata: 1906K->1906K(8192K) ClassSpace: >> 223K->223K(1048576K) >> >> >> and like this (i.e., what it was before the change) when coops are >> not enabled: >> >> [25.767s][info][gc,metaspace ] GC(28) Metaspace: 8107K->8107K(10240K) >> >> >> A couple of notes on the changes: >> >> - I?m open to different suggestions for what to call ClassMetadata >> and ClassSpace. :-) >> - I didn?t know where to put the HEAP_CHANGE_FORMAT and >> HEAP_CHANGE_FORMAT_ARGS macros, so that they can be easily shared, >> and I ended up putting them in heap.hpp. Let me know if there?s a >> better place for them. >> >> Tony >> >> >> ????? >> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.stuefe at gmail.com Wed May 8 20:07:36 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 8 May 2019 22:07:36 +0200 Subject: RFR(S): 8223575: add subspace transitions to gc+metaspace=info log lines In-Reply-To: References: Message-ID: Hi Tony, I think this makes sense. I agree, the output as it is now can be confusing. Note that users can still be confused since users may simply not be aware that the default ClassSpace reserved size is 1G. Can your change break scripts for folks parsing GC logs? I see that you only appended content and did not change the existing text, so I guess this is okay. --- Format and naming: Different forms are used in different places. In hs-err files, we print "Metaspace" and "Class Space", with the former being the total sum and the latter the class space part. But in other corners, eg. in the jcmd VM.metaspace command, we print non-class space and class space and maybe the sum in addition. Usually non-class space is called "Non-Class". I personally would prefer, in this case, the latter names: "Class Space" and "Non-Class Space" --- Note that for non-class space, reserved size can actually change when allocating and releasing metaspace, if VirtualSpaceNodes get purged or newly allocated. Hence total reserved size can change too. So we may want to print out reserved before and reserved after sizes. I guess that was also missing from the output before. ---- Since used = class used + nonclass used, I would simplify PreMetaspaceValues to _non_class_used and _class_used and calculate the sum on the fly. --- You added a new dependency to heap.hpp into metaspace.cpp for the format string. Is this really necessary? Can you move this define to metaspace.cpp? If not, you need at least add include heap.hpp (did it build this way without precompiled headers?) ---- The following issues are matters of taste, and I leave it up to you whether you want to change something: -> I wince a bit at yet another metaspace-sizes-structure - we have already ClassLoaderMetaspaceStatistics and MetaspaceCounters, and maybe others. JFR may also do something similar. Would it be possible to reuse one of those? (I guess ClassLoaderMetaspaceStatistics is too fine granular and too expensive to collect.) -> I would prefer the class PreMetaspaceValues to live inside an own header in metaspace folder, and in the metaspace namespace like we did for similar helpers. -> I would take the "Pre" out of the name and the member names and just name it "Snapshot" and the members "used" and "class used". -> I would probably have made PreMetaspaceValues a dumb structure and added a function "MetaspaceAux::get_dimensions()" instead of adding the implementation to the constructor and to the header file. At least I would prefer the ctor implementation to the cpp file. Cheers, Thomas On Wed, May 8, 2019 at 4:20 PM Tony Printezis wrote: > Any chance of someone taking a look at this? > > http://cr.openjdk.java.net/~tonyp/8223575/webrev.0/ > > As I said on the JIRA, users can be confused when they metaspace capacity > shows as over 1G even though they asked for 128M. Breaking down the spaces, > when coops are enabled, can address this confusion (and having the actual > occupancy of each space separately is also a lot more informative and > avoids guesswork). The output looks like this when coops are enabled: > > [10.090s][info][gc,metaspace ] GC(8) Metaspace: 2130K->2130K(1056768K) > ClassMetadata: 1906K->1906K(8192K) ClassSpace: 223K->223K(1048576K) > > and like this (i.e., what it was before the change) when coops are not > enabled: > > [25.767s][info][gc,metaspace ] GC(28) Metaspace: 8107K->8107K(10240K) > > A couple of notes on the changes: > > - I?m open to different suggestions for what to call ClassMetadata and > ClassSpace. :-) > - I didn?t know where to put the HEAP_CHANGE_FORMAT and > HEAP_CHANGE_FORMAT_ARGS macros, so that they can be easily shared, and I > ended up putting them in heap.hpp. Let me know if there?s a better place > for them. > > Tony > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aoqi at loongson.cn Thu May 9 01:06:31 2019 From: aoqi at loongson.cn (Ao Qi) Date: Thu, 9 May 2019 09:06:31 +0800 Subject: RFR: JDK-8222926: Unify cpu related part of shenandoahBarrierSetC1 and fix build when compiler1 is disabled In-Reply-To: <7a8892d4-97cb-a8d6-0c1c-05012abf71a9@redhat.com> References: <75d6106a-f162-73b6-c1c3-c622f47329fc@redhat.com> <61a3211c-21c5-340b-5823-3825cba20243@redhat.com> <4cb3d53c-8990-542d-dc7b-2d70979a23b4@redhat.com> <18f4ac22-6e90-9bcb-5cdf-73c7431afaf3@redhat.com> <06f822d8-f5c9-3184-49cc-f6b159257670@redhat.com> <7a8892d4-97cb-a8d6-0c1c-05012abf71a9@redhat.com> Message-ID: On Thu, May 9, 2019 at 12:10 AM Aleksey Shipilev wrote: > > On 5/8/19 5:10 PM, Aleksey Shipilev wrote: > > On 5/8/19 5:02 PM, Ao Qi wrote: > >> Agreed. Sorry for not testing that. Updated: > >> http://cr.openjdk.java.net/~aoqi/8222926/webrev.06/ > > > > Yup, I'll put it through some testing and push on success. > > Tests passed, pushed: > http://hg.openjdk.java.net/jdk/jdk/rev/2584e5772546 Thank you, Aleksey! > > -Aleksey > From kim.barrett at oracle.com Thu May 9 04:42:14 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 9 May 2019 00:42:14 -0400 Subject: RFR(M): 8220089: G1 wastes a significant amount of space in survivor regions In-Reply-To: <2dea4528-8c8b-76ab-d737-ad36d36dab0a@oracle.com> References: <11f5af23-bacf-267d-7e81-8cd408ae063b@oracle.com> <45151a57-4689-21e6-4bfb-a884d6b6c14d@oracle.com> <174f169f-4ccb-7567-7707-d6a7f7024aa8@oracle.com> <2dea4528-8c8b-76ab-d737-ad36d36dab0a@oracle.com> Message-ID: <3B084725-AAEC-4D78-9AC7-A16492E01613@oracle.com> > On Apr 30, 2019, at 2:06 AM, sangheon.kim at oracle.com wrote: > > Hi all, > > I got some comments from Thomas Schatzl, mostly about comment and assert changes. He also suggested to investigate combining current 2 steps of handling retaining survivor region into 1 step, so I filed JDK-8223106. > > Webrev: > http://cr.openjdk.java.net/~sangheki/8220089/webrev.2/ > http://cr.openjdk.java.net/~sangheki/8220089/webrev.2_inc/ > Testing: hs-tier1 ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1AllocRegion.cpp 367 bool SurvivorGCAllocRegion::should_retain(HeapRegion* region) { 368 if (region == NULL || region->free() < MinTLABSize) { 369 return false; 370 } 371 return true; 372 } Why does should_retain accept a NULL region? ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1AllocRegion.hpp 270 SurvivorGCAllocRegion(G1EvacStats* stats) 271 : G1GCAllocRegion("Survivor GC Alloc Region", false /* bot_updates */, stats, InCSetState::Young) { } 272 static bool should_retain(HeapRegion* region); Maybe add a blank line before should_retain declaration. ------------------------------------------------------------------------------ [discussed this offline with Sangheon and Thomas; mentioning it here in case anyone else is following email.] "root regions" representation was changed from an array of HeapRegion* to an array of HeapWord*, where the values are the captured "top" values from the corresponding regions. This was done because later allocation in survivor regions would change top, and we don't want that changed value. It seems like it might be easier and simpler to retain the existing representation, but capture the desired "top" value in NTAMS. Then in scan_root_region, for survivor regions scan from bottom to NTAMS. That means scan_root_region is different for old/survivor regions. But that's kind of already true; it's a bit of a hack that the current scan from NTAMS to top works for both. This also requires looking at and possibly adjusting other uses of NTAMS. ------------------------------------------------------------------------------ Consider renaming G1CollectedHeap::survivor_convert_to_eden() => convert_survivor_to_eden() ? ------------------------------------------------------------------------------ I dislike the "retained" region nomenclature for survivors. I didn't notice an explanation of it, and it's not obvious why that name was chosen or how it relates to how these regions are used. But I don't have a good suggestion for an alternative yet. It seems like that name was copied from MutatorAllocRegion, but I'm not sure the survivor case is really that similar. I also dislike some of the accounting kludges around retained regions, such as for used_bytes. Maybe the above discussed use of NTAMS might provide information that could simplify the accounting? ------------------------------------------------------------------------------ From tprintezis at twitter.com Thu May 9 16:38:35 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Thu, 9 May 2019 09:38:35 -0700 Subject: RFR(S): 8223575: add subspace transitions to gc+metaspace=info log lines In-Reply-To: References: Message-ID: Hi Thomas, Thanks for the detailed feedback and suggestions. Please see inline?. ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On May 8, 2019 at 4:07:49 PM, Thomas St?fe (thomas.stuefe at gmail.com) wrote: Hi Tony, I think this makes sense. I agree, the output as it is now can be confusing. Note that users can still be confused since users may simply not be aware that the default ClassSpace reserved size is 1G. Yes. What a user wants to size is (what I call - more below) the class metadata space. But, in the log, this is hidden in the total size. So, this creates confusion. FWIW, the older +PrintHeapAtGC output had the same issue: Metaspace used 5660K, capacity 5996K, committed 6144K, reserved 1056768K class space used 630K, capacity 734K, committed 768K, reserved 1048576K It shows the total size and the class space size (which the user might not know what it is) but doesn?t explicitly show the space the user actually wants to size. I think we can make it a bit easier for them. Can your change break scripts for folks parsing GC logs? I see that you only appended content and did not change the existing text, so I guess this is okay. Of course the change can break a GC log parsing script, if it assumes there?s only one size transition on gc,metaspace lines (but if it just parses the size transition and ignores that rest it will still work). But is there a hard requirement that we cannot change any log output? That?d be unfortunate, IMHO. In fact, the safepoint=info output changed almost totally between 11 and now. 11: [49.768s][info ][safepoint ] Entering safepoint region: Cleanup ... [49.769s][info ][safepoint ] Total time for which application threads were stopped: 0.0005466 seconds, Stopping threads took: 0.0001931 seconds now: [880.754s][info ][safepoint ] Safepoint "GenCollectForAllocation", Time since last: 990253904 ns, Reaching safepoint: 440316 ns, At safepoint: 3761268 ns, Total: 4201584 ns --- Format and naming: Different forms are used in different places. In hs-err files, we print "Metaspace" and "Class Space", with the former being the total sum and the latter the class space part. But in other corners, eg. in the jcmd VM.metaspace command, we print non-class space and class space and maybe the sum in addition. Usually non-class space is called "Non-Class". I personally would prefer, in this case, the latter names: "Class Space" and "Non-Class Space" Here?s another pet-peeve of mine (and I?m probably overthinking this) :-) : Users know that the metaspace holds class metadata. So, calling the area that actually holds the class metadata ?Non-Class Space? and the area that holds some internal JVM data structure ?Class Space? is counter-productive and confusing (IMHO). We don?t have to use the names I used. But any chance of coming up with something less confusing? --- Note that for non-class space, reserved size can actually change when allocating and releasing metaspace, if VirtualSpaceNodes get purged or newly allocated. Hence total reserved size can change too. So we may want to print out reserved before and reserved after sizes. I guess that was also missing from the output before. I?ll be very happy to add the before capacity to the output. You mean something like this, right? 2300K(4192K)->2100K(4192K) And we should also add it to the heap size transitions too, as they have the same issue (the heap can be resized during a GC). Of course, this will definitely require GC log parser changes. :-) But I think the change is worthwhile. Any objections? ---- Since used = class used + nonclass used, I would simplify PreMetaspaceValues to _non_class_used and _class_used and calculate the sum on the fly. Yeah, I thought about that. I just decided to just store the total used to avoid replicating the logic of how that?s calculated in that class too. --- You added a new dependency to heap.hpp into metaspace.cpp for the format string. Is this really necessary? Can you move this define to metaspace.cpp? If not, you need at least add include heap.hpp (did it build this way without precompiled headers?) I didn?t try to explicitly compile without precompiled headers. jdk submit job was successful. Is that usually evidence enough? If not, I?ll need to add the extra step to my workflow. As I eluded to earlier, if this change is accepted I?m going to suggest to also expand the gc,heap output with sub-space information for the young gen (eden / survivors). So I wanted to move the FORMAT macros to somewhere that can also be shared by GC files. I was not sure if heap.hpp was the best place for them. Maybe I can just put them in globalDefinitions.hpp, as there?s already some size formatting utilities there (proper_unit_for_byte_size(?), etc.)? ---- The following issues are matters of taste, and I leave it up to you whether you want to change something: -> I wince a bit at yet another metaspace-sizes-structure - we have already ClassLoaderMetaspaceStatistics and MetaspaceCounters, and maybe others. JFR may also do something similar. Would it be possible to reuse one of those? (I guess ClassLoaderMetaspaceStatistics is too fine granular and too expensive to collect.) I was not aware of the two you mentioned. Aren?t MetaspaceCounters only available when UsePerfData is enabled? I?d rather decouple this from UsePerfData. ClassLoaderMetaspaceStatistics does seem too fine-grain for what I want. And I modeled PreMetaspaceValues on PreGCValues used in ParallelGC, as it?s a nice way to package up what?s needed in an object and easily re-use it in the few places where it?s needed. The way is done now, i.e., just store the metaspace used size and pass it to the print method, is a bit ad-hoc (IMHO). -> I would prefer the class PreMetaspaceValues to live inside an own header in metaspace folder, and in the metaspace namespace like we did for similar helpers. I can definitely do that. -> I would take the "Pre" out of the name and the member names and just name it "Snapshot" and the members "used" and "class used". Again, I modeled it based on ParallelGC's PreGCValues class. But I can rename it to Snapshot to generalize its use. -> I would probably have made PreMetaspaceValues a dumb structure and added a function "MetaspaceAux::get_dimensions()" instead of adding the implementation to the constructor and to the header file. At least I would prefer the ctor implementation to the cpp file. I can move it to a cpp file. This will also eliminate a cyclic dependency I had to address by the class forward declaration. Tony Cheers, Thomas On Wed, May 8, 2019 at 4:20 PM Tony Printezis wrote: > Any chance of someone taking a look at this? > > http://cr.openjdk.java.net/~tonyp/8223575/webrev.0/ > > As I said on the JIRA, users can be confused when they metaspace capacity > shows as over 1G even though they asked for 128M. Breaking down the spaces, > when coops are enabled, can address this confusion (and having the actual > occupancy of each space separately is also a lot more informative and > avoids guesswork). The output looks like this when coops are enabled: > > [10.090s][info][gc,metaspace ] GC(8) Metaspace: 2130K->2130K(1056768K) > ClassMetadata: 1906K->1906K(8192K) ClassSpace: 223K->223K(1048576K) > > and like this (i.e., what it was before the change) when coops are not > enabled: > > [25.767s][info][gc,metaspace ] GC(28) Metaspace: 8107K->8107K(10240K) > > A couple of notes on the changes: > > - I?m open to different suggestions for what to call ClassMetadata and > ClassSpace. :-) > - I didn?t know where to put the HEAP_CHANGE_FORMAT and > HEAP_CHANGE_FORMAT_ARGS macros, so that they can be easily shared, and I > ended up putting them in heap.hpp. Let me know if there?s a better place > for them. > > Tony > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aoqi at loongson.cn Thu May 9 18:26:59 2019 From: aoqi at loongson.cn (Ao Qi) Date: Fri, 10 May 2019 02:26:59 +0800 Subject: RFR(trivial): JDK-8223651: Build fails with --with-jvm-features=-jfr and --disable-precompiled-headers Message-ID: Hi, Bugs: https://bugs.openjdk.java.net/browse/JDK-8223651 Webrev: http://cr.openjdk.java.net/~aoqi/8223651/webrev.00/ Some more details please see: https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2019-May/034244.html Thanks, Ao Qi From shade at redhat.com Thu May 9 18:29:06 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 9 May 2019 20:29:06 +0200 Subject: RFR(trivial): JDK-8223651: Build fails with --with-jvm-features=-jfr and --disable-precompiled-headers In-Reply-To: References: Message-ID: <96fafa39-ea59-3ed9-0592-5c3fa378c632@redhat.com> On 5/9/19 8:26 PM, Ao Qi wrote: > Bugs: > https://bugs.openjdk.java.net/browse/JDK-8223651 > > Webrev: > http://cr.openjdk.java.net/~aoqi/8223651/webrev.00/ Looks good to me. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Thu May 9 19:04:35 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 9 May 2019 21:04:35 +0200 Subject: RFR(trivial): JDK-8223651: Build fails with --with-jvm-features=-jfr and --disable-precompiled-headers In-Reply-To: <96fafa39-ea59-3ed9-0592-5c3fa378c632@redhat.com> References: <96fafa39-ea59-3ed9-0592-5c3fa378c632@redhat.com> Message-ID: <47412933-339a-6ab8-277d-53abf0ab6d95@redhat.com> On 5/9/19 8:29 PM, Aleksey Shipilev wrote: > On 5/9/19 8:26 PM, Ao Qi wrote: >> Bugs: >> https://bugs.openjdk.java.net/browse/JDK-8223651 >> >> Webrev: >> http://cr.openjdk.java.net/~aoqi/8223651/webrev.00/ > > Looks good to me. And yes, this is indeed trivial. I can push this for you. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From aoqi at loongson.cn Thu May 9 19:07:34 2019 From: aoqi at loongson.cn (Ao Qi) Date: Fri, 10 May 2019 03:07:34 +0800 Subject: RFR(trivial): JDK-8223651: Build fails with --with-jvm-features=-jfr and --disable-precompiled-headers In-Reply-To: <47412933-339a-6ab8-277d-53abf0ab6d95@redhat.com> References: <96fafa39-ea59-3ed9-0592-5c3fa378c632@redhat.com> <47412933-339a-6ab8-277d-53abf0ab6d95@redhat.com> Message-ID: On Fri, May 10, 2019 at 3:04 AM Aleksey Shipilev wrote: > > On 5/9/19 8:29 PM, Aleksey Shipilev wrote: > > On 5/9/19 8:26 PM, Ao Qi wrote: > >> Bugs: > >> https://bugs.openjdk.java.net/browse/JDK-8223651 > >> > >> Webrev: > >> http://cr.openjdk.java.net/~aoqi/8223651/webrev.00/ > > > > Looks good to me. > > And yes, this is indeed trivial. I can push this for you. Thank you, Aleksey! > > -Aleksey > From shade at redhat.com Thu May 9 19:11:40 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 9 May 2019 21:11:40 +0200 Subject: RFR(trivial): JDK-8223651: Build fails with --with-jvm-features=-jfr and --disable-precompiled-headers In-Reply-To: References: <96fafa39-ea59-3ed9-0592-5c3fa378c632@redhat.com> <47412933-339a-6ab8-277d-53abf0ab6d95@redhat.com> Message-ID: <4bf76aea-858b-0003-837c-cb9405fd4552@redhat.com> On 5/9/19 9:07 PM, Ao Qi wrote: > On Fri, May 10, 2019 at 3:04 AM Aleksey Shipilev wrote: >> >> On 5/9/19 8:29 PM, Aleksey Shipilev wrote: >>> On 5/9/19 8:26 PM, Ao Qi wrote: >>>> Bugs: >>>> https://bugs.openjdk.java.net/browse/JDK-8223651 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~aoqi/8223651/webrev.00/ >>> >>> Looks good to me. >> >> And yes, this is indeed trivial. I can push this for you. > > Thank you, Aleksey! Done: http://hg.openjdk.java.net/jdk/jdk/rev/9c18785a13bf -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From tprintezis at twitter.com Thu May 9 20:00:17 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Thu, 9 May 2019 13:00:17 -0700 Subject: RFR(S): 8223575: add subspace transitions to gc+metaspace=info log lines In-Reply-To: References: Message-ID: Thomas, Updated webrev: http://cr.openjdk.java.net/~tonyp/8223575/webrev.1/ with some of the suggested changes: * did some renaming * created hpp / cpp files for the new class (now called MetaspaceSizesSnapshot) * moved the formatting macros to globalDefinitions.hpp Still pending: * What should we call the spaces? * Should we add the before capacity in the output? (let me know if I missed anything?) Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On May 9, 2019 at 12:38:35 PM, Tony Printezis (tprintezis at twitter.com) wrote: Hi Thomas, Thanks for the detailed feedback and suggestions. Please see inline?. ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On May 8, 2019 at 4:07:49 PM, Thomas St?fe (thomas.stuefe at gmail.com) wrote: Hi Tony, I think this makes sense. I agree, the output as it is now can be confusing. Note that users can still be confused since users may simply not be aware that the default ClassSpace reserved size is 1G. Yes. What a user wants to size is (what I call - more below) the class metadata space. But, in the log, this is hidden in the total size. So, this creates confusion. FWIW, the older +PrintHeapAtGC output had the same issue: Metaspace used 5660K, capacity 5996K, committed 6144K, reserved 1056768K class space used 630K, capacity 734K, committed 768K, reserved 1048576K It shows the total size and the class space size (which the user might not know what it is) but doesn?t explicitly show the space the user actually wants to size. I think we can make it a bit easier for them. Can your change break scripts for folks parsing GC logs? I see that you only appended content and did not change the existing text, so I guess this is okay. Of course the change can break a GC log parsing script, if it assumes there?s only one size transition on gc,metaspace lines (but if it just parses the size transition and ignores that rest it will still work). But is there a hard requirement that we cannot change any log output? That?d be unfortunate, IMHO. In fact, the safepoint=info output changed almost totally between 11 and now. 11: [49.768s][info ][safepoint ] Entering safepoint region: Cleanup ... [49.769s][info ][safepoint ] Total time for which application threads were stopped: 0.0005466 seconds, Stopping threads took: 0.0001931 seconds now: [880.754s][info ][safepoint ] Safepoint "GenCollectForAllocation", Time since last: 990253904 ns, Reaching safepoint: 440316 ns, At safepoint: 3761268 ns, Total: 4201584 ns --- Format and naming: Different forms are used in different places. In hs-err files, we print "Metaspace" and "Class Space", with the former being the total sum and the latter the class space part. But in other corners, eg. in the jcmd VM.metaspace command, we print non-class space and class space and maybe the sum in addition. Usually non-class space is called "Non-Class". I personally would prefer, in this case, the latter names: "Class Space" and "Non-Class Space" Here?s another pet-peeve of mine (and I?m probably overthinking this) :-) : Users know that the metaspace holds class metadata. So, calling the area that actually holds the class metadata ?Non-Class Space? and the area that holds some internal JVM data structure ?Class Space? is counter-productive and confusing (IMHO). We don?t have to use the names I used. But any chance of coming up with something less confusing? --- Note that for non-class space, reserved size can actually change when allocating and releasing metaspace, if VirtualSpaceNodes get purged or newly allocated. Hence total reserved size can change too. So we may want to print out reserved before and reserved after sizes. I guess that was also missing from the output before. I?ll be very happy to add the before capacity to the output. You mean something like this, right? 2300K(4192K)->2100K(4192K) And we should also add it to the heap size transitions too, as they have the same issue (the heap can be resized during a GC). Of course, this will definitely require GC log parser changes. :-) But I think the change is worthwhile. Any objections? ---- Since used = class used + nonclass used, I would simplify PreMetaspaceValues to _non_class_used and _class_used and calculate the sum on the fly. Yeah, I thought about that. I just decided to just store the total used to avoid replicating the logic of how that?s calculated in that class too. --- You added a new dependency to heap.hpp into metaspace.cpp for the format string. Is this really necessary? Can you move this define to metaspace.cpp? If not, you need at least add include heap.hpp (did it build this way without precompiled headers?) I didn?t try to explicitly compile without precompiled headers. jdk submit job was successful. Is that usually evidence enough? If not, I?ll need to add the extra step to my workflow. As I eluded to earlier, if this change is accepted I?m going to suggest to also expand the gc,heap output with sub-space information for the young gen (eden / survivors). So I wanted to move the FORMAT macros to somewhere that can also be shared by GC files. I was not sure if heap.hpp was the best place for them. Maybe I can just put them in globalDefinitions.hpp, as there?s already some size formatting utilities there (proper_unit_for_byte_size(?), etc.)? ---- The following issues are matters of taste, and I leave it up to you whether you want to change something: -> I wince a bit at yet another metaspace-sizes-structure - we have already ClassLoaderMetaspaceStatistics and MetaspaceCounters, and maybe others. JFR may also do something similar. Would it be possible to reuse one of those? (I guess ClassLoaderMetaspaceStatistics is too fine granular and too expensive to collect.) I was not aware of the two you mentioned. Aren?t MetaspaceCounters only available when UsePerfData is enabled? I?d rather decouple this from UsePerfData. ClassLoaderMetaspaceStatistics does seem too fine-grain for what I want. And I modeled PreMetaspaceValues on PreGCValues used in ParallelGC, as it?s a nice way to package up what?s needed in an object and easily re-use it in the few places where it?s needed. The way is done now, i.e., just store the metaspace used size and pass it to the print method, is a bit ad-hoc (IMHO). -> I would prefer the class PreMetaspaceValues to live inside an own header in metaspace folder, and in the metaspace namespace like we did for similar helpers. I can definitely do that. -> I would take the "Pre" out of the name and the member names and just name it "Snapshot" and the members "used" and "class used". Again, I modeled it based on ParallelGC's PreGCValues class. But I can rename it to Snapshot to generalize its use. -> I would probably have made PreMetaspaceValues a dumb structure and added a function "MetaspaceAux::get_dimensions()" instead of adding the implementation to the constructor and to the header file. At least I would prefer the ctor implementation to the cpp file. I can move it to a cpp file. This will also eliminate a cyclic dependency I had to address by the class forward declaration. Tony Cheers, Thomas On Wed, May 8, 2019 at 4:20 PM Tony Printezis wrote: > Any chance of someone taking a look at this? > > http://cr.openjdk.java.net/~tonyp/8223575/webrev.0/ > > As I said on the JIRA, users can be confused when they metaspace capacity > shows as over 1G even though they asked for 128M. Breaking down the spaces, > when coops are enabled, can address this confusion (and having the actual > occupancy of each space separately is also a lot more informative and > avoids guesswork). The output looks like this when coops are enabled: > > [10.090s][info][gc,metaspace ] GC(8) Metaspace: 2130K->2130K(1056768K) > ClassMetadata: 1906K->1906K(8192K) ClassSpace: 223K->223K(1048576K) > > and like this (i.e., what it was before the change) when coops are not > enabled: > > [25.767s][info][gc,metaspace ] GC(28) Metaspace: 8107K->8107K(10240K) > > A couple of notes on the changes: > > - I?m open to different suggestions for what to call ClassMetadata and > ClassSpace. :-) > - I didn?t know where to put the HEAP_CHANGE_FORMAT and > HEAP_CHANGE_FORMAT_ARGS macros, so that they can be easily shared, and I > ended up putting them in heap.hpp. Let me know if there?s a better place > for them. > > Tony > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Fri May 10 00:49:26 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 9 May 2019 20:49:26 -0400 Subject: RFR (M): 8200545: Improve filter for enqueued deferred cards In-Reply-To: <5b600e3a6116009c4e326bbee277622fc3650471.camel@oracle.com> References: <81ba1000d2c8eff1b4a467cd7ed75a372857244f.camel@oracle.com> <6ca33ce2-28cc-f736-676e-67f16c8f2dc2@oracle.com> <5b600e3a6116009c4e326bbee277622fc3650471.camel@oracle.com> Message-ID: <65EF7FB0-ADA6-4504-A73C-AE2330C08DD9@oracle.com> > On May 6, 2019, at 5:51 AM, Thomas Schatzl wrote: > http://cr.openjdk.java.net/~tschatzl/8200545/webrev.1_to_2/ > http://cr.openjdk.java.net/~tschatzl/8200545/webrev.2/ > > (Test: local compilation) > > Thanks, > Thomas ============================================================================== http://cr.openjdk.java.net/~tschatzl/8200545/webrev/ ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1InCSetState.hpp 58 // The first two values are encoded in increasing generation order, which 59 // allows us to use them as index into arrays; they also start at zero. This part of the comment doesn't seem true. Unless NotInCSet (value == 0) is considered a "generation", but then there are 3 rather than 2. The comment formerly talked about the "positive values" (e.g. the values > 0). ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1InCSetState.hpp 67 static const in_cset_state_t NumGenInCSet = 3; // Number of generations that can be in the collection set. Again here, confusion about numbering. There are only 3 generations if NotInCSet is considered a "generation". And places that use these as array indices generally "waste" an entry for NotInCSet. I thought there was a CR for that, but couldn't find it. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1InCSetState.hpp 46 #ifdef SPARC 47 uint32_t _needs_remset_update; 48 #else 49 uint8_t _needs_remset_update; 50 #endif Is it actually worth conditionalizing the type of this "bool" member? Note that access via needs_remset_update() is going to be doing an integral to bool conversion. ------------------------------------------------------------------------------ I kind of wish some of the changes to g1InCSetState.hpp had been done separately. Oh well... ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1OopClosures.hpp 65 G1ScanCardClosure(G1CollectedHeap* g1h, 66 G1ParScanThreadState* pss) : Fix indentation on line 66. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/heapRegionRemSet.hpp 242 // log_error(gc)("Region %u set complete", _hr->hrm_index()); Some left-over debugging code? ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1CollectedHeap.cpp 2629 class RegisterHumongousWithInCSetFastTestClosure : public HeapRegionClosure { This class seems misnamed now, since it is now doing stuff for all regions (calls register_region_with_cset for non-humongous). (Maybe this is addressed in one of the other associated webrevs?) Similarly, record_fast_reclaim_humongous_stats() seems somewhat misnamed now, since the time value is for "register region". ------------------------------------------------------------------------------ From thomas.schatzl at oracle.com Fri May 10 10:18:33 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 10 May 2019 12:18:33 +0200 Subject: RFR (M): 8200545: Improve filter for enqueued deferred cards In-Reply-To: <65EF7FB0-ADA6-4504-A73C-AE2330C08DD9@oracle.com> References: <81ba1000d2c8eff1b4a467cd7ed75a372857244f.camel@oracle.com> <6ca33ce2-28cc-f736-676e-67f16c8f2dc2@oracle.com> <5b600e3a6116009c4e326bbee277622fc3650471.camel@oracle.com> <65EF7FB0-ADA6-4504-A73C-AE2330C08DD9@oracle.com> Message-ID: <9c6df03230a6c2a632908bbed556accf412f43d3.camel@oracle.com> Hi Kim, thanks for reviewing. On Thu, 2019-05-09 at 20:49 -0400, Kim Barrett wrote: > > On May 6, 2019, at 5:51 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > http://cr.openjdk.java.net/~tschatzl/8200545/webrev.1_to_2/ > > http://cr.openjdk.java.net/~tschatzl/8200545/webrev.2/ > > > > (Test: local compilation) > > > > Thanks, > > Thomas > > ===================================================================== > ========= > http://cr.openjdk.java.net/~tschatzl/8200545/webrev/ Sorry for any confusion I caused with this change. > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1InCSetState.hpp > 58 // The first two values are encoded in increasing generation > order, which > 59 // allows us to use them as index into arrays; they also start > at zero. > > This part of the comment doesn't seem true. Unless NotInCSet (value > == 0) is considered a "generation", but then there are 3 rather than > 2. The comment formerly talked about the "positive values" (e.g. the > values > 0). > Fixed in .2. > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1InCSetState.hpp > 67 static const in_cset_state_t NumGenInCSet = 3; // Number > of generations that can be in the collection set. > > Again here, confusion about numbering. There are only 3 generations > if NotInCSet is considered a "generation". > Yes, this is a misnomer. The G1HeapRegionAttr contents are copies/shortcuts for often used information during GC only. I reverted it to just "Num" as before. > And places that use these as array indices generally "waste" an entry > for NotInCSet. I thought there was a CR for that, but couldn't find > it. This had at least initially been intentional. I do not remember a CR for that. When originally introducing this table, I deliberately wanted to make the decision between in cset and not in cset a decision simple. Initially this has just been != 0 / == 0, but with the proliferation of information in that entry (first humongous reclaim candidate, then optional evacuation) it has changed to > 0. You are right that we might want to reconsider this decision now. It's not a big waste of space in the arrays. I filed JDK-8223693 for cleaning this up. > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1InCSetState.hpp > 46 #ifdef SPARC > 47 uint32_t _needs_remset_update; > 48 #else > 49 uint8_t _needs_remset_update; > 50 #endif > > Is it actually worth conditionalizing the type of this "bool" member? This has changed in .2; I conditionalized the entries to get minimum size of the element (i.e. sizeof(G1HeapRegionAttr)) for non-sparc, and word size for sparc for best performance. Note that I did not check SPARC performance, but I can see advantages to keep memory footprint low(er) for that table on everything else. (SPARC implementations I have tested are comparatively slow extracting parts of a word). > Note that access via needs_remset_update() is going to be doing an > integral to bool conversion. I fixed the implicit integral to bool conversion, that's indeed an unintentional mistake. > ------------------------------------------------------------------- > ----------- > > I kind of wish some of the changes to g1InCSetState.hpp had been done > separately. Oh well... What exactly do you suggest? Separate renaming from the functional change? Or first clean up InCSetState (typedefs, enum, JDK-8223693, ..), then rename and then add the new functionality? > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1OopClosures.hpp > 65 G1ScanCardClosure(G1CollectedHeap* g1h, > 66 G1ParScanThreadState* pss) : > > Fix indentation on line 66. Fixed in .2. > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/heapRegionRemSet.hpp > 242 // log_error(gc)("Region %u set complete", _hr- > >hrm_index()); > > Some left-over debugging code? Fixed in .2. > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1CollectedHeap.cpp > 2629 class RegisterHumongousWithInCSetFastTestClosure : public > HeapRegionClosure { > > This class seems misnamed now, since it is now doing stuff for all > regions (calls register_region_with_cset for non-humongous). (Maybe > this is addressed in one of the other associated webrevs?) > > Similarly, record_fast_reclaim_humongous_stats() seems somewhat > misnamed now, since the time value is for "register region". Fixed. New webrevs: http://cr.openjdk.java.net/~tschatzl/8200545/webrev.2_t o_3/ (diff) http://cr.openjdk.java.net/~tschatzl/8200545/webrev.3/ (full) testing: local compilation Thanks, Thomas From zgu at redhat.com Fri May 10 15:25:46 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Fri, 10 May 2019 11:25:46 -0400 Subject: RFR(XS) 8223215: Shenandoah: Support verifying subset of roots Message-ID: Please review this patch that allows to verify subset of roots. Bug: https://bugs.openjdk.java.net/browse/JDK-8223215 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8223215/webrev.00/index.html Test: hotspot_gc_shenandoah with -XX:+ShenandoahVerify (fastdebug and release) Thanks, -Zhengyu From rkennke at redhat.com Fri May 10 19:12:01 2019 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 10 May 2019 21:12:01 +0200 Subject: RFR(XS) 8223215: Shenandoah: Support verifying subset of roots In-Reply-To: References: Message-ID: <255952db-976e-fa5e-a847-cb92cc049a4a@redhat.com> Hi Zhengyu, this looks good. Maybe the root-verifier code should be in its own files? What's with the font it looks weird? ;-) Roman > Please review this patch that allows to verify subset of roots. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8223215 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8223215/webrev.00/index.html > > Test: > ? hotspot_gc_shenandoah with -XX:+ShenandoahVerify (fastdebug and release) > > Thanks, > > -Zhengyu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Fri May 10 21:14:00 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Fri, 10 May 2019 17:14:00 -0400 Subject: RFR(XS) 8223215: Shenandoah: Support verifying subset of roots In-Reply-To: <255952db-976e-fa5e-a847-cb92cc049a4a@redhat.com> References: <255952db-976e-fa5e-a847-cb92cc049a4a@redhat.com> Message-ID: <53165b5d-1303-9178-a1a6-34062203f2ad@redhat.com> On 5/10/19 3:12 PM, Roman Kennke wrote: > Hi Zhengyu, > > this looks good. > Maybe the root-verifier code should be in its own files? Updated: http://cr.openjdk.java.net/~zgu/JDK-8223215/webrev.01/ > What's with the font it looks weird? ;-) Not sure why, default language got switched to Chinese :-) -Zhengyu > > Roman > >> Please review this patch that allows to verify subset of roots. >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8223215 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8223215/webrev.00/index.html >> >> Test: >> ? hotspot_gc_shenandoah with -XX:+ShenandoahVerify (fastdebug and release) >> >> Thanks, >> >> -Zhengyu > From kim.barrett at oracle.com Fri May 10 22:53:43 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 10 May 2019 18:53:43 -0400 Subject: RFR (M): 8200545: Improve filter for enqueued deferred cards In-Reply-To: <9c6df03230a6c2a632908bbed556accf412f43d3.camel@oracle.com> References: <81ba1000d2c8eff1b4a467cd7ed75a372857244f.camel@oracle.com> <6ca33ce2-28cc-f736-676e-67f16c8f2dc2@oracle.com> <5b600e3a6116009c4e326bbee277622fc3650471.camel@oracle.com> <65EF7FB0-ADA6-4504-A73C-AE2330C08DD9@oracle.com> <9c6df03230a6c2a632908bbed556accf412f43d3.camel@oracle.com> Message-ID: <3DCA1C68-C1FB-49DB-8E96-791D38410600@oracle.com> > On May 10, 2019, at 6:18 AM, Thomas Schatzl wrote: > > On Thu, 2019-05-09 at 20:49 -0400, Kim Barrett wrote: >> ------------------------------------------------------------------- >> ----------- >> src/hotspot/share/gc/g1/g1InCSetState.hpp >> 58 // The first two values are encoded in increasing generation >> order, which >> 59 // allows us to use them as index into arrays; they also start >> at zero. >> >> This part of the comment doesn't seem true. Unless NotInCSet (value >> == 0) is considered a "generation", but then there are 3 rather than >> 2. The comment formerly talked about the "positive values" (e.g. the >> values > 0). >> > > Fixed in .2. Looking at webrev.3 (latest), I still see the above quoted comment that I think is wrong, or at least very confusing. >> And places that use these as array indices generally "waste" an entry >> for NotInCSet. I thought there was a CR for that, but couldn't find >> it. > > This had at least initially been intentional. I do not remember a CR > for that. > > When originally introducing this table, I deliberately wanted to make > the decision between in cset and not in cset a decision simple. > Initially this has just been != 0 / == 0, but with the proliferation of > information in that entry (first humongous reclaim candidate, then > optional evacuation) it has changed to > 0. > > You are right that we might want to reconsider this decision now. It's > not a big waste of space in the arrays. I filed JDK-8223693 for > cleaning this up. OK. Though I agree the space impact is likely small, and it might not be worth the trouble to fix anymore, despite it being potentially confusing. > >> ------------------------------------------------------------------- >> ----------- >> src/hotspot/share/gc/g1/g1InCSetState.hpp >> 46 #ifdef SPARC >> 47 uint32_t _needs_remset_update; >> 48 #else >> 49 uint8_t _needs_remset_update; >> 50 #endif >> >> Is it actually worth conditionalizing the type of this "bool" member? > > This has changed in .2; I conditionalized the entries to get minimum > size of the element (i.e. sizeof(G1HeapRegionAttr)) for non-sparc, and > word size for sparc for best performance. > > Note that I did not check SPARC performance, but I can see advantages > to keep memory footprint low(er) for that table on everything else. > (SPARC implementations I have tested are comparatively slow extracting > parts of a word). > >> Note that access via needs_remset_update() is going to be doing an >> integral to bool conversion. > > I fixed the implicit integral to bool conversion, that's indeed an > unintentional mistake. Why not just make the member type ?bool? and be done with it? Let the compiler decide the appropriate representation. >> I kind of wish some of the changes to g1InCSetState.hpp had been done >> separately. Oh well... > > What exactly do you suggest? Separate renaming from the functional > change? Or first clean up InCSetState (typedefs, enum, JDK-8223693, > ..), then rename and then add the new functionality? I think separating renamings from functional changes might have made reviewing easier, at least for me. Of course, that?s more work for you. >> src/hotspot/share/gc/g1/g1CollectedHeap.cpp >> 2629 class RegisterHumongousWithInCSetFastTestClosure : public >> HeapRegionClosure { >> >> This class seems misnamed now, since it is now doing stuff for all >> regions (calls register_region_with_cset for non-humongous). (Maybe >> this is addressed in one of the other associated webrevs?) >> >> Similarly, record_fast_reclaim_humongous_stats() seems somewhat >> misnamed now, since the time value is for "register region". > > Fixed. Thanks. > New webrevs: > > > http://cr.openjdk.java.net/~tschatzl/8200545/webrev.2_t o_3/ (diff) > > http://cr.openjdk.java.net/~tschatzl/8200545/webrev.3/ (full) > > testing: local compilation > > Thanks, > Thomas From thomas.stuefe at gmail.com Mon May 13 05:51:36 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 13 May 2019 07:51:36 +0200 Subject: RFR(S): 8223575: add subspace transitions to gc+metaspace=info log lines In-Reply-To: References: Message-ID: Hi Tony, On Thu, May 9, 2019 at 6:38 PM Tony Printezis wrote: > Hi Thomas, > > Thanks for the detailed feedback and suggestions. Please see inline?. > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > On May 8, 2019 at 4:07:49 PM, Thomas St?fe (thomas.stuefe at gmail.com) > wrote: > > > Hi Tony, > > I think this makes sense. I agree, the output as it is now can be > confusing. Note that users can still be confused since users may simply not > be aware that the default ClassSpace reserved size is 1G. > > > Yes. What a user wants to size is (what I call - more below) the class > metadata space. But, in the log, this is hidden in the total size. So, this > creates confusion. FWIW, the older +PrintHeapAtGC output had the same issue: > > Metaspace used 5660K, capacity 5996K, committed 6144K, reserved > 1056768K > > class space used 630K, capacity 734K, committed 768K, reserved > 1048576K > > It shows the total size and the class space size (which the user might not > know what it is) but doesn?t explicitly show the space the user actually > wants to size. > Not sure I follow. What the user could size is either MaxMetaspaceSize, which would be the sum of committed for all sub spaces, or CompressedClassSpaceSize, which would be the reserved size of the compressed class space. In this output, both numbers are written out. > I think we can make it a bit easier for them. > > > > Can your change break scripts for folks parsing GC logs? I see that you > only appended content and did not change the existing text, so I guess this > is okay. > > > Of course the change can break a GC log parsing script, if it assumes > there?s only one size transition on gc,metaspace lines (but if it just > parses the size transition and ignores that rest it will still work). But > is there a hard requirement that we cannot change any log output? That?d be > unfortunate, IMHO. In fact, the safepoint=info output changed almost > totally between 11 and now. > > 11: > > [49.768s][info ][safepoint ] Entering safepoint region: Cleanup > > ... > > [49.769s][info ][safepoint ] Total time for which application > threads were stopped: 0.0005466 seconds, Stopping threads took: 0.0001931 > seconds > > now: > > [880.754s][info ][safepoint ] Safepoint "GenCollectForAllocation", > Time since last: 990253904 ns, Reaching safepoint: 440316 ns, At safepoint: > 3761268 ns, Total: 4201584 ns > > > Okay, lets assume it is fine to break existing scripts if we can make the output better. > > --- > > Format and naming: Different forms are used in different places. In hs-err > files, we print "Metaspace" and "Class Space", with the former being the > total sum and the latter the class space part. > > But in other corners, eg. in the jcmd VM.metaspace command, we print > non-class space and class space and maybe the sum in addition. Usually > non-class space is called "Non-Class". > > I personally would prefer, in this case, the latter names: "Class Space" > and "Non-Class Space" > > > Here?s another pet-peeve of mine (and I?m probably overthinking this) :-) > : Users know that the metaspace holds class metadata. So, calling the area > that actually holds the class metadata ?Non-Class Space? and the area that > holds some internal JVM data structure ?Class Space? is counter-productive > and confusing (IMHO). We don?t have to use the names I used. But any chance > of coming up with something less confusing? > > > Well, I'd say both are class metadata if you see it as runtime data surrounding classes living outside the java heap. But I see what you mean. This is a result of the decision to call the part of the Metaspace containing the Klass structures "compressed class space", which is a slight misnomer, since it is neither compressed nor really the (one exclusive) Class Space. But then it follows that the other one is the "non-class space", or alternatively the "metaspace" which is not much clearer since the whole is also the Metaspace. I agree that this is not perfect, but I would not like to introduce new naming just now. That thing has been the "compressed class space" for a long time. We have parameters like "CompressedClassSpaceSize" and tools like "VM.metaspace" using that naming. We have numerous blogs and articles referencing those names. And the terminology is clear if a bit strange. So while I see your point, I would prefer to use the established naming scheme we have now instead of creating a new one, and leave the naming discussion for another day. Also: other metadata could move into the compressed class space in the future, if we decide that they should be addressed with compressed references. In theory, if some preconditions are met, we even could merge both spaces and let just all metadata live in one contiguous range. Also note that I currently work on a prototype for a JEP ( https://bugs.openjdk.java.net/browse/JDK-8221173) which revamps parts of the Metaspace allocator. One outcome would be simplified and more generic coding, with a lot less explicit decisions based on "class" or "non-class", which may make the code base more amenable to you :) > > --- > > Note that for non-class space, reserved size can actually change when > allocating and releasing metaspace, if VirtualSpaceNodes get purged or > newly allocated. Hence total reserved size can change too. So we may want > to print out reserved before and reserved after sizes. I guess that was > also missing from the output before. > > > I?ll be very happy to add the before capacity to the output. You mean > something like this, right? > > 2300K(4192K)->2100K(4192K) > > And we should also add it to the heap size transitions too, as they have > the same issue (the heap can be resized during a GC). Of course, this will > definitely require GC log parser changes. :-) But I think the change is > worthwhile. Any objections? > I had time to think about this a bit more and have some second thoughts. To me, Reserved Size actually seems rather unimportant. If we scratch backward compatibility and therefore can freely rework the message, lets think once more: We have four sizes standing out as candidates for reporting: a) used size - which is what we use for class metadata. b) committed size - which is what we actually spend on class metadata. This is the size that hits MaxMetaspaceSize. This is also "what the OS sees what we allocate" - the part of Metaspace contributing to RSS. It includes all kind of wastages which on unhappy days can be significant, so (b) can be larger than (a). c) reserved size. For non-class space the reserved size follows very closely the committed size, with a max delta of 4MB. For the compressed class space this size is just a constant which is usually not very informative since it 1G is normally way too large to ever hit it. d) the Maxima the user set - MaxMetaspaceSize and CompressedClassSpaceSize. The former is just a number, but if we want to display something like "used (maximum)" this would be a candidate. But it can be infinite. The latter is equal to (c). I find (b) the most interesting number, since it is what we pay for Metaspace, including all overhead. Does the normal gc log for the heap not also print "committed"? So maybe we should swap "used" for "committed" in the output. And then, the delta between used and committed is also interesting, since this waste can get large. Finally, (c) is interesting only if you want to know how much metaspace contributes to the virtual size of the process. All IMHO, since I am no GC log expert. What do you think? Maybe something like this: "Metaspace: -> (class: -> , non-class: -> )" or "Metaspace: / -> / (class: / -> /, non-class: / -> /)" or with reserved included as well: "Metaspace: // -> // (class: // -> //, non-class: // -> //)" > ---- > > Since used = class used + nonclass used, I would simplify > PreMetaspaceValues to _non_class_used and _class_used and calculate the sum > on the fly. > > > Yeah, I thought about that. I just decided to just store the total used to > avoid replicating the logic of how that?s calculated in that class too. > > > Ok. > > --- > > You added a new dependency to heap.hpp into metaspace.cpp for the format > string. Is this really necessary? Can you move this define to metaspace.cpp? > > If not, you need at least add include heap.hpp (did it build this way > without precompiled headers?) > > > I didn?t try to explicitly compile without precompiled headers. jdk submit > job was successful. Is that usually evidence enough? If not, I?ll need to > add the extra step to my workflow. > I think Oracle now testbuilds at least one no-pch build in jdk-submit. > As I eluded to earlier, if this change is accepted I?m going to suggest to > also expand the gc,heap output with sub-space information for the young gen > (eden / survivors). So I wanted to move the FORMAT macros to somewhere > that can also be shared by GC files. I was not sure if heap.hpp was the > best place for them. Maybe I can just put them in globalDefinitions.hpp, as > there?s already some size formatting utilities there > (proper_unit_for_byte_size(?), etc.)? > > > Alternatively you could move the printing code out of metaspace.cpp completely and add it to some heap cpp file, since arguably it seems more like metaspace printing on behalf of heap coding. I leave it up to you. If you keep including heap.hpp, please include it explicitly. I think you may pick it up by accident already which is why it did build for you. > > ---- > > The following issues are matters of taste, and I leave it up to you > whether you want to change something: > > -> I wince a bit at yet another metaspace-sizes-structure - we have > already ClassLoaderMetaspaceStatistics and MetaspaceCounters, and maybe > others. JFR may also do something similar. Would it be possible to reuse > one of those? (I guess ClassLoaderMetaspaceStatistics is too fine granular > and too expensive to collect.) > > > I was not aware of the two you mentioned. Aren?t MetaspaceCounters only > available when UsePerfData is enabled? I?d rather decouple this from > UsePerfData. ClassLoaderMetaspaceStatistics does seem too fine-grain for > what I want. And I modeled PreMetaspaceValues on PreGCValues used in > ParallelGC, as it?s a nice way to package up what?s needed in an object and > easily re-use it in the few places where it?s needed. The way is done now, > i.e., just store the metaspace used size and pass it to the print method, > is a bit ad-hoc (IMHO). > I agree. Using a structure here is way cleaner. My original thoughts went like this: a number of callers seem all to wish for a quick collection of metaspace statistics - those easy to obtain without locking and walking structures - so lets give them a generic simple API for it. But as I wrote, this is all a matter of taste, and consolidating the callers would be outside the scope of your change. Please go ahead and add your proposed structure and API; if we wish, we can at some future point consolidate the various callers to one API. Should not be too difficult. > -> I would prefer the class PreMetaspaceValues to live inside an own > header in metaspace folder, and in the metaspace namespace like we did for > similar helpers. > > > I can definitely do that. > See above. If we plan for a later brush up of this API, do as you see fit. > > > -> I would take the "Pre" out of the name and the member names and just > name it "Snapshot" and the members "used" and "class used". > > > Again, I modeled it based on ParallelGC's PreGCValues class. But I can > rename it to Snapshot to generalize its use. > > > > -> I would probably have made PreMetaspaceValues a dumb structure and > added a function "MetaspaceAux::get_dimensions()" instead of adding the > implementation to the constructor and to the header file. At least I would > prefer the ctor implementation to the cpp file. > > > I can move it to a cpp file. This will also eliminate a cyclic dependency > I had to address by the class forward declaration. > Thank you. Just occurred to me, could you add a regression test? Or does one already exist and need to be adapted? Cheers, Thomas > > Tony > > > > > Cheers, Thomas > > On Wed, May 8, 2019 at 4:20 PM Tony Printezis > wrote: > >> Any chance of someone taking a look at this? >> >> http://cr.openjdk.java.net/~tonyp/8223575/webrev.0/ >> >> As I said on the JIRA, users can be confused when they metaspace capacity >> shows as over 1G even though they asked for 128M. Breaking down the spaces, >> when coops are enabled, can address this confusion (and having the actual >> occupancy of each space separately is also a lot more informative and >> avoids guesswork). The output looks like this when coops are enabled: >> >> [10.090s][info][gc,metaspace ] GC(8) Metaspace: >> 2130K->2130K(1056768K) ClassMetadata: 1906K->1906K(8192K) ClassSpace: >> 223K->223K(1048576K) >> >> and like this (i.e., what it was before the change) when coops are not >> enabled: >> >> [25.767s][info][gc,metaspace ] GC(28) Metaspace: 8107K->8107K(10240K) >> >> A couple of notes on the changes: >> >> - I?m open to different suggestions for what to call ClassMetadata and >> ClassSpace. :-) >> - I didn?t know where to put the HEAP_CHANGE_FORMAT and >> HEAP_CHANGE_FORMAT_ARGS macros, so that they can be easily shared, and I >> ended up putting them in heap.hpp. Let me know if there?s a better place >> for them. >> >> Tony >> >> >> ????? >> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >> >> > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shade at redhat.com Mon May 13 08:50:46 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 13 May 2019 10:50:46 +0200 Subject: RFR (S) 8223759: Shenandoah should allow arbitrarily low initial heap size Message-ID: <1624bd21-2f99-9009-f603-f958579df062@redhat.com> RFE: https://bugs.openjdk.java.net/browse/JDK-8223759 Shenandoah heap regions checks verify that initial heap size covers at least 10 regions. This means the lowest uncommitted size is at least 10*RegionSize, which is a lot for large heaps. This seems to be the reminiscent of old resizing scheme, where Shenandoah could have run with "initial heap" for a long time, before GC code decided to resize. There is little sense it in now: Shenandoah would commit regions up to max_capacity if needed, including during the initial barrage of allocations. We should instead check that "max" heap size covers at least 10 regions. Patch: http://cr.openjdk.java.net/~shade/8223759/webrev.01/ Testing: hotspot_gc_shenandoah -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Mon May 13 09:48:55 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 13 May 2019 11:48:55 +0200 Subject: RFR (M): 8200545: Improve filter for enqueued deferred cards In-Reply-To: <3DCA1C68-C1FB-49DB-8E96-791D38410600@oracle.com> References: <81ba1000d2c8eff1b4a467cd7ed75a372857244f.camel@oracle.com> <6ca33ce2-28cc-f736-676e-67f16c8f2dc2@oracle.com> <5b600e3a6116009c4e326bbee277622fc3650471.camel@oracle.com> <65EF7FB0-ADA6-4504-A73C-AE2330C08DD9@oracle.com> <9c6df03230a6c2a632908bbed556accf412f43d3.camel@oracle.com> <3DCA1C68-C1FB-49DB-8E96-791D38410600@oracle.com> Message-ID: Hi, On Fri, 2019-05-10 at 18:53 -0400, Kim Barrett wrote: > > On May 10, 2019, at 6:18 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > On Thu, 2019-05-09 at 20:49 -0400, Kim Barrett wrote: > > > -------------------------------------------------------------- > > > src/hotspot/share/gc/g1/g1InCSetState.hpp > > > 58 // The first two values are encoded in increasing > > > generation order, which > > > 59 // allows us to use them as index into arrays; they also > > > start at zero. > > > > > > This part of the comment doesn't seem true. Unless NotInCSet > > > (value == 0) is considered a "generation", but then there are 3 > > > rather than 2. The comment formerly talked about the "positive > > > values" (e.g. the values > 0). > > > > > > > Fixed in .2. > > Looking at webrev.3 (latest), I still see the above quoted comment > that I think is wrong, or at least very confusing. I removed the comment; it does not seem necessary, and does not answer any real question from what I can see. I assigned JDK-8223693 to me and will fix that asap to remove the problem with the NotInCSet value. > > > And places that use these as array indices generally "waste" an > > > entry for NotInCSet. I thought there was a CR for that, but > > > couldn't find it. > > > > This had at least initially been intentional. I do not remember a > > CR for that. > > When originally introducing this table, I deliberately wanted to > > make the decision between in cset and not in cset a decision > > simple. Initially this has just been != 0 / == 0, but with the > > proliferation of information in that entry (first humongous reclaim > > candidate, then optional evacuation) it has changed to > 0. > > > > You are right that we might want to reconsider this decision now. > > It's not a big waste of space in the arrays. I filed JDK-8223693 > > for cleaning this up. > > OK. Though I agree the space impact is likely small, and it might > not be worth the trouble to fix anymore, despite it being potentially > confusing. It's like three or four words per gc thread during evacuation afaict. > > > --------------------------------------------------------------- > > > > > > src/hotspot/share/gc/g1/g1InCSetState.hpp > > > 46 #ifdef SPARC > > > 47 uint32_t _needs_remset_update; > > > 48 #else > > > 49 uint8_t _needs_remset_update; > > > 50 #endif > > > > > > Is it actually worth conditionalizing the type of this "bool" > > > member? > > > > This has changed in .2; I conditionalized the entries to get > > minimum size of the element (i.e. sizeof(G1HeapRegionAttr)) for > > non-sparc, and word size for sparc for best performance. > > > > Note that I did not check SPARC performance, but I can see > > advantages to keep memory footprint low(er) for that table on > > everything else. > > (SPARC implementations I have tested are comparatively slow > > extracting parts of a word). > > > > > Note that access via needs_remset_update() is going to be doing > > > an integral to bool conversion. > > > > I fixed the implicit integral to bool conversion, that's indeed an > > unintentional mistake. > > Why not just make the member type ?bool? and be done with it? > Let the compiler decide the appropriate representation. We want to have control over the size of the G1HeapRegionAttr data structure to minimize its size due to already being very cache/memory bound during evacuation. So the intent is to not exacerbate the issue further by increasing the size of this table more than necessary. SPARC is the exception due to its very large performance penalty when accessing sub-word memory locations and/or the compiler doing really weird things. Initially the webrev used the LSB of the existing value member, i.e. no separate "needs_remset_update" member for this reason. I backed away from doing so before posting the webrev for clarity, and I intend to add information in the future anyway to require 16 bits. Some perf tests comparing 8 to 16 bits element entries did not show a measurable difference. [Background/anecdotal evidence: - During initial introduction of InCSetState I wrote "[...] for SPARC we define _value as intptr_t instead of int8_t. This avoids the mentioned shifts at the cost of some memory. However we want to use int8_t on other architectures because tests indicated that it is (slightly) faster. [...]" http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2014-December/011557.html - some a kind of G1 clone for some different use case I worked on in the 2008-2011 timeframe also had a similar (16 bit entry; with some more fields using C-bitfields) table for the same reasons. I remember doing a lot of measurements until being satisfied and convinced that it is worth the complexity.] > > > I kind of wish some of the changes to g1InCSetState.hpp had been > > > done separately. Oh well... > > > > What exactly do you suggest? Separate renaming from the functional > > change? Or first clean up InCSetState (typedefs, enum, JDK-8223693, > > ..), then rename and then add the new functionality? > > I think separating renamings from functional changes might have made > reviewing easier, at least for me. Of course, that?s more work for > you. There has been such a separation for the initial webrev (the "webrev" vs. the "webrev.rename" webrevs), but I did not continue that further down on assuming that after that the renaming would be accepted and I was not required to continue providing two webrevs for each. I thought this would be enough, but it looks like I erred here. I will separate these changes out more obviously in the future. Apologies. New webrevs: http://cr.openjdk.java.net/~tschatzl/8200545/webrev.3_to_4/ (diff) http://cr.openjdk.java.net/~tschatzl/8200545/webrev.4/ (full) Thanks, Thomas From shade at redhat.com Mon May 13 10:30:41 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 13 May 2019 12:30:41 +0200 Subject: RFR (S) 8223762: Shenandoah: overflows in calculations involving heap capacity Message-ID: <98b79f57-cb8e-1885-d9c4-c99729ed7840@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8223762 Heap capacity is in size_t. Which means expressions involving it are subject to overflow if done without care. Consider the x86_32 case with max_capacity = 1G and ShenandoahMinFreeThreshold >= 5. this overflows this expression here on 32-bit platforms: size_t min_threshold = ShenandoahMinFreeThreshold * heap->max_capacity() / 100; This readily happens on x86_32 and wreck up heuristics calculations. Fix: http://cr.openjdk.java.net/~shade/8223762/webrev.01/ Testing: hotspot_gc_shenandoah (x86_64, x86_32), plus ad-hoc x86_32 tests -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Mon May 13 10:35:53 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 13 May 2019 12:35:53 +0200 Subject: RFR (S) 8223759: Shenandoah should allow arbitrarily low initial heap size In-Reply-To: <1624bd21-2f99-9009-f603-f958579df062@redhat.com> References: <1624bd21-2f99-9009-f603-f958579df062@redhat.com> Message-ID: Ok. Thanks, Roman > RFE: > https://bugs.openjdk.java.net/browse/JDK-8223759 > > Shenandoah heap regions checks verify that initial heap size covers at least 10 regions. This means > the lowest uncommitted size is at least 10*RegionSize, which is a lot for large heaps. This seems to > be the reminiscent of old resizing scheme, where Shenandoah could have run with "initial heap" for a > long time, before GC code decided to resize. There is little sense it in now: Shenandoah would > commit regions up to max_capacity if needed, including during the initial barrage of allocations. We > should instead check that "max" heap size covers at least 10 regions. > > Patch: > http://cr.openjdk.java.net/~shade/8223759/webrev.01/ > > Testing: hotspot_gc_shenandoah > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Mon May 13 10:37:34 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 13 May 2019 12:37:34 +0200 Subject: RFR (S) 8223762: Shenandoah: overflows in calculations involving heap capacity In-Reply-To: <98b79f57-cb8e-1885-d9c4-c99729ed7840@redhat.com> References: <98b79f57-cb8e-1885-d9c4-c99729ed7840@redhat.com> Message-ID: <648db01b-ac1d-bd4e-5784-40b90a6ab01f@redhat.com> Looks good! Thanks! Roman > Bug: > https://bugs.openjdk.java.net/browse/JDK-8223762 > > Heap capacity is in size_t. Which means expressions involving it are subject to overflow if done > without care. Consider the x86_32 case with max_capacity = 1G and ShenandoahMinFreeThreshold >= 5. > this overflows this expression here on 32-bit platforms: > size_t min_threshold = ShenandoahMinFreeThreshold * heap->max_capacity() / 100; > > This readily happens on x86_32 and wreck up heuristics calculations. > > Fix: > http://cr.openjdk.java.net/~shade/8223762/webrev.01/ > > Testing: hotspot_gc_shenandoah (x86_64, x86_32), plus ad-hoc x86_32 tests > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Mon May 13 10:39:43 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 13 May 2019 12:39:43 +0200 Subject: RFR(XS) 8223215: Shenandoah: Support verifying subset of roots In-Reply-To: <53165b5d-1303-9178-a1a6-34062203f2ad@redhat.com> References: <255952db-976e-fa5e-a847-cb92cc049a4a@redhat.com> <53165b5d-1303-9178-a1a6-34062203f2ad@redhat.com> Message-ID: <596c24c0-95fc-e593-26a5-2bcb498223c4@redhat.com> On 5/10/19 11:14 PM, Zhengyu Gu wrote: > > On 5/10/19 3:12 PM, Roman Kennke wrote: >> Hi Zhengyu, >> >> this looks good. >> Maybe the root-verifier code should be in its own files? > > Updated: http://cr.openjdk.java.net/~zgu/JDK-8223215/webrev.01/ Stylistic nits: *) New line is missing here in shenandoahRootVerifier.cpp? 54 } 55 if (verify(CLDGRoots)) { *) You sure uint is not too narrow here? I think it's fine, but maybe casting through uintptr_t is safer? 42 void ShenandoahRootVerifier::excludes(RootTypes types) { 43 _types = static_cast(static_cast(_types) & (~static_cast(types))); 44 } -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From thomas.stuefe at gmail.com Mon May 13 10:50:32 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 13 May 2019 12:50:32 +0200 Subject: RFR(S): 8223575: add subspace transitions to gc+metaspace=info log lines In-Reply-To: References: Message-ID: Hi Tony, had a look at your second webrev. Seems fine, modulo the discussion points from my mail from earlier today. Small nit, I personally prefer pointers vs references for output structures, but I leave it up to you whether you want to change it. Cheers, Thomas On Thu, May 9, 2019 at 10:00 PM Tony Printezis wrote: > Thomas, > > Updated webrev: > > http://cr.openjdk.java.net/~tonyp/8223575/webrev.1/ > > with some of the suggested changes: > > * did some renaming > * created hpp / cpp files for the new class (now called > MetaspaceSizesSnapshot) > * moved the formatting macros to globalDefinitions.hpp > > Still pending: > > * What should we call the spaces? > * Should we add the before capacity in the output? > > (let me know if I missed anything?) > > Tony > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > On May 9, 2019 at 12:38:35 PM, Tony Printezis (tprintezis at twitter.com) > wrote: > > Hi Thomas, > > Thanks for the detailed feedback and suggestions. Please see inline?. > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > On May 8, 2019 at 4:07:49 PM, Thomas St?fe (thomas.stuefe at gmail.com) > wrote: > > > Hi Tony, > > I think this makes sense. I agree, the output as it is now can be > confusing. Note that users can still be confused since users may simply not > be aware that the default ClassSpace reserved size is 1G. > > > Yes. What a user wants to size is (what I call - more below) the class > metadata space. But, in the log, this is hidden in the total size. So, this > creates confusion. FWIW, the older +PrintHeapAtGC output had the same issue: > > Metaspace used 5660K, capacity 5996K, committed 6144K, reserved > 1056768K > > class space used 630K, capacity 734K, committed 768K, reserved > 1048576K > > It shows the total size and the class space size (which the user might not > know what it is) but doesn?t explicitly show the space the user actually > wants to size. > > I think we can make it a bit easier for them. > > > > Can your change break scripts for folks parsing GC logs? I see that you > only appended content and did not change the existing text, so I guess this > is okay. > > > Of course the change can break a GC log parsing script, if it assumes > there?s only one size transition on gc,metaspace lines (but if it just > parses the size transition and ignores that rest it will still work). But > is there a hard requirement that we cannot change any log output? That?d be > unfortunate, IMHO. In fact, the safepoint=info output changed almost > totally between 11 and now. > > 11: > > [49.768s][info ][safepoint ] Entering safepoint region: Cleanup > > ... > > [49.769s][info ][safepoint ] Total time for which application > threads were stopped: 0.0005466 seconds, Stopping threads took: 0.0001931 > seconds > > now: > > [880.754s][info ][safepoint ] Safepoint "GenCollectForAllocation", > Time since last: 990253904 ns, Reaching safepoint: 440316 ns, At safepoint: > 3761268 ns, Total: 4201584 ns > > > > --- > > Format and naming: Different forms are used in different places. In hs-err > files, we print "Metaspace" and "Class Space", with the former being the > total sum and the latter the class space part. > > But in other corners, eg. in the jcmd VM.metaspace command, we print > non-class space and class space and maybe the sum in addition. Usually > non-class space is called "Non-Class". > > I personally would prefer, in this case, the latter names: "Class Space" > and "Non-Class Space" > > > Here?s another pet-peeve of mine (and I?m probably overthinking this) :-) > : Users know that the metaspace holds class metadata. So, calling the area > that actually holds the class metadata ?Non-Class Space? and the area that > holds some internal JVM data structure ?Class Space? is counter-productive > and confusing (IMHO). We don?t have to use the names I used. But any chance > of coming up with something less confusing? > > > > --- > > Note that for non-class space, reserved size can actually change when > allocating and releasing metaspace, if VirtualSpaceNodes get purged or > newly allocated. Hence total reserved size can change too. So we may want > to print out reserved before and reserved after sizes. I guess that was > also missing from the output before. > > > I?ll be very happy to add the before capacity to the output. You mean > something like this, right? > > 2300K(4192K)->2100K(4192K) > > And we should also add it to the heap size transitions too, as they have > the same issue (the heap can be resized during a GC). Of course, this will > definitely require GC log parser changes. :-) But I think the change is > worthwhile. Any objections? > > > > ---- > > Since used = class used + nonclass used, I would simplify > PreMetaspaceValues to _non_class_used and _class_used and calculate the sum > on the fly. > > > Yeah, I thought about that. I just decided to just store the total used to > avoid replicating the logic of how that?s calculated in that class too. > > > > --- > > You added a new dependency to heap.hpp into metaspace.cpp for the format > string. Is this really necessary? Can you move this define to metaspace.cpp? > > If not, you need at least add include heap.hpp (did it build this way > without precompiled headers?) > > > I didn?t try to explicitly compile without precompiled headers. jdk submit > job was successful. Is that usually evidence enough? If not, I?ll need to > add the extra step to my workflow. > > As I eluded to earlier, if this change is accepted I?m going to suggest to > also expand the gc,heap output with sub-space information for the young gen > (eden / survivors). So I wanted to move the FORMAT macros to somewhere > that can also be shared by GC files. I was not sure if heap.hpp was the > best place for them. Maybe I can just put them in globalDefinitions.hpp, as > there?s already some size formatting utilities there > (proper_unit_for_byte_size(?), etc.)? > > > > ---- > > The following issues are matters of taste, and I leave it up to you > whether you want to change something: > > -> I wince a bit at yet another metaspace-sizes-structure - we have > already ClassLoaderMetaspaceStatistics and MetaspaceCounters, and maybe > others. JFR may also do something similar. Would it be possible to reuse > one of those? (I guess ClassLoaderMetaspaceStatistics is too fine granular > and too expensive to collect.) > > > I was not aware of the two you mentioned. Aren?t MetaspaceCounters only > available when UsePerfData is enabled? I?d rather decouple this from > UsePerfData. ClassLoaderMetaspaceStatistics does seem too fine-grain for > what I want. And I modeled PreMetaspaceValues on PreGCValues used in > ParallelGC, as it?s a nice way to package up what?s needed in an object and > easily re-use it in the few places where it?s needed. The way is done now, > i.e., just store the metaspace used size and pass it to the print method, > is a bit ad-hoc (IMHO). > > > > -> I would prefer the class PreMetaspaceValues to live inside an own > header in metaspace folder, and in the metaspace namespace like we did for > similar helpers. > > > I can definitely do that. > > > > -> I would take the "Pre" out of the name and the member names and just > name it "Snapshot" and the members "used" and "class used". > > > Again, I modeled it based on ParallelGC's PreGCValues class. But I can > rename it to Snapshot to generalize its use. > > > > -> I would probably have made PreMetaspaceValues a dumb structure and > added a function "MetaspaceAux::get_dimensions()" instead of adding the > implementation to the constructor and to the header file. At least I would > prefer the ctor implementation to the cpp file. > > > I can move it to a cpp file. This will also eliminate a cyclic dependency > I had to address by the class forward declaration. > > > Tony > > > > > Cheers, Thomas > > On Wed, May 8, 2019 at 4:20 PM Tony Printezis > wrote: > >> Any chance of someone taking a look at this? >> >> http://cr.openjdk.java.net/~tonyp/8223575/webrev.0/ >> >> As I said on the JIRA, users can be confused when they metaspace capacity >> shows as over 1G even though they asked for 128M. Breaking down the spaces, >> when coops are enabled, can address this confusion (and having the actual >> occupancy of each space separately is also a lot more informative and >> avoids guesswork). The output looks like this when coops are enabled: >> >> [10.090s][info][gc,metaspace ] GC(8) Metaspace: >> 2130K->2130K(1056768K) ClassMetadata: 1906K->1906K(8192K) ClassSpace: >> 223K->223K(1048576K) >> >> and like this (i.e., what it was before the change) when coops are not >> enabled: >> >> [25.767s][info][gc,metaspace ] GC(28) Metaspace: 8107K->8107K(10240K) >> >> A couple of notes on the changes: >> >> - I?m open to different suggestions for what to call ClassMetadata and >> ClassSpace. :-) >> - I didn?t know where to put the HEAP_CHANGE_FORMAT and >> HEAP_CHANGE_FORMAT_ARGS macros, so that they can be easily shared, and I >> ended up putting them in heap.hpp. Let me know if there?s a better place >> for them. >> >> Tony >> >> >> ????? >> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >> >> > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zgu at redhat.com Mon May 13 12:43:07 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 13 May 2019 08:43:07 -0400 Subject: RFR(XS) 8223215: Shenandoah: Support verifying subset of roots In-Reply-To: <596c24c0-95fc-e593-26a5-2bcb498223c4@redhat.com> References: <255952db-976e-fa5e-a847-cb92cc049a4a@redhat.com> <53165b5d-1303-9178-a1a6-34062203f2ad@redhat.com> <596c24c0-95fc-e593-26a5-2bcb498223c4@redhat.com> Message-ID: <9c2b67ac-fa03-93fe-650f-be3faded7625@redhat.com> Hi Aleksey, On 5/13/19 6:39 AM, Aleksey Shipilev wrote: > On 5/10/19 11:14 PM, Zhengyu Gu wrote: >> >> On 5/10/19 3:12 PM, Roman Kennke wrote: >>> Hi Zhengyu, >>> >>> this looks good. >>> Maybe the root-verifier code should be in its own files? >> >> Updated: http://cr.openjdk.java.net/~zgu/JDK-8223215/webrev.01/ > > Stylistic nits: > > *) New line is missing here in shenandoahRootVerifier.cpp? > > 54 } > 55 if (verify(CLDGRoots)) { Fixed. > > *) You sure uint is not too narrow here? I think it's fine, but maybe casting through uintptr_t is > safer? > > 42 void ShenandoahRootVerifier::excludes(RootTypes types) { > 43 _types = static_cast(static_cast(_types) & > (~static_cast(types))); > 44 } > Change to uintptr_t won't do any good for newly supported 32-bit platforms :-) Added STATIC_ASSERT instead. --- a/src/hotspot/share/gc/shenandoah/shenandoahRootVerifier.cpp Mon May 13 08:14:19 2019 -0400 +++ b/src/hotspot/share/gc/shenandoah/shenandoahRootVerifier.cpp Mon May 13 08:29:56 2019 -0400 @@ -36,6 +36,10 @@ #include "memory/universe.hpp" #include "runtime/thread.hpp" #include "services/management.hpp" +#include "utilities/debug.hpp" + +// Check for overflow of number of root types. +STATIC_ASSERT((static_cast(ShenandoahRootVerifier::AllRoots) + 1) > static_cast(ShenandoahRootVerifier::AllRoots)); Thanks, -Zhengyu > > -Aleksey > From bob.vandette at oracle.com Mon May 13 14:03:10 2019 From: bob.vandette at oracle.com (Bob Vandette) Date: Mon, 13 May 2019 10:03:10 -0400 Subject: RFR: 8222252 - Java ergonomics limits heap to 128GB with disabled compressed oops Message-ID: <46E18910-9D4E-4219-803C-A883F48AD43B@oracle.com> Please review this suggested fix for correcting Heap size selection when -XX:{Max,Min,Initial)RAMPercentage and their Factional equivalent options are used. There are two bugs filed that are related to this issue (8222252 and 8213175) SUMMARY of FIX: The fix corrects what I believe is an incorrect implementation of the *Percentage/*Fraction options. These options should not have caused the heap size to be fractionally based on MaxRAM or limited by MaxRAM. They instead should be based on the host memory. This is what I assume was meant by ?real memory?. product(double, MaxRAMPercentage, 25.0, \ "Maximum percentage of real memory used for maximum heap size") \ range(0.0, 100.0) When these options are selected, they should take precedence over UseCompressedOops unless UseCompressedOops is also specified on the command line. WEBREV: http://cr.openjdk.java.net/~bobv/8222252/webrev.01 BUGS: https://bugs.openjdk.java.net/browse/JDK-8222252 Java ergonomics limits heap to 128GB with disabled compressed oops https://bugs.openjdk.java.net/browse/JDK-8213175 Java ergonomics limits heap to 32GB to allow compressed oops RESULTS TABLE: On a System with 148G of RAM ./java -XX:MaxRAM=192g -XX:+PrintFlagsFinal -version | grep MaxHeapSize size_t MaxHeapSize = 32178700288 {product} {ergonomic} ./java -XX:MaxRAM=192g -XX:+PrintFlagsFinal -version | grep UseCompressedOops bool UseCompressedOops = true {lp64_product} {ergonomic} ./java -XX:MaxRAM=192g -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep MaxHeapSize size_t MaxHeapSize [ORIG] = 32178700288 {product} {ergonomic} size_t MaxHeapSize [NEW] = 113883742208 {product} {ergonomic} ./java -XX:MaxRAM=192g -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep UseCompressedOops bool UseCompressedOops [ORIG] = true {lp64_product} {ergonomic} bool UseCompressedOops [NEW] = false {lp64_product} {ergonomic} ./java -XX:MaxRAMPercentage=10 -XX:+PrintFlagsFinal -version | grep MaxHeapSize size_t MaxHeapSize [ORIG] = 13744734208 {product} {ergonomic} size_t MaxHeapSize [NEW] = 15183380480 {product} {ergonomic} ./java -XX:MaxRAMPercentage=10 -XX:+PrintFlagsFinal -version | grep UseCompressedOops bool UseCompressedOops = true {lp64_product} {ergonomic} ./java -XX:+PrintFlagsFinal -version | grep MaxHeapSize size_t MaxHeapSize = 32178700288 {product} {ergonomic} ./java -XX:+PrintFlagsFinal -version | grep UseCompressedOops bool UseCompressedOops = true {lp64_product} {ergonomic} ./java -XX:-UseCompressedOops -XX:+PrintFlagsFinal -version | grep MaxHeapSize size_t MaxHeapSize = 34359738368 {product} {ergonomic} ./java -XX:-UseCompressedOops -XX:+PrintFlagsFinal -version | grep UseCompressedOops bool UseCompressedOops = false {lp64_product} {command line} ./java -XX:+UseCompressedOops -XX:MaxRAM=128g -XX:+PrintFlagsFinal -version | grep MaxHeapSize size_t MaxHeapSize = 32178700288 {product} {ergonomic} ./java -XX:+UseCompressedOops -XX:MaxRAM=128g -XX:+PrintFlagsFinal -version | grep UseCompressedOops bool UseCompressedOops = true {lp64_product} {command line} ./java -XX:+UseCompressedOops -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep MaxHeapSize size_t MaxHeapSize = 32178700288 {product} {ergonomic} ./java -XX:+UseCompressedOops -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep UseCompressedOops bool UseCompressedOops = true {lp64_product} {command line} Bob. From zgu at redhat.com Mon May 13 15:00:48 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 13 May 2019 11:00:48 -0400 Subject: RFR 8223774: Shenandoah: Refactor ShenandoahRootProcessor and family Message-ID: I would like to refactor ShenandoahRootProcessor and family, because it is bit hard to track and reason the combinations of incoming parameters, and combinations are likely to increase when moving more processing to concurrent phases. Also, fixed a traversal bug that did not do StringDedup::gc_prologue()/gc_epilogue() in ShenandoahInitTraversalCollectionTask, bug showed up with refactored code. The refacoring removed some verification assertions, but we now have equivalent functionality in verifier. Bug: https://bugs.openjdk.java.net/browse/JDK-8223774 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8223774/webrev.00/index.html Test: hotspot_gc_shenandoah (fastdebug and release) Thanks, -Zhengyu From boris.ulasevich at bell-sw.com Mon May 13 15:27:37 2019 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Mon, 13 May 2019 18:27:37 +0300 Subject: RFR: 8223767: Shenandoah fails to build on Solaris x86_64 Message-ID: Hi all, Please review a set of simple changes to satisfy Oracle Solaris Studio 12.4 which complains about last comma in enums, private access specifier for the given enum used outside, and finds names conflict between function parameter and struct field names. http://bugs.openjdk.java.net/browse/JDK-8223767 http://cr.openjdk.java.net/~bulasevich/8223767/webrev.00 With the given change release and fastdebug builds are Ok, and hotspot_gc_shenandoah tests works good. thanks, Boris From rkennke at redhat.com Mon May 13 15:34:13 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 13 May 2019 17:34:13 +0200 Subject: RFR 8223774: Shenandoah: Refactor ShenandoahRootProcessor and family In-Reply-To: References: Message-ID: The webrev lists both: rev 54809 : 8223215: Shenandoah: Support verifying subset of roots rev 54810 : 8223774: Shenandoah: Refactor ShenandoahRootProcessor and family Not a big deal though. The templatization of ShenandoahRootUpdater... wouldn't it make sense to also do this for the other root iterators? Right now, the interesting stuff e.g. Thread::oops_do() is not templatized, but I suspect it would benefit if it did, and avoid virtual calls in the thread scanning code. Other than that, looks good to me. Let Aleksey also have a look. Roman > I would like to refactor ShenandoahRootProcessor and family, because it > is bit hard to track and reason the combinations of incoming parameters, > and combinations are likely to increase when moving more processing to > concurrent phases. > > Also, fixed a traversal bug that did not do > StringDedup::gc_prologue()/gc_epilogue() in > ShenandoahInitTraversalCollectionTask, bug showed up with refactored code. > > The refacoring removed some verification assertions, but we now have > equivalent functionality in verifier. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8223774 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8223774/webrev.00/index.html > > Test: > ? hotspot_gc_shenandoah (fastdebug and release) > > Thanks, > > -Zhengyu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Mon May 13 15:43:49 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 13 May 2019 17:43:49 +0200 Subject: RFR: 8223767: Shenandoah fails to build on Solaris x86_64 In-Reply-To: References: Message-ID: <87dcfaac-f351-b8d5-ba30-4029124e33f6@redhat.com> On 5/13/19 5:27 PM, Boris Ulasevich wrote: > http://bugs.openjdk.java.net/browse/JDK-8223767 > http://cr.openjdk.java.net/~bulasevich/8223767/webrev.00 Thank you, this looks good. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Mon May 13 16:05:49 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 13 May 2019 18:05:49 +0200 Subject: RFR: 8223767: Shenandoah fails to build on Solaris x86_64 In-Reply-To: References: Message-ID: > Please review a set of simple changes to satisfy Oracle Solaris Studio > 12.4 which complains about last comma in enums, private access specifier > for the given enum used outside, and finds names conflict between > function parameter and struct field names. > > http://bugs.openjdk.java.net/browse/JDK-8223767 > http://cr.openjdk.java.net/~bulasevich/8223767/webrev.00 > > With the given change release and fastdebug builds are Ok, and > hotspot_gc_shenandoah tests works good. Looks good to me! Thank you!! Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From kim.barrett at oracle.com Mon May 13 22:30:21 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 13 May 2019 18:30:21 -0400 Subject: RFR (M): 8200545: Improve filter for enqueued deferred cards In-Reply-To: References: <81ba1000d2c8eff1b4a467cd7ed75a372857244f.camel@oracle.com> <6ca33ce2-28cc-f736-676e-67f16c8f2dc2@oracle.com> <5b600e3a6116009c4e326bbee277622fc3650471.camel@oracle.com> <65EF7FB0-ADA6-4504-A73C-AE2330C08DD9@oracle.com> <9c6df03230a6c2a632908bbed556accf412f43d3.camel@oracle.com> <3DCA1C68-C1FB-49DB-8E96-791D38410600@oracle.com> Message-ID: <3B094DD3-F636-457F-B5DE-317183AF646D@oracle.com> On May 13, 2019, at 5:48 AM, Thomas Schatzl wrote: > On Fri, 2019-05-10 at 18:53 -0400, Kim Barrett wrote: >>>> And places that use these as array indices generally "waste" an >>>> entry for NotInCSet. I thought there was a CR for that, but >>>> couldn't find it. >>> >>> This had at least initially been intentional. I do not remember a >>> CR for that. >>> When originally introducing this table, I deliberately wanted to >>> make the decision between in cset and not in cset a decision >>> simple. Initially this has just been != 0 / == 0, but with the >>> proliferation of information in that entry (first humongous reclaim >>> candidate, then optional evacuation) it has changed to > 0. >>> >>> You are right that we might want to reconsider this decision now. >>> It's not a big waste of space in the arrays. I filed JDK-8223693 >>> for cleaning this up. >> >> OK. Though I agree the space impact is likely small, and it might >> not be worth the trouble to fix anymore, despite it being potentially >> confusing. > > It's like three or four words per gc thread during evacuation afaict. Right, which seems like not a lot. I think the main issue is the chance of confusion; not sure it?s worth the development effort (esp. perf. testing) needed to change it at this point. >>> I fixed the implicit integral to bool conversion, that's indeed an >>> unintentional mistake. >> >> Why not just make the member type ?bool? and be done with it? >> Let the compiler decide the appropriate representation. > > We want to have control over the size of the G1HeapRegionAttr data > structure to minimize its size due to already being very cache/memory > bound during evacuation. So the intent is to not exacerbate the issue > further by increasing the size of this table more than necessary. SPARC > is the exception due to its very large performance penalty when > accessing sub-word memory locations and/or the compiler doing really > weird things. I'm curious where that's an issue, but we can talk about it offline. I also wonder whether "old" performance measurements might have been affected / spoiled by old value passing ABI, since the state type is not POD but is trivially copyable / destructable. Note that sizeof(bool) is typically 1 (though the standard explicitly does not guarantee that). But if there are going to be more bits coming, then okay. >>>> I kind of wish some of the changes to g1InCSetState.hpp had been >>>> done separately. Oh well... >>> >>> What exactly do you suggest? Separate renaming from the functional >>> change? Or first clean up InCSetState (typedefs, enum, JDK-8223693, >>> ..), then rename and then add the new functionality? >> >> I think separating renamings from functional changes might have made >> reviewing easier, at least for me. Of course, that?s more work for >> you. > > There has been such a separation for the initial webrev (the "webrev" > vs. the "webrev.rename" webrevs), but I did not continue that further > down on assuming that after that the renaming would be accepted and I > was not required to continue providing two webrevs for each. > > I thought this would be enough, but it looks like I erred here. I will > separate these changes out more obviously in the future. Apologies. I joined the discussion after revisions had been made, and started reviewing with webrev.2. In conjunction with how updates were provided, that led to trouble? > New webrevs: > > http://cr.openjdk.java.net/~tschatzl/8200545/webrev.3_to_4/ (diff) > > http://cr.openjdk.java.net/~tschatzl/8200545/webrev.4/ (full) Looks good. From boris.ulasevich at bell-sw.com Tue May 14 04:39:01 2019 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Tue, 14 May 2019 07:39:01 +0300 Subject: RFR: 8223767: Shenandoah fails to build on Solaris x86_64 In-Reply-To: <87dcfaac-f351-b8d5-ba30-4029124e33f6@redhat.com> References: <87dcfaac-f351-b8d5-ba30-4029124e33f6@redhat.com> Message-ID: <3387252d-af57-0f52-5219-98abccdf810b@bell-sw.com> Thanks for the review! 13.05.2019 18:43, Aleksey Shipilev: > On 5/13/19 5:27 PM, Boris Ulasevich wrote: >> http://bugs.openjdk.java.net/browse/JDK-8223767 >> http://cr.openjdk.java.net/~bulasevich/8223767/webrev.00 > Thank you, this looks good. > > -Aleksey From fujie at loongson.cn Tue May 14 07:35:09 2019 From: fujie at loongson.cn (Jie Fu) Date: Tue, 14 May 2019 15:35:09 +0800 Subject: RFR: 8223822: Add gc IDs in the log of gc verification Message-ID: Hi all, JBS: https://bugs.openjdk.java.net/browse/JDK-8223822 Please review the following patch which improves the readability of gc verification log. The change is helpful for gc analysis and debugging. ----------------------------------- diff -r 6a60270af76b src/hotspot/share/logging/logPrefix.hpp --- a/src/hotspot/share/logging/logPrefix.hpp?? Mon May 13 20:45:03 2019 -0700 +++ b/src/hotspot/share/logging/logPrefix.hpp?? Tue May 14 13:38:11 2019 +0800 @@ -71,6 +71,8 @@ ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, ref)) \ ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, start)) \ ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, task)) \ +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, verify)) \ +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, verify, start)) \ ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, plab)) \ ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, promotion)) \ ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, region)) \ @@ -88,6 +90,8 @@ ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, task, time)) \ ?? DEBUG_ONLY(LOG_PREFIX(Test_log_prefix_prefixer, LOG_TAGS(logging, test))) \ ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, tlab)) \ +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, verify)) \ +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, verify, start)) \ ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, workgang)) ----------------------------------- For more info, please see the JBS. Testing: ? - make test TEST="jtreg:hotspot_gc" CONF=release Thanks a lot. Best regards, Jie From stefan.karlsson at oracle.com Tue May 14 08:33:11 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 14 May 2019 10:33:11 +0200 Subject: RFR: 8223822: Add gc IDs in the log of gc verification In-Reply-To: References: Message-ID: Hi Jie, This looks good to me. I've run with fastdebug and verified that we get the GC IDs as expected. Thanks, StefanK On 2019-05-14 09:35, Jie Fu wrote: > Hi all, > > JBS: https://bugs.openjdk.java.net/browse/JDK-8223822 > > Please review the following patch which improves the readability of gc > verification log. > The change is helpful for gc analysis and debugging. > ----------------------------------- > diff -r 6a60270af76b src/hotspot/share/logging/logPrefix.hpp > --- a/src/hotspot/share/logging/logPrefix.hpp?? Mon May 13 20:45:03 2019 > -0700 > +++ b/src/hotspot/share/logging/logPrefix.hpp?? Tue May 14 13:38:11 2019 > +0800 > @@ -71,6 +71,8 @@ > ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, ref)) \ > ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, start)) \ > ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, task)) \ > +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, verify)) \ > +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, verify, start)) \ > ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, plab)) \ > ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, promotion)) \ > ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, region)) \ > @@ -88,6 +90,8 @@ > ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, task, time)) \ > ?? DEBUG_ONLY(LOG_PREFIX(Test_log_prefix_prefixer, LOG_TAGS(logging, > test))) \ > ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, tlab)) \ > +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, verify)) \ > +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, verify, start)) \ > ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, workgang)) > ----------------------------------- > For more info, please see the JBS. > > Testing: > ? - make test TEST="jtreg:hotspot_gc" CONF=release > > Thanks a lot. > > Best regards, > Jie > > From fujie at loongson.cn Tue May 14 09:04:06 2019 From: fujie at loongson.cn (Jie Fu) Date: Tue, 14 May 2019 17:04:06 +0800 Subject: RFR: 8223822: Add gc IDs in the log of gc verification In-Reply-To: References: Message-ID: <0dcd2634-3fbc-bb7f-b813-d2b81d72be1a@loongson.cn> Hi StefanK, Thanks for your review. Do you think it's trivial? If so, could you please sponsor it? Thanks a lot. Best regards, Jie On 2019/5/14 ??4:33, Stefan Karlsson wrote: > Hi Jie, > > This looks good to me. I've run with fastdebug and verified that we > get the GC IDs as expected. > > Thanks, > StefanK > > On 2019-05-14 09:35, Jie Fu wrote: >> Hi all, >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8223822 >> >> Please review the following patch which improves the readability of >> gc verification log. >> The change is helpful for gc analysis and debugging. >> ----------------------------------- >> diff -r 6a60270af76b src/hotspot/share/logging/logPrefix.hpp >> --- a/src/hotspot/share/logging/logPrefix.hpp?? Mon May 13 20:45:03 >> 2019 -0700 >> +++ b/src/hotspot/share/logging/logPrefix.hpp?? Tue May 14 13:38:11 >> 2019 +0800 >> @@ -71,6 +71,8 @@ >> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, ref)) \ >> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, start)) \ >> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, task)) \ >> +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, verify)) \ >> +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, verify, start)) \ >> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, plab)) \ >> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, promotion)) \ >> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, region)) \ >> @@ -88,6 +90,8 @@ >> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, task, time)) \ >> ??? DEBUG_ONLY(LOG_PREFIX(Test_log_prefix_prefixer, LOG_TAGS(logging, >> test))) \ >> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, tlab)) \ >> +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, verify)) \ >> +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, verify, start)) \ >> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, workgang)) >> ----------------------------------- >> For more info, please see the JBS. >> >> Testing: >> ?? - make test TEST="jtreg:hotspot_gc" CONF=release >> >> Thanks a lot. >> >> Best regards, >> Jie >> >> From stefan.karlsson at oracle.com Tue May 14 09:06:18 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 14 May 2019 11:06:18 +0200 Subject: RFR: 8223822: Add gc IDs in the log of gc verification In-Reply-To: <0dcd2634-3fbc-bb7f-b813-d2b81d72be1a@loongson.cn> References: <0dcd2634-3fbc-bb7f-b813-d2b81d72be1a@loongson.cn> Message-ID: <7d77f973-5e34-f404-9f53-25d0acd6ecee@oracle.com> On 2019-05-14 11:04, Jie Fu wrote: > Hi StefanK, > > Thanks for your review. > > Do you think it's trivial? > If so, could you please sponsor it? I think it's simple enough, and I'll sponsor it but let's wait for a second reviewer while I run this through our testing infrastructure. Thanks, StefanK > > Thanks a lot. > Best regards, > Jie > > On 2019/5/14 ??4:33, Stefan Karlsson wrote: >> Hi Jie, >> >> This looks good to me. I've run with fastdebug and verified that we >> get the GC IDs as expected. >> >> Thanks, >> StefanK >> >> On 2019-05-14 09:35, Jie Fu wrote: >>> Hi all, >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8223822 >>> >>> Please review the following patch which improves the readability of >>> gc verification log. >>> The change is helpful for gc analysis and debugging. >>> ----------------------------------- >>> diff -r 6a60270af76b src/hotspot/share/logging/logPrefix.hpp >>> --- a/src/hotspot/share/logging/logPrefix.hpp?? Mon May 13 20:45:03 >>> 2019 -0700 >>> +++ b/src/hotspot/share/logging/logPrefix.hpp?? Tue May 14 13:38:11 >>> 2019 +0800 >>> @@ -71,6 +71,8 @@ >>> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, ref)) \ >>> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, start)) \ >>> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, task)) \ >>> +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, verify)) \ >>> +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, verify, start)) \ >>> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, plab)) \ >>> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, promotion)) \ >>> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, region)) \ >>> @@ -88,6 +90,8 @@ >>> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, task, time)) \ >>> ??? DEBUG_ONLY(LOG_PREFIX(Test_log_prefix_prefixer, LOG_TAGS(logging, >>> test))) \ >>> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, tlab)) \ >>> +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, verify)) \ >>> +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, verify, start)) \ >>> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, workgang)) >>> ----------------------------------- >>> For more info, please see the JBS. >>> >>> Testing: >>> ?? - make test TEST="jtreg:hotspot_gc" CONF=release >>> >>> Thanks a lot. >>> >>> Best regards, >>> Jie >>> >>> > From shade at redhat.com Tue May 14 09:09:18 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 14 May 2019 11:09:18 +0200 Subject: RFR: 8223767: Shenandoah fails to build on Solaris x86_64 In-Reply-To: <3387252d-af57-0f52-5219-98abccdf810b@bell-sw.com> References: <87dcfaac-f351-b8d5-ba30-4029124e33f6@redhat.com> <3387252d-af57-0f52-5219-98abccdf810b@bell-sw.com> Message-ID: <9d14f592-9866-8b16-9600-37868e4b2407@redhat.com> Feel free to push it to jdk/jdk, whenever you are ready. -Aleksey On 5/14/19 6:39 AM, Boris Ulasevich wrote: > Thanks for the review! > > 13.05.2019 18:43, Aleksey Shipilev: >> On 5/13/19 5:27 PM, Boris Ulasevich wrote: >>> http://bugs.openjdk.java.net/browse/JDK-8223767 >>> http://cr.openjdk.java.net/~bulasevich/8223767/webrev.00 >> Thank you, this looks good. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Tue May 14 09:25:08 2019 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 14 May 2019 11:25:08 +0200 Subject: RFR: 8223767: Shenandoah fails to build on Solaris x86_64 In-Reply-To: <3387252d-af57-0f52-5219-98abccdf810b@bell-sw.com> References: <87dcfaac-f351-b8d5-ba30-4029124e33f6@redhat.com> <3387252d-af57-0f52-5219-98abccdf810b@bell-sw.com> Message-ID: Hi Boris, Would you help backporting the patch to sh/jdk11 and sh/jdk8 at some point? It would be cool to have there, but we have no way to test it... Thanks, Roman > Thanks for the review! > > 13.05.2019 18:43, Aleksey Shipilev: >> On 5/13/19 5:27 PM, Boris Ulasevich wrote: >>> http://bugs.openjdk.java.net/browse/JDK-8223767 >>> http://cr.openjdk.java.net/~bulasevich/8223767/webrev.00 >> Thank you, this looks good. >> >> -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Tue May 14 09:29:10 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 14 May 2019 11:29:10 +0200 Subject: RFR: 8223822: Add gc IDs in the log of gc verification In-Reply-To: References: Message-ID: <68076584daa7ffbac3795e499a3ae3b84aa2c54a.camel@oracle.com> Hi, On Tue, 2019-05-14 at 15:35 +0800, Jie Fu wrote: > Hi all, > > JBS: https://bugs.openjdk.java.net/browse/JDK-8223822 > > Please review the following patch which improves the readability of > gc verification log. > The change is helpful for gc analysis and debugging. > ----------------------------------- > diff -r 6a60270af76b src/hotspot/share/logging/logPrefix.hpp > --- a/src/hotspot/share/logging/logPrefix.hpp Mon May 13 20:45:03 > 2019 looks good. Thomas From david.holmes at oracle.com Tue May 14 10:10:45 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 14 May 2019 20:10:45 +1000 Subject: RFR: 8222252 - Java ergonomics limits heap to 128GB with disabled compressed oops In-Reply-To: <46E18910-9D4E-4219-803C-A883F48AD43B@oracle.com> References: <46E18910-9D4E-4219-803C-A883F48AD43B@oracle.com> Message-ID: Hi Bob, On 14/05/2019 12:03 am, Bob Vandette wrote: > Please review this suggested fix for correcting Heap size selection when > -XX:{Max,Min,Initial)RAMPercentage and their Factional equivalent options > are used. > > There are two bugs filed that are related to this issue (8222252 and 8213175) > > SUMMARY of FIX: > The fix corrects what I believe is an incorrect implementation of the *Percentage/*Fraction > options. These options should not have caused the heap size to be fractionally based on MaxRAM > or limited by MaxRAM. They instead should be based on the host memory. This is what I assume > was meant by ?real memory?. > > product(double, MaxRAMPercentage, 25.0, \ > "Maximum percentage of real memory used for maximum heap size") \ > range(0.0, 100.0) Perhaps - but if I'm reading things correctly this change will now potentially result in a different amount of memory being used if physical memory > MaxRAM default - right? In which case this is a behavioural change that will require a CSR request. Further, I think if someone sets MaxRAM and one of the "fraction" flags then they may well want the fraction to apply to the MaxRAM value they gave - so I think you're use of physical_memory() should also require FLAG_IS_DEFAULT(MaxRAM). Cheers, David ----- > When these options are selected, they should take precedence over UseCompressedOops > unless UseCompressedOops is also specified on the command line. > > > WEBREV: > http://cr.openjdk.java.net/~bobv/8222252/webrev.01 > > BUGS: > https://bugs.openjdk.java.net/browse/JDK-8222252 > Java ergonomics limits heap to 128GB with disabled compressed oops > > https://bugs.openjdk.java.net/browse/JDK-8213175 > Java ergonomics limits heap to 32GB to allow compressed oops > > > RESULTS TABLE: > > On a System with 148G of RAM > > ./java -XX:MaxRAM=192g -XX:+PrintFlagsFinal -version | grep MaxHeapSize > size_t MaxHeapSize = 32178700288 {product} {ergonomic} > ./java -XX:MaxRAM=192g -XX:+PrintFlagsFinal -version | grep UseCompressedOops > bool UseCompressedOops = true {lp64_product} {ergonomic} > > ./java -XX:MaxRAM=192g -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep MaxHeapSize > size_t MaxHeapSize [ORIG] = 32178700288 {product} {ergonomic} > size_t MaxHeapSize [NEW] = 113883742208 {product} {ergonomic} > ./java -XX:MaxRAM=192g -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep UseCompressedOops > bool UseCompressedOops [ORIG] = true {lp64_product} {ergonomic} > bool UseCompressedOops [NEW] = false {lp64_product} {ergonomic} > > ./java -XX:MaxRAMPercentage=10 -XX:+PrintFlagsFinal -version | grep MaxHeapSize > size_t MaxHeapSize [ORIG] = 13744734208 {product} {ergonomic} > size_t MaxHeapSize [NEW] = 15183380480 {product} {ergonomic} > ./java -XX:MaxRAMPercentage=10 -XX:+PrintFlagsFinal -version | grep UseCompressedOops > bool UseCompressedOops = true {lp64_product} {ergonomic} > > ./java -XX:+PrintFlagsFinal -version | grep MaxHeapSize > size_t MaxHeapSize = 32178700288 {product} {ergonomic} > ./java -XX:+PrintFlagsFinal -version | grep UseCompressedOops > bool UseCompressedOops = true {lp64_product} {ergonomic} > > ./java -XX:-UseCompressedOops -XX:+PrintFlagsFinal -version | grep MaxHeapSize > size_t MaxHeapSize = 34359738368 {product} {ergonomic} > ./java -XX:-UseCompressedOops -XX:+PrintFlagsFinal -version | grep UseCompressedOops > bool UseCompressedOops = false {lp64_product} {command line} > > ./java -XX:+UseCompressedOops -XX:MaxRAM=128g -XX:+PrintFlagsFinal -version | grep MaxHeapSize > size_t MaxHeapSize = 32178700288 {product} {ergonomic} > ./java -XX:+UseCompressedOops -XX:MaxRAM=128g -XX:+PrintFlagsFinal -version | grep UseCompressedOops > bool UseCompressedOops = true {lp64_product} {command line} > > ./java -XX:+UseCompressedOops -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep MaxHeapSize > size_t MaxHeapSize = 32178700288 {product} {ergonomic} > ./java -XX:+UseCompressedOops -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep UseCompressedOops > bool UseCompressedOops = true {lp64_product} {command line} > > > > Bob. > From fujie at loongson.cn Tue May 14 11:03:51 2019 From: fujie at loongson.cn (Jie Fu) Date: Tue, 14 May 2019 19:03:51 +0800 Subject: RFR: 8223822: Add gc IDs in the log of gc verification In-Reply-To: <68076584daa7ffbac3795e499a3ae3b84aa2c54a.camel@oracle.com> References: <68076584daa7ffbac3795e499a3ae3b84aa2c54a.camel@oracle.com> Message-ID: <6712e42d-a4f7-5e7d-1ea2-658add25b001@loongson.cn> Thanks Thomas for your review. On 2019/5/14 ??5:29, Thomas Schatzl wrote: > Hi, > > On Tue, 2019-05-14 at 15:35 +0800, Jie Fu wrote: >> Hi all, >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8223822 >> >> Please review the following patch which improves the readability of >> gc verification log. >> The change is helpful for gc analysis and debugging. >> ----------------------------------- >> diff -r 6a60270af76b src/hotspot/share/logging/logPrefix.hpp >> --- a/src/hotspot/share/logging/logPrefix.hpp Mon May 13 20:45:03 >> 2019 > looks good. > > Thomas > > From stefan.karlsson at oracle.com Tue May 14 11:53:46 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 14 May 2019 13:53:46 +0200 Subject: RFR: 8223822: Add gc IDs in the log of gc verification In-Reply-To: References: Message-ID: <90fd081d-86e2-d852-910b-705c8f8cc1af@oracle.com> The patch has now been pushed: http://hg.openjdk.java.net/jdk/jdk/rev/a82655619efd Thanks, StefanK On 2019-05-14 09:35, Jie Fu wrote: > Hi all, > > JBS: https://bugs.openjdk.java.net/browse/JDK-8223822 > > Please review the following patch which improves the readability of gc > verification log. > The change is helpful for gc analysis and debugging. > ----------------------------------- > diff -r 6a60270af76b src/hotspot/share/logging/logPrefix.hpp > --- a/src/hotspot/share/logging/logPrefix.hpp?? Mon May 13 20:45:03 2019 > -0700 > +++ b/src/hotspot/share/logging/logPrefix.hpp?? Tue May 14 13:38:11 2019 > +0800 > @@ -71,6 +71,8 @@ > ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, ref)) \ > ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, start)) \ > ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, task)) \ > +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, verify)) \ > +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, verify, start)) \ > ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, plab)) \ > ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, promotion)) \ > ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, region)) \ > @@ -88,6 +90,8 @@ > ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, task, time)) \ > ?? DEBUG_ONLY(LOG_PREFIX(Test_log_prefix_prefixer, LOG_TAGS(logging, > test))) \ > ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, tlab)) \ > +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, verify)) \ > +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, verify, start)) \ > ?? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, workgang)) > ----------------------------------- > For more info, please see the JBS. > > Testing: > ? - make test TEST="jtreg:hotspot_gc" CONF=release > > Thanks a lot. > > Best regards, > Jie > > From fujie at loongson.cn Tue May 14 11:57:31 2019 From: fujie at loongson.cn (Jie Fu) Date: Tue, 14 May 2019 19:57:31 +0800 Subject: RFR: 8223822: Add gc IDs in the log of gc verification In-Reply-To: <90fd081d-86e2-d852-910b-705c8f8cc1af@oracle.com> References: <90fd081d-86e2-d852-910b-705c8f8cc1af@oracle.com> Message-ID: Thank you so much, StefanK. On 2019/5/14 ??7:53, Stefan Karlsson wrote: > The patch has now been pushed: > http://hg.openjdk.java.net/jdk/jdk/rev/a82655619efd > > Thanks, > StefanK > > On 2019-05-14 09:35, Jie Fu wrote: >> Hi all, >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8223822 >> >> Please review the following patch which improves the readability of >> gc verification log. >> The change is helpful for gc analysis and debugging. >> ----------------------------------- >> diff -r 6a60270af76b src/hotspot/share/logging/logPrefix.hpp >> --- a/src/hotspot/share/logging/logPrefix.hpp?? Mon May 13 20:45:03 >> 2019 -0700 >> +++ b/src/hotspot/share/logging/logPrefix.hpp?? Tue May 14 13:38:11 >> 2019 +0800 >> @@ -71,6 +71,8 @@ >> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, ref)) \ >> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, start)) \ >> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, task)) \ >> +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, verify)) \ >> +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, phases, verify, start)) \ >> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, plab)) \ >> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, promotion)) \ >> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, region)) \ >> @@ -88,6 +90,8 @@ >> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, task, time)) \ >> ??? DEBUG_ONLY(LOG_PREFIX(Test_log_prefix_prefixer, LOG_TAGS(logging, >> test))) \ >> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, tlab)) \ >> +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, verify)) \ >> +? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, verify, start)) \ >> ??? LOG_PREFIX(GCId::print_prefix, LOG_TAGS(gc, workgang)) >> ----------------------------------- >> For more info, please see the JBS. >> >> Testing: >> ?? - make test TEST="jtreg:hotspot_gc" CONF=release >> >> Thanks a lot. >> >> Best regards, >> Jie >> >> From thomas.schatzl at oracle.com Tue May 14 13:26:05 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 14 May 2019 15:26:05 +0200 Subject: RFR: 8222252 - Java ergonomics limits heap to 128GB with disabled compressed oops In-Reply-To: <46E18910-9D4E-4219-803C-A883F48AD43B@oracle.com> References: <46E18910-9D4E-4219-803C-A883F48AD43B@oracle.com> Message-ID: Hi, On Mon, 2019-05-13 at 10:03 -0400, Bob Vandette wrote: > Please review this suggested fix for correcting Heap size selection > when -XX:{Max,Min,Initial)RAMPercentage and their Factional > equivalent options are used. > > There are two bugs filed that are related to this issue (8222252 and > 8213175) > > SUMMARY of FIX: > The fix corrects what I believe is an incorrect implementation of the > *Percentage/*Fraction options. These options should not have caused > the heap size to be fractionally based on MaxRAM or limited by > MaxRAM. They instead should be based on the host memory. This is > what I assume was meant by ?real memory?. > > product(double, MaxRAMPercentage, > 25.0, \ > "Maximum percentage of real memory used for maximum heap > size") \ > range(0.0, 100.0) > > When these options are selected, they should take precedence over > UseCompressedOops unless UseCompressedOops is also specified on the > command line. > > > WEBREV: > http://cr.openjdk.java.net/~bobv/8222252/webrev.01 - this is a style preference: I would assign the result of the condition in the if (arguments.cpp:1721+) to the initialization of use_os_mem_limit directly instead of first setting it to false, and then to true again if needed. Feel free to ignore. - as David pointed out, this change needs a CSR. - the interaction between MaxRAM and MaxRAMPercentage is a bit unclear; if they are independent as implemented now, then the update of one should probably update the other. Or the VM should fail if they disagree? If they are dependent, if David suggests (i.e. MaxRAMPercentage uses MaxRAM as maximum real memory if set; this is what I would tend to prefer), then not of course. - there should be regressions tests verifying this in cases where possible. I.e. it should be doable to get current os::physical_memory using e.g. whitebox and then run a few tests verifying the results of MaxHeapSize and coop mode. Particularly the interaction between MaxRAM, MaxRAMPercentage and coops mode would be interesting. Thanks, Thomas From bob.vandette at oracle.com Tue May 14 13:49:33 2019 From: bob.vandette at oracle.com (Bob Vandette) Date: Tue, 14 May 2019 09:49:33 -0400 Subject: RFR: 8222252 - Java ergonomics limits heap to 128GB with disabled compressed oops In-Reply-To: References: <46E18910-9D4E-4219-803C-A883F48AD43B@oracle.com> Message-ID: <4889284C-A2C3-4CF3-9B81-1FA0CBAF4E1B@oracle.com> > On May 14, 2019, at 9:26 AM, Thomas Schatzl wrote: > > Hi, > > On Mon, 2019-05-13 at 10:03 -0400, Bob Vandette wrote: >> Please review this suggested fix for correcting Heap size selection >> when -XX:{Max,Min,Initial)RAMPercentage and their Factional >> equivalent options are used. >> >> There are two bugs filed that are related to this issue (8222252 and >> 8213175) >> >> SUMMARY of FIX: >> The fix corrects what I believe is an incorrect implementation of the >> *Percentage/*Fraction options. These options should not have caused >> the heap size to be fractionally based on MaxRAM or limited by >> MaxRAM. They instead should be based on the host memory. This is >> what I assume was meant by ?real memory?. >> >> product(double, MaxRAMPercentage, >> 25.0, \ >> "Maximum percentage of real memory used for maximum heap >> size") \ >> range(0.0, 100.0) >> >> When these options are selected, they should take precedence over >> UseCompressedOops unless UseCompressedOops is also specified on the >> command line. >> >> >> WEBREV: >> http://cr.openjdk.java.net/~bobv/8222252/webrev.01 > > - this is a style preference: I would assign the result of the > condition in the if (arguments.cpp:1721+) to the initialization of > use_os_mem_limit directly instead of first setting it to false, and > then to true again if needed. Feel free to ignore. > > - as David pointed out, this change needs a CSR. > > - the interaction between MaxRAM and MaxRAMPercentage is a bit unclear; > if they are independent as implemented now, then the update of one > should probably update the other. Or the VM should fail if they > disagree? > > If they are dependent, if David suggests (i.e. MaxRAMPercentage uses > MaxRAM as maximum real memory if set; this is what I would tend to > prefer), then not of course. After Davids comment, I?m thinking that the user setting of MaxRAM should have the same side affect as the MaxRAMPercentage in that it should override coop_limit. If you agree then I?ll change the flag name to something like ?override_coop_limit?. I agree that I should set MaxRAM to phys_mem if the MaxRAMPercentage flags are specified. I didn?t realize that this flag was referenced outside of set_heap_size function. > > - there should be regressions tests verifying this in cases where > possible. I?ll look into that assuming I get a CSR approval on my proposed changes. Bob. > > I.e. it should be doable to get current os::physical_memory using e.g. > whitebox and then run a few tests verifying the results of MaxHeapSize > and coop mode. > > Particularly the interaction between MaxRAM, MaxRAMPercentage and coops > mode would be interesting. > > Thanks, > Thomas > > From tprintezis at twitter.com Wed May 15 08:11:06 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Wed, 15 May 2019 01:11:06 -0700 Subject: RFR(S): 8223575: add subspace transitions to gc+metaspace=info log lines In-Reply-To: References: Message-ID: Hi Thomas, Inline. ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On May 13, 2019 at 2:51:48 PM, Thomas St?fe (thomas.stuefe at gmail.com) wrote: Hi Tony, On Thu, May 9, 2019 at 6:38 PM Tony Printezis wrote: > Hi Thomas, > > Thanks for the detailed feedback and suggestions. Please see inline?. > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > On May 8, 2019 at 4:07:49 PM, Thomas St?fe (thomas.stuefe at gmail.com) > wrote: > > > Hi Tony, > > I think this makes sense. I agree, the output as it is now can be > confusing. Note that users can still be confused since users may simply not > be aware that the default ClassSpace reserved size is 1G. > > > Yes. What a user wants to size is (what I call - more below) the class > metadata space. But, in the log, this is hidden in the total size. So, this > creates confusion. FWIW, the older +PrintHeapAtGC output had the same issue: > > Metaspace used 5660K, capacity 5996K, committed 6144K, reserved > 1056768K > > class space used 630K, capacity 734K, committed 768K, reserved > 1048576K > > It shows the total size and the class space size (which the user might not > know what it is) but doesn?t explicitly show the space the user actually > wants to size. > Not sure I follow. What the user could size is either MaxMetaspaceSize, which would be the sum of committed for all sub spaces, or CompressedClassSpaceSize, which would be the reserved size of the compressed class space. In this output, both numbers are written out. This was my bad. I had convinced myself that MaxMetaspaceSize only sets the ?non-class space? max size. But, my original arguments holds, though: It?s nice to see the breakdown. > I think we can make it a bit easier for them. > > > > Can your change break scripts for folks parsing GC logs? I see that you > only appended content and did not change the existing text, so I guess this > is okay. > > > Of course the change can break a GC log parsing script, if it assumes > there?s only one size transition on gc,metaspace lines (but if it just > parses the size transition and ignores that rest it will still work). But > is there a hard requirement that we cannot change any log output? That?d be > unfortunate, IMHO. In fact, the safepoint=info output changed almost > totally between 11 and now. > > 11: > > [49.768s][info ][safepoint ] Entering safepoint region: Cleanup > > ... > > [49.769s][info ][safepoint ] Total time for which application > threads were stopped: 0.0005466 seconds, Stopping threads took: 0.0001931 > seconds > > now: > > [880.754s][info ][safepoint ] Safepoint "GenCollectForAllocation", > Time since last: 990253904 ns, Reaching safepoint: 440316 ns, At safepoint: > 3761268 ns, Total: 4201584 ns > > > Okay, lets assume it is fine to break existing scripts if we can make the output better. I agree. > > --- > > Format and naming: Different forms are used in different places. In hs-err > files, we print "Metaspace" and "Class Space", with the former being the > total sum and the latter the class space part. > > But in other corners, eg. in the jcmd VM.metaspace command, we print > non-class space and class space and maybe the sum in addition. Usually > non-class space is called "Non-Class". > > I personally would prefer, in this case, the latter names: "Class Space" > and "Non-Class Space" > > > Here?s another pet-peeve of mine (and I?m probably overthinking this) :-) > : Users know that the metaspace holds class metadata. So, calling the area > that actually holds the class metadata ?Non-Class Space? and the area that > holds some internal JVM data structure ?Class Space? is counter-productive > and confusing (IMHO). We don?t have to use the names I used. But any chance > of coming up with something less confusing? > > > Well, I'd say both are class metadata if you see it as runtime data surrounding classes living outside the java heap. But I see what you mean. This is a result of the decision to call the part of the Metaspace containing the Klass structures "compressed class space", which is a slight misnomer, since it is neither compressed nor really the (one exclusive) Class Space. :-) But then it follows that the other one is the "non-class space", or alternatively the "metaspace" which is not much clearer since the whole is also the Metaspace. Yeah, but ?non-class? is totally misleading here. Yes, yes, yes, I know I?m being pedantic. But having to explain that to people has been a pain and it?s confusing. I agree that this is not perfect, but I would not like to introduce new naming just now. That thing has been the "compressed class space" for a long time. We have parameters like "CompressedClassSpaceSize" and tools like "VM.metaspace" using that naming. We have numerous blogs and articles referencing those names. And the terminology is clear if a bit strange. So while I see your point, I would prefer to use the established naming scheme we have now instead of creating a new one, and leave the naming discussion for another day. OK, I can go with Non-Class and Class, but in protest. :-) Also: other metadata could move into the compressed class space in the future, if we decide that they should be addressed with compressed references. In theory, if some preconditions are met, we even could merge both spaces and let just all metadata live in one contiguous range. Also note that I currently work on a prototype for a JEP ( https://bugs.openjdk.java.net/browse/JDK-8221173) which revamps parts of the Metaspace allocator. One outcome would be simplified and more generic coding, with a lot less explicit decisions based on "class" or "non-class", which may make the code base more amenable to you :) We?ll see. :-) > > --- > > Note that for non-class space, reserved size can actually change when > allocating and releasing metaspace, if VirtualSpaceNodes get purged or > newly allocated. Hence total reserved size can change too. So we may want > to print out reserved before and reserved after sizes. I guess that was > also missing from the output before. > > > I?ll be very happy to add the before capacity to the output. You mean > something like this, right? > > 2300K(4192K)->2100K(4192K) > > And we should also add it to the heap size transitions too, as they have > the same issue (the heap can be resized during a GC). Of course, this will > definitely require GC log parser changes. :-) But I think the change is > worthwhile. Any objections? > I had time to think about this a bit more and have some second thoughts. To me, Reserved Size actually seems rather unimportant. If we scratch backward compatibility and therefore can freely rework the message, lets think once more: We have four sizes standing out as candidates for reporting: a) used size - which is what we use for class metadata. b) committed size - which is what we actually spend on class metadata. This is the size that hits MaxMetaspaceSize. This is also "what the OS sees what we allocate" - the part of Metaspace contributing to RSS. It includes all kind of wastages which on unhappy days can be significant, so (b) can be larger than (a). c) reserved size. For non-class space the reserved size follows very closely the committed size, with a max delta of 4MB. For the compressed class space this size is just a constant which is usually not very informative since it 1G is normally way too large to ever hit it. d) the Maxima the user set - MaxMetaspaceSize and CompressedClassSpaceSize. The former is just a number, but if we want to display something like "used (maximum)" this would be a candidate. But it can be infinite. The latter is equal to (c). I find (b) the most interesting number, since it is what we pay for Metaspace, including all overhead. I agree that b) is a lot more interesting than c) and d) for the metaspace. But a) and its difference from b) are the most interesting of all, IMHO. Does the normal gc log for the heap not also print "committed?? Yes, used / committed. So maybe we should swap "used" for "committed" in the output. It might work for the metaspace but not for the GC. A lot of users, like our devs at Twitter, run with a fixed heap size, which makes the used size most interesting to report (the committed size is basically flat). The difference between used and committed, as you say below, is also interesting, but mostly for us, not the users (e.g., to keep an eye on fragmentation I guess?). And it?d be nice to report the same metrics in both cases otherwise it will create confusion (IMHO, at least). So, I?d suggest used / committed for both metaspace and GC. And then, the delta between used and committed is also interesting, since this waste can get large. Finally, (c) is interesting only if you want to know how much metaspace contributes to the virtual size of the process. All IMHO, since I am no GC log expert. What do you think? Maybe something like this: "Metaspace: -> (class: -> , non-class: -> )" or "Metaspace: / -> / (class: / -> /, non-class: / -> /)" I?d go with this. You prefer used/committed to used(committed)? I?m OK either way. But we should also change all GC output to the same form. It will be unfortunate if they are different. Also, can I suggest we skip the round brackets, as it will just unnecessary overhead when trying to parse it. So: Metaspace: used/committed->used/committed NonClass: used/committed->used/committed Class: used/committed->used/committed (NonClass is the non-class space, right? Should we have that first, as it?s the more interesting space?) or with reserved included as well: "Metaspace: // -> // (class: // -> //, non-class: // -> //)" > ---- > > Since used = class used + nonclass used, I would simplify > PreMetaspaceValues to _non_class_used and _class_used and calculate the sum > on the fly. > > > Yeah, I thought about that. I just decided to just store the total used to > avoid replicating the logic of how that?s calculated in that class too. > > > Ok. > > --- > > You added a new dependency to heap.hpp into metaspace.cpp for the format > string. Is this really necessary? Can you move this define to metaspace.cpp? > > If not, you need at least add include heap.hpp (did it build this way > without precompiled headers?) > > > I didn?t try to explicitly compile without precompiled headers. jdk submit > job was successful. Is that usually evidence enough? If not, I?ll need to > add the extra step to my workflow. > I think Oracle now testbuilds at least one no-pch build in jdk-submit. OK, cool. > As I eluded to earlier, if this change is accepted I?m going to suggest to > also expand the gc,heap output with sub-space information for the young gen > (eden / survivors). So I wanted to move the FORMAT macros to somewhere > that can also be shared by GC files. I was not sure if heap.hpp was the > best place for them. Maybe I can just put them in globalDefinitions.hpp, as > there?s already some size formatting utilities there > (proper_unit_for_byte_size(?), etc.)? > > > Alternatively you could move the printing code out of metaspace.cpp completely and add it to some heap cpp file, since arguably it seems more like metaspace printing on behalf of heap coding. I leave it up to you. If you keep including heap.hpp, please include it explicitly. I think you may pick it up by accident already which is why it did build for you. > > ---- > > The following issues are matters of taste, and I leave it up to you > whether you want to change something: > > -> I wince a bit at yet another metaspace-sizes-structure - we have > already ClassLoaderMetaspaceStatistics and MetaspaceCounters, and maybe > others. JFR may also do something similar. Would it be possible to reuse > one of those? (I guess ClassLoaderMetaspaceStatistics is too fine granular > and too expensive to collect.) > > > I was not aware of the two you mentioned. Aren?t MetaspaceCounters only > available when UsePerfData is enabled? I?d rather decouple this from > UsePerfData. ClassLoaderMetaspaceStatistics does seem too fine-grain for > what I want. And I modeled PreMetaspaceValues on PreGCValues used in > ParallelGC, as it?s a nice way to package up what?s needed in an object and > easily re-use it in the few places where it?s needed. The way is done now, > i.e., just store the metaspace used size and pass it to the print method, > is a bit ad-hoc (IMHO). > I agree. Using a structure here is way cleaner. My original thoughts went like this: a number of callers seem all to wish for a quick collection of metaspace statistics - those easy to obtain without locking and walking structures - so lets give them a generic simple API for it. But as I wrote, this is all a matter of taste, and consolidating the callers would be outside the scope of your change. Please go ahead and add your proposed structure and API; if we wish, we can at some future point consolidate the various callers to one API. Should not be too difficult. > -> I would prefer the class PreMetaspaceValues to live inside an own > header in metaspace folder, and in the metaspace namespace like we did for > similar helpers. > > > I can definitely do that. > See above. If we plan for a later brush up of this API, do as you see fit. > > > -> I would take the "Pre" out of the name and the member names and just > name it "Snapshot" and the members "used" and "class used". > > > Again, I modeled it based on ParallelGC's PreGCValues class. But I can > rename it to Snapshot to generalize its use. > > > > -> I would probably have made PreMetaspaceValues a dumb structure and > added a function "MetaspaceAux::get_dimensions()" instead of adding the > implementation to the constructor and to the header file. At least I would > prefer the ctor implementation to the cpp file. > > > I can move it to a cpp file. This will also eliminate a cyclic dependency > I had to address by the class forward declaration. > Thank you. Just occurred to me, could you add a regression test? Or does one already exist and need to be adapted? Good point! I?ll look into that? (IIRC, I did run all the regressions tests and got no failures; but I?ll double check?) Tony Cheers, Thomas > > Tony > > > > > Cheers, Thomas > > On Wed, May 8, 2019 at 4:20 PM Tony Printezis > wrote: > >> Any chance of someone taking a look at this? >> >> http://cr.openjdk.java.net/~tonyp/8223575/webrev.0/ >> >> As I said on the JIRA, users can be confused when they metaspace capacity >> shows as over 1G even though they asked for 128M. Breaking down the spaces, >> when coops are enabled, can address this confusion (and having the actual >> occupancy of each space separately is also a lot more informative and >> avoids guesswork). The output looks like this when coops are enabled: >> >> [10.090s][info][gc,metaspace ] GC(8) Metaspace: >> 2130K->2130K(1056768K) ClassMetadata: 1906K->1906K(8192K) ClassSpace: >> 223K->223K(1048576K) >> >> and like this (i.e., what it was before the change) when coops are not >> enabled: >> >> [25.767s][info][gc,metaspace ] GC(28) Metaspace: 8107K->8107K(10240K) >> >> A couple of notes on the changes: >> >> - I?m open to different suggestions for what to call ClassMetadata and >> ClassSpace. :-) >> - I didn?t know where to put the HEAP_CHANGE_FORMAT and >> HEAP_CHANGE_FORMAT_ARGS macros, so that they can be easily shared, and I >> ended up putting them in heap.hpp. Let me know if there?s a better place >> for them. >> >> Tony >> >> >> ????? >> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >> >> > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tprintezis at twitter.com Wed May 15 08:15:20 2019 From: tprintezis at twitter.com (Tony Printezis) Date: Wed, 15 May 2019 01:15:20 -0700 Subject: RFR(S): 8223575: add subspace transitions to gc+metaspace=info log lines In-Reply-To: References: Message-ID: Thomas, Yeah, I?ll update the space names as discussed in the previous e-mail and rework the size transition format (unless I hear otherwise!). Re: I generally use references for const arguments and pointers for arguments the method modifies (to make it a bit clearer what will be modified and what it will not). Quick related question: I saw that some files seem to have SAP (and also RedHat?) copyright notices in addition to the usual Oracle ones. So, I can add a Twitter copyright line for any new files we add? I?m on vacation this week and next, so it might be a few days before I post a new webrev. Thanks again for the comments! Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On May 13, 2019 at 7:50:44 PM, Thomas St?fe (thomas.stuefe at gmail.com) wrote: Hi Tony, had a look at your second webrev. Seems fine, modulo the discussion points from my mail from earlier today. Small nit, I personally prefer pointers vs references for output structures, but I leave it up to you whether you want to change it. Cheers, Thomas On Thu, May 9, 2019 at 10:00 PM Tony Printezis wrote: > Thomas, > > Updated webrev: > > http://cr.openjdk.java.net/~tonyp/8223575/webrev.1/ > > with some of the suggested changes: > > * did some renaming > * created hpp / cpp files for the new class (now called > MetaspaceSizesSnapshot) > * moved the formatting macros to globalDefinitions.hpp > > Still pending: > > * What should we call the spaces? > * Should we add the before capacity in the output? > > (let me know if I missed anything?) > > Tony > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > On May 9, 2019 at 12:38:35 PM, Tony Printezis (tprintezis at twitter.com) > wrote: > > Hi Thomas, > > Thanks for the detailed feedback and suggestions. Please see inline?. > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > On May 8, 2019 at 4:07:49 PM, Thomas St?fe (thomas.stuefe at gmail.com) > wrote: > > > Hi Tony, > > I think this makes sense. I agree, the output as it is now can be > confusing. Note that users can still be confused since users may simply not > be aware that the default ClassSpace reserved size is 1G. > > > Yes. What a user wants to size is (what I call - more below) the class > metadata space. But, in the log, this is hidden in the total size. So, this > creates confusion. FWIW, the older +PrintHeapAtGC output had the same issue: > > Metaspace used 5660K, capacity 5996K, committed 6144K, reserved > 1056768K > > class space used 630K, capacity 734K, committed 768K, reserved > 1048576K > > It shows the total size and the class space size (which the user might not > know what it is) but doesn?t explicitly show the space the user actually > wants to size. > > I think we can make it a bit easier for them. > > > > Can your change break scripts for folks parsing GC logs? I see that you > only appended content and did not change the existing text, so I guess this > is okay. > > > Of course the change can break a GC log parsing script, if it assumes > there?s only one size transition on gc,metaspace lines (but if it just > parses the size transition and ignores that rest it will still work). But > is there a hard requirement that we cannot change any log output? That?d be > unfortunate, IMHO. In fact, the safepoint=info output changed almost > totally between 11 and now. > > 11: > > [49.768s][info ][safepoint ] Entering safepoint region: Cleanup > > ... > > [49.769s][info ][safepoint ] Total time for which application > threads were stopped: 0.0005466 seconds, Stopping threads took: 0.0001931 > seconds > > now: > > [880.754s][info ][safepoint ] Safepoint "GenCollectForAllocation", > Time since last: 990253904 ns, Reaching safepoint: 440316 ns, At safepoint: > 3761268 ns, Total: 4201584 ns > > > > --- > > Format and naming: Different forms are used in different places. In hs-err > files, we print "Metaspace" and "Class Space", with the former being the > total sum and the latter the class space part. > > But in other corners, eg. in the jcmd VM.metaspace command, we print > non-class space and class space and maybe the sum in addition. Usually > non-class space is called "Non-Class". > > I personally would prefer, in this case, the latter names: "Class Space" > and "Non-Class Space" > > > Here?s another pet-peeve of mine (and I?m probably overthinking this) :-) > : Users know that the metaspace holds class metadata. So, calling the area > that actually holds the class metadata ?Non-Class Space? and the area that > holds some internal JVM data structure ?Class Space? is counter-productive > and confusing (IMHO). We don?t have to use the names I used. But any chance > of coming up with something less confusing? > > > > --- > > Note that for non-class space, reserved size can actually change when > allocating and releasing metaspace, if VirtualSpaceNodes get purged or > newly allocated. Hence total reserved size can change too. So we may want > to print out reserved before and reserved after sizes. I guess that was > also missing from the output before. > > > I?ll be very happy to add the before capacity to the output. You mean > something like this, right? > > 2300K(4192K)->2100K(4192K) > > And we should also add it to the heap size transitions too, as they have > the same issue (the heap can be resized during a GC). Of course, this will > definitely require GC log parser changes. :-) But I think the change is > worthwhile. Any objections? > > > > ---- > > Since used = class used + nonclass used, I would simplify > PreMetaspaceValues to _non_class_used and _class_used and calculate the sum > on the fly. > > > Yeah, I thought about that. I just decided to just store the total used to > avoid replicating the logic of how that?s calculated in that class too. > > > > --- > > You added a new dependency to heap.hpp into metaspace.cpp for the format > string. Is this really necessary? Can you move this define to metaspace.cpp? > > If not, you need at least add include heap.hpp (did it build this way > without precompiled headers?) > > > I didn?t try to explicitly compile without precompiled headers. jdk submit > job was successful. Is that usually evidence enough? If not, I?ll need to > add the extra step to my workflow. > > As I eluded to earlier, if this change is accepted I?m going to suggest to > also expand the gc,heap output with sub-space information for the young gen > (eden / survivors). So I wanted to move the FORMAT macros to somewhere > that can also be shared by GC files. I was not sure if heap.hpp was the > best place for them. Maybe I can just put them in globalDefinitions.hpp, as > there?s already some size formatting utilities there > (proper_unit_for_byte_size(?), etc.)? > > > > ---- > > The following issues are matters of taste, and I leave it up to you > whether you want to change something: > > -> I wince a bit at yet another metaspace-sizes-structure - we have > already ClassLoaderMetaspaceStatistics and MetaspaceCounters, and maybe > others. JFR may also do something similar. Would it be possible to reuse > one of those? (I guess ClassLoaderMetaspaceStatistics is too fine granular > and too expensive to collect.) > > > I was not aware of the two you mentioned. Aren?t MetaspaceCounters only > available when UsePerfData is enabled? I?d rather decouple this from > UsePerfData. ClassLoaderMetaspaceStatistics does seem too fine-grain for > what I want. And I modeled PreMetaspaceValues on PreGCValues used in > ParallelGC, as it?s a nice way to package up what?s needed in an object and > easily re-use it in the few places where it?s needed. The way is done now, > i.e., just store the metaspace used size and pass it to the print method, > is a bit ad-hoc (IMHO). > > > > -> I would prefer the class PreMetaspaceValues to live inside an own > header in metaspace folder, and in the metaspace namespace like we did for > similar helpers. > > > I can definitely do that. > > > > -> I would take the "Pre" out of the name and the member names and just > name it "Snapshot" and the members "used" and "class used". > > > Again, I modeled it based on ParallelGC's PreGCValues class. But I can > rename it to Snapshot to generalize its use. > > > > -> I would probably have made PreMetaspaceValues a dumb structure and > added a function "MetaspaceAux::get_dimensions()" instead of adding the > implementation to the constructor and to the header file. At least I would > prefer the ctor implementation to the cpp file. > > > I can move it to a cpp file. This will also eliminate a cyclic dependency > I had to address by the class forward declaration. > > > Tony > > > > > Cheers, Thomas > > On Wed, May 8, 2019 at 4:20 PM Tony Printezis > wrote: > >> Any chance of someone taking a look at this? >> >> http://cr.openjdk.java.net/~tonyp/8223575/webrev.0/ >> >> As I said on the JIRA, users can be confused when they metaspace capacity >> shows as over 1G even though they asked for 128M. Breaking down the spaces, >> when coops are enabled, can address this confusion (and having the actual >> occupancy of each space separately is also a lot more informative and >> avoids guesswork). The output looks like this when coops are enabled: >> >> [10.090s][info][gc,metaspace ] GC(8) Metaspace: >> 2130K->2130K(1056768K) ClassMetadata: 1906K->1906K(8192K) ClassSpace: >> 223K->223K(1048576K) >> >> and like this (i.e., what it was before the change) when coops are not >> enabled: >> >> [25.767s][info][gc,metaspace ] GC(28) Metaspace: 8107K->8107K(10240K) >> >> A couple of notes on the changes: >> >> - I?m open to different suggestions for what to call ClassMetadata and >> ClassSpace. :-) >> - I didn?t know where to put the HEAP_CHANGE_FORMAT and >> HEAP_CHANGE_FORMAT_ARGS macros, so that they can be easily shared, and I >> ended up putting them in heap.hpp. Let me know if there?s a better place >> for them. >> >> Tony >> >> >> ????? >> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >> >> > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.stuefe at gmail.com Wed May 15 08:26:06 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 15 May 2019 10:26:06 +0200 Subject: RFR(S): 8223575: add subspace transitions to gc+metaspace=info log lines In-Reply-To: References: Message-ID: Hi Tony, I combine my answer to both your mails up into this one mail: - I am fine with printing "used" and "committed" and leaving out "reserved". - Choose whatever format you like, I like them all :) more inline: On Wed, May 15, 2019 at 10:15 AM Tony Printezis wrote: > Thomas, > > Yeah, I?ll update the space names as discussed in the previous e-mail and > rework the size transition format (unless I hear otherwise!). > > Re: I generally use references for const arguments and pointers for > arguments the method modifies (to make it a bit clearer what will be > modified and what it will not). > Yes, I prefer that too. > > Quick related question: I saw that some files seem to have SAP (and also > RedHat?) copyright notices in addition to the usual Oracle ones. So, I can > add a Twitter copyright line for any new files we add? > I usually add SAP copyrights for files I introduce and which have a reasonable large original content made by us. > > I?m on vacation this week and next, so it might be a few days before I > post a new webrev. Thanks again for the comments! > Np. Enjoy your time off. Cheers, Thomas > > Tony > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > On May 13, 2019 at 7:50:44 PM, Thomas St?fe (thomas.stuefe at gmail.com) > wrote: > > Hi Tony, > > had a look at your second webrev. Seems fine, modulo the discussion points > from my mail from earlier today. > > Small nit, I personally prefer pointers vs references for output > structures, but I leave it up to you whether you want to change it. > > Cheers, Thomas > > On Thu, May 9, 2019 at 10:00 PM Tony Printezis > wrote: > >> Thomas, >> >> Updated webrev: >> >> http://cr.openjdk.java.net/~tonyp/8223575/webrev.1/ >> >> with some of the suggested changes: >> >> * did some renaming >> * created hpp / cpp files for the new class (now called >> MetaspaceSizesSnapshot) >> * moved the formatting macros to globalDefinitions.hpp >> >> Still pending: >> >> * What should we call the spaces? >> * Should we add the before capacity in the output? >> >> (let me know if I missed anything?) >> >> Tony >> >> >> ????? >> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >> >> >> On May 9, 2019 at 12:38:35 PM, Tony Printezis (tprintezis at twitter.com) >> wrote: >> >> Hi Thomas, >> >> Thanks for the detailed feedback and suggestions. Please see inline?. >> >> >> ????? >> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >> >> >> On May 8, 2019 at 4:07:49 PM, Thomas St?fe (thomas.stuefe at gmail.com) >> wrote: >> >> >> Hi Tony, >> >> I think this makes sense. I agree, the output as it is now can be >> confusing. Note that users can still be confused since users may simply not >> be aware that the default ClassSpace reserved size is 1G. >> >> >> Yes. What a user wants to size is (what I call - more below) the class >> metadata space. But, in the log, this is hidden in the total size. So, this >> creates confusion. FWIW, the older +PrintHeapAtGC output had the same issue: >> >> Metaspace used 5660K, capacity 5996K, committed 6144K, reserved >> 1056768K >> >> class space used 630K, capacity 734K, committed 768K, reserved >> 1048576K >> >> It shows the total size and the class space size (which the user might >> not know what it is) but doesn?t explicitly show the space the user >> actually wants to size. >> >> I think we can make it a bit easier for them. >> >> >> >> Can your change break scripts for folks parsing GC logs? I see that you >> only appended content and did not change the existing text, so I guess this >> is okay. >> >> >> Of course the change can break a GC log parsing script, if it assumes >> there?s only one size transition on gc,metaspace lines (but if it just >> parses the size transition and ignores that rest it will still work). But >> is there a hard requirement that we cannot change any log output? That?d be >> unfortunate, IMHO. In fact, the safepoint=info output changed almost >> totally between 11 and now. >> >> 11: >> >> [49.768s][info ][safepoint ] Entering safepoint region: Cleanup >> >> ... >> >> [49.769s][info ][safepoint ] Total time for which application >> threads were stopped: 0.0005466 seconds, Stopping threads took: 0.0001931 >> seconds >> >> now: >> >> [880.754s][info ][safepoint ] Safepoint "GenCollectForAllocation", >> Time since last: 990253904 ns, Reaching safepoint: 440316 ns, At safepoint: >> 3761268 ns, Total: 4201584 ns >> >> >> >> --- >> >> Format and naming: Different forms are used in different places. In >> hs-err files, we print "Metaspace" and "Class Space", with the former being >> the total sum and the latter the class space part. >> >> But in other corners, eg. in the jcmd VM.metaspace command, we print >> non-class space and class space and maybe the sum in addition. Usually >> non-class space is called "Non-Class". >> >> I personally would prefer, in this case, the latter names: "Class Space" >> and "Non-Class Space" >> >> >> Here?s another pet-peeve of mine (and I?m probably overthinking this) :-) >> : Users know that the metaspace holds class metadata. So, calling the area >> that actually holds the class metadata ?Non-Class Space? and the area that >> holds some internal JVM data structure ?Class Space? is counter-productive >> and confusing (IMHO). We don?t have to use the names I used. But any chance >> of coming up with something less confusing? >> >> >> >> --- >> >> Note that for non-class space, reserved size can actually change when >> allocating and releasing metaspace, if VirtualSpaceNodes get purged or >> newly allocated. Hence total reserved size can change too. So we may want >> to print out reserved before and reserved after sizes. I guess that was >> also missing from the output before. >> >> >> I?ll be very happy to add the before capacity to the output. You mean >> something like this, right? >> >> 2300K(4192K)->2100K(4192K) >> >> And we should also add it to the heap size transitions too, as they have >> the same issue (the heap can be resized during a GC). Of course, this will >> definitely require GC log parser changes. :-) But I think the change is >> worthwhile. Any objections? >> >> >> >> ---- >> >> Since used = class used + nonclass used, I would simplify >> PreMetaspaceValues to _non_class_used and _class_used and calculate the sum >> on the fly. >> >> >> Yeah, I thought about that. I just decided to just store the total used >> to avoid replicating the logic of how that?s calculated in that class too. >> >> >> >> --- >> >> You added a new dependency to heap.hpp into metaspace.cpp for the format >> string. Is this really necessary? Can you move this define to metaspace.cpp? >> >> If not, you need at least add include heap.hpp (did it build this way >> without precompiled headers?) >> >> >> I didn?t try to explicitly compile without precompiled headers. jdk >> submit job was successful. Is that usually evidence enough? If not, I?ll >> need to add the extra step to my workflow. >> >> As I eluded to earlier, if this change is accepted I?m going to suggest >> to also expand the gc,heap output with sub-space information for the young >> gen (eden / survivors). So I wanted to move the FORMAT macros to somewhere >> that can also be shared by GC files. I was not sure if heap.hpp was the >> best place for them. Maybe I can just put them in globalDefinitions.hpp, as >> there?s already some size formatting utilities there >> (proper_unit_for_byte_size(?), etc.)? >> >> >> >> ---- >> >> The following issues are matters of taste, and I leave it up to you >> whether you want to change something: >> >> -> I wince a bit at yet another metaspace-sizes-structure - we have >> already ClassLoaderMetaspaceStatistics and MetaspaceCounters, and maybe >> others. JFR may also do something similar. Would it be possible to reuse >> one of those? (I guess ClassLoaderMetaspaceStatistics is too fine granular >> and too expensive to collect.) >> >> >> I was not aware of the two you mentioned. Aren?t MetaspaceCounters only >> available when UsePerfData is enabled? I?d rather decouple this from >> UsePerfData. ClassLoaderMetaspaceStatistics does seem too fine-grain for >> what I want. And I modeled PreMetaspaceValues on PreGCValues used in >> ParallelGC, as it?s a nice way to package up what?s needed in an object and >> easily re-use it in the few places where it?s needed. The way is done now, >> i.e., just store the metaspace used size and pass it to the print method, >> is a bit ad-hoc (IMHO). >> >> >> >> -> I would prefer the class PreMetaspaceValues to live inside an own >> header in metaspace folder, and in the metaspace namespace like we did for >> similar helpers. >> >> >> I can definitely do that. >> >> >> >> -> I would take the "Pre" out of the name and the member names and just >> name it "Snapshot" and the members "used" and "class used". >> >> >> Again, I modeled it based on ParallelGC's PreGCValues class. But I can >> rename it to Snapshot to generalize its use. >> >> >> >> -> I would probably have made PreMetaspaceValues a dumb structure and >> added a function "MetaspaceAux::get_dimensions()" instead of adding the >> implementation to the constructor and to the header file. At least I would >> prefer the ctor implementation to the cpp file. >> >> >> I can move it to a cpp file. This will also eliminate a cyclic dependency >> I had to address by the class forward declaration. >> >> >> Tony >> >> >> >> >> Cheers, Thomas >> >> On Wed, May 8, 2019 at 4:20 PM Tony Printezis >> wrote: >> >>> Any chance of someone taking a look at this? >>> >>> http://cr.openjdk.java.net/~tonyp/8223575/webrev.0/ >>> >>> As I said on the JIRA, users can be confused when they metaspace >>> capacity shows as over 1G even though they asked for 128M. Breaking down >>> the spaces, when coops are enabled, can address this confusion (and having >>> the actual occupancy of each space separately is also a lot more >>> informative and avoids guesswork). The output looks like this when coops >>> are enabled: >>> >>> [10.090s][info][gc,metaspace ] GC(8) Metaspace: >>> 2130K->2130K(1056768K) ClassMetadata: 1906K->1906K(8192K) ClassSpace: >>> 223K->223K(1048576K) >>> >>> and like this (i.e., what it was before the change) when coops are not >>> enabled: >>> >>> [25.767s][info][gc,metaspace ] GC(28) Metaspace: 8107K->8107K(10240K) >>> >>> A couple of notes on the changes: >>> >>> - I?m open to different suggestions for what to call ClassMetadata and >>> ClassSpace. :-) >>> - I didn?t know where to put the HEAP_CHANGE_FORMAT and >>> HEAP_CHANGE_FORMAT_ARGS macros, so that they can be easily shared, and I >>> ended up putting them in heap.hpp. Let me know if there?s a better place >>> for them. >>> >>> Tony >>> >>> >>> ????? >>> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >>> >>> >> >> >> >> >> >> >> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From fujie at loongson.cn Wed May 15 14:33:21 2019 From: fujie at loongson.cn (Jie Fu) Date: Wed, 15 May 2019 22:33:21 +0800 Subject: RFR: 8223961: ZGC: Unexpected behaviour was observed caused by the incorrect update of _nticks in ZMetronome::wait_for_tick() Message-ID: <6fa35e69-6d44-45f5-f921-de788cb2d073@loongson.cn> Hi all, JBS: https://bugs.openjdk.java.net/browse/JDK-8223961 Please review the following patch which fixes the update of "_nticks" in ZMetronome::wait_for_tick() ----------------------------------- diff -r b0b20413d853 src/hotspot/share/gc/z/zMetronome.cpp --- a/src/hotspot/share/gc/z/zMetronome.cpp Wed May 15 11:53:47 2019 +0200 +++ b/src/hotspot/share/gc/z/zMetronome.cpp Wed May 15 20:15:39 2019 +0800 @@ -57,6 +57,8 @@ if (!_stopped && timeout_ms > 0) { // Wait ml.wait(timeout_ms); + } else if (timeout_ms < -1 * (int64_t)_interval_ms) { + _nticks += -timeout_ms / _interval_ms; } else { // Tick return !_stopped; ----------------------------------- For more info, please see the JBS. Testing: - make test TEST="hotspot_gc" CONF=release Thanks a lot. Best regards, Jie From bob.vandette at oracle.com Wed May 15 14:52:25 2019 From: bob.vandette at oracle.com (Bob Vandette) Date: Wed, 15 May 2019 10:52:25 -0400 Subject: RFR: 8222252 - Java ergonomics limits heap to 128GB with disabled compressed oops In-Reply-To: References: <46E18910-9D4E-4219-803C-A883F48AD43B@oracle.com> Message-ID: <772C07C5-F2E7-4878-A94C-A99927ED204A@oracle.com> I?ve updated the webrev based on your comments. http://cr.openjdk.java.net/~bobv/8222252/webrev.02/ Changes: 1. MaxRAM is updated if fractional flags cause us to use OS memory size for heap selection. 2. If MaxRAM is specified with the other fractional based flags, we will use MaxRAM as upper limit instead of OS memory size. Here?s a proposed CSR for this behavioral change. https://bugs.openjdk.java.net/browse/JDK-8223957 Bob. > On May 14, 2019, at 9:26 AM, Thomas Schatzl wrote: > > Hi, > > On Mon, 2019-05-13 at 10:03 -0400, Bob Vandette wrote: >> Please review this suggested fix for correcting Heap size selection >> when -XX:{Max,Min,Initial)RAMPercentage and their Factional >> equivalent options are used. >> >> There are two bugs filed that are related to this issue (8222252 and >> 8213175) >> >> SUMMARY of FIX: >> The fix corrects what I believe is an incorrect implementation of the >> *Percentage/*Fraction options. These options should not have caused >> the heap size to be fractionally based on MaxRAM or limited by >> MaxRAM. They instead should be based on the host memory. This is >> what I assume was meant by ?real memory?. >> >> product(double, MaxRAMPercentage, >> 25.0, \ >> "Maximum percentage of real memory used for maximum heap >> size") \ >> range(0.0, 100.0) >> >> When these options are selected, they should take precedence over >> UseCompressedOops unless UseCompressedOops is also specified on the >> command line. >> >> >> WEBREV: >> http://cr.openjdk.java.net/~bobv/8222252/webrev.01 > > - this is a style preference: I would assign the result of the > condition in the if (arguments.cpp:1721+) to the initialization of > use_os_mem_limit directly instead of first setting it to false, and > then to true again if needed. Feel free to ignore. > > - as David pointed out, this change needs a CSR. > > - the interaction between MaxRAM and MaxRAMPercentage is a bit unclear; > if they are independent as implemented now, then the update of one > should probably update the other. Or the VM should fail if they > disagree? > > If they are dependent, if David suggests (i.e. MaxRAMPercentage uses > MaxRAM as maximum real memory if set; this is what I would tend to > prefer), then not of course. > > - there should be regressions tests verifying this in cases where > possible. > > I.e. it should be doable to get current os::physical_memory using e.g. > whitebox and then run a few tests verifying the results of MaxHeapSize > and coop mode. > > Particularly the interaction between MaxRAM, MaxRAMPercentage and coops > mode would be interesting. > > Thanks, > Thomas > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shade at redhat.com Wed May 15 17:52:30 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 15 May 2019 19:52:30 +0200 Subject: RFR (S) 8223980: Shenandoah: Refactor and fix ObjArrayChunkedTask verification Message-ID: <7b6f57bc-6d63-86ec-fe1f-f98d62fd9a6f@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8223980 Fix: http://cr.openjdk.java.net/~shade/8223980/webrev.01/ Testing on old Solaris exposed funny bugs. We need verification done more consistently here. The change is mostly equivalent, except the startup check moved to ShenandoahHeap where we know the heap region addresses. Testing: hotspot_gc_shenandoah; eyeballing failure messages after increasing {chunk|pow}_bits -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed May 15 18:32:13 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 15 May 2019 20:32:13 +0200 Subject: RFR (S) 8223980: Shenandoah: Refactor and fix ObjArrayChunkedTask verification In-Reply-To: <7b6f57bc-6d63-86ec-fe1f-f98d62fd9a6f@redhat.com> References: <7b6f57bc-6d63-86ec-fe1f-f98d62fd9a6f@redhat.com> Message-ID: Ok. Roman > Bug: > https://bugs.openjdk.java.net/browse/JDK-8223980 > > Fix: > http://cr.openjdk.java.net/~shade/8223980/webrev.01/ > > Testing on old Solaris exposed funny bugs. We need verification done more consistently here. The > change is mostly equivalent, except the startup check moved to ShenandoahHeap where we know the heap > region addresses. > > Testing: hotspot_gc_shenandoah; eyeballing failure messages after increasing {chunk|pow}_bits > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From sangheon.kim at oracle.com Wed May 15 22:08:39 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Wed, 15 May 2019 15:08:39 -0700 Subject: RFR(M): 8220089: G1 wastes a significant amount of space in survivor regions In-Reply-To: <3B084725-AAEC-4D78-9AC7-A16492E01613@oracle.com> References: <11f5af23-bacf-267d-7e81-8cd408ae063b@oracle.com> <45151a57-4689-21e6-4bfb-a884d6b6c14d@oracle.com> <174f169f-4ccb-7567-7707-d6a7f7024aa8@oracle.com> <2dea4528-8c8b-76ab-d737-ad36d36dab0a@oracle.com> <3B084725-AAEC-4D78-9AC7-A16492E01613@oracle.com> Message-ID: Hi Kim, Thanks for reviewing this. On 5/8/19 9:42 PM, Kim Barrett wrote: >> On Apr 30, 2019, at 2:06 AM,sangheon.kim at oracle.com wrote: >> >> Hi all, >> >> I got some comments from Thomas Schatzl, mostly about comment and assert changes. He also suggested to investigate combining current 2 steps of handling retaining survivor region into 1 step, so I filed JDK-8223106. >> >> Webrev: >> http://cr.openjdk.java.net/~sangheki/8220089/webrev.2/ >> http://cr.openjdk.java.net/~sangheki/8220089/webrev.2_inc/ >> Testing: hs-tier1 > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1AllocRegion.cpp > 367 bool SurvivorGCAllocRegion::should_retain(HeapRegion* region) { > 368 if (region == NULL || region->free() < MinTLABSize) { > 369 return false; > 370 } > 371 return true; > 372 } > > Why does should_retain accept a NULL region? Removed NULL check at should_retain() and added NULL check before calling should_retain. G1AllocRegion::release() which used as an input of should_retain() may return NULL. > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1AllocRegion.hpp > > 270 SurvivorGCAllocRegion(G1EvacStats* stats) > 271 : G1GCAllocRegion("Survivor GC Alloc Region", false /* bot_updates */, stats, InCSetState::Young) { } > 272 static bool should_retain(HeapRegion* region); > > Maybe add a blank line before should_retain declaration. Done. > ------------------------------------------------------------------------------ > > [discussed this offline with Sangheon and Thomas; mentioning it here > in case anyone else is following email.] I discussed with Kim and Thomas. And we decided not to pursue this nTAMS idea. nTAMS is used in many locations and currently it is updated mostly at the start/end of marking. And changing the value conditionally seems riskier. So decided to use current approach which is more localized. > "root regions" representation was changed from an array of HeapRegion* > to an array of HeapWord*, where the values are the captured "top" > values from the corresponding regions. This was done because later > allocation in survivor regions would change top, and we don't want > that changed value. > > It seems like it might be easier and simpler to retain the existing > representation, but capture the desired "top" value in NTAMS. Then in > scan_root_region, for survivor regions scan from bottom to NTAMS. That > means scan_root_region is different for old/survivor regions. But > that's kind of already true; it's a bit of a hack that the current > scan from NTAMS to top works for both. > > This also requires looking at and possibly adjusting other uses of NTAMS. > > ------------------------------------------------------------------------------ > > Consider renaming > G1CollectedHeap::survivor_convert_to_eden() => convert_survivor_to_eden() ? Done. > ------------------------------------------------------------------------------ > > I dislike the "retained" region nomenclature for survivors. I didn't > notice an explanation of it, and it's not obvious why that name was > chosen or how it relates to how these regions are used. But I don't > have a good suggestion for an alternative yet. It seems like that name > was copied from MutatorAllocRegion, but I'm not sure the survivor case > is really that similar. Yeah, I agree that it is not that similar. But as I don't have better name, I copied from mutator case. Initially I named 'reuse' but seems not better than 'retain'. Unless there is a better naming suggestion, I would like to proceed as is. > I also dislike some of the accounting kludges around retained regions, > such as for used_bytes. Maybe the above discussed use of NTAMS might > provide information that could simplify the accounting? N/A. http://cr.openjdk.java.net/~sangheki/8220089/webrev.3 http://cr.openjdk.java.net/~sangheki/8220089/webrev.3_inc/ Testing: hs-tier1 ~ 5, tier7-gc Thanks, Sangheon -------------- next part -------------- An HTML attachment was scrubbed... URL: From per.liden at oracle.com Thu May 16 06:25:18 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 16 May 2019 08:25:18 +0200 Subject: RFR: 8223961: ZGC: Unexpected behaviour was observed caused by the incorrect update of _nticks in ZMetronome::wait_for_tick() In-Reply-To: <6fa35e69-6d44-45f5-f921-de788cb2d073@loongson.cn> References: <6fa35e69-6d44-45f5-f921-de788cb2d073@loongson.cn> Message-ID: <3854c380-d99e-2995-4ce1-30ba4dbb3a11@oracle.com> Hi, There is code in ZStat that assumes ZMetronome::ntick() never skips over ticks. That would need to be addressed if we do something like this. Otherwise we risk not getting statistics print outs just because the metronome overslept. I'll take a look at it. cheers, Per On 05/15/2019 04:33 PM, Jie Fu wrote: > Hi all, > > JBS: https://bugs.openjdk.java.net/browse/JDK-8223961 > > Please review the following patch which fixes the update of "_nticks" in > ZMetronome::wait_for_tick() > ----------------------------------- > diff -r b0b20413d853 src/hotspot/share/gc/z/zMetronome.cpp > --- a/src/hotspot/share/gc/z/zMetronome.cpp Wed May 15 11:53:47 2019 > +0200 > +++ b/src/hotspot/share/gc/z/zMetronome.cpp Wed May 15 20:15:39 2019 > +0800 > @@ -57,6 +57,8 @@ > if (!_stopped && timeout_ms > 0) { > // Wait > ml.wait(timeout_ms); > + } else if (timeout_ms < -1 * (int64_t)_interval_ms) { > + _nticks += -timeout_ms / _interval_ms; > } else { > // Tick > return !_stopped; > ----------------------------------- > For more info, please see the JBS. > > Testing: > - make test TEST="hotspot_gc" CONF=release > > Thanks a lot. > Best regards, > Jie > > From per.liden at oracle.com Thu May 16 08:40:11 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 16 May 2019 10:40:11 +0200 Subject: RFR: 8223961: ZGC: Unexpected behaviour was observed caused by the incorrect update of _nticks in ZMetronome::wait_for_tick() In-Reply-To: <3854c380-d99e-2995-4ce1-30ba4dbb3a11@oracle.com> References: <6fa35e69-6d44-45f5-f921-de788cb2d073@loongson.cn> <3854c380-d99e-2995-4ce1-30ba4dbb3a11@oracle.com> Message-ID: On 05/16/2019 08:25 AM, Per Liden wrote: > Hi, > > There is code in ZStat that assumes ZMetronome::ntick() never skips over > ticks. That would need to be addressed if we do something like this. > Otherwise we risk not getting statistics print outs just because the > metronome overslept. I'll take a look at it. I was thinking something like this: diff --git a/src/hotspot/share/gc/z/zMetronome.cpp b/src/hotspot/share/gc/z/zMetronome.cpp --- a/src/hotspot/share/gc/z/zMetronome.cpp +++ b/src/hotspot/share/gc/z/zMetronome.cpp @@ -45,7 +45,9 @@ _start_ms = TimeHelper::counter_to_millis(now.value()); } - for (;;) { + MonitorLocker ml(&_monitor, Monitor::_no_safepoint_check_flag); + + while (!_stopped) { // We might wake up spuriously from wait, so always recalculate // the timeout after a wakeup to see if we need to wait again. const Ticks now = Ticks::now(); @@ -53,15 +55,24 @@ const uint64_t next_ms = _start_ms + (_interval_ms * _nticks); const int64_t timeout_ms = next_ms - now_ms; - MonitorLocker ml(&_monitor, Monitor::_no_safepoint_check_flag); - if (!_stopped && timeout_ms > 0) { + if (timeout_ms > 0) { // Wait ml.wait(timeout_ms); } else { // Tick - return !_stopped; + if (timeout_ms < -_interval_ms) { + // Missed one or more ticks. Bump _nticks accordingly to + // avoid firing a string of immediate ticks to make up + // for the ones we missed. + _nticks += -timeout_ms / _interval_ms; + } + + return true; } } + + // Stopped + return false; } void ZMetronome::stop() { diff --git a/src/hotspot/share/gc/z/zMetronome.hpp b/src/hotspot/share/gc/z/zMetronome.hpp --- a/src/hotspot/share/gc/z/zMetronome.hpp +++ b/src/hotspot/share/gc/z/zMetronome.hpp @@ -29,11 +29,11 @@ class ZMetronome : public StackObj { private: - Monitor _monitor; - const uint64_t _interval_ms; - uint64_t _start_ms; - uint64_t _nticks; - bool _stopped; + Monitor _monitor; + const int64_t _interval_ms; + uint64_t _start_ms; + uint64_t _nticks; + bool _stopped; public: ZMetronome(uint64_t hz); diff --git a/src/hotspot/share/gc/z/zStat.cpp b/src/hotspot/share/gc/z/zStat.cpp --- a/src/hotspot/share/gc/z/zStat.cpp +++ b/src/hotspot/share/gc/z/zStat.cpp @@ -850,7 +850,16 @@ } bool ZStat::should_print(LogTargetHandle log) const { - return log.is_enabled() && (_metronome.nticks() % ZStatisticsInterval == 0); + static uint64_t print_at_nticks = ZStatisticsInterval; + uint64_t nticks = _metronome.nticks(); + + if (nticks < print_at_nticks) { + return false; + } + + print_at_nticks = ((nticks / ZStatisticsInterval) * ZStatisticsInterval) + ZStatisticsInterval; + + return log.is_enabled(); } void ZStat::print(LogTargetHandle log, const ZStatSamplerHistory* history) const { cheers, Per > > cheers, > Per > > On 05/15/2019 04:33 PM, Jie Fu wrote: >> Hi all, >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8223961 >> >> Please review the following patch which fixes the update of "_nticks" >> in ZMetronome::wait_for_tick() >> ----------------------------------- >> diff -r b0b20413d853 src/hotspot/share/gc/z/zMetronome.cpp >> --- a/src/hotspot/share/gc/z/zMetronome.cpp Wed May 15 11:53:47 >> 2019 +0200 >> +++ b/src/hotspot/share/gc/z/zMetronome.cpp Wed May 15 20:15:39 >> 2019 +0800 >> @@ -57,6 +57,8 @@ >> if (!_stopped && timeout_ms > 0) { >> // Wait >> ml.wait(timeout_ms); >> + } else if (timeout_ms < -1 * (int64_t)_interval_ms) { >> + _nticks += -timeout_ms / _interval_ms; >> } else { >> // Tick >> return !_stopped; >> ----------------------------------- >> For more info, please see the JBS. >> >> Testing: >> - make test TEST="hotspot_gc" CONF=release >> >> Thanks a lot. >> Best regards, >> Jie >> >> From fujie at loongson.cn Thu May 16 09:21:32 2019 From: fujie at loongson.cn (Jie Fu) Date: Thu, 16 May 2019 17:21:32 +0800 Subject: RFR: 8223961: ZGC: Unexpected behaviour was observed caused by the incorrect update of _nticks in ZMetronome::wait_for_tick() In-Reply-To: References: <6fa35e69-6d44-45f5-f921-de788cb2d073@loongson.cn> <3854c380-d99e-2995-4ce1-30ba4dbb3a11@oracle.com> Message-ID: <6b2e2fc3-4f39-14ea-e247-a8f4c6b8819d@loongson.cn> It's much clearer and more reasonable. I like your patch. Thanks Per. On 2019/5/16 ??4:40, Per Liden wrote: > On 05/16/2019 08:25 AM, Per Liden wrote: >> Hi, >> >> There is code in ZStat that assumes ZMetronome::ntick() never skips >> over ticks. That would need to be addressed if we do something like >> this. Otherwise we risk not getting statistics print outs just >> because the metronome overslept. I'll take a look at it. > > I was thinking something like this: > > diff --git a/src/hotspot/share/gc/z/zMetronome.cpp > b/src/hotspot/share/gc/z/zMetronome.cpp > --- a/src/hotspot/share/gc/z/zMetronome.cpp > +++ b/src/hotspot/share/gc/z/zMetronome.cpp > @@ -45,7 +45,9 @@ > ???? _start_ms = TimeHelper::counter_to_millis(now.value()); > ?? } > > -? for (;;) { > +? MonitorLocker ml(&_monitor, Monitor::_no_safepoint_check_flag); > + > +? while (!_stopped) { > ???? // We might wake up spuriously from wait, so always recalculate > ???? // the timeout after a wakeup to see if we need to wait again. > ???? const Ticks now = Ticks::now(); > @@ -53,15 +55,24 @@ > ???? const uint64_t next_ms = _start_ms + (_interval_ms * _nticks); > ???? const int64_t timeout_ms = next_ms - now_ms; > > -??? MonitorLocker ml(&_monitor, Monitor::_no_safepoint_check_flag); > -??? if (!_stopped && timeout_ms > 0) { > +??? if (timeout_ms > 0) { > ?????? // Wait > ?????? ml.wait(timeout_ms); > ???? } else { > ?????? // Tick > -????? return !_stopped; > +????? if (timeout_ms < -_interval_ms) { > +??????? // Missed one or more ticks. Bump _nticks accordingly to > +??????? // avoid firing a string of immediate ticks to make up > +??????? // for the ones we missed. > +??????? _nticks += -timeout_ms / _interval_ms; > +????? } > + > +????? return true; > ???? } > ?? } > + > +? // Stopped > +? return false; > ?} > > ?void ZMetronome::stop() { > diff --git a/src/hotspot/share/gc/z/zMetronome.hpp > b/src/hotspot/share/gc/z/zMetronome.hpp > --- a/src/hotspot/share/gc/z/zMetronome.hpp > +++ b/src/hotspot/share/gc/z/zMetronome.hpp > @@ -29,11 +29,11 @@ > > ?class ZMetronome : public StackObj { > ?private: > -? Monitor??????? _monitor; > -? const uint64_t _interval_ms; > -? uint64_t?????? _start_ms; > -? uint64_t?????? _nticks; > -? bool?????????? _stopped; > +? Monitor?????? _monitor; > +? const int64_t _interval_ms; > +? uint64_t????? _start_ms; > +? uint64_t????? _nticks; > +? bool????????? _stopped; > > ?public: > ?? ZMetronome(uint64_t hz); > diff --git a/src/hotspot/share/gc/z/zStat.cpp > b/src/hotspot/share/gc/z/zStat.cpp > --- a/src/hotspot/share/gc/z/zStat.cpp > +++ b/src/hotspot/share/gc/z/zStat.cpp > @@ -850,7 +850,16 @@ > ?} > > ?bool ZStat::should_print(LogTargetHandle log) const { > -? return log.is_enabled() && (_metronome.nticks() % > ZStatisticsInterval == 0); > +? static uint64_t print_at_nticks = ZStatisticsInterval; > +? uint64_t nticks = _metronome.nticks(); > + > +? if (nticks < print_at_nticks) { > +??? return false; > +? } > + > +? print_at_nticks = ((nticks / ZStatisticsInterval) * > ZStatisticsInterval) + ZStatisticsInterval; > + > +? return log.is_enabled(); > ?} > > ?void ZStat::print(LogTargetHandle log, const ZStatSamplerHistory* > history) const { > > > cheers, > Per > >> >> cheers, >> Per >> >> On 05/15/2019 04:33 PM, Jie Fu wrote: >>> Hi all, >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8223961 >>> >>> Please review the following patch which fixes the update of >>> "_nticks" in ZMetronome::wait_for_tick() >>> ----------------------------------- >>> diff -r b0b20413d853 src/hotspot/share/gc/z/zMetronome.cpp >>> --- a/src/hotspot/share/gc/z/zMetronome.cpp???? Wed May 15 11:53:47 >>> 2019 +0200 >>> +++ b/src/hotspot/share/gc/z/zMetronome.cpp???? Wed May 15 20:15:39 >>> 2019 +0800 >>> @@ -57,6 +57,8 @@ >>> ????? if (!_stopped && timeout_ms > 0) { >>> ??????? // Wait >>> ??????? ml.wait(timeout_ms); >>> +??? } else if (timeout_ms < -1 * (int64_t)_interval_ms) { >>> +????? _nticks += -timeout_ms / _interval_ms; >>> ????? } else { >>> ??????? // Tick >>> ??????? return !_stopped; >>> ----------------------------------- >>> For more info, please see the JBS. >>> >>> Testing: >>> ?? - make test TEST="hotspot_gc" CONF=release >>> >>> Thanks a lot. >>> Best regards, >>> Jie >>> >>> From shade at redhat.com Thu May 16 10:09:30 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 16 May 2019 12:09:30 +0200 Subject: RFR 8223774: Shenandoah: Refactor ShenandoahRootProcessor and family In-Reply-To: References: Message-ID: On 5/13/19 5:00 PM, Zhengyu Gu wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8223774 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8223774/webrev.00/index.html Oh, this is a cute refactoring! First sweep: *) Maybe make ShenandoahRoot{Processor,Evacuator,Updater} the superclass and give them ShenandoahHeap in a const field? Then replace uses of ShenandoahHeap::heap() everywhere in them. Maybe even move phase_timings() and safepoint asserts there. *) Is there really a significant difference between ShenandoahRootAdjuster and ShenandoahRootUpdater? Current thing is fine, just wanted to understand if they are common enough to merge. *) I think names should be "Root*s*" for e.g. ShenandoahClassLoaderDataRoot, etc. *) "const" on new fields, where applicable? *) shenandoahArguments.cpp: this block is better be in ShenandoahTraversalHeuristics? 186 if (strcmp(ShenandoahGCHeuristics, "traversal") == 0) { 187 FLAG_SET_DEFAULT(ShenandoahConcurrentScanCodeRoots, false); 188 } 189 *) This comment mentions ShenandoahConcurrentScanCodeRoots, but there are no uses in the subsequent block. Need to move point (c) to somewhere else? 118 // c. With ShenandoahConcurrentScanCodeRoots, we avoid scanning the entire code cache here, 119 // and instead do that in concurrent phase under the relevant lock. This saves init mark 120 // pause time. *) Indents: 154 ShenandoahRootProcessor::ShenandoahRootProcessor(uint n_workers, ShenandoahPhaseTimings::Phase phase) : 155 _thread_roots(n_workers > 1), 156 _phase(phase) { 157 assert(SafepointSynchronize::is_at_safepoint(), "Must at safepoint"); *) Stick with one message here? E.g. "Should be during class unloading". 178 assert(!ShenandoahHeap::heap()->unload_classes(), "Use without class unloading"); ... 192 assert(ShenandoahHeap::heap()->unload_classes(), "Use with class unloading cycle"); -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Thu May 16 12:49:47 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 16 May 2019 14:49:47 +0200 Subject: RFR (XS): 8224038: Fix remaining InCSetState mentions Message-ID: <2e9a7f8cd5d04470b24f02bb82b6478b73ba179c.camel@oracle.com> Hi all, in JDK-8200545 there has been some renaming of the InCSetState class to G1RegionAttr class. Some comments and class names still mention it. This change fixes this. CR: https://bugs.openjdk.java.net/browse/JDK-8224038 Webrev: http://cr.openjdk.java.net/~tschatzl/8224038/webrev/ Testing: local compilation Thanks, Thomas From zgu at redhat.com Thu May 16 13:13:59 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 16 May 2019 09:13:59 -0400 Subject: RFR 8223774: Shenandoah: Refactor ShenandoahRootProcessor and family In-Reply-To: References: Message-ID: <451b6348-8c37-9a46-31d3-6f6ea960de4a@redhat.com> Thanks for reviewing. On 5/16/19 6:09 AM, Aleksey Shipilev wrote: > On 5/13/19 5:00 PM, Zhengyu Gu wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8223774 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8223774/webrev.00/index.html > > Oh, this is a cute refactoring! > > First sweep: > > *) Maybe make ShenandoahRoot{Processor,Evacuator,Updater} the superclass and give them > ShenandoahHeap in a const field? Then replace uses of ShenandoahHeap::heap() everywhere in them. > Maybe even move phase_timings() and safepoint asserts there. > Done. > *) Is there really a significant difference between ShenandoahRootAdjuster and > ShenandoahRootUpdater? Current thing is fine, just wanted to understand if they are common enough to > merge. Not at this point, but will be with concurrent stuffs. Let's revisit this after that. > > *) I think names should be "Root*s*" for e.g. ShenandoahClassLoaderDataRoot, etc. > Done. > *) "const" on new fields, where applicable? Done > > *) shenandoahArguments.cpp: this block is better be in ShenandoahTraversalHeuristics? > > 186 if (strcmp(ShenandoahGCHeuristics, "traversal") == 0) { > 187 FLAG_SET_DEFAULT(ShenandoahConcurrentScanCodeRoots, false); > 188 } > 189 We don't have this heuristic, let's do it in follow up RFE. > > *) This comment mentions ShenandoahConcurrentScanCodeRoots, but there are no uses in the subsequent > block. Need to move point (c) to somewhere else? > > 118 // c. With ShenandoahConcurrentScanCodeRoots, we avoid scanning the entire code cache here, > 119 // and instead do that in concurrent phase under the relevant lock. This saves init mark > 120 // pause time. > Moved to right place. > > *) Indents: > > 154 ShenandoahRootProcessor::ShenandoahRootProcessor(uint n_workers, ShenandoahPhaseTimings::Phase > phase) : > 155 _thread_roots(n_workers > 1), > 156 _phase(phase) { > 157 assert(SafepointSynchronize::is_at_safepoint(), "Must at safepoint"); > > *) Stick with one message here? E.g. "Should be during class unloading". > > 178 assert(!ShenandoahHeap::heap()->unload_classes(), "Use without class unloading"); > ... > 192 assert(ShenandoahHeap::heap()->unload_classes(), "Use with class unloading cycle"); > Done. Updated: http://cr.openjdk.java.net/~zgu/JDK-8223774/webrev.01/ Reran hotspot_gc_shenandoah (fastdebug and release) Thanks, -Zhengyu > > -Aleksey > From rkennke at redhat.com Thu May 16 13:30:25 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 16 May 2019 15:30:25 +0200 Subject: RFR 8223774: Shenandoah: Refactor ShenandoahRootProcessor and family In-Reply-To: <451b6348-8c37-9a46-31d3-6f6ea960de4a@redhat.com> References: <451b6348-8c37-9a46-31d3-6f6ea960de4a@redhat.com> Message-ID: <57c55b42-c03e-2649-e8c5-37a9101e93a1@redhat.com> >> *) shenandoahArguments.cpp: this block is better be in >> ShenandoahTraversalHeuristics? >> >> ? 186?? if (strcmp(ShenandoahGCHeuristics, "traversal") == 0) { >> ? 187???? FLAG_SET_DEFAULT(ShenandoahConcurrentScanCodeRoots, false); >> ? 188?? } >> ? 189 > > We don't have this heuristic, let's do it in follow up RFE. We do: src/hotspot/share/gc/shenandoah/heuristics/shenandoahTraversalHeuristics.hpp and src/hotspot/share/gc/shenandoah/heuristics/shenandoahTraversalHeuristics.cpp Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Thu May 16 14:25:54 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 16 May 2019 10:25:54 -0400 Subject: RFR(S) 8223951: Shenandoah: Only need to update thread roots during final update refs Message-ID: <52773c59-62fc-d763-0e9d-b79131eeba19@redhat.com> After switching to strong to-space invariant and pre-evacuating all root, thread roots are only places that may contain to-space oops after updating references during normal concurrent cycles, we only need to update them during final updaterefs pause, which should reduce pause time. Bug: https://bugs.openjdk.java.net/browse/JDK-8223951 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8223951/webrev.00/ Test: hotspot_gc_shenandoah (fastdebug and release) Thanks, -Zhengyu From zgu at redhat.com Thu May 16 14:33:23 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 16 May 2019 10:33:23 -0400 Subject: RFR 8223774: Shenandoah: Refactor ShenandoahRootProcessor and family In-Reply-To: <57c55b42-c03e-2649-e8c5-37a9101e93a1@redhat.com> References: <451b6348-8c37-9a46-31d3-6f6ea960de4a@redhat.com> <57c55b42-c03e-2649-e8c5-37a9101e93a1@redhat.com> Message-ID: <102078db-5b2b-804b-556c-b8114bd40bff@redhat.com> On 5/16/19 9:30 AM, Roman Kennke wrote: >>> *) shenandoahArguments.cpp: this block is better be in >>> ShenandoahTraversalHeuristics? >>> >>> ? 186?? if (strcmp(ShenandoahGCHeuristics, "traversal") == 0) { >>> ? 187???? FLAG_SET_DEFAULT(ShenandoahConcurrentScanCodeRoots, false); >>> ? 188?? } >>> ? 189 >> >> We don't have this heuristic, let's do it in follow up RFE. > > We do: > src/hotspot/share/gc/shenandoah/heuristics/shenandoahTraversalHeuristics.hpp > > and > > src/hotspot/share/gc/shenandoah/heuristics/shenandoahTraversalHeuristics.cpp > Sorry, I meant can_concurrent_scan_code_roots() in heuristics ... -Zhengyu > Roman > From rkennke at redhat.com Thu May 16 14:39:39 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 16 May 2019 16:39:39 +0200 Subject: RFR 8223774: Shenandoah: Refactor ShenandoahRootProcessor and family In-Reply-To: <102078db-5b2b-804b-556c-b8114bd40bff@redhat.com> References: <451b6348-8c37-9a46-31d3-6f6ea960de4a@redhat.com> <57c55b42-c03e-2649-e8c5-37a9101e93a1@redhat.com> <102078db-5b2b-804b-556c-b8114bd40bff@redhat.com> Message-ID: <83d496a1-c767-755e-13af-46ee4102dc97@redhat.com> > On 5/16/19 9:30 AM, Roman Kennke wrote: >>>> *) shenandoahArguments.cpp: this block is better be in >>>> ShenandoahTraversalHeuristics? >>>> >>>> ?? 186?? if (strcmp(ShenandoahGCHeuristics, "traversal") == 0) { >>>> ?? 187???? FLAG_SET_DEFAULT(ShenandoahConcurrentScanCodeRoots, false); >>>> ?? 188?? } >>>> ?? 189 >>> >>> We don't have this heuristic, let's do it in follow up RFE. >> >> We do: >> src/hotspot/share/gc/shenandoah/heuristics/shenandoahTraversalHeuristics.hpp >> >> >> and >> >> src/hotspot/share/gc/shenandoah/heuristics/shenandoahTraversalHeuristics.cpp >> >> > > Sorry, I meant can_concurrent_scan_code_roots() in heuristics ... > But you can put the line: FLAG_SET_DEFAULT(ShenandoahConcurrentScanCodeRoots, false); in the constructor of the ShenandoahTraversalHeuristics, together with other similar lines. Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From boris.ulasevich at bell-sw.com Thu May 16 14:43:37 2019 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Thu, 16 May 2019 17:43:37 +0300 Subject: RFR: 8223767: Shenandoah fails to build on Solaris x86_64 In-Reply-To: References: <87dcfaac-f351-b8d5-ba30-4029124e33f6@redhat.com> <3387252d-af57-0f52-5219-98abccdf810b@bell-sw.com> Message-ID: <55bb8152-3ec6-dcc0-65fe-3b7efd0b7919@bell-sw.com> Roman, I prepared backporting patch to sh/jdk11 and sh/jdk8: http://cr.openjdk.java.net/~bulasevich/8223767/webrev.00.shenandoah.jdk11 - added minor change in shared code: share/opto/compile.hpp http://cr.openjdk.java.net/~bulasevich/8223767/webrev.00.shenandoah.jdk8.hotspot - stdarg.h include added to satisfy va_list in shenandoahLogging.hpp Both backports passed hotspot_gc_shenandoah on Solaris 11. For Solaris 10 JDK-8223980 (Refactor and fix ObjArrayChunkedTask verification) needs to be backported as well. What is the correct backport procedure? regards, Boris On 14.05.2019 12:25, Roman Kennke wrote: > Hi Boris, > > Would you help backporting the patch to sh/jdk11 and sh/jdk8 at some > point? It would be cool to have there, but we have no way to test it... > > Thanks, > Roman > >> Thanks for the review! >> >> 13.05.2019 18:43, Aleksey Shipilev: >>> On 5/13/19 5:27 PM, Boris Ulasevich wrote: >>>> http://bugs.openjdk.java.net/browse/JDK-8223767 >>>> http://cr.openjdk.java.net/~bulasevich/8223767/webrev.00 >>> Thank you, this looks good. >>> >>> -Aleksey > From zgu at redhat.com Thu May 16 15:09:17 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 16 May 2019 11:09:17 -0400 Subject: RFR 8223774: Shenandoah: Refactor ShenandoahRootProcessor and family In-Reply-To: <83d496a1-c767-755e-13af-46ee4102dc97@redhat.com> References: <451b6348-8c37-9a46-31d3-6f6ea960de4a@redhat.com> <57c55b42-c03e-2649-e8c5-37a9101e93a1@redhat.com> <102078db-5b2b-804b-556c-b8114bd40bff@redhat.com> <83d496a1-c767-755e-13af-46ee4102dc97@redhat.com> Message-ID: <60b7f75e-b8ec-7122-4e39-f7dca14a2ea8@redhat.com> >> > > But you can put the line: > > FLAG_SET_DEFAULT(ShenandoahConcurrentScanCodeRoots, false); > > in the constructor of the ShenandoahTraversalHeuristics, together with > other similar lines. Ah, I misunderstood. Updated: http://cr.openjdk.java.net/~zgu/JDK-8223774/webrev.02/ -Zhengyu > > Roman > > From rkennke at redhat.com Thu May 16 15:43:22 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 16 May 2019 17:43:22 +0200 Subject: RFR: 8223767: Shenandoah fails to build on Solaris x86_64 In-Reply-To: <55bb8152-3ec6-dcc0-65fe-3b7efd0b7919@bell-sw.com> References: <87dcfaac-f351-b8d5-ba30-4029124e33f6@redhat.com> <3387252d-af57-0f52-5219-98abccdf810b@bell-sw.com> <55bb8152-3ec6-dcc0-65fe-3b7efd0b7919@bell-sw.com> Message-ID: <596a0613-97f1-5673-01c9-06f63f8edbcb@redhat.com> Hi Boris, > I prepared backporting patch to sh/jdk11 and sh/jdk8: > > http://cr.openjdk.java.net/~bulasevich/8223767/webrev.00.shenandoah.jdk11 > - added minor change in shared code: share/opto/compile.hpp > > http://cr.openjdk.java.net/~bulasevich/8223767/webrev.00.shenandoah.jdk8.hotspot > > - stdarg.h include added to satisfy va_list in shenandoahLogging.hpp > > Both backports passed hotspot_gc_shenandoah on Solaris 11. For Solaris > 10 JDK-8223980 (Refactor and fix ObjArrayChunkedTask verification) needs > to be backported as well. Cool! Thanks you!! > What is the correct backport procedure? I think for the Shenandoah parts it's fine to post them here for review, and one of us can integrate them to shenandoah/jdk11 and shenandoah/jdk8 for you. Please prepare the changesets with prefixed [backport] in the commit msg, e.g.: [backport] 8223767: Shenandoah fails to build on Solaris x86_64 To be honest, I am not sure about the rather trivial change in compile.hpp. I guess ideally this would have to go via separate change through jdk-updates/jdk11u. Thanks, Roman > regards, > Boris > > On 14.05.2019 12:25, Roman Kennke wrote: >> Hi Boris, >> >> Would you help backporting the patch to sh/jdk11 and sh/jdk8 at some >> point? It would be cool to have there, but we have no way to test it... >> >> Thanks, >> Roman >> >>> Thanks for the review! >>> >>> 13.05.2019 18:43, Aleksey Shipilev: >>>> On 5/13/19 5:27 PM, Boris Ulasevich wrote: >>>>> http://bugs.openjdk.java.net/browse/JDK-8223767 >>>>> http://cr.openjdk.java.net/~bulasevich/8223767/webrev.00 >>>> Thank you, this looks good. >>>> >>>> -Aleksey >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Thu May 16 16:02:47 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 16 May 2019 18:02:47 +0200 Subject: RFR: JDK-8224043: Shenandoah: Remove clear_claimed_marks() from start of concurrent_traversal() Message-ID: Currently we clear claimed marks in the CLD at the start of concurrent traversal phase. This is (no more) necessary, and in fact leads to marking through CLD structures during traversal again, even when they have already been seen during init-traversal. This changeset simply removes the little block. I've tested by running all of hotspot_gc_shenandoah on it, plus a bunch of applications including specjvm with traversal and +ShenandoahVerify. Looks all good. http://cr.openjdk.java.net/~rkennke/JDK-8224043/webrev.00/ Ok? Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Thu May 16 16:47:39 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 16 May 2019 18:47:39 +0200 Subject: RFR: JDK-8224043: Shenandoah: Remove clear_claimed_marks() from start of concurrent_traversal() In-Reply-To: References: Message-ID: <8986dbe1-8de8-6840-c7fc-a9a318de1bba@redhat.com> On 5/16/19 6:02 PM, Roman Kennke wrote: > http://cr.openjdk.java.net/~rkennke/JDK-8224043/webrev.00/ Okay then. We should have done it from the beginning! Eh. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Thu May 16 16:53:34 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 16 May 2019 18:53:34 +0200 Subject: RFR: 8223767: Shenandoah fails to build on Solaris x86_64 In-Reply-To: <596a0613-97f1-5673-01c9-06f63f8edbcb@redhat.com> References: <87dcfaac-f351-b8d5-ba30-4029124e33f6@redhat.com> <3387252d-af57-0f52-5219-98abccdf810b@bell-sw.com> <55bb8152-3ec6-dcc0-65fe-3b7efd0b7919@bell-sw.com> <596a0613-97f1-5673-01c9-06f63f8edbcb@redhat.com> Message-ID: <9f832ee6-2f06-d29f-807b-125e1f853646@redhat.com> On 5/16/19 5:43 PM, Roman Kennke wrote: >> What is the correct backport procedure? > > I think for the Shenandoah parts it's fine to post them here for review, > and one of us can integrate them to shenandoah/jdk11 and shenandoah/jdk8 > for you. Please prepare the changesets with prefixed [backport] in the > commit msg, e.g.: > > [backport] 8223767: Shenandoah fails to build on Solaris x86_64 I can do this myself while sponsoring fixes for Boris. Thanks for the backport! > To be honest, I am not sure about the rather trivial change in > compile.hpp. I guess ideally this would have to go via separate change > through jdk-updates/jdk11u. This change is our own code, no need (or use) to go via upstream: https://builds.shipilev.net/patch-openjdk-shenandoah-jdk11/src/hotspot/share/opto/compile.hpp.sdiff.html -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Thu May 16 18:00:18 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 16 May 2019 20:00:18 +0200 Subject: RFR: 8223767: Shenandoah fails to build on Solaris x86_64 In-Reply-To: <9f832ee6-2f06-d29f-807b-125e1f853646@redhat.com> References: <87dcfaac-f351-b8d5-ba30-4029124e33f6@redhat.com> <3387252d-af57-0f52-5219-98abccdf810b@bell-sw.com> <55bb8152-3ec6-dcc0-65fe-3b7efd0b7919@bell-sw.com> <596a0613-97f1-5673-01c9-06f63f8edbcb@redhat.com> <9f832ee6-2f06-d29f-807b-125e1f853646@redhat.com> Message-ID: <27b5d599-b25f-073d-0609-1c84c2433d1d@redhat.com> On 5/16/19 6:53 PM, Aleksey Shipilev wrote: > On 5/16/19 5:43 PM, Roman Kennke wrote: >>> What is the correct backport procedure? >> >> I think for the Shenandoah parts it's fine to post them here for review, >> and one of us can integrate them to shenandoah/jdk11 and shenandoah/jdk8 >> for you. Please prepare the changesets with prefixed [backport] in the >> commit msg, e.g.: >> >> [backport] 8223767: Shenandoah fails to build on Solaris x86_64 > > I can do this myself while sponsoring fixes for Boris. Thanks for the backport! There: http://hg.openjdk.java.net/shenandoah/jdk11/rev/159ea4412948 http://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/468dd92b90f9 Pushed them with "author" set to "bulasevich" to give credit. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From kim.barrett at oracle.com Thu May 16 18:10:55 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 16 May 2019 14:10:55 -0400 Subject: RFR (XS): 8224038: Fix remaining InCSetState mentions In-Reply-To: <2e9a7f8cd5d04470b24f02bb82b6478b73ba179c.camel@oracle.com> References: <2e9a7f8cd5d04470b24f02bb82b6478b73ba179c.camel@oracle.com> Message-ID: On May 16, 2019, at 8:49 AM, Thomas Schatzl wrote: > > Hi all, > > in JDK-8200545 there has been some renaming of the InCSetState class > to G1RegionAttr class. > > Some comments and class names still mention it. > > This change fixes this. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8224038 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8224038/webrev/ > Testing: > local compilation > > Thanks, > Thomas > > Looks good, and trivial. From kim.barrett at oracle.com Thu May 16 22:45:04 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 16 May 2019 18:45:04 -0400 Subject: RFR (S/M): 8222492: G1 unnecessarily scans remembered set cards for regions that already have been evacuated In-Reply-To: <3e061b852275110f67a309f66211dd5832f0e5b4.camel@oracle.com> References: <3e061b852275110f67a309f66211dd5832f0e5b4.camel@oracle.com> Message-ID: <5D369410-424A-4550-9B02-B09725C502B8@oracle.com> On Apr 26, 2019, at 8:27 AM, Thomas Schatzl wrote: > CR: > https://bugs.openjdk.java.net/browse/JDK-8222492 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8222492/webrev/ > Testing: > hs-tier1-5, running the crashing test of JDK-8222426 for 5k times > without failures (the crash has an occurrence of 1-2 times/1000 runs). > > Thanks, > Thomas > Looks good. One minor nit: src/hotspot/share/gc/g1/g1OopClosures.inline.hpp 190 // Entries in the optional collection set may become to be located in the collection 191 // set during incremental collection of the old generation. That comment needs some fixing. From david.holmes at oracle.com Fri May 17 03:34:53 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 17 May 2019 13:34:53 +1000 Subject: RFR: 8222252 - Java ergonomics limits heap to 128GB with disabled compressed oops In-Reply-To: <772C07C5-F2E7-4878-A94C-A99927ED204A@oracle.com> References: <46E18910-9D4E-4219-803C-A883F48AD43B@oracle.com> <772C07C5-F2E7-4878-A94C-A99927ED204A@oracle.com> Message-ID: <3b410422-68b0-a143-78f5-dd0a141d1353@oracle.com> Hi Bob, On 16/05/2019 12:52 am, Bob Vandette wrote: > I?ve updated the webrev based on your comments. > > http://cr.openjdk.java.net/~bobv/8222252/webrev.02/ > > Changes: > > 1. MaxRAM is updated if fractional flags cause us to use OS memory size > for heap selection. > 2. If MaxRAM is specified with the other fractional based flags, we will > use MaxRAM as > ? ?upper limit instead of OS memory size. That seems fine, but here: 1799 if (FLAG_IS_ERGO(UseCompressedOops) && use_os_mem_limit) { Shouldn't this also be checking for FLAG_IS_DEFAULT(MaxRAM) ? Or better that check would fold into the use_os_mem_limit determination: bool use_os_mem_limit = (!FLAG_IS_DEFAULT(MaxRAMPercentage) || !FLAG_IS_DEFAULT(MaxRAMFraction) || !FLAG_IS_DEFAULT(MinRAMPercentage) || !FLAG_IS_DEFAULT(MinRAMFraction) || !FLAG_IS_DEFAULT(InitialRAMPercentage) || !FLAG_IS_DEFAULT(InitialRAMFraction)) && FLAG_IS_DEFAULT(MaxRAM); if (use_os_mem_limit) { phys_mem = os::physical_memory(); FLAG_SET_ERGO(uint64_t, MaxRAM, (uint64_t)phys_mem); } else { phys_mem = FLAG_IS_DEFAULT(MaxRAM) ? MIN2(os::physical_memory(), (julong)MaxRAM) : (julong)MaxRAM; } ? Thanks, David ----- > > Here?s a proposed CSR for this behavioral change. > > https://bugs.openjdk.java.net/browse/JDK-8223957 > > > Bob. > >> On May 14, 2019, at 9:26 AM, Thomas Schatzl > > wrote: >> >> Hi, >> >> On Mon, 2019-05-13 at 10:03 -0400, Bob Vandette wrote: >>> Please review this suggested fix for correcting Heap size selection >>> when -XX:{Max,Min,Initial)RAMPercentage and their Factional >>> equivalent options are used. >>> >>> There are two bugs filed that are related to this issue (8222252 and >>> 8213175) >>> >>> SUMMARY of FIX: >>> The fix corrects what I believe is an incorrect implementation of the >>> *Percentage/*Fraction options. ?These options should not have caused >>> the heap size to be fractionally based on MaxRAM or limited by >>> MaxRAM. ?They instead should be based on the host memory. This is >>> what I assume was meant by ?real memory?. >>> >>> product(double, MaxRAMPercentage, >>> 25.0, ??????????????????????????????????\ >>> ????????"Maximum percentage of real memory used for maximum heap >>> size") \ >>> ????????range(0.0, 100.0) >>> >>> When these options are selected, they should take precedence over >>> UseCompressedOops unless UseCompressedOops is also specified on the >>> command line. >>> >>> >>> WEBREV: >>> http://cr.openjdk.java.net/~bobv/8222252/webrev.01 >> >> - this is a style preference: I would assign the result of the >> condition in the if (arguments.cpp:1721+) to the initialization of >> use_os_mem_limit directly instead of first setting it to false, and >> then to true again if needed. Feel free to ignore. >> >> - as David pointed out, this change needs a CSR. >> >> - the interaction between MaxRAM and MaxRAMPercentage is a bit unclear; >> if they are independent as implemented now, then the update of one >> should probably update the other. Or the VM should fail if they >> disagree? >> >> If they are dependent, if David suggests (i.e. MaxRAMPercentage uses >> MaxRAM as maximum real memory if set; this is what I would tend to >> prefer), then not of course. >> >> - there should be regressions tests verifying this in cases where >> possible. >> >> I.e. it should be doable to get current os::physical_memory ?using e.g. >> whitebox and then run a few tests verifying the results of MaxHeapSize >> and coop mode. >> >> Particularly the interaction between MaxRAM, MaxRAMPercentage and coops >> mode would be interesting. >> >> Thanks, >> ?Thomas >> >> > From chengjingwei1 at huawei.com Fri May 17 03:54:08 2019 From: chengjingwei1 at huawei.com (chengjingwei (A)) Date: Fri, 17 May 2019 03:54:08 +0000 Subject: PING: RFR: 8217432: MetaspaceGC::_capacity_until_GC exceeds MaxMetaspaceSize Message-ID: Hi, After this patch was applied (since jdk8u212 and jdk-11.0.3, to be specific), we started to get test failures with jtreg. The failed test case was jdk8u/hotspot/test/gc/metaspace/TestCapacityUntilGCWrapAround.java. It always fails on windows-32bit platform, and fails on linux-32bit if -XX:MaxMetaspaceSize is set. The failure message was: java.lang.IllegalStateException: WB_IncMetaspaceCapacityUntilGC: could not increase capacity until GC due to contention with another thread at sun.hotspot.WhiteBox.incMetaspaceCapacityUntilGC(Native Method) at TestCapacityUntilGCWrapAround.main(TestCapacityUntilGCWrapAround.java:51) I checked the code, and found that MetaspaceGC::inc_capacity_until_GC was passed a value of 4G-page_size-1, which caused addition overflow when computing new_value, and new_value always set to "align_size_down(max_uintx, Metaspace::commit_alignment())". So when MaxMetaspaceSize defaults to or is set to a value less than it, the function will return false to WB_IncMetaspaceCapacityUntilGC, which will then throw an Exception, causing the test failure. On win-32 platform, the default value of MaxMetaspaceSize is 4G - allocation_granularity, which is 4G - 64K (according to https://devblogs.microsoft.com/oldnewthing/20031008-00/?p=42223), this is very likely to be less than "new_value" and cause test failure. Do you have any idea how to fix this? -------------- next part -------------- An HTML attachment was scrubbed... URL: From per.liden at oracle.com Fri May 17 06:14:26 2019 From: per.liden at oracle.com (Per Liden) Date: Fri, 17 May 2019 08:14:26 +0200 Subject: RFR: 8223961: ZGC: Unexpected behaviour was observed caused by the incorrect update of _nticks in ZMetronome::wait_for_tick() In-Reply-To: <6b2e2fc3-4f39-14ea-e247-a8f4c6b8819d@loongson.cn> References: <6fa35e69-6d44-45f5-f921-de788cb2d073@loongson.cn> <3854c380-d99e-2995-4ce1-30ba4dbb3a11@oracle.com> <6b2e2fc3-4f39-14ea-e247-a8f4c6b8819d@loongson.cn> Message-ID: <4db7be28-440f-59ee-7c61-681ee802ead8@oracle.com> Let's wait for a second Reviewer to give thumbs up, then I'll go ahead and sponsor/push this. cheers, Per On 5/16/19 11:21 AM, Jie Fu wrote: > It's much clearer and more reasonable. > I like your patch. > Thanks Per. > > On 2019/5/16 ??4:40, Per Liden wrote: >> On 05/16/2019 08:25 AM, Per Liden wrote: >>> Hi, >>> >>> There is code in ZStat that assumes ZMetronome::ntick() never skips >>> over ticks. That would need to be addressed if we do something like >>> this. Otherwise we risk not getting statistics print outs just >>> because the metronome overslept. I'll take a look at it. >> >> I was thinking something like this: >> >> diff --git a/src/hotspot/share/gc/z/zMetronome.cpp >> b/src/hotspot/share/gc/z/zMetronome.cpp >> --- a/src/hotspot/share/gc/z/zMetronome.cpp >> +++ b/src/hotspot/share/gc/z/zMetronome.cpp >> @@ -45,7 +45,9 @@ >> ???? _start_ms = TimeHelper::counter_to_millis(now.value()); >> ?? } >> >> -? for (;;) { >> +? MonitorLocker ml(&_monitor, Monitor::_no_safepoint_check_flag); >> + >> +? while (!_stopped) { >> ???? // We might wake up spuriously from wait, so always recalculate >> ???? // the timeout after a wakeup to see if we need to wait again. >> ???? const Ticks now = Ticks::now(); >> @@ -53,15 +55,24 @@ >> ???? const uint64_t next_ms = _start_ms + (_interval_ms * _nticks); >> ???? const int64_t timeout_ms = next_ms - now_ms; >> >> -??? MonitorLocker ml(&_monitor, Monitor::_no_safepoint_check_flag); >> -??? if (!_stopped && timeout_ms > 0) { >> +??? if (timeout_ms > 0) { >> ?????? // Wait >> ?????? ml.wait(timeout_ms); >> ???? } else { >> ?????? // Tick >> -????? return !_stopped; >> +????? if (timeout_ms < -_interval_ms) { >> +??????? // Missed one or more ticks. Bump _nticks accordingly to >> +??????? // avoid firing a string of immediate ticks to make up >> +??????? // for the ones we missed. >> +??????? _nticks += -timeout_ms / _interval_ms; >> +????? } >> + >> +????? return true; >> ???? } >> ?? } >> + >> +? // Stopped >> +? return false; >> ?} >> >> ?void ZMetronome::stop() { >> diff --git a/src/hotspot/share/gc/z/zMetronome.hpp >> b/src/hotspot/share/gc/z/zMetronome.hpp >> --- a/src/hotspot/share/gc/z/zMetronome.hpp >> +++ b/src/hotspot/share/gc/z/zMetronome.hpp >> @@ -29,11 +29,11 @@ >> >> ?class ZMetronome : public StackObj { >> ?private: >> -? Monitor??????? _monitor; >> -? const uint64_t _interval_ms; >> -? uint64_t?????? _start_ms; >> -? uint64_t?????? _nticks; >> -? bool?????????? _stopped; >> +? Monitor?????? _monitor; >> +? const int64_t _interval_ms; >> +? uint64_t????? _start_ms; >> +? uint64_t????? _nticks; >> +? bool????????? _stopped; >> >> ?public: >> ?? ZMetronome(uint64_t hz); >> diff --git a/src/hotspot/share/gc/z/zStat.cpp >> b/src/hotspot/share/gc/z/zStat.cpp >> --- a/src/hotspot/share/gc/z/zStat.cpp >> +++ b/src/hotspot/share/gc/z/zStat.cpp >> @@ -850,7 +850,16 @@ >> ?} >> >> ?bool ZStat::should_print(LogTargetHandle log) const { >> -? return log.is_enabled() && (_metronome.nticks() % >> ZStatisticsInterval == 0); >> +? static uint64_t print_at_nticks = ZStatisticsInterval; >> +? uint64_t nticks = _metronome.nticks(); >> + >> +? if (nticks < print_at_nticks) { >> +??? return false; >> +? } >> + >> +? print_at_nticks = ((nticks / ZStatisticsInterval) * >> ZStatisticsInterval) + ZStatisticsInterval; >> + >> +? return log.is_enabled(); >> ?} >> >> ?void ZStat::print(LogTargetHandle log, const ZStatSamplerHistory* >> history) const { >> >> >> cheers, >> Per >> >>> >>> cheers, >>> Per >>> >>> On 05/15/2019 04:33 PM, Jie Fu wrote: >>>> Hi all, >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8223961 >>>> >>>> Please review the following patch which fixes the update of >>>> "_nticks" in ZMetronome::wait_for_tick() >>>> ----------------------------------- >>>> diff -r b0b20413d853 src/hotspot/share/gc/z/zMetronome.cpp >>>> --- a/src/hotspot/share/gc/z/zMetronome.cpp???? Wed May 15 11:53:47 >>>> 2019 +0200 >>>> +++ b/src/hotspot/share/gc/z/zMetronome.cpp???? Wed May 15 20:15:39 >>>> 2019 +0800 >>>> @@ -57,6 +57,8 @@ >>>> ????? if (!_stopped && timeout_ms > 0) { >>>> ??????? // Wait >>>> ??????? ml.wait(timeout_ms); >>>> +??? } else if (timeout_ms < -1 * (int64_t)_interval_ms) { >>>> +????? _nticks += -timeout_ms / _interval_ms; >>>> ????? } else { >>>> ??????? // Tick >>>> ??????? return !_stopped; >>>> ----------------------------------- >>>> For more info, please see the JBS. >>>> >>>> Testing: >>>> ?? - make test TEST="hotspot_gc" CONF=release >>>> >>>> Thanks a lot. >>>> Best regards, >>>> Jie >>>> >>>> > From fujie at loongson.cn Fri May 17 07:15:01 2019 From: fujie at loongson.cn (Jie Fu) Date: Fri, 17 May 2019 15:15:01 +0800 Subject: RFR: 8223961: ZGC: Unexpected behaviour was observed caused by the incorrect update of _nticks in ZMetronome::wait_for_tick() In-Reply-To: <4db7be28-440f-59ee-7c61-681ee802ead8@oracle.com> References: <6fa35e69-6d44-45f5-f921-de788cb2d073@loongson.cn> <3854c380-d99e-2995-4ce1-30ba4dbb3a11@oracle.com> <6b2e2fc3-4f39-14ea-e247-a8f4c6b8819d@loongson.cn> <4db7be28-440f-59ee-7c61-681ee802ead8@oracle.com> Message-ID: <00035b17-3934-be2d-41e1-07497e6af3f8@loongson.cn> Thank you so much, Per. Hope to be reviewed soon. Best regards, Jie On 2019/5/17 ??2:14, Per Liden wrote: > Let's wait for a second Reviewer to give thumbs up, then I'll go ahead > and sponsor/push this. > > cheers, > Per From per.liden at oracle.com Fri May 17 08:03:32 2019 From: per.liden at oracle.com (Per Liden) Date: Fri, 17 May 2019 10:03:32 +0200 Subject: RFR: 8223961: ZGC: Unexpected behaviour was observed caused by the incorrect update of _nticks in ZMetronome::wait_for_tick() In-Reply-To: <4db7be28-440f-59ee-7c61-681ee802ead8@oracle.com> References: <6fa35e69-6d44-45f5-f921-de788cb2d073@loongson.cn> <3854c380-d99e-2995-4ce1-30ba4dbb3a11@oracle.com> <6b2e2fc3-4f39-14ea-e247-a8f4c6b8819d@loongson.cn> <4db7be28-440f-59ee-7c61-681ee802ead8@oracle.com> Message-ID: <3762e14f-8943-f659-d287-1f89a95ba2bb@oracle.com> A slight refinement and also removing ZMetronome::nticks() as it's not very useful anymore. http://cr.openjdk.java.net/~pliden/8223961/webrev.0 /Per On 5/17/19 8:14 AM, Per Liden wrote: > Let's wait for a second Reviewer to give thumbs up, then I'll go ahead > and sponsor/push this. > > cheers, > Per > > On 5/16/19 11:21 AM, Jie Fu wrote: >> It's much clearer and more reasonable. >> I like your patch. >> Thanks Per. >> >> On 2019/5/16 ??4:40, Per Liden wrote: >>> On 05/16/2019 08:25 AM, Per Liden wrote: >>>> Hi, >>>> >>>> There is code in ZStat that assumes ZMetronome::ntick() never skips >>>> over ticks. That would need to be addressed if we do something like >>>> this. Otherwise we risk not getting statistics print outs just >>>> because the metronome overslept. I'll take a look at it. >>> >>> I was thinking something like this: >>> >>> diff --git a/src/hotspot/share/gc/z/zMetronome.cpp >>> b/src/hotspot/share/gc/z/zMetronome.cpp >>> --- a/src/hotspot/share/gc/z/zMetronome.cpp >>> +++ b/src/hotspot/share/gc/z/zMetronome.cpp >>> @@ -45,7 +45,9 @@ >>> ???? _start_ms = TimeHelper::counter_to_millis(now.value()); >>> ?? } >>> >>> -? for (;;) { >>> +? MonitorLocker ml(&_monitor, Monitor::_no_safepoint_check_flag); >>> + >>> +? while (!_stopped) { >>> ???? // We might wake up spuriously from wait, so always recalculate >>> ???? // the timeout after a wakeup to see if we need to wait again. >>> ???? const Ticks now = Ticks::now(); >>> @@ -53,15 +55,24 @@ >>> ???? const uint64_t next_ms = _start_ms + (_interval_ms * _nticks); >>> ???? const int64_t timeout_ms = next_ms - now_ms; >>> >>> -??? MonitorLocker ml(&_monitor, Monitor::_no_safepoint_check_flag); >>> -??? if (!_stopped && timeout_ms > 0) { >>> +??? if (timeout_ms > 0) { >>> ?????? // Wait >>> ?????? ml.wait(timeout_ms); >>> ???? } else { >>> ?????? // Tick >>> -????? return !_stopped; >>> +????? if (timeout_ms < -_interval_ms) { >>> +??????? // Missed one or more ticks. Bump _nticks accordingly to >>> +??????? // avoid firing a string of immediate ticks to make up >>> +??????? // for the ones we missed. >>> +??????? _nticks += -timeout_ms / _interval_ms; >>> +????? } >>> + >>> +????? return true; >>> ???? } >>> ?? } >>> + >>> +? // Stopped >>> +? return false; >>> ?} >>> >>> ?void ZMetronome::stop() { >>> diff --git a/src/hotspot/share/gc/z/zMetronome.hpp >>> b/src/hotspot/share/gc/z/zMetronome.hpp >>> --- a/src/hotspot/share/gc/z/zMetronome.hpp >>> +++ b/src/hotspot/share/gc/z/zMetronome.hpp >>> @@ -29,11 +29,11 @@ >>> >>> ?class ZMetronome : public StackObj { >>> ?private: >>> -? Monitor??????? _monitor; >>> -? const uint64_t _interval_ms; >>> -? uint64_t?????? _start_ms; >>> -? uint64_t?????? _nticks; >>> -? bool?????????? _stopped; >>> +? Monitor?????? _monitor; >>> +? const int64_t _interval_ms; >>> +? uint64_t????? _start_ms; >>> +? uint64_t????? _nticks; >>> +? bool????????? _stopped; >>> >>> ?public: >>> ?? ZMetronome(uint64_t hz); >>> diff --git a/src/hotspot/share/gc/z/zStat.cpp >>> b/src/hotspot/share/gc/z/zStat.cpp >>> --- a/src/hotspot/share/gc/z/zStat.cpp >>> +++ b/src/hotspot/share/gc/z/zStat.cpp >>> @@ -850,7 +850,16 @@ >>> ?} >>> >>> ?bool ZStat::should_print(LogTargetHandle log) const { >>> -? return log.is_enabled() && (_metronome.nticks() % >>> ZStatisticsInterval == 0); >>> +? static uint64_t print_at_nticks = ZStatisticsInterval; >>> +? uint64_t nticks = _metronome.nticks(); >>> + >>> +? if (nticks < print_at_nticks) { >>> +??? return false; >>> +? } >>> + >>> +? print_at_nticks = ((nticks / ZStatisticsInterval) * >>> ZStatisticsInterval) + ZStatisticsInterval; >>> + >>> +? return log.is_enabled(); >>> ?} >>> >>> ?void ZStat::print(LogTargetHandle log, const ZStatSamplerHistory* >>> history) const { >>> >>> >>> cheers, >>> Per >>> >>>> >>>> cheers, >>>> Per >>>> >>>> On 05/15/2019 04:33 PM, Jie Fu wrote: >>>>> Hi all, >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8223961 >>>>> >>>>> Please review the following patch which fixes the update of >>>>> "_nticks" in ZMetronome::wait_for_tick() >>>>> ----------------------------------- >>>>> diff -r b0b20413d853 src/hotspot/share/gc/z/zMetronome.cpp >>>>> --- a/src/hotspot/share/gc/z/zMetronome.cpp???? Wed May 15 11:53:47 >>>>> 2019 +0200 >>>>> +++ b/src/hotspot/share/gc/z/zMetronome.cpp???? Wed May 15 20:15:39 >>>>> 2019 +0800 >>>>> @@ -57,6 +57,8 @@ >>>>> ????? if (!_stopped && timeout_ms > 0) { >>>>> ??????? // Wait >>>>> ??????? ml.wait(timeout_ms); >>>>> +??? } else if (timeout_ms < -1 * (int64_t)_interval_ms) { >>>>> +????? _nticks += -timeout_ms / _interval_ms; >>>>> ????? } else { >>>>> ??????? // Tick >>>>> ??????? return !_stopped; >>>>> ----------------------------------- >>>>> For more info, please see the JBS. >>>>> >>>>> Testing: >>>>> ?? - make test TEST="hotspot_gc" CONF=release >>>>> >>>>> Thanks a lot. >>>>> Best regards, >>>>> Jie >>>>> >>>>> >> From boris.ulasevich at bell-sw.com Fri May 17 08:05:24 2019 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Fri, 17 May 2019 11:05:24 +0300 Subject: RFR: 8223767: Shenandoah fails to build on Solaris x86_64 In-Reply-To: <27b5d599-b25f-073d-0609-1c84c2433d1d@redhat.com> References: <87dcfaac-f351-b8d5-ba30-4029124e33f6@redhat.com> <3387252d-af57-0f52-5219-98abccdf810b@bell-sw.com> <55bb8152-3ec6-dcc0-65fe-3b7efd0b7919@bell-sw.com> <596a0613-97f1-5673-01c9-06f63f8edbcb@redhat.com> <9f832ee6-2f06-d29f-807b-125e1f853646@redhat.com> <27b5d599-b25f-073d-0609-1c84c2433d1d@redhat.com> Message-ID: <9fcac029-e789-dc33-b58f-365e4084b7da@bell-sw.com> Thank you! On 16.05.2019 21:00, Aleksey Shipilev wrote: > On 5/16/19 6:53 PM, Aleksey Shipilev wrote: >> On 5/16/19 5:43 PM, Roman Kennke wrote: >>>> What is the correct backport procedure? >>> >>> I think for the Shenandoah parts it's fine to post them here for review, >>> and one of us can integrate them to shenandoah/jdk11 and shenandoah/jdk8 >>> for you. Please prepare the changesets with prefixed [backport] in the >>> commit msg, e.g.: >>> >>> [backport] 8223767: Shenandoah fails to build on Solaris x86_64 >> >> I can do this myself while sponsoring fixes for Boris. Thanks for the backport! > > There: > http://hg.openjdk.java.net/shenandoah/jdk11/rev/159ea4412948 > http://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/468dd92b90f9 > > Pushed them with "author" set to "bulasevich" to give credit. > > -Aleksey > From stefan.karlsson at oracle.com Fri May 17 08:05:36 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 17 May 2019 10:05:36 +0200 Subject: RFR: 8223961: ZGC: Unexpected behaviour was observed caused by the incorrect update of _nticks in ZMetronome::wait_for_tick() In-Reply-To: <3762e14f-8943-f659-d287-1f89a95ba2bb@oracle.com> References: <6fa35e69-6d44-45f5-f921-de788cb2d073@loongson.cn> <3854c380-d99e-2995-4ce1-30ba4dbb3a11@oracle.com> <6b2e2fc3-4f39-14ea-e247-a8f4c6b8819d@loongson.cn> <4db7be28-440f-59ee-7c61-681ee802ead8@oracle.com> <3762e14f-8943-f659-d287-1f89a95ba2bb@oracle.com> Message-ID: <4d20f994-9bc0-52ac-2654-c65b22e0b08b@oracle.com> Looks good. StefanK On 2019-05-17 10:03, Per Liden wrote: > A slight refinement and also removing ZMetronome::nticks() as it's not > very useful anymore. > > http://cr.openjdk.java.net/~pliden/8223961/webrev.0 > > /Per > > On 5/17/19 8:14 AM, Per Liden wrote: >> Let's wait for a second Reviewer to give thumbs up, then I'll go ahead >> and sponsor/push this. >> >> cheers, >> Per >> >> On 5/16/19 11:21 AM, Jie Fu wrote: >>> It's much clearer and more reasonable. >>> I like your patch. >>> Thanks Per. >>> >>> On 2019/5/16 ??4:40, Per Liden wrote: >>>> On 05/16/2019 08:25 AM, Per Liden wrote: >>>>> Hi, >>>>> >>>>> There is code in ZStat that assumes ZMetronome::ntick() never skips >>>>> over ticks. That would need to be addressed if we do something like >>>>> this. Otherwise we risk not getting statistics print outs just >>>>> because the metronome overslept. I'll take a look at it. >>>> >>>> I was thinking something like this: >>>> >>>> diff --git a/src/hotspot/share/gc/z/zMetronome.cpp >>>> b/src/hotspot/share/gc/z/zMetronome.cpp >>>> --- a/src/hotspot/share/gc/z/zMetronome.cpp >>>> +++ b/src/hotspot/share/gc/z/zMetronome.cpp >>>> @@ -45,7 +45,9 @@ >>>> ???? _start_ms = TimeHelper::counter_to_millis(now.value()); >>>> ?? } >>>> >>>> -? for (;;) { >>>> +? MonitorLocker ml(&_monitor, Monitor::_no_safepoint_check_flag); >>>> + >>>> +? while (!_stopped) { >>>> ???? // We might wake up spuriously from wait, so always recalculate >>>> ???? // the timeout after a wakeup to see if we need to wait again. >>>> ???? const Ticks now = Ticks::now(); >>>> @@ -53,15 +55,24 @@ >>>> ???? const uint64_t next_ms = _start_ms + (_interval_ms * _nticks); >>>> ???? const int64_t timeout_ms = next_ms - now_ms; >>>> >>>> -??? MonitorLocker ml(&_monitor, Monitor::_no_safepoint_check_flag); >>>> -??? if (!_stopped && timeout_ms > 0) { >>>> +??? if (timeout_ms > 0) { >>>> ?????? // Wait >>>> ?????? ml.wait(timeout_ms); >>>> ???? } else { >>>> ?????? // Tick >>>> -????? return !_stopped; >>>> +????? if (timeout_ms < -_interval_ms) { >>>> +??????? // Missed one or more ticks. Bump _nticks accordingly to >>>> +??????? // avoid firing a string of immediate ticks to make up >>>> +??????? // for the ones we missed. >>>> +??????? _nticks += -timeout_ms / _interval_ms; >>>> +????? } >>>> + >>>> +????? return true; >>>> ???? } >>>> ?? } >>>> + >>>> +? // Stopped >>>> +? return false; >>>> ?} >>>> >>>> ?void ZMetronome::stop() { >>>> diff --git a/src/hotspot/share/gc/z/zMetronome.hpp >>>> b/src/hotspot/share/gc/z/zMetronome.hpp >>>> --- a/src/hotspot/share/gc/z/zMetronome.hpp >>>> +++ b/src/hotspot/share/gc/z/zMetronome.hpp >>>> @@ -29,11 +29,11 @@ >>>> >>>> ?class ZMetronome : public StackObj { >>>> ?private: >>>> -? Monitor??????? _monitor; >>>> -? const uint64_t _interval_ms; >>>> -? uint64_t?????? _start_ms; >>>> -? uint64_t?????? _nticks; >>>> -? bool?????????? _stopped; >>>> +? Monitor?????? _monitor; >>>> +? const int64_t _interval_ms; >>>> +? uint64_t????? _start_ms; >>>> +? uint64_t????? _nticks; >>>> +? bool????????? _stopped; >>>> >>>> ?public: >>>> ?? ZMetronome(uint64_t hz); >>>> diff --git a/src/hotspot/share/gc/z/zStat.cpp >>>> b/src/hotspot/share/gc/z/zStat.cpp >>>> --- a/src/hotspot/share/gc/z/zStat.cpp >>>> +++ b/src/hotspot/share/gc/z/zStat.cpp >>>> @@ -850,7 +850,16 @@ >>>> ?} >>>> >>>> ?bool ZStat::should_print(LogTargetHandle log) const { >>>> -? return log.is_enabled() && (_metronome.nticks() % >>>> ZStatisticsInterval == 0); >>>> +? static uint64_t print_at_nticks = ZStatisticsInterval; >>>> +? uint64_t nticks = _metronome.nticks(); >>>> + >>>> +? if (nticks < print_at_nticks) { >>>> +??? return false; >>>> +? } >>>> + >>>> +? print_at_nticks = ((nticks / ZStatisticsInterval) * >>>> ZStatisticsInterval) + ZStatisticsInterval; >>>> + >>>> +? return log.is_enabled(); >>>> ?} >>>> >>>> ?void ZStat::print(LogTargetHandle log, const ZStatSamplerHistory* >>>> history) const { >>>> >>>> >>>> cheers, >>>> Per >>>> >>>>> >>>>> cheers, >>>>> Per >>>>> >>>>> On 05/15/2019 04:33 PM, Jie Fu wrote: >>>>>> Hi all, >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8223961 >>>>>> >>>>>> Please review the following patch which fixes the update of >>>>>> "_nticks" in ZMetronome::wait_for_tick() >>>>>> ----------------------------------- >>>>>> diff -r b0b20413d853 src/hotspot/share/gc/z/zMetronome.cpp >>>>>> --- a/src/hotspot/share/gc/z/zMetronome.cpp???? Wed May 15 >>>>>> 11:53:47 2019 +0200 >>>>>> +++ b/src/hotspot/share/gc/z/zMetronome.cpp???? Wed May 15 >>>>>> 20:15:39 2019 +0800 >>>>>> @@ -57,6 +57,8 @@ >>>>>> ????? if (!_stopped && timeout_ms > 0) { >>>>>> ??????? // Wait >>>>>> ??????? ml.wait(timeout_ms); >>>>>> +??? } else if (timeout_ms < -1 * (int64_t)_interval_ms) { >>>>>> +????? _nticks += -timeout_ms / _interval_ms; >>>>>> ????? } else { >>>>>> ??????? // Tick >>>>>> ??????? return !_stopped; >>>>>> ----------------------------------- >>>>>> For more info, please see the JBS. >>>>>> >>>>>> Testing: >>>>>> ?? - make test TEST="hotspot_gc" CONF=release >>>>>> >>>>>> Thanks a lot. >>>>>> Best regards, >>>>>> Jie >>>>>> >>>>>> >>> From per.liden at oracle.com Fri May 17 08:06:19 2019 From: per.liden at oracle.com (Per Liden) Date: Fri, 17 May 2019 10:06:19 +0200 Subject: RFR: 8223961: ZGC: Unexpected behaviour was observed caused by the incorrect update of _nticks in ZMetronome::wait_for_tick() In-Reply-To: <4d20f994-9bc0-52ac-2654-c65b22e0b08b@oracle.com> References: <6fa35e69-6d44-45f5-f921-de788cb2d073@loongson.cn> <3854c380-d99e-2995-4ce1-30ba4dbb3a11@oracle.com> <6b2e2fc3-4f39-14ea-e247-a8f4c6b8819d@loongson.cn> <4db7be28-440f-59ee-7c61-681ee802ead8@oracle.com> <3762e14f-8943-f659-d287-1f89a95ba2bb@oracle.com> <4d20f994-9bc0-52ac-2654-c65b22e0b08b@oracle.com> Message-ID: <653ff97f-75ca-ecb7-b2c9-ff3c0f4e3f83@oracle.com> Thanks Stefan! /Per On 5/17/19 10:05 AM, Stefan Karlsson wrote: > Looks good. > > StefanK > > On 2019-05-17 10:03, Per Liden wrote: >> A slight refinement and also removing ZMetronome::nticks() as it's not >> very useful anymore. >> >> http://cr.openjdk.java.net/~pliden/8223961/webrev.0 >> >> /Per >> >> On 5/17/19 8:14 AM, Per Liden wrote: >>> Let's wait for a second Reviewer to give thumbs up, then I'll go >>> ahead and sponsor/push this. >>> >>> cheers, >>> Per >>> >>> On 5/16/19 11:21 AM, Jie Fu wrote: >>>> It's much clearer and more reasonable. >>>> I like your patch. >>>> Thanks Per. >>>> >>>> On 2019/5/16 ??4:40, Per Liden wrote: >>>>> On 05/16/2019 08:25 AM, Per Liden wrote: >>>>>> Hi, >>>>>> >>>>>> There is code in ZStat that assumes ZMetronome::ntick() never >>>>>> skips over ticks. That would need to be addressed if we do >>>>>> something like this. Otherwise we risk not getting statistics >>>>>> print outs just because the metronome overslept. I'll take a look >>>>>> at it. >>>>> >>>>> I was thinking something like this: >>>>> >>>>> diff --git a/src/hotspot/share/gc/z/zMetronome.cpp >>>>> b/src/hotspot/share/gc/z/zMetronome.cpp >>>>> --- a/src/hotspot/share/gc/z/zMetronome.cpp >>>>> +++ b/src/hotspot/share/gc/z/zMetronome.cpp >>>>> @@ -45,7 +45,9 @@ >>>>> ???? _start_ms = TimeHelper::counter_to_millis(now.value()); >>>>> ?? } >>>>> >>>>> -? for (;;) { >>>>> +? MonitorLocker ml(&_monitor, Monitor::_no_safepoint_check_flag); >>>>> + >>>>> +? while (!_stopped) { >>>>> ???? // We might wake up spuriously from wait, so always recalculate >>>>> ???? // the timeout after a wakeup to see if we need to wait again. >>>>> ???? const Ticks now = Ticks::now(); >>>>> @@ -53,15 +55,24 @@ >>>>> ???? const uint64_t next_ms = _start_ms + (_interval_ms * _nticks); >>>>> ???? const int64_t timeout_ms = next_ms - now_ms; >>>>> >>>>> -??? MonitorLocker ml(&_monitor, Monitor::_no_safepoint_check_flag); >>>>> -??? if (!_stopped && timeout_ms > 0) { >>>>> +??? if (timeout_ms > 0) { >>>>> ?????? // Wait >>>>> ?????? ml.wait(timeout_ms); >>>>> ???? } else { >>>>> ?????? // Tick >>>>> -????? return !_stopped; >>>>> +????? if (timeout_ms < -_interval_ms) { >>>>> +??????? // Missed one or more ticks. Bump _nticks accordingly to >>>>> +??????? // avoid firing a string of immediate ticks to make up >>>>> +??????? // for the ones we missed. >>>>> +??????? _nticks += -timeout_ms / _interval_ms; >>>>> +????? } >>>>> + >>>>> +????? return true; >>>>> ???? } >>>>> ?? } >>>>> + >>>>> +? // Stopped >>>>> +? return false; >>>>> ?} >>>>> >>>>> ?void ZMetronome::stop() { >>>>> diff --git a/src/hotspot/share/gc/z/zMetronome.hpp >>>>> b/src/hotspot/share/gc/z/zMetronome.hpp >>>>> --- a/src/hotspot/share/gc/z/zMetronome.hpp >>>>> +++ b/src/hotspot/share/gc/z/zMetronome.hpp >>>>> @@ -29,11 +29,11 @@ >>>>> >>>>> ?class ZMetronome : public StackObj { >>>>> ?private: >>>>> -? Monitor??????? _monitor; >>>>> -? const uint64_t _interval_ms; >>>>> -? uint64_t?????? _start_ms; >>>>> -? uint64_t?????? _nticks; >>>>> -? bool?????????? _stopped; >>>>> +? Monitor?????? _monitor; >>>>> +? const int64_t _interval_ms; >>>>> +? uint64_t????? _start_ms; >>>>> +? uint64_t????? _nticks; >>>>> +? bool????????? _stopped; >>>>> >>>>> ?public: >>>>> ?? ZMetronome(uint64_t hz); >>>>> diff --git a/src/hotspot/share/gc/z/zStat.cpp >>>>> b/src/hotspot/share/gc/z/zStat.cpp >>>>> --- a/src/hotspot/share/gc/z/zStat.cpp >>>>> +++ b/src/hotspot/share/gc/z/zStat.cpp >>>>> @@ -850,7 +850,16 @@ >>>>> ?} >>>>> >>>>> ?bool ZStat::should_print(LogTargetHandle log) const { >>>>> -? return log.is_enabled() && (_metronome.nticks() % >>>>> ZStatisticsInterval == 0); >>>>> +? static uint64_t print_at_nticks = ZStatisticsInterval; >>>>> +? uint64_t nticks = _metronome.nticks(); >>>>> + >>>>> +? if (nticks < print_at_nticks) { >>>>> +??? return false; >>>>> +? } >>>>> + >>>>> +? print_at_nticks = ((nticks / ZStatisticsInterval) * >>>>> ZStatisticsInterval) + ZStatisticsInterval; >>>>> + >>>>> +? return log.is_enabled(); >>>>> ?} >>>>> >>>>> ?void ZStat::print(LogTargetHandle log, const ZStatSamplerHistory* >>>>> history) const { >>>>> >>>>> >>>>> cheers, >>>>> Per >>>>> >>>>>> >>>>>> cheers, >>>>>> Per >>>>>> >>>>>> On 05/15/2019 04:33 PM, Jie Fu wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8223961 >>>>>>> >>>>>>> Please review the following patch which fixes the update of >>>>>>> "_nticks" in ZMetronome::wait_for_tick() >>>>>>> ----------------------------------- >>>>>>> diff -r b0b20413d853 src/hotspot/share/gc/z/zMetronome.cpp >>>>>>> --- a/src/hotspot/share/gc/z/zMetronome.cpp???? Wed May 15 >>>>>>> 11:53:47 2019 +0200 >>>>>>> +++ b/src/hotspot/share/gc/z/zMetronome.cpp???? Wed May 15 >>>>>>> 20:15:39 2019 +0800 >>>>>>> @@ -57,6 +57,8 @@ >>>>>>> ????? if (!_stopped && timeout_ms > 0) { >>>>>>> ??????? // Wait >>>>>>> ??????? ml.wait(timeout_ms); >>>>>>> +??? } else if (timeout_ms < -1 * (int64_t)_interval_ms) { >>>>>>> +????? _nticks += -timeout_ms / _interval_ms; >>>>>>> ????? } else { >>>>>>> ??????? // Tick >>>>>>> ??????? return !_stopped; >>>>>>> ----------------------------------- >>>>>>> For more info, please see the JBS. >>>>>>> >>>>>>> Testing: >>>>>>> ?? - make test TEST="hotspot_gc" CONF=release >>>>>>> >>>>>>> Thanks a lot. >>>>>>> Best regards, >>>>>>> Jie >>>>>>> >>>>>>> >>>> From fujie at loongson.cn Fri May 17 08:32:54 2019 From: fujie at loongson.cn (Jie Fu) Date: Fri, 17 May 2019 16:32:54 +0800 Subject: RFR: 8223961: ZGC: Unexpected behaviour was observed caused by the incorrect update of _nticks in ZMetronome::wait_for_tick() In-Reply-To: <3762e14f-8943-f659-d287-1f89a95ba2bb@oracle.com> References: <6fa35e69-6d44-45f5-f921-de788cb2d073@loongson.cn> <3854c380-d99e-2995-4ce1-30ba4dbb3a11@oracle.com> <6b2e2fc3-4f39-14ea-e247-a8f4c6b8819d@loongson.cn> <4db7be28-440f-59ee-7c61-681ee802ead8@oracle.com> <3762e14f-8943-f659-d287-1f89a95ba2bb@oracle.com> Message-ID: <7e519aa0-6789-4259-7fa1-c2401368e14f@loongson.cn> Very nice. Feel much better when debugging now. Thank you, Per. Best regards, Jie On 2019/5/17 ??4:03, Per Liden wrote: > A slight refinement and also removing ZMetronome::nticks() as it's not > very useful anymore. > > http://cr.openjdk.java.net/~pliden/8223961/webrev.0 > > /Per > > On 5/17/19 8:14 AM, Per Liden wrote: >> Let's wait for a second Reviewer to give thumbs up, then I'll go >> ahead and sponsor/push this. >> >> cheers, >> Per >> >> On 5/16/19 11:21 AM, Jie Fu wrote: >>> It's much clearer and more reasonable. >>> I like your patch. >>> Thanks Per. >>> >>> On 2019/5/16 ??4:40, Per Liden wrote: >>>> On 05/16/2019 08:25 AM, Per Liden wrote: >>>>> Hi, >>>>> >>>>> There is code in ZStat that assumes ZMetronome::ntick() never >>>>> skips over ticks. That would need to be addressed if we do >>>>> something like this. Otherwise we risk not getting statistics >>>>> print outs just because the metronome overslept. I'll take a look >>>>> at it. >>>> >>>> I was thinking something like this: >>>> >>>> diff --git a/src/hotspot/share/gc/z/zMetronome.cpp >>>> b/src/hotspot/share/gc/z/zMetronome.cpp >>>> --- a/src/hotspot/share/gc/z/zMetronome.cpp >>>> +++ b/src/hotspot/share/gc/z/zMetronome.cpp >>>> @@ -45,7 +45,9 @@ >>>> ???? _start_ms = TimeHelper::counter_to_millis(now.value()); >>>> ?? } >>>> >>>> -? for (;;) { >>>> +? MonitorLocker ml(&_monitor, Monitor::_no_safepoint_check_flag); >>>> + >>>> +? while (!_stopped) { >>>> ???? // We might wake up spuriously from wait, so always recalculate >>>> ???? // the timeout after a wakeup to see if we need to wait again. >>>> ???? const Ticks now = Ticks::now(); >>>> @@ -53,15 +55,24 @@ >>>> ???? const uint64_t next_ms = _start_ms + (_interval_ms * _nticks); >>>> ???? const int64_t timeout_ms = next_ms - now_ms; >>>> >>>> -??? MonitorLocker ml(&_monitor, Monitor::_no_safepoint_check_flag); >>>> -??? if (!_stopped && timeout_ms > 0) { >>>> +??? if (timeout_ms > 0) { >>>> ?????? // Wait >>>> ?????? ml.wait(timeout_ms); >>>> ???? } else { >>>> ?????? // Tick >>>> -????? return !_stopped; >>>> +????? if (timeout_ms < -_interval_ms) { >>>> +??????? // Missed one or more ticks. Bump _nticks accordingly to >>>> +??????? // avoid firing a string of immediate ticks to make up >>>> +??????? // for the ones we missed. >>>> +??????? _nticks += -timeout_ms / _interval_ms; >>>> +????? } >>>> + >>>> +????? return true; >>>> ???? } >>>> ?? } >>>> + >>>> +? // Stopped >>>> +? return false; >>>> ?} >>>> >>>> ?void ZMetronome::stop() { >>>> diff --git a/src/hotspot/share/gc/z/zMetronome.hpp >>>> b/src/hotspot/share/gc/z/zMetronome.hpp >>>> --- a/src/hotspot/share/gc/z/zMetronome.hpp >>>> +++ b/src/hotspot/share/gc/z/zMetronome.hpp >>>> @@ -29,11 +29,11 @@ >>>> >>>> ?class ZMetronome : public StackObj { >>>> ?private: >>>> -? Monitor??????? _monitor; >>>> -? const uint64_t _interval_ms; >>>> -? uint64_t?????? _start_ms; >>>> -? uint64_t?????? _nticks; >>>> -? bool?????????? _stopped; >>>> +? Monitor?????? _monitor; >>>> +? const int64_t _interval_ms; >>>> +? uint64_t????? _start_ms; >>>> +? uint64_t????? _nticks; >>>> +? bool????????? _stopped; >>>> >>>> ?public: >>>> ?? ZMetronome(uint64_t hz); >>>> diff --git a/src/hotspot/share/gc/z/zStat.cpp >>>> b/src/hotspot/share/gc/z/zStat.cpp >>>> --- a/src/hotspot/share/gc/z/zStat.cpp >>>> +++ b/src/hotspot/share/gc/z/zStat.cpp >>>> @@ -850,7 +850,16 @@ >>>> ?} >>>> >>>> ?bool ZStat::should_print(LogTargetHandle log) const { >>>> -? return log.is_enabled() && (_metronome.nticks() % >>>> ZStatisticsInterval == 0); >>>> +? static uint64_t print_at_nticks = ZStatisticsInterval; >>>> +? uint64_t nticks = _metronome.nticks(); >>>> + >>>> +? if (nticks < print_at_nticks) { >>>> +??? return false; >>>> +? } >>>> + >>>> +? print_at_nticks = ((nticks / ZStatisticsInterval) * >>>> ZStatisticsInterval) + ZStatisticsInterval; >>>> + >>>> +? return log.is_enabled(); >>>> ?} >>>> >>>> ?void ZStat::print(LogTargetHandle log, const ZStatSamplerHistory* >>>> history) const { >>>> >>>> >>>> cheers, >>>> Per >>>> >>>>> >>>>> cheers, >>>>> Per >>>>> >>>>> On 05/15/2019 04:33 PM, Jie Fu wrote: >>>>>> Hi all, >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8223961 >>>>>> >>>>>> Please review the following patch which fixes the update of >>>>>> "_nticks" in ZMetronome::wait_for_tick() >>>>>> ----------------------------------- >>>>>> diff -r b0b20413d853 src/hotspot/share/gc/z/zMetronome.cpp >>>>>> --- a/src/hotspot/share/gc/z/zMetronome.cpp???? Wed May 15 >>>>>> 11:53:47 2019 +0200 >>>>>> +++ b/src/hotspot/share/gc/z/zMetronome.cpp???? Wed May 15 >>>>>> 20:15:39 2019 +0800 >>>>>> @@ -57,6 +57,8 @@ >>>>>> ????? if (!_stopped && timeout_ms > 0) { >>>>>> ??????? // Wait >>>>>> ??????? ml.wait(timeout_ms); >>>>>> +??? } else if (timeout_ms < -1 * (int64_t)_interval_ms) { >>>>>> +????? _nticks += -timeout_ms / _interval_ms; >>>>>> ????? } else { >>>>>> ??????? // Tick >>>>>> ??????? return !_stopped; >>>>>> ----------------------------------- >>>>>> For more info, please see the JBS. >>>>>> >>>>>> Testing: >>>>>> ?? - make test TEST="hotspot_gc" CONF=release >>>>>> >>>>>> Thanks a lot. >>>>>> Best regards, >>>>>> Jie >>>>>> >>>>>> >>> From thomas.schatzl at oracle.com Fri May 17 10:27:08 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 17 May 2019 12:27:08 +0200 Subject: RFR (XS): 8224038: Fix remaining InCSetState mentions In-Reply-To: References: <2e9a7f8cd5d04470b24f02bb82b6478b73ba179c.camel@oracle.com> Message-ID: <94090e42749695f0f6960e49e228b67b2160a66b.camel@oracle.com> Hi Kim, On Thu, 2019-05-16 at 14:10 -0400, Kim Barrett wrote: > On May 16, 2019, at 8:49 AM, Thomas Schatzl < > thomas.schatzl at oracle.com> wrote: > > > > Hi all, > > [...] > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8224038 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8224038/webrev/ > > Testing: > > local compilation > > > > Thanks, > > Thomas > > > > > > Looks good, and trivial. > thanks for your review. Will push. Thanks, Thomas From thomas.schatzl at oracle.com Fri May 17 10:31:47 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 17 May 2019 12:31:47 +0200 Subject: RFR (S/M): 8222492: G1 unnecessarily scans remembered set cards for regions that already have been evacuated In-Reply-To: <5D369410-424A-4550-9B02-B09725C502B8@oracle.com> References: <3e061b852275110f67a309f66211dd5832f0e5b4.camel@oracle.com> <5D369410-424A-4550-9B02-B09725C502B8@oracle.com> Message-ID: Hi Kim, thanks for your review. On Thu, 2019-05-16 at 18:45 -0400, Kim Barrett wrote: > On Apr 26, 2019, at 8:27 AM, Thomas Schatzl < > thomas.schatzl at oracle.com> wrote: > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8222492 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8222492/webrev/ > > Testing: > > hs-tier1-5, running the crashing test of JDK-8222426 for 5k times > > without failures (the crash has an occurrence of 1-2 times/1000 > > runs). > > > > Thanks, > > Thomas > > > > Looks good. > > One minor nit: > > src/hotspot/share/gc/g1/g1OopClosures.inline.hpp > 190 // Entries in the optional collection set may become to be > located in the collection > 191 // set during incremental collection of the old generation. > > That comment needs some fixing. > I changed the comment to the following and refreshed the webrev: // Entries in the optional collection set may start to originate from the collection // set after one or more increments. In this case, previously optional regions // became actual collection set regions. Filter them out here. Thanks, Thomas From shade at redhat.com Fri May 17 11:59:34 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 17 May 2019 13:59:34 +0200 Subject: RFR 8223774: Shenandoah: Refactor ShenandoahRootProcessor and family In-Reply-To: <60b7f75e-b8ec-7122-4e39-f7dca14a2ea8@redhat.com> References: <451b6348-8c37-9a46-31d3-6f6ea960de4a@redhat.com> <57c55b42-c03e-2649-e8c5-37a9101e93a1@redhat.com> <102078db-5b2b-804b-556c-b8114bd40bff@redhat.com> <83d496a1-c767-755e-13af-46ee4102dc97@redhat.com> <60b7f75e-b8ec-7122-4e39-f7dca14a2ea8@redhat.com> Message-ID: On 5/16/19 5:09 PM, Zhengyu Gu wrote: > Updated: http://cr.openjdk.java.net/~zgu/JDK-8223774/webrev.02/ I like it. *) No, this block should be in ShenandoahTraversalGC*Heuristics*, not in ShenandoahTraversalGC itself, look: http://hg.openjdk.java.net/jdk/jdk/file/23837d614c17/src/hotspot/share/gc/shenandoah/heuristics/shenandoahTraversalHeuristics.cpp#l36 313 // Traversal does not support concurrent code root scanning 314 FLAG_SET_DEFAULT(ShenandoahConcurrentScanCodeRoots, false); *) One more naming suggestion (see if it makes sense!). Current hierarchy is: ShenandoahRootProcessingPhase -> ShenandoahRootProcessor -> ShenandoahRootEvacuator -> ShenandoahRootUpdater -> ShenandoahRootAdjuster It leads to awkward superclass name. Maybe we should rename ShenandoahRootProcessor to reflect what it does to also match Evacuator/Updater/Adjuster? This frees ShenandoahRootProcessor to become the superclass. For example: ShenandoahRootProcessor -> ShenandoahRootScanner [Does it *only* scan, though? Dunno.] -> ShenandoahRootEvacuator -> ShenandoahRootUpdater -> ShenandoahRootAdjuster -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Fri May 17 12:15:21 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Fri, 17 May 2019 08:15:21 -0400 Subject: RFR 8223774: Shenandoah: Refactor ShenandoahRootProcessor and family In-Reply-To: References: <451b6348-8c37-9a46-31d3-6f6ea960de4a@redhat.com> <57c55b42-c03e-2649-e8c5-37a9101e93a1@redhat.com> <102078db-5b2b-804b-556c-b8114bd40bff@redhat.com> <83d496a1-c767-755e-13af-46ee4102dc97@redhat.com> <60b7f75e-b8ec-7122-4e39-f7dca14a2ea8@redhat.com> Message-ID: On 5/17/19 7:59 AM, Aleksey Shipilev wrote: > On 5/16/19 5:09 PM, Zhengyu Gu wrote: >> Updated: http://cr.openjdk.java.net/~zgu/JDK-8223774/webrev.02/ > > I like it. > > *) No, this block should be in ShenandoahTraversalGC*Heuristics*, not in ShenandoahTraversalGC > itself, look: > http://hg.openjdk.java.net/jdk/jdk/file/23837d614c17/src/hotspot/share/gc/shenandoah/heuristics/shenandoahTraversalHeuristics.cpp#l36 > > 313 // Traversal does not support concurrent code root scanning > 314 FLAG_SET_DEFAULT(ShenandoahConcurrentScanCodeRoots, false); > > *) One more naming suggestion (see if it makes sense!). Current hierarchy is: > > ShenandoahRootProcessingPhase > -> ShenandoahRootProcessor > -> ShenandoahRootEvacuator > -> ShenandoahRootUpdater > -> ShenandoahRootAdjuster > > It leads to awkward superclass name. Maybe we should rename ShenandoahRootProcessor to reflect what > it does to also match Evacuator/Updater/Adjuster? This frees ShenandoahRootProcessor to become the > superclass. For example: > > ShenandoahRootProcessor > -> ShenandoahRootScanner [Does it *only* scan, though? Dunno.] > -> ShenandoahRootEvacuator > -> ShenandoahRootUpdater > -> ShenandoahRootAdjuster Was in my mind too, too lazy to rename :-( will do before push. Thanks, -Zhengyu > > -Aleksey > > From rkennke at redhat.com Fri May 17 12:33:46 2019 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 17 May 2019 14:33:46 +0200 Subject: RFR 8223774: Shenandoah: Refactor ShenandoahRootProcessor and family In-Reply-To: References: <451b6348-8c37-9a46-31d3-6f6ea960de4a@redhat.com> <57c55b42-c03e-2649-e8c5-37a9101e93a1@redhat.com> <102078db-5b2b-804b-556c-b8114bd40bff@redhat.com> <83d496a1-c767-755e-13af-46ee4102dc97@redhat.com> <60b7f75e-b8ec-7122-4e39-f7dca14a2ea8@redhat.com> Message-ID: <37c69240-f89c-42f3-9ed7-d835c5b6f0be@redhat.com> >>> Updated: http://cr.openjdk.java.net/~zgu/JDK-8223774/webrev.02/ Looks good (with Aleksey's suggestion). Thanks & cheers, Roman >> >> I like it. >> >> *) No, this block should be in ShenandoahTraversalGC*Heuristics*, not >> in ShenandoahTraversalGC >> itself, look: >> http://hg.openjdk.java.net/jdk/jdk/file/23837d614c17/src/hotspot/share/gc/shenandoah/heuristics/shenandoahTraversalHeuristics.cpp#l36 >> >> >> ? 313?? // Traversal does not support concurrent code root scanning >> ? 314?? FLAG_SET_DEFAULT(ShenandoahConcurrentScanCodeRoots, false); >> >> *) One more naming suggestion (see if it makes sense!). Current >> hierarchy is: >> >> ? ShenandoahRootProcessingPhase >> ??? -> ShenandoahRootProcessor >> ??? -> ShenandoahRootEvacuator >> ??? -> ShenandoahRootUpdater >> ??? -> ShenandoahRootAdjuster >> >> It leads to awkward superclass name. Maybe we should rename >> ShenandoahRootProcessor to reflect what >> it does to also match Evacuator/Updater/Adjuster? This frees >> ShenandoahRootProcessor to become the >> superclass. For example: >> >> ? ShenandoahRootProcessor >> ??? -> ShenandoahRootScanner [Does it *only* scan, though? Dunno.] >> ??? -> ShenandoahRootEvacuator >> ??? -> ShenandoahRootUpdater >> ??? -> ShenandoahRootAdjuster > Was in my mind too, too lazy to rename :-( will do before push. > > Thanks, > > -Zhengyu > >> >> -Aleksey >> >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Fri May 17 14:12:33 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 17 May 2019 16:12:33 +0200 Subject: RFR(S) 8223951: Shenandoah: Only need to update thread roots during final update refs In-Reply-To: <52773c59-62fc-d763-0e9d-b79131eeba19@redhat.com> References: <52773c59-62fc-d763-0e9d-b79131eeba19@redhat.com> Message-ID: On 5/16/19 4:25 PM, Zhengyu Gu wrote: > After switching to strong to-space invariant and pre-evacuating all root, thread roots are only > places that may contain to-space oops after updating references during normal concurrent cycles, we > only need to update them during final updaterefs pause, which should reduce pause time. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8223951 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8223951/webrev.00/ Looks fine to me. Aside: I wonder how would be get the thread root with from-space ptr if we don't have safepoints between loads and the LRBs that give us to-space ptrs after potential evac. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Fri May 17 14:17:51 2019 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 17 May 2019 16:17:51 +0200 Subject: RFR(S) 8223951: Shenandoah: Only need to update thread roots during final update refs In-Reply-To: References: <52773c59-62fc-d763-0e9d-b79131eeba19@redhat.com> Message-ID: <9ff8320d-8750-1ce0-9130-207e4d49c617@redhat.com> >> After switching to strong to-space invariant and pre-evacuating all root, thread roots are only >> places that may contain to-space oops after updating references during normal concurrent cycles, we >> only need to update them during final updaterefs pause, which should reduce pause time. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8223951 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8223951/webrev.00/ > > Looks fine to me. > > Aside: I wonder how would be get the thread root with from-space ptr if we don't have safepoints > between loads and the LRBs that give us to-space ptrs after potential evac. I don't think we cann guarantee that we don't get a safepoint between load and LRB. Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Fri May 17 14:20:25 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 17 May 2019 16:20:25 +0200 Subject: RFR(S) 8223951: Shenandoah: Only need to update thread roots during final update refs In-Reply-To: <9ff8320d-8750-1ce0-9130-207e4d49c617@redhat.com> References: <52773c59-62fc-d763-0e9d-b79131eeba19@redhat.com> <9ff8320d-8750-1ce0-9130-207e4d49c617@redhat.com> Message-ID: <52d5756d-eb04-1dc1-45ca-d669045d6370@redhat.com> On 5/17/19 4:17 PM, Roman Kennke wrote: >>> After switching to strong to-space invariant and pre-evacuating all root, thread roots are only >>> places that may contain to-space oops after updating references during normal concurrent cycles, we >>> only need to update them during final updaterefs pause, which should reduce pause time. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8223951 >>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8223951/webrev.00/ >> >> Looks fine to me. >> >> Aside: I wonder how would be get the thread root with from-space ptr if we don't have safepoints >> between loads and the LRBs that give us to-space ptrs after potential evac. > > I don't think we cann guarantee that we don't get a safepoint between > load and LRB. Ah, that sucks. Oh well! But that is the only plausible way we can get the non-updated thread root, right? -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From stuart.monteith at linaro.org Fri May 17 14:28:58 2019 From: stuart.monteith at linaro.org (Stuart Monteith) Date: Fri, 17 May 2019 15:28:58 +0100 Subject: RFR: JDK-8214527 AArch64: ZGC for Aarch64 Message-ID: Hello, This patch implements ZGC for AArch64. Unlike the previous set of patches, this duplicates the x86 multimapping scheme rather than using the Aarch64 TBI feature, where the top 8 bits are ignored by the hardware. I didn't measure any conclusive performance problems using multimapping. It would necessitate changes to SA, JNI, and wasn't going to be compatible with Graal or JVMCI in general until they were also able to switch to 64-bit literals dynamically. In addition, there are up and coming features in the AArch64 architecture that exploit the TBI area that we should keep open, such as MTE (Memory Tagging Extensions) that open the door to lightweight hardware ASAN. With MTE using 4 of the 8 bits, and ZGC and MTE perhaps needing more than 4 bits, future conflict looks likely. JTreg testing hasn't demonstrated any significant differences from x86 ZGC in my testing. The behaviour should be the same as is expected from x86_64, including for the behaviour with command line arguments. Currently ZGC requires -XX:+UnlockExperimentalVMOptions, that should be the case for AArch64 this release. If x86_64 were to do have ZGC designated as not experimental, we must ensure it still applies for ZGC on AArch64. The associated bug is: https://bugs.openjdk.java.net/browse/JDK-8214527 The patchset is: http://cr.openjdk.java.net/~smonteith/8214527/webrev.0/ There is a patch contributed by Roland Westerlin required for fixing memory barriers, http://cr.openjdk.java.net/~smonteith/zgc-mm/20190430/membar/ I'll follow up with a separate RFR patch to address that. Thanks, Stuart From shade at redhat.com Fri May 17 14:41:07 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 17 May 2019 16:41:07 +0200 Subject: RFR: JDK-8214527 AArch64: ZGC for Aarch64 In-Reply-To: References: Message-ID: <5fbb0b46-e644-490f-ecef-d5b219fc5128@redhat.com> On 5/17/19 4:28 PM, Stuart Monteith wrote: > The patchset is: > http://cr.openjdk.java.net/~smonteith/8214527/webrev.0/ I think that the extension of arraycopy_prologue interface is better to be handled and tested separately, to avoid surprises. GC interface should give us clean specific-GC-only patches for specific-GC features. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From stuart.monteith at linaro.org Fri May 17 14:55:37 2019 From: stuart.monteith at linaro.org (Stuart Monteith) Date: Fri, 17 May 2019 15:55:37 +0100 Subject: RFR: JDK-8214527 AArch64: ZGC for Aarch64 In-Reply-To: <5fbb0b46-e644-490f-ecef-d5b219fc5128@redhat.com> References: <5fbb0b46-e644-490f-ecef-d5b219fc5128@redhat.com> Message-ID: Thanks for looking Aleksey. I'll generate a separate patch. Are you happy that I deliberately left arraycopy_epilogue as it was? On Fri, 17 May 2019 at 15:41, Aleksey Shipilev wrote: > > On 5/17/19 4:28 PM, Stuart Monteith wrote: > > The patchset is: > > http://cr.openjdk.java.net/~smonteith/8214527/webrev.0/ > > I think that the extension of arraycopy_prologue interface is better to be handled and tested > separately, to avoid surprises. GC interface should give us clean specific-GC-only patches for > specific-GC features. > > -Aleksey > > From rkennke at redhat.com Fri May 17 14:55:49 2019 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 17 May 2019 16:55:49 +0200 Subject: RFR(S) 8223951: Shenandoah: Only need to update thread roots during final update refs In-Reply-To: <52d5756d-eb04-1dc1-45ca-d669045d6370@redhat.com> References: <52773c59-62fc-d763-0e9d-b79131eeba19@redhat.com> <9ff8320d-8750-1ce0-9130-207e4d49c617@redhat.com> <52d5756d-eb04-1dc1-45ca-d669045d6370@redhat.com> Message-ID: <9c44bd43-92a0-d81b-ea4d-6fca1ee91727@redhat.com> >>>> After switching to strong to-space invariant and pre-evacuating all root, thread roots are only >>>> places that may contain to-space oops after updating references during normal concurrent cycles, we >>>> only need to update them during final updaterefs pause, which should reduce pause time. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8223951 >>>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8223951/webrev.00/ >>> >>> Looks fine to me. >>> >>> Aside: I wonder how would be get the thread root with from-space ptr if we don't have safepoints >>> between loads and the LRBs that give us to-space ptrs after potential evac. >> >> I don't think we cann guarantee that we don't get a safepoint between >> load and LRB. > > Ah, that sucks. Oh well! Positively speaking: it allows C2 more freedom and move barriers across safepoints :-) OTOH, with LRB it probably doesn't matter that much. We may actually attempt to prevent it and save this extra pass. It may not actually be very difficult: wire up current ctrl in ShLoadBarrierNode instead of NULL. Knowing C2 I can see all hell break loose though... ;-) > But that is the only plausible way we can get the non-updated thread root, > right? Well, yeah. I think/hope so ;-) Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Fri May 17 14:58:30 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 17 May 2019 16:58:30 +0200 Subject: RFR: JDK-8214527 AArch64: ZGC for Aarch64 In-Reply-To: References: <5fbb0b46-e644-490f-ecef-d5b219fc5128@redhat.com> Message-ID: <73f3be77-a177-4f8d-3fa9-98de57580e4c@redhat.com> On 5/17/19 4:55 PM, Stuart Monteith wrote: > Are you happy that I deliberately left arraycopy_epilogue as it was? I have no problem with that. GC interface can evolve incrementally :) -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Fri May 17 15:05:42 2019 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 17 May 2019 17:05:42 +0200 Subject: RFR: JDK-8214527 AArch64: ZGC for Aarch64 In-Reply-To: References: <5fbb0b46-e644-490f-ecef-d5b219fc5128@redhat.com> Message-ID: That's actually going in the same direction of what I already proposed: https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2019-May/025693.html We may want to pick up that discussion. Last thing I heard was that Erik suggested to broaden the proposed API changes to let the GC own the whole arraycopy stub generation. I.e. instead of: GC-interface-prologue actual arraycopy (without barriers) GC-interface-epilogue make one call to: GC-interface-arraycopy and let that generate prologue, copy and epilogue, or something totally different. The variant that I proposed was to keep the current structure, but allow the prologue to skip the body: GC-interface-prologue (possibly jump to exit) actual arraycopy (without barriers) GC-interface-epilogue exit: ... I didn't have time to follow-up on that yet, but I definitely want to. WDYT? Roman > Thanks for looking Aleksey. I'll generate a separate patch. > > Are you happy that I deliberately left arraycopy_epilogue as it was? > > On Fri, 17 May 2019 at 15:41, Aleksey Shipilev wrote: >> >> On 5/17/19 4:28 PM, Stuart Monteith wrote: >>> The patchset is: >>> http://cr.openjdk.java.net/~smonteith/8214527/webrev.0/ >> >> I think that the extension of arraycopy_prologue interface is better to be handled and tested >> separately, to avoid surprises. GC interface should give us clean specific-GC-only patches for >> specific-GC features. >> >> -Aleksey >> >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From erik.osterlund at oracle.com Fri May 17 15:42:18 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 17 May 2019 17:42:18 +0200 Subject: RFR: JDK-8214527 AArch64: ZGC for Aarch64 In-Reply-To: References: <5fbb0b46-e644-490f-ecef-d5b219fc5128@redhat.com> Message-ID: Hi, I still do believe it's better to let the GC override both the barriers and the accesses. In fact this is the way we consistently deal with all access calls, except arraycopy: the GC sprinkles barriers as they wish and calls the basic access in the appropriate places. By doing this for arraycopy as well, the structure will be more clear, and it will be more flexible for the GC. Not just in theory, but in a way that is actually useful in reality as well. Rickard has for example played around with materializing wide bad masks, and bulk applying the load barrier in the copying loop, which showed great benefit. But it fell short exactly because we did not have the right interface in place to allow that. Thanks, /Erik On 2019-05-17 17:05, Roman Kennke wrote: > That's actually going in the same direction of what I already proposed: > > https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2019-May/025693.html > > We may want to pick up that discussion. Last thing I heard was that Erik > suggested to broaden the proposed API changes to let the GC own the > whole arraycopy stub generation. I.e. instead of: > > GC-interface-prologue > actual arraycopy (without barriers) > GC-interface-epilogue > > make one call to: > > GC-interface-arraycopy > > and let that generate prologue, copy and epilogue, or something totally > different. > > The variant that I proposed was to keep the current structure, but allow > the prologue to skip the body: > > GC-interface-prologue (possibly jump to exit) > actual arraycopy (without barriers) > GC-interface-epilogue > exit: ... > > I didn't have time to follow-up on that yet, but I definitely want to. > > WDYT? > > Roman > >> Thanks for looking Aleksey. I'll generate a separate patch. >> >> Are you happy that I deliberately left arraycopy_epilogue as it was? >> >> On Fri, 17 May 2019 at 15:41, Aleksey Shipilev wrote: >>> On 5/17/19 4:28 PM, Stuart Monteith wrote: >>>> The patchset is: >>>> http://cr.openjdk.java.net/~smonteith/8214527/webrev.0/ >>> I think that the extension of arraycopy_prologue interface is better to be handled and tested >>> separately, to avoid surprises. GC interface should give us clean specific-GC-only patches for >>> specific-GC features. >>> >>> -Aleksey >>> >>> From rkennke at redhat.com Fri May 17 15:52:11 2019 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 17 May 2019 17:52:11 +0200 Subject: RFR: JDK-8214527 AArch64: ZGC for Aarch64 In-Reply-To: References: <5fbb0b46-e644-490f-ecef-d5b219fc5128@redhat.com> Message-ID: Hi Erik That is fine by me. We have 3 cases lined up now that require a more flexible interface for arraycopy, and what you propose sounds reasonable. But it requires work, and right now I'm terribly side-tracked. So this has to wait a little bit from my side. If anybody is free to pick it up, feel free. I suggest to just use the existing issue to cover it: https://bugs.openjdk.java.net/browse/JDK-8223240 Roman > I still do believe it's better to let the GC override both the barriers > and the accesses. In fact this is the way we consistently deal with all > access calls, except arraycopy: the GC sprinkles barriers as they wish > and calls the basic access in the appropriate places. > By doing this for arraycopy as well, the structure will be more clear, > and it will be more flexible for the GC. Not just in theory, but in a > way that is actually useful in reality as well. Rickard has for example > played around with materializing wide bad masks, and bulk applying the > load barrier in the copying loop, which showed great benefit. But it > fell short exactly because we did not have the right interface in place > to allow that. > > Thanks, > /Erik > > On 2019-05-17 17:05, Roman Kennke wrote: >> That's actually going in the same direction of what I already proposed: >> >> https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2019-May/025693.html >> >> >> We may want to pick up that discussion. Last thing I heard was that Erik >> suggested to broaden the proposed API changes to let the GC own the >> whole arraycopy stub generation. I.e. instead of: >> >> GC-interface-prologue >> actual arraycopy (without barriers) >> GC-interface-epilogue >> >> make one call to: >> >> GC-interface-arraycopy >> >> and let that generate prologue, copy and epilogue, or something totally >> different. >> >> The variant that I proposed was to keep the current structure, but allow >> the prologue to skip the body: >> >> GC-interface-prologue (possibly jump to exit) >> actual arraycopy (without barriers) >> GC-interface-epilogue >> exit: ... >> >> I didn't have time to follow-up on that yet, but I definitely want to. >> >> WDYT? >> >> Roman >> >>> Thanks for looking Aleksey.? I'll generate a separate patch. >>> >>> Are you happy that I deliberately left arraycopy_epilogue as it was? >>> >>> On Fri, 17 May 2019 at 15:41, Aleksey Shipilev wrote: >>>> On 5/17/19 4:28 PM, Stuart Monteith wrote: >>>>> The patchset is: >>>>> ?????? http://cr.openjdk.java.net/~smonteith/8214527/webrev.0/ >>>> I think that the extension of arraycopy_prologue interface is better >>>> to be handled and tested >>>> separately, to avoid surprises. GC interface should give us clean >>>> specific-GC-only patches for >>>> specific-GC features. >>>> >>>> -Aleksey >>>> >>>> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Fri May 17 15:57:47 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Fri, 17 May 2019 11:57:47 -0400 Subject: RFR(M) 8224115: Shenandoah: Eliminate RWLock that protects recorded nmethod data array Message-ID: <3f736be4-bf6f-1559-f8fe-66902ce97c9f@redhat.com> I would like to eliminate RWLock that used to protect recorded nmethod data array in ShenandoahCodeRoots, because the methods that update the array, are already guarded by CodeCache_lock. The patch also helps concurrent class unloading work, by avoiding two lock dances, which are deadlock prone. Bug: https://bugs.openjdk.java.net/browse/JDK-8224115 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224115/webrev.00/ Test: hotspot_gc_shenandoah (fastdebug and release) Thanks, -Zhengyu From sangheon.kim at oracle.com Fri May 17 19:35:50 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Fri, 17 May 2019 12:35:50 -0700 Subject: RFR (S/M): 8222492: G1 unnecessarily scans remembered set cards for regions that already have been evacuated In-Reply-To: References: <3e061b852275110f67a309f66211dd5832f0e5b4.camel@oracle.com> <5D369410-424A-4550-9B02-B09725C502B8@oracle.com> Message-ID: <86c0be76-db38-299f-44f2-83a8929c5135@oracle.com> Hi Thomas, On 5/17/19 3:31 AM, Thomas Schatzl wrote: > Hi Kim, > > thanks for your review. > > On Thu, 2019-05-16 at 18:45 -0400, Kim Barrett wrote: >> On Apr 26, 2019, at 8:27 AM, Thomas Schatzl < >> thomas.schatzl at oracle.com> wrote: >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8222492 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8222492/webrev/ >>> Testing: >>> hs-tier1-5, running the crashing test of JDK-8222426 for 5k times >>> without failures (the crash has an occurrence of 1-2 times/1000 >>> runs). >>> >>> Thanks, >>> Thomas >>> >> Looks good. >> >> One minor nit: >> >> src/hotspot/share/gc/g1/g1OopClosures.inline.hpp >> 190 // Entries in the optional collection set may become to be >> located in the collection >> 191 // set during incremental collection of the old generation. >> >> That comment needs some fixing. >> > I changed the comment to the following and refreshed the webrev: > > // Entries in the optional collection set may start to originate from > the collection > // set after one or more increments. In this case, previously > optional regions > // became actual collection set regions. Filter them out here. Looks good to me too. Thanks, Sangheon > > Thanks, > Thomas > > From thomas.schatzl at oracle.com Fri May 17 19:57:25 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 17 May 2019 21:57:25 +0200 Subject: RFR (S/M): 8222492: G1 unnecessarily scans remembered set cards for regions that already have been evacuated In-Reply-To: <86c0be76-db38-299f-44f2-83a8929c5135@oracle.com> References: <3e061b852275110f67a309f66211dd5832f0e5b4.camel@oracle.com> <5D369410-424A-4550-9B02-B09725C502B8@oracle.com> <86c0be76-db38-299f-44f2-83a8929c5135@oracle.com> Message-ID: <171f8e3ecb3f79662309b44f7f06339fe0da7be8.camel@oracle.com> Hi Sangheon, On Fri, 2019-05-17 at 12:35 -0700, sangheon.kim at oracle.com wrote: > Hi Thomas, > > On 5/17/19 3:31 AM, Thomas Schatzl wrote: > > Hi Kim, > > > > thanks for your review. > > > > On Thu, 2019-05-16 at 18:45 -0400, Kim Barrett wrote: > > > On Apr 26, 2019, at 8:27 AM, Thomas Schatzl < > > > thomas.schatzl at oracle.com> wrote: > > > > CR: > > > > https://bugs.openjdk.java.net/browse/JDK-8222492 > > > > Webrev: > > > > http://cr.openjdk.java.net/~tschatzl/8222492/webrev/ > > > > Testing: > > > > hs-tier1-5, running the crashing test of JDK-8222426 for 5k > > > > times without failures (the crash has an occurrence of 1-2 > > > > times/1000 runs). > > > > > > > > Thanks, > > > > Thomas > > > > > > > [...] > Looks good to me too. Thanks for your review. Thomas From zgu at redhat.com Mon May 20 00:59:59 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Sun, 19 May 2019 20:59:59 -0400 Subject: RFR 8224179: Shenandoah: CTW test failures with traversal GC Message-ID: <0a13c972-9a15-917a-ef0a-7cfe351ea15b@redhat.com> The failures were due to nested nmethod::oops_do_marking_prologue()/nmethod::oops_do_marking_epilogue() calls. This patch is split from early JDK-8224115. Bug: https://bugs.openjdk.java.net/browse/JDK-8224179 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224179/webrev.00/ Test: hotspot_gc_shenandoah (fastdebug and release) applications/ctw/modules/ fastdebug. Thanks, -Zhengyu From shade at redhat.com Mon May 20 07:35:57 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 20 May 2019 09:35:57 +0200 Subject: RFR 8224179: Shenandoah: CTW test failures with traversal GC In-Reply-To: <0a13c972-9a15-917a-ef0a-7cfe351ea15b@redhat.com> References: <0a13c972-9a15-917a-ef0a-7cfe351ea15b@redhat.com> Message-ID: <5f521f87-28f1-811b-3aad-9dff2457d9bb@redhat.com> On 5/20/19 2:59 AM, Zhengyu Gu wrote: > The failures were due to nested > nmethod::oops_do_marking_prologue()/nmethod::oops_do_marking_epilogue() calls. This patch is split > from early JDK-8224115. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8224179 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224179/webrev.00/ Looks okay. The synopsis for the change does not reflect what is going on: it should mention what the change does, i.e. "Introduce all- and cset-specialized ShenandoahRootScanner"? The patch does lots of code moves. Are those moves necessary to fix the failure? Ideally, the refactoring and the functional change should go separately. In other words, the fix should make it absolutely clear what is done, not obscure most of it with reshuffling the code. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Mon May 20 08:11:44 2019 From: per.liden at oracle.com (Per Liden) Date: Mon, 20 May 2019 10:11:44 +0200 Subject: RFR: 8224185: ZGC: Introduce "High Usage" rule Message-ID: <0e87a5ac-3a38-4301-5762-58d0d7fa1d94@oracle.com> Start a GC cycle if the amount of free memory is 5% or less. This is a preventive measure in the case where the application has a very low allocation rate, such that the allocation rate rule doesn't trigger, but the amount of free memory is still slowly but surely heading towards zero. In this situation, we should start a GC cycle to avoid a potential allocation stall later. Testing: New jtreg test added. Currently running tier 1-6. Bug: https://bugs.openjdk.java.net/browse/JDK-8224185 Webrev: http://cr.openjdk.java.net/~pliden/8224185/webrev.0 /Per From stuart.monteith at linaro.org Mon May 20 09:17:34 2019 From: stuart.monteith at linaro.org (Stuart Monteith) Date: Mon, 20 May 2019 10:17:34 +0100 Subject: RFR: JDK-8214527 AArch64: ZGC for Aarch64 In-Reply-To: References: <5fbb0b46-e644-490f-ecef-d5b219fc5128@redhat.com> Message-ID: Hello Roman, I'm not sure what the time frame for JDK-822340 , but I'd like ZGC for aarch64 to be in JDK 13. I've opened JDK-8224187 with just the changes to the array_prologue signature, as I'll at least need that for ZGC. BR, Stuart On Fri, 17 May 2019 at 16:52, Roman Kennke wrote: > > Hi Erik > > That is fine by me. We have 3 cases lined up now that require a more > flexible interface for arraycopy, and what you propose sounds > reasonable. But it requires work, and right now I'm terribly > side-tracked. So this has to wait a little bit from my side. If anybody > is free to pick it up, feel free. I suggest to just use the existing > issue to cover it: > > https://bugs.openjdk.java.net/browse/JDK-8223240 > > Roman > > > I still do believe it's better to let the GC override both the barriers > > and the accesses. In fact this is the way we consistently deal with all > > access calls, except arraycopy: the GC sprinkles barriers as they wish > > and calls the basic access in the appropriate places. > > By doing this for arraycopy as well, the structure will be more clear, > > and it will be more flexible for the GC. Not just in theory, but in a > > way that is actually useful in reality as well. Rickard has for example > > played around with materializing wide bad masks, and bulk applying the > > load barrier in the copying loop, which showed great benefit. But it > > fell short exactly because we did not have the right interface in place > > to allow that. > > > > Thanks, > > /Erik > > > > On 2019-05-17 17:05, Roman Kennke wrote: > >> That's actually going in the same direction of what I already proposed: > >> > >> https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2019-May/025693.html > >> > >> > >> We may want to pick up that discussion. Last thing I heard was that Erik > >> suggested to broaden the proposed API changes to let the GC own the > >> whole arraycopy stub generation. I.e. instead of: > >> > >> GC-interface-prologue > >> actual arraycopy (without barriers) > >> GC-interface-epilogue > >> > >> make one call to: > >> > >> GC-interface-arraycopy > >> > >> and let that generate prologue, copy and epilogue, or something totally > >> different. > >> > >> The variant that I proposed was to keep the current structure, but allow > >> the prologue to skip the body: > >> > >> GC-interface-prologue (possibly jump to exit) > >> actual arraycopy (without barriers) > >> GC-interface-epilogue > >> exit: ... > >> > >> I didn't have time to follow-up on that yet, but I definitely want to. > >> > >> WDYT? > >> > >> Roman > >> > >>> Thanks for looking Aleksey. I'll generate a separate patch. > >>> > >>> Are you happy that I deliberately left arraycopy_epilogue as it was? > >>> > >>> On Fri, 17 May 2019 at 15:41, Aleksey Shipilev wrote: > >>>> On 5/17/19 4:28 PM, Stuart Monteith wrote: > >>>>> The patchset is: > >>>>> http://cr.openjdk.java.net/~smonteith/8214527/webrev.0/ > >>>> I think that the extension of arraycopy_prologue interface is better > >>>> to be handled and tested > >>>> separately, to avoid surprises. GC interface should give us clean > >>>> specific-GC-only patches for > >>>> specific-GC features. > >>>> > >>>> -Aleksey > >>>> > >>>> > > > From erik.osterlund at oracle.com Mon May 20 09:27:07 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 20 May 2019 11:27:07 +0200 Subject: RFR: JDK-8214527 AArch64: ZGC for Aarch64 In-Reply-To: References: <5fbb0b46-e644-490f-ecef-d5b219fc5128@redhat.com> Message-ID: <0a67463c-99f1-cbdb-e6ed-ff106a80d651@oracle.com> Hi Stuart, I am okay with just changing the signature of the prologue to have both src and dst for now. This is after all what x86, SPARC, PPC and S390 all do. So making that small change unifies the way we do it today on the various different platforms. I can volunteer to improve the interface to be even more flexible as discussed, in a separate follow-up RFE. But I don't see any reason why you would have to wait for that. Thanks, /Erik On 2019-05-20 11:17, Stuart Monteith wrote: > Hello Roman, > I'm not sure what the time frame for JDK-822340 , but I'd like ZGC > for aarch64 to be in JDK 13. I've opened JDK-8224187 with just the > changes to the array_prologue signature, as I'll at least need that > for ZGC. > > BR, > Stuart > > On Fri, 17 May 2019 at 16:52, Roman Kennke wrote: >> Hi Erik >> >> That is fine by me. We have 3 cases lined up now that require a more >> flexible interface for arraycopy, and what you propose sounds >> reasonable. But it requires work, and right now I'm terribly >> side-tracked. So this has to wait a little bit from my side. If anybody >> is free to pick it up, feel free. I suggest to just use the existing >> issue to cover it: >> >> https://bugs.openjdk.java.net/browse/JDK-8223240 >> >> Roman >> >>> I still do believe it's better to let the GC override both the barriers >>> and the accesses. In fact this is the way we consistently deal with all >>> access calls, except arraycopy: the GC sprinkles barriers as they wish >>> and calls the basic access in the appropriate places. >>> By doing this for arraycopy as well, the structure will be more clear, >>> and it will be more flexible for the GC. Not just in theory, but in a >>> way that is actually useful in reality as well. Rickard has for example >>> played around with materializing wide bad masks, and bulk applying the >>> load barrier in the copying loop, which showed great benefit. But it >>> fell short exactly because we did not have the right interface in place >>> to allow that. >>> >>> Thanks, >>> /Erik >>> >>> On 2019-05-17 17:05, Roman Kennke wrote: >>>> That's actually going in the same direction of what I already proposed: >>>> >>>> https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2019-May/025693.html >>>> >>>> >>>> We may want to pick up that discussion. Last thing I heard was that Erik >>>> suggested to broaden the proposed API changes to let the GC own the >>>> whole arraycopy stub generation. I.e. instead of: >>>> >>>> GC-interface-prologue >>>> actual arraycopy (without barriers) >>>> GC-interface-epilogue >>>> >>>> make one call to: >>>> >>>> GC-interface-arraycopy >>>> >>>> and let that generate prologue, copy and epilogue, or something totally >>>> different. >>>> >>>> The variant that I proposed was to keep the current structure, but allow >>>> the prologue to skip the body: >>>> >>>> GC-interface-prologue (possibly jump to exit) >>>> actual arraycopy (without barriers) >>>> GC-interface-epilogue >>>> exit: ... >>>> >>>> I didn't have time to follow-up on that yet, but I definitely want to. >>>> >>>> WDYT? >>>> >>>> Roman >>>> >>>>> Thanks for looking Aleksey. I'll generate a separate patch. >>>>> >>>>> Are you happy that I deliberately left arraycopy_epilogue as it was? >>>>> >>>>> On Fri, 17 May 2019 at 15:41, Aleksey Shipilev wrote: >>>>>> On 5/17/19 4:28 PM, Stuart Monteith wrote: >>>>>>> The patchset is: >>>>>>> http://cr.openjdk.java.net/~smonteith/8214527/webrev.0/ >>>>>> I think that the extension of arraycopy_prologue interface is better >>>>>> to be handled and tested >>>>>> separately, to avoid surprises. GC interface should give us clean >>>>>> specific-GC-only patches for >>>>>> specific-GC features. >>>>>> >>>>>> -Aleksey >>>>>> >>>>>> From thomas.schatzl at oracle.com Mon May 20 09:34:12 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 20 May 2019 11:34:12 +0200 Subject: RFR: 8224185: ZGC: Introduce "High Usage" rule In-Reply-To: <0e87a5ac-3a38-4301-5762-58d0d7fa1d94@oracle.com> References: <0e87a5ac-3a38-4301-5762-58d0d7fa1d94@oracle.com> Message-ID: <06c6cff49119376145dafc56bc2e493ee726b333.camel@oracle.com> Hi Per, On Mon, 2019-05-20 at 10:11 +0200, Per Liden wrote: > Start a GC cycle if the amount of free memory is 5% or less. This is > a preventive measure in the case where the application has a very > low allocation rate, such that the allocation rate rule doesn't > trigger, but the amount of free memory is still slowly but surely > heading towards zero. In this situation, we should start a GC cycle > to avoid a potential allocation stall later. > > Testing: New jtreg test added. Currently running tier 1-6. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8224185 > Webrev: http://cr.openjdk.java.net/~pliden/8224185/webrev.0 > just pointing out the obvious: this means that ZGC will potentially never use 5-Epsilon% of the heap. I do not recommend such a threshold being a fixed, compiled in constant size: while I understand that you want to have as few options as possible with ZGC, 5% of 100GB are still 5GB which may be better used for use in the application in some cases. I.e. G1 has a similar (10%) threshold, and we've run in a few situations where this has been too much and unneeded, particularly with large heaps. Now I know that ZGC pauses are by far less intrusive than G1's, but still the concurrent marking has its cost. Making it configurable would allow people getting the last bit of performance out of it if they want. Thanks, Thomas From per.liden at oracle.com Mon May 20 09:44:53 2019 From: per.liden at oracle.com (Per Liden) Date: Mon, 20 May 2019 11:44:53 +0200 Subject: RFR: 8224185: ZGC: Introduce "High Usage" rule In-Reply-To: <06c6cff49119376145dafc56bc2e493ee726b333.camel@oracle.com> References: <0e87a5ac-3a38-4301-5762-58d0d7fa1d94@oracle.com> <06c6cff49119376145dafc56bc2e493ee726b333.camel@oracle.com> Message-ID: Hi Thomas, On 5/20/19 11:34 AM, Thomas Schatzl wrote: > Hi Per, > > On Mon, 2019-05-20 at 10:11 +0200, Per Liden wrote: >> Start a GC cycle if the amount of free memory is 5% or less. This is >> a preventive measure in the case where the application has a very >> low allocation rate, such that the allocation rate rule doesn't >> trigger, but the amount of free memory is still slowly but surely >> heading towards zero. In this situation, we should start a GC cycle >> to avoid a potential allocation stall later. >> >> Testing: New jtreg test added. Currently running tier 1-6. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8224185 >> Webrev: http://cr.openjdk.java.net/~pliden/8224185/webrev.0 >> > > just pointing out the obvious: this means that ZGC will potentially > never use 5-Epsilon% of the heap. > > I do not recommend such a threshold being a fixed, compiled in constant > size: while I understand that you want to have as few options as > possible with ZGC, 5% of 100GB are still 5GB which may be better used > for use in the application in some cases. > > I.e. G1 has a similar (10%) threshold, and we've run in a few > situations where this has been too much and unneeded, particularly with > large heaps. > Now I know that ZGC pauses are by far less intrusive than G1's, but > still the concurrent marking has its cost. Making it configurable would > allow people getting the last bit of performance out of it if they > want. The situation is slightly different for ZGC. This rule typically kicks in when the allocation rate is very low. If the allocation rate is very high, we typically collect a lot earlier than 95% full heap anyway. Regarding a flag. We've discussed this within the team, and decided to not have one for now. It can be added later if it turns out to be a need for it. cheers, Per From thomas.schatzl at oracle.com Mon May 20 09:54:38 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 20 May 2019 11:54:38 +0200 Subject: RFR: 8224185: ZGC: Introduce "High Usage" rule In-Reply-To: References: <0e87a5ac-3a38-4301-5762-58d0d7fa1d94@oracle.com> <06c6cff49119376145dafc56bc2e493ee726b333.camel@oracle.com> Message-ID: <28c1d8326dfed46507e2752a5acae955cfcfd663.camel@oracle.com> Hi Per, On Mon, 2019-05-20 at 11:44 +0200, Per Liden wrote: > Hi Thomas, > > On 5/20/19 11:34 AM, Thomas Schatzl wrote: > > Hi Per, > > > > On Mon, 2019-05-20 at 10:11 +0200, Per Liden wrote: > > > Start a GC cycle if the amount of free memory is 5% or less. This > > > is a preventive measure in the case where the application has a > > > very low allocation rate, such that the allocation rate rule > > > doesn't trigger, but the amount of free memory is still slowly > > > but surely heading towards zero. In this situation, we should > > > start a GC cycle to avoid a potential allocation stall later. > > > > > > Testing: New jtreg test added. Currently running tier 1-6. > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8224185 > > > Webrev: http://cr.openjdk.java.net/~pliden/8224185/webrev.0 > > > > > [...] > > > > I.e. G1 has a similar (10%) threshold, and we've run in a few > > situations where this has been too much and unneeded, particularly > > with large heaps. > > Now I know that ZGC pauses are by far less intrusive than G1's, but > > still the concurrent marking has its cost. Making it configurable > > would allow people getting the last bit of performance out of it if > > they want. > > The situation is slightly different for ZGC. This rule typically > kicks in when the allocation rate is very low. If the allocation rate > is very high, we typically collect a lot earlier than 95% full heap > anyway. This is the same situation G1 uses G1ReservePercent for determining start of concurrent cycle (it won't poll it in the background though, so it's a little different) in such applications. G1ReservePercent's current description does not spell out the impact as nicely as this threshold does, but it automatically follows from that. > Regarding a flag. We've discussed this within the team, and decided > to not have one for now. It can be added later if it turns out to be > a need for it. That's fine, I was merely pointing out this potential caveat for introducing such a threshold. Thanks, Thomas From erik.osterlund at oracle.com Mon May 20 09:59:03 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 20 May 2019 11:59:03 +0200 Subject: RFR: 8224185: ZGC: Introduce "High Usage" rule In-Reply-To: References: <0e87a5ac-3a38-4301-5762-58d0d7fa1d94@oracle.com> <06c6cff49119376145dafc56bc2e493ee726b333.camel@oracle.com> Message-ID: <5790ee54-17e8-9699-c288-5c0c6184e845@oracle.com> Hi Per and Thomas, Looks good to me. I also would not like to see a JVM flag for this. We have a quite high bar before we introduce new JVM flags, to avoid JVM flag hell. Especially for policies that are likely to change as the algorithm evolves, such that the flag might become useless. And this did not seem to warrant a flag yet. Thanks, /Erik On 2019-05-20 11:44, Per Liden wrote: > Hi Thomas, > > On 5/20/19 11:34 AM, Thomas Schatzl wrote: >> Hi Per, >> >> On Mon, 2019-05-20 at 10:11 +0200, Per Liden wrote: >>> Start a GC cycle if the amount of free memory is 5% or less. This is >>> a preventive measure in the case where the application has a very >>> low allocation rate, such that the allocation rate rule doesn't >>> trigger, but the amount of free memory is still slowly but surely >>> heading towards zero. In this situation, we should start a GC cycle >>> to avoid a potential allocation stall later. >>> >>> Testing: New jtreg test added. Currently running tier 1-6. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8224185 >>> Webrev: http://cr.openjdk.java.net/~pliden/8224185/webrev.0 >>> >> >> ?? just pointing out the obvious: this means that ZGC will potentially >> never use 5-Epsilon% of the heap. >> >> I do not recommend such a threshold being a fixed, compiled in constant >> size: while I understand that you want to have as few options as >> possible with ZGC, 5% of 100GB are still 5GB which may be better used >> for use in the application in some cases. >> >> I.e. G1 has a similar (10%) threshold, and we've run in a few >> situations where this has been too much and unneeded, particularly with >> large heaps. >> Now I know that ZGC pauses are by far less intrusive than G1's, but >> still the concurrent marking has its cost. Making it configurable would >> allow people getting the last bit of performance out of it if they >> want. > > The situation is slightly different for ZGC. This rule typically kicks > in when the allocation rate is very low. If the allocation rate is > very high, we typically collect a lot earlier than 95% full heap anyway. > > Regarding a flag. We've discussed this within the team, and decided to > not have one for now. It can be added later if it turns out to be a > need for it. > > cheers, > Per From stuart.monteith at linaro.org Mon May 20 10:23:08 2019 From: stuart.monteith at linaro.org (Stuart Monteith) Date: Mon, 20 May 2019 11:23:08 +0100 Subject: RFR: JDK-8214527 AArch64: ZGC for Aarch64 In-Reply-To: <0a67463c-99f1-cbdb-e6ed-ff106a80d651@oracle.com> References: <5fbb0b46-e644-490f-ecef-d5b219fc5128@redhat.com> <0a67463c-99f1-cbdb-e6ed-ff106a80d651@oracle.com> Message-ID: Thanks Erik. I've uploaded: http://cr.openjdk.java.net/~smonteith/8224187/webrev.0/ for: https://bugs.openjdk.java.net/browse/JDK-8224187 I'll send out a separate RFR for that. I've updated the ZGC patch to rely on 8224187: http://cr.openjdk.java.net/~smonteith/8214527/webrev.1/ BR, Stuart On Mon, 20 May 2019 at 10:26, Erik ?sterlund wrote: > > Hi Stuart, > > I am okay with just changing the signature of the prologue to have both > src and dst for now. This is after all what x86, SPARC, PPC and S390 all do. > So making that small change unifies the way we do it today on the > various different platforms. > > I can volunteer to improve the interface to be even more flexible as > discussed, in a separate follow-up RFE. > But I don't see any reason why you would have to wait for that. > > Thanks, > /Erik > > On 2019-05-20 11:17, Stuart Monteith wrote: > > Hello Roman, > > I'm not sure what the time frame for JDK-822340 , but I'd like ZGC > > for aarch64 to be in JDK 13. I've opened JDK-8224187 with just the > > changes to the array_prologue signature, as I'll at least need that > > for ZGC. > > > > BR, > > Stuart > > > > On Fri, 17 May 2019 at 16:52, Roman Kennke wrote: > >> Hi Erik > >> > >> That is fine by me. We have 3 cases lined up now that require a more > >> flexible interface for arraycopy, and what you propose sounds > >> reasonable. But it requires work, and right now I'm terribly > >> side-tracked. So this has to wait a little bit from my side. If anybody > >> is free to pick it up, feel free. I suggest to just use the existing > >> issue to cover it: > >> > >> https://bugs.openjdk.java.net/browse/JDK-8223240 > >> > >> Roman > >> > >>> I still do believe it's better to let the GC override both the barriers > >>> and the accesses. In fact this is the way we consistently deal with all > >>> access calls, except arraycopy: the GC sprinkles barriers as they wish > >>> and calls the basic access in the appropriate places. > >>> By doing this for arraycopy as well, the structure will be more clear, > >>> and it will be more flexible for the GC. Not just in theory, but in a > >>> way that is actually useful in reality as well. Rickard has for example > >>> played around with materializing wide bad masks, and bulk applying the > >>> load barrier in the copying loop, which showed great benefit. But it > >>> fell short exactly because we did not have the right interface in place > >>> to allow that. > >>> > >>> Thanks, > >>> /Erik > >>> > >>> On 2019-05-17 17:05, Roman Kennke wrote: > >>>> That's actually going in the same direction of what I already proposed: > >>>> > >>>> https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2019-May/025693.html > >>>> > >>>> > >>>> We may want to pick up that discussion. Last thing I heard was that Erik > >>>> suggested to broaden the proposed API changes to let the GC own the > >>>> whole arraycopy stub generation. I.e. instead of: > >>>> > >>>> GC-interface-prologue > >>>> actual arraycopy (without barriers) > >>>> GC-interface-epilogue > >>>> > >>>> make one call to: > >>>> > >>>> GC-interface-arraycopy > >>>> > >>>> and let that generate prologue, copy and epilogue, or something totally > >>>> different. > >>>> > >>>> The variant that I proposed was to keep the current structure, but allow > >>>> the prologue to skip the body: > >>>> > >>>> GC-interface-prologue (possibly jump to exit) > >>>> actual arraycopy (without barriers) > >>>> GC-interface-epilogue > >>>> exit: ... > >>>> > >>>> I didn't have time to follow-up on that yet, but I definitely want to. > >>>> > >>>> WDYT? > >>>> > >>>> Roman > >>>> > >>>>> Thanks for looking Aleksey. I'll generate a separate patch. > >>>>> > >>>>> Are you happy that I deliberately left arraycopy_epilogue as it was? > >>>>> > >>>>> On Fri, 17 May 2019 at 15:41, Aleksey Shipilev wrote: > >>>>>> On 5/17/19 4:28 PM, Stuart Monteith wrote: > >>>>>>> The patchset is: > >>>>>>> http://cr.openjdk.java.net/~smonteith/8214527/webrev.0/ > >>>>>> I think that the extension of arraycopy_prologue interface is better > >>>>>> to be handled and tested > >>>>>> separately, to avoid surprises. GC interface should give us clean > >>>>>> specific-GC-only patches for > >>>>>> specific-GC features. > >>>>>> > >>>>>> -Aleksey > >>>>>> > >>>>>> > From stuart.monteith at linaro.org Mon May 20 10:27:01 2019 From: stuart.monteith at linaro.org (Stuart Monteith) Date: Mon, 20 May 2019 11:27:01 +0100 Subject: RFR: 8224187 Refactor arraycopy_prologue to allow ZGC read barriers on arraycopy Message-ID: Hello, On aarch64 the signature for barrierSetAssembler::arraycopy_prologue needs to be changed in order accept the source as well as the destination array for ZGC's barriers. The bug: https://bugs.openjdk.java.net/browse/JDK-8224187 and the patch: http://cr.openjdk.java.net/~smonteith/8224187/webrev.0/ BR, Stuart From rkennke at redhat.com Mon May 20 10:40:05 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 20 May 2019 12:40:05 +0200 Subject: RFR: 8224187 Refactor arraycopy_prologue to allow ZGC read barriers on arraycopy In-Reply-To: References: Message-ID: <3ac5ea67-039b-cae8-f50d-5fe289255ae0@redhat.com> I'd rather see Erik's proposal implemented to make the GC generate the whole path to begin with. Otherwise we'd band-aid for ZGC now, and later have to band-aid again for Shenandoah. See discussion for JDK-8223240 and in your aarch64 on ZGC thread. Roman > Hello, > On aarch64 the signature for > barrierSetAssembler::arraycopy_prologue needs to be changed in order > accept the source as well as the destination array for ZGC's barriers. > > The bug: > https://bugs.openjdk.java.net/browse/JDK-8224187 > and the patch: > http://cr.openjdk.java.net/~smonteith/8224187/webrev.0/ > > > BR, > Stuart > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Mon May 20 11:26:47 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 20 May 2019 07:26:47 -0400 Subject: RFR(XS) 8224115: Shenandoah: Eliminate RWLock that protects recorded nmethod data array In-Reply-To: <3f736be4-bf6f-1559-f8fe-66902ce97c9f@redhat.com> References: <3f736be4-bf6f-1559-f8fe-66902ce97c9f@redhat.com> Message-ID: <25e4be93-8496-3c6b-127e-30d2ef9fc4e8@redhat.com> Split the original patch, moved root processing changes that deal with traversal GC problem into JDK-8224179. Updated webrev: http://cr.openjdk.java.net/~zgu/JDK-8224115/webrev.01/ Test: reran hotspot_gc_shenandoah (fastdebug and release) applications/ctw/modules (fastdebug) Thanks, -Zhengyu On 5/17/19 11:57 AM, Zhengyu Gu wrote: > I would like to eliminate RWLock that used to protect recorded nmethod > data array in ShenandoahCodeRoots, because the methods that update the > array, are already guarded by CodeCache_lock. > > The patch also helps concurrent class unloading work, by avoiding two > lock dances, which are deadlock prone. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8224115 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224115/webrev.00/ > > Test: > ? hotspot_gc_shenandoah (fastdebug and release) > > Thanks, > > -Zhengyu From stuart.monteith at linaro.org Mon May 20 12:02:13 2019 From: stuart.monteith at linaro.org (Stuart Monteith) Date: Mon, 20 May 2019 13:02:13 +0100 Subject: RFR: 8224187 Refactor arraycopy_prologue to allow ZGC read barriers on arraycopy In-Reply-To: <3ac5ea67-039b-cae8-f50d-5fe289255ae0@redhat.com> References: <3ac5ea67-039b-cae8-f50d-5fe289255ae0@redhat.com> Message-ID: Hello Roman, I don't think there is any disagreement about Erik's proposal. However, I'd like to understand what the time-frame is for Erik's proposal would be before abandoning this patch. I am aiming for ZGC for aarch64 to get merged before rampdown phase 1 in mid-June in order for it to be in JDK 13 as an experimental option.. I believe this change is minimal, it wouldn't add any additional effort onto Erik's proposal, and if nothing else brings aarch64 more in line with the other platforms. BR, Stuart On Mon, 20 May 2019 at 11:40, Roman Kennke wrote: > > I'd rather see Erik's proposal implemented to make the GC generate the > whole path to begin with. Otherwise we'd band-aid for ZGC now, and later > have to band-aid again for Shenandoah. See discussion for JDK-8223240 > and in your aarch64 on ZGC thread. > > Roman > > > Hello, > > On aarch64 the signature for > > barrierSetAssembler::arraycopy_prologue needs to be changed in order > > accept the source as well as the destination array for ZGC's barriers. > > > > The bug: > > https://bugs.openjdk.java.net/browse/JDK-8224187 > > and the patch: > > http://cr.openjdk.java.net/~smonteith/8224187/webrev.0/ > > > > > > BR, > > Stuart > > > From rkennke at redhat.com Mon May 20 12:03:28 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 20 May 2019 14:03:28 +0200 Subject: RFR: JDK-8214527 AArch64: ZGC for Aarch64 In-Reply-To: <0a67463c-99f1-cbdb-e6ed-ff106a80d651@oracle.com> References: <5fbb0b46-e644-490f-ecef-d5b219fc5128@redhat.com> <0a67463c-99f1-cbdb-e6ed-ff106a80d651@oracle.com> Message-ID: Hi Erik, This is great! To me the question is, would changing the interfaces a little more to also add a 'Register klass, Label& exit' parameters as temporary measure be acceptable too? Like I proposed here: https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2019-May/025693.html and that is, for the same reasons. We would like to make progress on a bunch of dependend issues and get it into jdk13, but can't currently afford a full-out reworking of the corresponding GC interfaces? It also seems like a less-risky measure overall. Roman > Hi Stuart, > > I am okay with just changing the signature of the prologue to have both > src and dst for now. This is after all what x86, SPARC, PPC and S390 all > do. > So making that small change unifies the way we do it today on the > various different platforms. > > I can volunteer to improve the interface to be even more flexible as > discussed, in a separate follow-up RFE. > But I don't see any reason why you would have to wait for that. > > Thanks, > /Erik > > On 2019-05-20 11:17, Stuart Monteith wrote: >> Hello Roman, >> ???? I'm not sure what the time frame for JDK-822340 , but I'd like ZGC >> for aarch64 to be in JDK 13. I've opened JDK-8224187 with just the >> changes to the array_prologue signature, as I'll at least need that >> for ZGC. >> >> BR, >> ??? Stuart >> >> On Fri, 17 May 2019 at 16:52, Roman Kennke wrote: >>> Hi Erik >>> >>> That is fine by me. We have 3 cases lined up now that require a more >>> flexible interface for arraycopy, and what you propose sounds >>> reasonable. But it requires work, and right now I'm terribly >>> side-tracked. So this has to wait a little bit from my side. If anybody >>> is free to pick it up, feel free. I suggest to just use the existing >>> issue to cover it: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8223240 >>> >>> Roman >>> >>>> I still do believe it's better to let the GC override both the barriers >>>> and the accesses. In fact this is the way we consistently deal with all >>>> access calls, except arraycopy: the GC sprinkles barriers as they wish >>>> and calls the basic access in the appropriate places. >>>> By doing this for arraycopy as well, the structure will be more clear, >>>> and it will be more flexible for the GC. Not just in theory, but in a >>>> way that is actually useful in reality as well. Rickard has for example >>>> played around with materializing wide bad masks, and bulk applying the >>>> load barrier in the copying loop, which showed great benefit. But it >>>> fell short exactly because we did not have the right interface in place >>>> to allow that. >>>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2019-05-17 17:05, Roman Kennke wrote: >>>>> That's actually going in the same direction of what I already >>>>> proposed: >>>>> >>>>> https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2019-May/025693.html >>>>> >>>>> >>>>> >>>>> We may want to pick up that discussion. Last thing I heard was that >>>>> Erik >>>>> suggested to broaden the proposed API changes to let the GC own the >>>>> whole arraycopy stub generation. I.e. instead of: >>>>> >>>>> GC-interface-prologue >>>>> actual arraycopy (without barriers) >>>>> GC-interface-epilogue >>>>> >>>>> make one call to: >>>>> >>>>> GC-interface-arraycopy >>>>> >>>>> and let that generate prologue, copy and epilogue, or something >>>>> totally >>>>> different. >>>>> >>>>> The variant that I proposed was to keep the current structure, but >>>>> allow >>>>> the prologue to skip the body: >>>>> >>>>> GC-interface-prologue (possibly jump to exit) >>>>> actual arraycopy (without barriers) >>>>> GC-interface-epilogue >>>>> exit: ... >>>>> >>>>> I didn't have time to follow-up on that yet, but I definitely want to. >>>>> >>>>> WDYT? >>>>> >>>>> Roman >>>>> >>>>>> Thanks for looking Aleksey.? I'll generate a separate patch. >>>>>> >>>>>> Are you happy that I deliberately left arraycopy_epilogue as it was? >>>>>> >>>>>> On Fri, 17 May 2019 at 15:41, Aleksey Shipilev >>>>>> wrote: >>>>>>> On 5/17/19 4:28 PM, Stuart Monteith wrote: >>>>>>>> The patchset is: >>>>>>>> ??????? http://cr.openjdk.java.net/~smonteith/8214527/webrev.0/ >>>>>>> I think that the extension of arraycopy_prologue interface is better >>>>>>> to be handled and tested >>>>>>> separately, to avoid surprises. GC interface should give us clean >>>>>>> specific-GC-only patches for >>>>>>> specific-GC features. >>>>>>> >>>>>>> -Aleksey >>>>>>> >>>>>>> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Mon May 20 12:18:26 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 20 May 2019 14:18:26 +0200 Subject: RFR: 8224187 Refactor arraycopy_prologue to allow ZGC read barriers on arraycopy In-Reply-To: References: <3ac5ea67-039b-cae8-f50d-5fe289255ae0@redhat.com> Message-ID: > Hello Roman, > I don't think there is any disagreement about Erik's proposal. > However, I'd like to understand what the time-frame is for Erik's > proposal would be before abandoning this patch. I am aiming for ZGC > for aarch64 to get merged before rampdown phase 1 in mid-June in order > for it to be in JDK 13 as an experimental option.. I believe this > change is minimal, it wouldn't add any additional effort onto Erik's > proposal, and if nothing else brings aarch64 more in line with the > other platforms. Right. I would like to propose that we take it a tiny step further and do: https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2019-May/025693.html Such that we (Shenandoah team) can make progress too. Then Erik can work on the full arraycopy GC interface without extra pressure. Roman > BR, > Stuart > > On Mon, 20 May 2019 at 11:40, Roman Kennke wrote: >> >> I'd rather see Erik's proposal implemented to make the GC generate the >> whole path to begin with. Otherwise we'd band-aid for ZGC now, and later >> have to band-aid again for Shenandoah. See discussion for JDK-8223240 >> and in your aarch64 on ZGC thread. >> >> Roman >> >>> Hello, >>> On aarch64 the signature for >>> barrierSetAssembler::arraycopy_prologue needs to be changed in order >>> accept the source as well as the destination array for ZGC's barriers. >>> >>> The bug: >>> https://bugs.openjdk.java.net/browse/JDK-8224187 >>> and the patch: >>> http://cr.openjdk.java.net/~smonteith/8224187/webrev.0/ >>> >>> >>> BR, >>> Stuart >>> >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Mon May 20 13:11:53 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 20 May 2019 09:11:53 -0400 Subject: RFR(S) 8224210: Shenandoah: Refactor ShenandoahRootScanner to support scanning CSet codecache roots Message-ID: Split the refactoring from early JDK-8224179 patch, as suggested by Aleksey. Bug: https://bugs.openjdk.java.net/browse/JDK-8224210 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224210/webrev.00/ Test: hotspot_gc_shenandoah (fastdebug and release) Failed CTW tests in traversal mode as expected. Thanks, -Zhengyu From shade at redhat.com Mon May 20 13:16:47 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 20 May 2019 15:16:47 +0200 Subject: RFR(S) 8224210: Shenandoah: Refactor ShenandoahRootScanner to support scanning CSet codecache roots In-Reply-To: References: Message-ID: On 5/20/19 3:11 PM, Zhengyu Gu wrote: > Split the refactoring from early JDK-8224179 patch, as suggested by Aleksey. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8224210 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224210/webrev.00/ Looks fine to me. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Mon May 20 13:36:13 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 20 May 2019 09:36:13 -0400 Subject: RFR 8224179: Shenandoah: CTW test failures with traversal GC In-Reply-To: <5f521f87-28f1-811b-3aad-9dff2457d9bb@redhat.com> References: <0a13c972-9a15-917a-ef0a-7cfe351ea15b@redhat.com> <5f521f87-28f1-811b-3aad-9dff2457d9bb@redhat.com> Message-ID: <1143128f-a899-5ddd-02f2-b02ee1645615@redhat.com> On 5/20/19 3:35 AM, Aleksey Shipilev wrote: > On 5/20/19 2:59 AM, Zhengyu Gu wrote: >> The failures were due to nested >> nmethod::oops_do_marking_prologue()/nmethod::oops_do_marking_epilogue() calls. This patch is split >> from early JDK-8224115. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8224179 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224179/webrev.00/ > > Looks okay. The synopsis for the change does not reflect what is going on: it should mention what > the change does, i.e. "Introduce all- and cset-specialized ShenandoahRootScanner"? Split the changeset, ShenandoahRootScanner refactoring now is done by JDK-8224210. Updated webrev: http://cr.openjdk.java.net/~zgu/JDK-8224179/webrev.01/ Test: hotspot_gc_shenandoah and CTW test in traversal mode. Thanks, -Zhengyu > > The patch does lots of code moves. Are those moves necessary to fix the failure? Ideally, the > refactoring and the functional change should go separately. In other words, the fix should make it > absolutely clear what is done, not obscure most of it with reshuffling the code. > > -Aleksey > From shade at redhat.com Mon May 20 13:38:51 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 20 May 2019 15:38:51 +0200 Subject: RFR 8224179: Shenandoah: CTW test failures with traversal GC In-Reply-To: <1143128f-a899-5ddd-02f2-b02ee1645615@redhat.com> References: <0a13c972-9a15-917a-ef0a-7cfe351ea15b@redhat.com> <5f521f87-28f1-811b-3aad-9dff2457d9bb@redhat.com> <1143128f-a899-5ddd-02f2-b02ee1645615@redhat.com> Message-ID: On 5/20/19 3:36 PM, Zhengyu Gu wrote: > Updated webrev: http://cr.openjdk.java.net/~zgu/JDK-8224179/webrev.01/ Excellent. So we clearly see that !unload_classes path only does not scan CLDs. Coderoots are scanned normally by passing the code_cl to the same RootScanner. Thumbs up! -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From stuart.monteith at linaro.org Mon May 20 13:40:34 2019 From: stuart.monteith at linaro.org (Stuart Monteith) Date: Mon, 20 May 2019 14:40:34 +0100 Subject: RFR: 8224187 Refactor arraycopy_prologue to allow ZGC read barriers on arraycopy In-Reply-To: References: <3ac5ea67-039b-cae8-f50d-5fe289255ae0@redhat.com> Message-ID: Hello Roman, You proposal is a superset of what I've proposed, so I'm happy to go with your patch. I can close my bug and rebase on yours. Thanks, Stuart On Mon, 20 May 2019 at 13:18, Roman Kennke wrote: > > > Hello Roman, > > I don't think there is any disagreement about Erik's proposal. > > However, I'd like to understand what the time-frame is for Erik's > > proposal would be before abandoning this patch. I am aiming for ZGC > > for aarch64 to get merged before rampdown phase 1 in mid-June in order > > for it to be in JDK 13 as an experimental option.. I believe this > > change is minimal, it wouldn't add any additional effort onto Erik's > > proposal, and if nothing else brings aarch64 more in line with the > > other platforms. > > Right. I would like to propose that we take it a tiny step further and do: > > https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2019-May/025693.html > > > Such that we (Shenandoah team) can make progress too. Then Erik can work > on the full arraycopy GC interface without extra pressure. > > Roman > > > > > BR, > > Stuart > > > > On Mon, 20 May 2019 at 11:40, Roman Kennke wrote: > >> > >> I'd rather see Erik's proposal implemented to make the GC generate the > >> whole path to begin with. Otherwise we'd band-aid for ZGC now, and later > >> have to band-aid again for Shenandoah. See discussion for JDK-8223240 > >> and in your aarch64 on ZGC thread. > >> > >> Roman > >> > >>> Hello, > >>> On aarch64 the signature for > >>> barrierSetAssembler::arraycopy_prologue needs to be changed in order > >>> accept the source as well as the destination array for ZGC's barriers. > >>> > >>> The bug: > >>> https://bugs.openjdk.java.net/browse/JDK-8224187 > >>> and the patch: > >>> http://cr.openjdk.java.net/~smonteith/8224187/webrev.0/ > >>> > >>> > >>> BR, > >>> Stuart > >>> > >> > From stuart.monteith at linaro.org Mon May 20 13:45:48 2019 From: stuart.monteith at linaro.org (Stuart Monteith) Date: Mon, 20 May 2019 14:45:48 +0100 Subject: RFR: JDK-8223244: Extend arraycopy stub GC interface to allow GC replacement of the whole loop In-Reply-To: References: Message-ID: This looks good to me. (as a non-committer). On Thu, 2 May 2019 at 22:43, Roman Kennke wrote: > > Currently, when doing oop arraycopy from C2 and C1, we call into runtime > for pre-barrier, then do the arraycopy as fast-blt-loop, then call into > runtime again for post-barrier. > > In Shenandoah we would like to call into runtime once, and do the > arraycopy in a single-loop, when GC is active, and only do the > fast-blt-loop outside of GC. > > This requires some GC interface changes to allow skipping the fast-loop > when it's not needed, and to also pass the element type for doing the > arraycopy loop and checkcasts in runtime. > > For the most part, the change is to add the klass Register, and the > L_exit label to jump to. In aarch64, it also involves to actually > add+bind those labels, plus a very minor shuffling of code, in order to > preserve original element count before diving into the prologue. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8223240 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8223244/webrev.00/ > > Testing: hotspot_gc_shenandoah, hotspot/tier1 (x86_64 and aarch64) will > jdk-submit before pushing. Also, tried with the prototype for > JDK-8222859 both x86_64 and aarch64. > > Can I please get reviews? > > Thanks, > Roman > From thomas.schatzl at oracle.com Mon May 20 14:33:21 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 20 May 2019 16:33:21 +0200 Subject: RFR (S): 8224160: Move G1RemSetScanClosure into g1RemSet.cpp file Message-ID: Hi all, can I have reviews for this cleanup that moves the G1RemSetScanClosure class from a hpp file close to the only use in the cpp file. This has been made possible by the earlier JDK-8218668 change. CR: https://bugs.openjdk.java.net/browse/JDK-8224160 Webrev: http://cr.openjdk.java.net/~tschatzl/8224160/webrev/index.html Testing: hs-tier1 Thanks, Thomas From erik.osterlund at oracle.com Mon May 20 15:53:32 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 20 May 2019 17:53:32 +0200 Subject: RFR: 8224187 Refactor arraycopy_prologue to allow ZGC read barriers on arraycopy In-Reply-To: <3ac5ea67-039b-cae8-f50d-5fe289255ae0@redhat.com> References: <3ac5ea67-039b-cae8-f50d-5fe289255ae0@redhat.com> Message-ID: <1d9569d1-520e-2083-c7c6-acd1492732a3@oracle.com> Hi Roman, I don't think I would call it a band-aid for ZGC to add the src parameter to the prologue. That is just harmonizing the arraycopy interface model we already have on all platforms today, to do on AArch64 what the other platforms have always been doing since the API was introduced, since before ZGC was integrated. However, having the actual copying being performed sometimes inside of the public pre-barrier, which may or may not throw a checkcast exception, seems like a very different beast to jam into the public interface. Apart from being a rather ugly abstraction, I don't feel like I can assess if that approach is correct or not (w.r.t. checkcast exceptions) without seeing the proposed shenandoah code that uses it. In your patch I just see passing around of values that are never used. I also don't know how that will play with windows that has an ABI-adapter from windows ABI to Sys-V used to wrap the arraycopy code. The ABI wrapper saves windows callee saved registers on the thread, assuming that the copying code itself does not nest... I don't know if it still does not nest with this approach, as the ABI conversion scope is set up outside the barrier set calls, and inside of the prebarrier in the runtime that calls arraycopy back into the stub code again, I think the result on windows is that it might overwrite the callee saved registers in a nasty and intermittent way that is hard to reproduce. /Erik On 2019-05-20 12:40, Roman Kennke wrote: > I'd rather see Erik's proposal implemented to make the GC generate the > whole path to begin with. Otherwise we'd band-aid for ZGC now, and later > have to band-aid again for Shenandoah. See discussion for JDK-8223240 > and in your aarch64 on ZGC thread. > > Roman > >> Hello, >> On aarch64 the signature for >> barrierSetAssembler::arraycopy_prologue needs to be changed in order >> accept the source as well as the destination array for ZGC's barriers. >> >> The bug: >> https://bugs.openjdk.java.net/browse/JDK-8224187 >> and the patch: >> http://cr.openjdk.java.net/~smonteith/8224187/webrev.0/ >> >> >> BR, >> Stuart >> From erik.osterlund at oracle.com Mon May 20 16:13:08 2019 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Mon, 20 May 2019 18:13:08 +0200 Subject: RFR: 8224187 Refactor arraycopy_prologue to allow ZGC read barriers on arraycopy In-Reply-To: References: Message-ID: <21B3E617-5F2D-4791-81D7-74B5637F781A@oracle.com> Hi Stuart, Looks good to me. Thanks, /Erik > On 20 May 2019, at 12:27, Stuart Monteith wrote: > > Hello, > On aarch64 the signature for > barrierSetAssembler::arraycopy_prologue needs to be changed in order > accept the source as well as the destination array for ZGC's barriers. > > The bug: > https://bugs.openjdk.java.net/browse/JDK-8224187 > and the patch: > http://cr.openjdk.java.net/~smonteith/8224187/webrev.0/ > > > BR, > Stuart From kim.barrett at oracle.com Mon May 20 20:46:36 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 20 May 2019 16:46:36 -0400 Subject: RFR (S): 8224160: Move G1RemSetScanClosure into g1RemSet.cpp file In-Reply-To: References: Message-ID: > On May 20, 2019, at 10:33 AM, Thomas Schatzl wrote: > > Hi all, > > can I have reviews for this cleanup that moves the > G1RemSetScanClosure class from a hpp file close to the only use in the > cpp file. This has been made possible by the earlier JDK-8218668 > change. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8224160 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8224160/webrev/index.html > Testing: > hs-tier1 > > Thanks, > Thomas Looks good. From thomas.schatzl at oracle.com Mon May 20 21:02:09 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 20 May 2019 23:02:09 +0200 Subject: RFR (S): 8224160: Move G1RemSetScanClosure into g1RemSet.cpp file In-Reply-To: References: Message-ID: Hi Kim, On Mon, 2019-05-20 at 16:46 -0400, Kim Barrett wrote: > > On May 20, 2019, at 10:33 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > Hi all, > > > > can I have reviews for this cleanup that moves the > > G1RemSetScanClosure class from a hpp file close to the only use in > > the cpp file. This has been made possible by the earlier JDK- > > 8218668 change. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8224160 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8224160/webrev/index.html > > Testing: > > hs-tier1 > > > > Thanks, > > Thomas > > Looks good. > thanks for your review. Thomas From sangheon.kim at oracle.com Mon May 20 21:12:05 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Mon, 20 May 2019 14:12:05 -0700 Subject: RFR (S): 8224160: Move G1RemSetScanClosure into g1RemSet.cpp file In-Reply-To: References: Message-ID: <2bae2ebc-8798-aeee-4b6e-5955a54e7dcb@oracle.com> Hi Thomas, On 5/20/19 7:33 AM, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this cleanup that moves the > G1RemSetScanClosure class from a hpp file close to the only use in the > cpp file. This has been made possible by the earlier JDK-8218668 > change. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8224160 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8224160/webrev/index.html Looks good. Thanks, Sangheon > Testing: > hs-tier1 > > Thanks, > Thomas > From kim.barrett at oracle.com Mon May 20 21:15:38 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 20 May 2019 17:15:38 -0400 Subject: RFR: 8224167: Refactor PtrQueue completed buffer processing Message-ID: <219A1332-768F-4A7A-94A1-582F5BF10C8C@oracle.com> Please review this change to how PtrQueue handles completed (full) buffers. The code has been refactored to have a new virtual function (handle_completed_buffer) that is applicable to and implemented by all derived classes. This replaces a pair of virtual functions that were each only relevant to one derived class (should_enqueue_buffer and mut_process_buffer). This allowed some configuration parameters that were only applicable to G1 dirty card handling to be moved from PtrQueueSet to G1DirtyCardQueueSet. Some related cleanup work: PtrQueue::set_capacity has been removed. The one place it was being called instead inlines the relevant code and comments it as the bootstrapping kludge that it is; see JDK-8221361. Added some utility functions. PtrQueueSet::_n_completed_buffers declared volatile, since it has racy reads outside the lock. Shenandoah is affected by this refactoring, and has been updated accordingly, but I've only very lightly tested it. CR: https://bugs.openjdk.java.net/browse/JDK-8224167 Webrev: http://cr.openjdk.java.net/~kbarrett/8224167/open.00/ Testing: mach5 tier1-5 Local (linux-x64) hs-tier1 with shenandoah included. From chengjingwei1 at huawei.com Tue May 21 01:50:41 2019 From: chengjingwei1 at huawei.com (chengjingwei (A)) Date: Tue, 21 May 2019 01:50:41 +0000 Subject: PING: RFR: 8217432: MetaspaceGC::_capacity_until_GC exceeds MaxMetaspaceSize Message-ID: By the way, the link of the patch that caused problems was https://bugs.openjdk.java.net/browse/JDK-8217432. The test would fail with 32-bit jdk even on 64-bit platforms. > Hi, > > After this patch was applied (since jdk8u212 and jdk-11.0.3, to be specific), we started to get test failures with jtreg. > The failed test case was jdk8u/hotspot/test/gc/metaspace/TestCapacityUntilGCWrapAround.java. > > It always fails on windows-32bit platform, and fails on linux-32bit if -XX:MaxMetaspaceSize is set. > > The failure message was: > java.lang.IllegalStateException: WB_IncMetaspaceCapacityUntilGC: could not increase capacity until GC due to contention with another thread > at sun.hotspot.WhiteBox.incMetaspaceCapacityUntilGC(Native Method) > at TestCapacityUntilGCWrapAround.main(TestCapacityUntilGCWrapAround.java:51) > > I checked the code, and found that MetaspaceGC::inc_capacity_until_GC was passed a value of 4G-page_size-1, > which caused addition overflow when computing new_value, and new_value always set to "align_size_down(max_uintx, Metaspace::commit_alignment())". > So when MaxMetaspaceSize defaults to or is set to a value less than it, the function will return false to WB_IncMetaspaceCapacityUntilGC, > which will then throw an Exception, causing the test failure. > > On win-32 platform, the default value of MaxMetaspaceSize is 4G - allocation_granularity, > which is 4G - 64K (according to https://devblogs.microsoft.com/oldnewthing/20031008-00/?p=42223), > this is very likely to be less than "new_value" and cause test failure. > > Do you have any idea how to fix this? -------------- next part -------------- An HTML attachment was scrubbed... URL: From yasuenag at gmail.com Tue May 21 02:41:12 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Tue, 21 May 2019 11:41:12 +0900 Subject: PING: RFR: 8217432: MetaspaceGC::_capacity_until_GC exceeds MaxMetaspaceSize In-Reply-To: References: Message-ID: Hi, I think the result which you saw is correct because 32 bit platform cannot obtain 4GB memory. IMHO you need to remove or modify this testcase. Thanks, Yasumasa 2019?5?21?(?) 10:50 chengjingwei (A) : > > By the way, the link of the patch that caused problems was https://bugs.openjdk.java.net/browse/JDK-8217432. > > The test would fail with 32-bit jdk even on 64-bit platforms. > > > > > Hi, > > > > > > After this patch was applied (since jdk8u212 and jdk-11.0.3, to be specific), we started to get test failures with jtreg. > > > The failed test case was jdk8u/hotspot/test/gc/metaspace/TestCapacityUntilGCWrapAround.java. > > > > > > It always fails on windows-32bit platform, and fails on linux-32bit if -XX:MaxMetaspaceSize is set. > > > > > > The failure message was: > > > java.lang.IllegalStateException: WB_IncMetaspaceCapacityUntilGC: could not increase capacity until GC due to contention with another thread > > > at sun.hotspot.WhiteBox.incMetaspaceCapacityUntilGC(Native Method) > > > at TestCapacityUntilGCWrapAround.main(TestCapacityUntilGCWrapAround.java:51) > > > > > > I checked the code, and found that MetaspaceGC::inc_capacity_until_GC was passed a value of 4G-page_size-1, > > > which caused addition overflow when computing new_value, and new_value always set to "align_size_down(max_uintx, Metaspace::commit_alignment())". > > > So when MaxMetaspaceSize defaults to or is set to a value less than it, the function will return false to WB_IncMetaspaceCapacityUntilGC, > > > which will then throw an Exception, causing the test failure. > > > > > > On win-32 platform, the default value of MaxMetaspaceSize is 4G - allocation_granularity, > > > which is 4G - 64K (according to https://devblogs.microsoft.com/oldnewthing/20031008-00/?p=42223), > > > this is very likely to be less than "new_value" and cause test failure. > > > > > > Do you have any idea how to fix this? > > From per.liden at oracle.com Tue May 21 08:28:10 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 21 May 2019 10:28:10 +0200 Subject: [aarch64-port-dev ] RFR: 8224187 Refactor arraycopy_prologue to allow ZGC read barriers on arraycopy In-Reply-To: References: Message-ID: <56d60878-63ec-9540-12c2-8d4da4064048@oracle.com> Looks good to me. This brings aarch64 in sync with all other platforms. /Per On 2019-05-20 12:27, Stuart Monteith wrote: > Hello, > On aarch64 the signature for > barrierSetAssembler::arraycopy_prologue needs to be changed in order > accept the source as well as the destination array for ZGC's barriers. > > The bug: > https://bugs.openjdk.java.net/browse/JDK-8224187 > and the patch: > http://cr.openjdk.java.net/~smonteith/8224187/webrev.0/ > > > BR, > Stuart > From shade at redhat.com Tue May 21 08:35:28 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 21 May 2019 10:35:28 +0200 Subject: RFR: 8224167: Refactor PtrQueue completed buffer processing In-Reply-To: <219A1332-768F-4A7A-94A1-582F5BF10C8C@oracle.com> References: <219A1332-768F-4A7A-94A1-582F5BF10C8C@oracle.com> Message-ID: <24f7865e-64be-1676-718f-72fdbd5e150a@redhat.com> On 5/20/19 11:15 PM, Kim Barrett wrote: > Webrev: > http://cr.openjdk.java.net/~kbarrett/8224167/open.00/ Shenandoah parts look good. They do not seem to affect performance in adverse way. Let me understand: SATBMarkQueue::handle_completed_buffer does reply either empty buffer (after full filtering/enqueueing), or semi-full buffer if filters were applied and it still decided not to enqueue it. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Tue May 21 08:36:25 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 21 May 2019 10:36:25 +0200 Subject: RFR: 8224187 Refactor arraycopy_prologue to allow ZGC read barriers on arraycopy In-Reply-To: <1d9569d1-520e-2083-c7c6-acd1492732a3@oracle.com> References: <3ac5ea67-039b-cae8-f50d-5fe289255ae0@redhat.com> <1d9569d1-520e-2083-c7c6-acd1492732a3@oracle.com> Message-ID: <0ba5a6ac-40d5-7acf-a93a-44e8aaa690f3@oracle.com> Hi Roman, I have to agree. This is not a about adding a band-aid for ZGC, it's about fixing aarch64's barrier set. I don't think this should be conflated with the other patch you proposed, since that's is a different review/discussion to solve a different problem. cheers, Per On 2019-05-20 17:53, Erik ?sterlund wrote: > Hi Roman, > > I don't think I would call it a band-aid for ZGC to add the src > parameter to the prologue. That is just harmonizing the arraycopy > interface model we already have on all platforms today, to do on AArch64 > what the other platforms have always been doing since the API was > introduced, since before ZGC was integrated. > However, having the actual copying being performed sometimes inside of > the public pre-barrier, which may or may not throw a checkcast > exception, seems like a very different beast to jam into the public > interface. > Apart from being a rather ugly abstraction, I don't feel like I can > assess if that approach is correct or not (w.r.t. checkcast exceptions) > without seeing the proposed shenandoah code that uses it. In your patch > I just see passing around of values that are never used. I also don't > know how that will play with windows that has an ABI-adapter from > windows ABI to Sys-V used to wrap the arraycopy code. The ABI wrapper > saves windows callee saved registers on the thread, assuming that the > copying code itself does not nest... I don't know if it still does not > nest with this approach, as the ABI conversion scope is set up outside > the barrier set calls, and inside of the prebarrier in the runtime that > calls arraycopy back into the stub code again, I think the result on > windows is that it might overwrite the callee saved registers in a nasty > and intermittent way that is hard to reproduce. > > /Erik > > On 2019-05-20 12:40, Roman Kennke wrote: >> I'd rather see Erik's proposal implemented to make the GC generate the >> whole path to begin with. Otherwise we'd band-aid for ZGC now, and later >> have to band-aid again for Shenandoah. See discussion for JDK-8223240 >> and in your aarch64 on ZGC thread. >> >> Roman >> >>> Hello, >>> ??? On aarch64 the signature for >>> barrierSetAssembler::arraycopy_prologue needs to be changed in order >>> accept the source as well as the destination array for ZGC's barriers. >>> >>> The bug: >>> ? https://bugs.openjdk.java.net/browse/JDK-8224187 >>> and the patch: >>> ?? http://cr.openjdk.java.net/~smonteith/8224187/webrev.0/ >>> >>> >>> BR, >>> ??? Stuart >>> > From thomas.schatzl at oracle.com Tue May 21 08:49:27 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 21 May 2019 10:49:27 +0200 Subject: RFR (S): 8224160: Move G1RemSetScanClosure into g1RemSet.cpp file In-Reply-To: <2bae2ebc-8798-aeee-4b6e-5955a54e7dcb@oracle.com> References: <2bae2ebc-8798-aeee-4b6e-5955a54e7dcb@oracle.com> Message-ID: Hi, On Mon, 2019-05-20 at 14:12 -0700, sangheon.kim at oracle.com wrote: > Hi Thomas, > > On 5/20/19 7:33 AM, Thomas Schatzl wrote: > > Hi all, > > > > can I have reviews for this cleanup that moves the > > G1RemSetScanClosure class from a hpp file close to the only use in > > the cpp file. This has been made possible by the earlier JDK- > > 8218668 change. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8224160 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8224160/webrev/index.html > > Looks good. > thanks for your review. Thomas From thomas.schatzl at oracle.com Tue May 21 09:05:58 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 21 May 2019 11:05:58 +0200 Subject: RFR: 8224167: Refactor PtrQueue completed buffer processing In-Reply-To: <219A1332-768F-4A7A-94A1-582F5BF10C8C@oracle.com> References: <219A1332-768F-4A7A-94A1-582F5BF10C8C@oracle.com> Message-ID: <43c83061a512dc99f4900b1fd4ac9250339e7a4e.camel@oracle.com> Hi, On Mon, 2019-05-20 at 17:15 -0400, Kim Barrett wrote: > Please review this change to how PtrQueue handles completed (full) > buffers. The code has been refactored to have a new virtual function > [...] > > Shenandoah is affected by this refactoring, and has been updated > accordingly, but I've only very lightly tested it. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8224167 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8224167/open.00/ > > Testing: > mach5 tier1-5 > Local (linux-x64) hs-tier1 with shenandoah included. > thanks for this cleanup. Looks good. Thomas From thomas.schatzl at oracle.com Tue May 21 09:44:42 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 21 May 2019 11:44:42 +0200 Subject: RFR(M): 8220089: G1 wastes a significant amount of space in survivor regions In-Reply-To: References: <11f5af23-bacf-267d-7e81-8cd408ae063b@oracle.com> <45151a57-4689-21e6-4bfb-a884d6b6c14d@oracle.com> <174f169f-4ccb-7567-7707-d6a7f7024aa8@oracle.com> <2dea4528-8c8b-76ab-d737-ad36d36dab0a@oracle.com> <3B084725-AAEC-4D78-9AC7-A16492E01613@oracle.com> Message-ID: <6d77557059312613475c84a00290b994b65185b7.camel@oracle.com> Hi Sangheon, On Wed, 2019-05-15 at 15:08 -0700, sangheon.kim at oracle.com wrote: > Hi Kim, > > Thanks for reviewing this. > > On 5/8/19 9:42 PM, Kim Barrett wrote: > > > On Apr 30, 2019, at 2:06 AM, sangheon.kim at oracle.com wrote: > > > > > > Hi all, > > > > > > I got some comments from Thomas Schatzl, mostly about comment and > > > assert changes. He also suggested to investigate combining > > > current 2 steps of handling retaining survivor region into 1 > > > step, so I filed JDK-8223106. > > > > > > Webrev: > > > http://cr.openjdk.java.net/~sangheki/8220089/webrev.2/ > > > http://cr.openjdk.java.net/~sangheki/8220089/webrev.2_inc/ > > > Testing: hs-tier1 > > > > ----------------------------------------------------------------- > > ------------- > > src/hotspot/share/gc/g1/g1AllocRegion.cpp > > 367 bool SurvivorGCAllocRegion::should_retain(HeapRegion* region) { > > 368 if (region == NULL || region->free() < MinTLABSize) { > > 369 return false; > > 370 } > > 371 return true; > > 372 } > > > > Why does should_retain accept a NULL region? > Removed NULL check at should_retain() and added NULL check before > calling should_retain. > G1AllocRegion::release() which used as an input of should_retain() > may return NULL. Could you change this to "return region != NULL && region->free() >= MinTLABSize;" ? > > > > I dislike the "retained" region nomenclature for survivors. I > > didn't notice an explanation of it, and it's not obvious why that > > name was chosen or how it relates to how these regions are used. > > But I don't have a good suggestion for an alternative yet. It seems > > like that name was copied from MutatorAllocRegion, but I'm not sure > > the survivor case is really that similar. > Yeah, I agree that it is not that similar. But as I don't have > better name, I copied from mutator case. Initially I named 'reuse' > but seems not better than 'retain'. Unless there is a better naming > suggestion, I would like to proceed as is. That may be revisited, and may go away with the CR filed by Sangheon to convert all survivors to eden at the end of gc. - there is an extra newline at the end of g1SurvivorRegions.cpp. Looks good otherwise. Thanks, Thomas From rkennke at redhat.com Tue May 21 10:17:56 2019 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 21 May 2019 12:17:56 +0200 Subject: RFR: 8224187 Refactor arraycopy_prologue to allow ZGC read barriers on arraycopy In-Reply-To: <0ba5a6ac-40d5-7acf-a93a-44e8aaa690f3@oracle.com> References: <3ac5ea67-039b-cae8-f50d-5fe289255ae0@redhat.com> <1d9569d1-520e-2083-c7c6-acd1492732a3@oracle.com> <0ba5a6ac-40d5-7acf-a93a-44e8aaa690f3@oracle.com> Message-ID: <3366d0bd-1810-74ac-8bb1-a892b449e86d@redhat.com> Well, I disagree. But it's not important enough to hold anything back, so please go ahead. Roman > Hi Roman, > > I have to agree. This is not a about adding a band-aid for ZGC, it's > about fixing aarch64's barrier set. > > I don't think this should be conflated with the other patch you > proposed, since that's is a different review/discussion to solve a > different problem. > > cheers, > Per > > On 2019-05-20 17:53, Erik ?sterlund wrote: >> Hi Roman, >> >> I don't think I would call it a band-aid for ZGC to add the src >> parameter to the prologue. That is just harmonizing the arraycopy >> interface model we already have on all platforms today, to do on >> AArch64 what the other platforms have always been doing since the API >> was introduced, since before ZGC was integrated. >> However, having the actual copying being performed sometimes inside of >> the public pre-barrier, which may or may not throw a checkcast >> exception, seems like a very different beast to jam into the public >> interface. >> Apart from being a rather ugly abstraction, I don't feel like I can >> assess if that approach is correct or not (w.r.t. checkcast >> exceptions) without seeing the proposed shenandoah code that uses it. >> In your patch I just see passing around of values that are never used. >> I also don't know how that will play with windows that has an >> ABI-adapter from windows ABI to Sys-V used to wrap the arraycopy code. >> The ABI wrapper saves windows callee saved registers on the thread, >> assuming that the copying code itself does not nest... I don't know if >> it still does not nest with this approach, as the ABI conversion scope >> is set up outside the barrier set calls, and inside of the prebarrier >> in the runtime that calls arraycopy back into the stub code again, I >> think the result on windows is that it might overwrite the callee >> saved registers in a nasty and intermittent way that is hard to >> reproduce. >> >> /Erik >> >> On 2019-05-20 12:40, Roman Kennke wrote: >>> I'd rather see Erik's proposal implemented to make the GC generate the >>> whole path to begin with. Otherwise we'd band-aid for ZGC now, and later >>> have to band-aid again for Shenandoah. See discussion for JDK-8223240 >>> and in your aarch64 on ZGC thread. >>> >>> Roman >>> >>>> Hello, >>>> ??? On aarch64 the signature for >>>> barrierSetAssembler::arraycopy_prologue needs to be changed in order >>>> accept the source as well as the destination array for ZGC's barriers. >>>> >>>> The bug: >>>> ? https://bugs.openjdk.java.net/browse/JDK-8224187 >>>> and the patch: >>>> ?? http://cr.openjdk.java.net/~smonteith/8224187/webrev.0/ >>>> >>>> >>>> BR, >>>> ??? Stuart >>>> >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From stuart.monteith at linaro.org Tue May 21 10:22:01 2019 From: stuart.monteith at linaro.org (Stuart Monteith) Date: Tue, 21 May 2019 11:22:01 +0100 Subject: RFR: 8224187 Refactor arraycopy_prologue to allow ZGC read barriers on arraycopy In-Reply-To: <0ba5a6ac-40d5-7acf-a93a-44e8aaa690f3@oracle.com> References: <3ac5ea67-039b-cae8-f50d-5fe289255ae0@redhat.com> <1d9569d1-520e-2083-c7c6-acd1492732a3@oracle.com> <0ba5a6ac-40d5-7acf-a93a-44e8aaa690f3@oracle.com> Message-ID: Hello, I've reopened JDK-8224187, as 8223240 is another discussion. I don't have committer's writes to push commit 8224187. BR, Stuart On Tue, 21 May 2019 at 09:36, Per Liden wrote: > > Hi Roman, > > I have to agree. This is not a about adding a band-aid for ZGC, it's > about fixing aarch64's barrier set. > > I don't think this should be conflated with the other patch you > proposed, since that's is a different review/discussion to solve a > different problem. > > cheers, > Per > > On 2019-05-20 17:53, Erik ?sterlund wrote: > > Hi Roman, > > > > I don't think I would call it a band-aid for ZGC to add the src > > parameter to the prologue. That is just harmonizing the arraycopy > > interface model we already have on all platforms today, to do on AArch64 > > what the other platforms have always been doing since the API was > > introduced, since before ZGC was integrated. > > However, having the actual copying being performed sometimes inside of > > the public pre-barrier, which may or may not throw a checkcast > > exception, seems like a very different beast to jam into the public > > interface. > > Apart from being a rather ugly abstraction, I don't feel like I can > > assess if that approach is correct or not (w.r.t. checkcast exceptions) > > without seeing the proposed shenandoah code that uses it. In your patch > > I just see passing around of values that are never used. I also don't > > know how that will play with windows that has an ABI-adapter from > > windows ABI to Sys-V used to wrap the arraycopy code. The ABI wrapper > > saves windows callee saved registers on the thread, assuming that the > > copying code itself does not nest... I don't know if it still does not > > nest with this approach, as the ABI conversion scope is set up outside > > the barrier set calls, and inside of the prebarrier in the runtime that > > calls arraycopy back into the stub code again, I think the result on > > windows is that it might overwrite the callee saved registers in a nasty > > and intermittent way that is hard to reproduce. > > > > /Erik > > > > On 2019-05-20 12:40, Roman Kennke wrote: > >> I'd rather see Erik's proposal implemented to make the GC generate the > >> whole path to begin with. Otherwise we'd band-aid for ZGC now, and later > >> have to band-aid again for Shenandoah. See discussion for JDK-8223240 > >> and in your aarch64 on ZGC thread. > >> > >> Roman > >> > >>> Hello, > >>> On aarch64 the signature for > >>> barrierSetAssembler::arraycopy_prologue needs to be changed in order > >>> accept the source as well as the destination array for ZGC's barriers. > >>> > >>> The bug: > >>> https://bugs.openjdk.java.net/browse/JDK-8224187 > >>> and the patch: > >>> http://cr.openjdk.java.net/~smonteith/8224187/webrev.0/ > >>> > >>> > >>> BR, > >>> Stuart > >>> > > From per.liden at oracle.com Tue May 21 11:05:09 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 21 May 2019 13:05:09 +0200 Subject: RFR: JDK-8214527 AArch64: ZGC for Aarch64 In-Reply-To: References: Message-ID: <2abfb1d9-5235-e55f-6d90-ba8640cb197c@oracle.com> Hi Stuart, On 05/17/2019 04:28 PM, Stuart Monteith wrote: > Hello, > This patch implements ZGC for AArch64. Unlike the previous set of > patches, this duplicates the x86 multimapping scheme rather than using > the Aarch64 TBI feature, where the top 8 bits are ignored by the > hardware. I didn't measure any conclusive performance problems using > multimapping. It would necessitate changes to SA, JNI, and wasn't > going to be compatible with Graal or JVMCI in general until they were > also able to switch to 64-bit literals dynamically. In addition, there > are up and coming features in the AArch64 architecture that exploit > the TBI area that we should keep open, such as MTE (Memory Tagging > Extensions) that open the door to lightweight hardware ASAN. With MTE > using 4 of the 8 bits, and ZGC and MTE perhaps needing more than 4 > bits, future conflict looks likely. > > JTreg testing hasn't demonstrated any significant differences from x86 > ZGC in my testing. > > The behaviour should be the same as is expected from x86_64, including > for the behaviour with command line arguments. Currently ZGC requires > -XX:+UnlockExperimentalVMOptions, that should be the case for AArch64 > this release. If x86_64 were to do have ZGC designated as not > experimental, we must ensure it still applies for ZGC on AArch64. I don't think there's a good way to declare the a flag product on one platform, and experimental on another. But I guess zArguments::initialize_platform() on aarch64 could check for and warn/exit if UnlockExperimentalVMOptions is false. > > The associated bug is: > https://bugs.openjdk.java.net/browse/JDK-8214527 > The patchset is: > http://cr.openjdk.java.net/~smonteith/8214527/webrev.0/ src/hotspot/cpu/aarch64/gc/z/z_aarch64.ad ----------------------------------------- I'd suggest you break out the duplicate parts of these into a function, like we have done in z_x86.ad. src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingFile_linux_aarch64.cpp src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingFile_linux_aarch64.hpp src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingPath_linux_aarch64.cpp src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingPath_linux_aarch64.hpp src/hotspot/os_cpu/linux_aarch64/gc/z/zGlobals_linux_aarch64.cpp src/hotspot/os_cpu/linux_aarch64/gc/z/zGlobals_linux_aarch64.hpp src/hotspot/os_cpu/linux_aarch64/gc/z/zPhysicalMemoryBacking_linux_aarch64.cpp src/hotspot/os_cpu/linux_aarch64/gc/z/zPhysicalMemoryBacking_linux_aarch64.hpp -------------------------------------------------------------------- I'd would like to deduplicate most of these at some point, but I'm ok with doing that after this has been merged, as we might need/want to slice this slightly differently (to cater for the different syscall numbers etc). cheers, Per > > There is a patch contributed by Roland Westerlin required for fixing > memory barriers, > http://cr.openjdk.java.net/~smonteith/zgc-mm/20190430/membar/ > I'll follow up with a separate RFR patch to address that. > > Thanks, > Stuart > From stuart.monteith at linaro.org Tue May 21 13:44:14 2019 From: stuart.monteith at linaro.org (Stuart Monteith) Date: Tue, 21 May 2019 14:44:14 +0100 Subject: RFR: JDK-8214527 AArch64: ZGC for Aarch64 In-Reply-To: <2abfb1d9-5235-e55f-6d90-ba8640cb197c@oracle.com> References: <2abfb1d9-5235-e55f-6d90-ba8640cb197c@oracle.com> Message-ID: Hi Per, Thanks for taking a look at that. I'll wait until ZGC on x86_64 isn't experimental. When that happens, if aarch64 isn't ready for production use, I'd expect we'd make the definition of "UseZGC" conditional, something like: ---- #if defined(AARCH64) #define USEZGC_COND(type, name, flag, descr) experimental((type),(name),(flag),(descr)) #else #define USEZGC_COND(type, name, flag, descr) product((type),(name),(flag),(descr)) #endif USEZGC_COND(bool, UseZGC, false, \ "Use the Z garbage collector") \ ---- That'll need more thought, but I'd like a general solution that could be applied more widely if, indeed, that were even necessary. I've done the refactoring of z_aarch64.ad as you suggested here: http://cr.openjdk.java.net/~smonteith/8214527/webrev.2/ Apart from the system calls, I agree with you about the deduplication. I presume there is no expectation for there to be a SPARC linux port, or other architectures were there might be a desire for a different implementation. BR, Stuart On Tue, 21 May 2019 at 12:05, Per Liden wrote: > > Hi Stuart, > > On 05/17/2019 04:28 PM, Stuart Monteith wrote: > > Hello, > > This patch implements ZGC for AArch64. Unlike the previous set of > > patches, this duplicates the x86 multimapping scheme rather than using > > the Aarch64 TBI feature, where the top 8 bits are ignored by the > > hardware. I didn't measure any conclusive performance problems using > > multimapping. It would necessitate changes to SA, JNI, and wasn't > > going to be compatible with Graal or JVMCI in general until they were > > also able to switch to 64-bit literals dynamically. In addition, there > > are up and coming features in the AArch64 architecture that exploit > > the TBI area that we should keep open, such as MTE (Memory Tagging > > Extensions) that open the door to lightweight hardware ASAN. With MTE > > using 4 of the 8 bits, and ZGC and MTE perhaps needing more than 4 > > bits, future conflict looks likely. > > > > JTreg testing hasn't demonstrated any significant differences from x86 > > ZGC in my testing. > > > > The behaviour should be the same as is expected from x86_64, including > > for the behaviour with command line arguments. Currently ZGC requires > > -XX:+UnlockExperimentalVMOptions, that should be the case for AArch64 > > this release. If x86_64 were to do have ZGC designated as not > > experimental, we must ensure it still applies for ZGC on AArch64. > > I don't think there's a good way to declare the a flag product on one > platform, and experimental on another. But I guess > zArguments::initialize_platform() on aarch64 could check for and > warn/exit if UnlockExperimentalVMOptions is false. > > > > > The associated bug is: > > https://bugs.openjdk.java.net/browse/JDK-8214527 > > The patchset is: > > http://cr.openjdk.java.net/~smonteith/8214527/webrev.0/ > > src/hotspot/cpu/aarch64/gc/z/z_aarch64.ad > ----------------------------------------- > I'd suggest you break out the duplicate parts of these into a function, > like we have done in z_x86.ad. > > > src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingFile_linux_aarch64.cpp > src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingFile_linux_aarch64.hpp > src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingPath_linux_aarch64.cpp > src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingPath_linux_aarch64.hpp > src/hotspot/os_cpu/linux_aarch64/gc/z/zGlobals_linux_aarch64.cpp > src/hotspot/os_cpu/linux_aarch64/gc/z/zGlobals_linux_aarch64.hpp > src/hotspot/os_cpu/linux_aarch64/gc/z/zPhysicalMemoryBacking_linux_aarch64.cpp > src/hotspot/os_cpu/linux_aarch64/gc/z/zPhysicalMemoryBacking_linux_aarch64.hpp > > -------------------------------------------------------------------- > I'd would like to deduplicate most of these at some point, but I'm ok > with doing that after this has been merged, as we might need/want to > slice this slightly differently (to cater for the different syscall > numbers etc). > > > cheers, > Per > > > > > There is a patch contributed by Roland Westerlin required for fixing > > memory barriers, > > http://cr.openjdk.java.net/~smonteith/zgc-mm/20190430/membar/ > > I'll follow up with a separate RFR patch to address that. > > > > Thanks, > > Stuart > > From per.liden at oracle.com Tue May 21 13:48:43 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 21 May 2019 15:48:43 +0200 Subject: RFR: 8224185: ZGC: Introduce "High Usage" rule In-Reply-To: <5790ee54-17e8-9699-c288-5c0c6184e845@oracle.com> References: <0e87a5ac-3a38-4301-5762-58d0d7fa1d94@oracle.com> <06c6cff49119376145dafc56bc2e493ee726b333.camel@oracle.com> <5790ee54-17e8-9699-c288-5c0c6184e845@oracle.com> Message-ID: <9f129198-29cd-5f5b-3bad-cbf39b0d6c2c@oracle.com> Tweaked the test slightly. Diff: http://cr.openjdk.java.net/~pliden/8224185/webrev.1-diff Full: http://cr.openjdk.java.net/~pliden/8224185/webrev.1 /Per On 5/20/19 11:59 AM, Erik ?sterlund wrote: > Hi Per and Thomas, > > Looks good to me. I also would not like to see a JVM flag for this. We > have a quite high bar before we introduce new JVM flags, to avoid JVM > flag hell. Especially for policies that are likely to change as the > algorithm evolves, such that the flag might become useless. And this did > not seem to warrant a flag yet. > > Thanks, > /Erik > > On 2019-05-20 11:44, Per Liden wrote: >> Hi Thomas, >> >> On 5/20/19 11:34 AM, Thomas Schatzl wrote: >>> Hi Per, >>> >>> On Mon, 2019-05-20 at 10:11 +0200, Per Liden wrote: >>>> Start a GC cycle if the amount of free memory is 5% or less. This is >>>> a preventive measure in the case where the application has a very >>>> low allocation rate, such that the allocation rate rule doesn't >>>> trigger, but the amount of free memory is still slowly but surely >>>> heading towards zero. In this situation, we should start a GC cycle >>>> to avoid a potential allocation stall later. >>>> >>>> Testing: New jtreg test added. Currently running tier 1-6. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8224185 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8224185/webrev.0 >>>> >>> >>> ?? just pointing out the obvious: this means that ZGC will potentially >>> never use 5-Epsilon% of the heap. >>> >>> I do not recommend such a threshold being a fixed, compiled in constant >>> size: while I understand that you want to have as few options as >>> possible with ZGC, 5% of 100GB are still 5GB which may be better used >>> for use in the application in some cases. >>> >>> I.e. G1 has a similar (10%) threshold, and we've run in a few >>> situations where this has been too much and unneeded, particularly with >>> large heaps. >>> Now I know that ZGC pauses are by far less intrusive than G1's, but >>> still the concurrent marking has its cost. Making it configurable would >>> allow people getting the last bit of performance out of it if they >>> want. >> >> The situation is slightly different for ZGC. This rule typically kicks >> in when the allocation rate is very low. If the allocation rate is >> very high, we typically collect a lot earlier than 95% full heap anyway. >> >> Regarding a flag. We've discussed this within the team, and decided to >> not have one for now. It can be added later if it turns out to be a >> need for it. >> >> cheers, >> Per > From erik.osterlund at oracle.com Tue May 21 14:00:29 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 21 May 2019 16:00:29 +0200 Subject: RFR: 8224185: ZGC: Introduce "High Usage" rule In-Reply-To: <9f129198-29cd-5f5b-3bad-cbf39b0d6c2c@oracle.com> References: <0e87a5ac-3a38-4301-5762-58d0d7fa1d94@oracle.com> <06c6cff49119376145dafc56bc2e493ee726b333.camel@oracle.com> <5790ee54-17e8-9699-c288-5c0c6184e845@oracle.com> <9f129198-29cd-5f5b-3bad-cbf39b0d6c2c@oracle.com> Message-ID: <1128d53d-f1a7-d560-9bcb-184feb3608dc@oracle.com> Hi Per, Looks good. /Erik On 2019-05-21 15:48, Per Liden wrote: > Tweaked the test slightly. > > Diff: http://cr.openjdk.java.net/~pliden/8224185/webrev.1-diff > Full: http://cr.openjdk.java.net/~pliden/8224185/webrev.1 > > /Per > > On 5/20/19 11:59 AM, Erik ?sterlund wrote: >> Hi Per and Thomas, >> >> Looks good to me. I also would not like to see a JVM flag for this. >> We have a quite high bar before we introduce new JVM flags, to avoid >> JVM flag hell. Especially for policies that are likely to change as >> the algorithm evolves, such that the flag might become useless. And >> this did not seem to warrant a flag yet. >> >> Thanks, >> /Erik >> >> On 2019-05-20 11:44, Per Liden wrote: >>> Hi Thomas, >>> >>> On 5/20/19 11:34 AM, Thomas Schatzl wrote: >>>> Hi Per, >>>> >>>> On Mon, 2019-05-20 at 10:11 +0200, Per Liden wrote: >>>>> Start a GC cycle if the amount of free memory is 5% or less. This is >>>>> a preventive measure in the case where the application has a very >>>>> low allocation rate, such that the allocation rate rule doesn't >>>>> trigger, but the amount of free memory is still slowly but surely >>>>> heading towards zero. In this situation, we should start a GC cycle >>>>> to avoid a potential allocation stall later. >>>>> >>>>> Testing: New jtreg test added. Currently running tier 1-6. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8224185 >>>>> Webrev: http://cr.openjdk.java.net/~pliden/8224185/webrev.0 >>>>> >>>> >>>> ?? just pointing out the obvious: this means that ZGC will potentially >>>> never use 5-Epsilon% of the heap. >>>> >>>> I do not recommend such a threshold being a fixed, compiled in >>>> constant >>>> size: while I understand that you want to have as few options as >>>> possible with ZGC, 5% of 100GB are still 5GB which may be better used >>>> for use in the application in some cases. >>>> >>>> I.e. G1 has a similar (10%) threshold, and we've run in a few >>>> situations where this has been too much and unneeded, particularly >>>> with >>>> large heaps. >>>> Now I know that ZGC pauses are by far less intrusive than G1's, but >>>> still the concurrent marking has its cost. Making it configurable >>>> would >>>> allow people getting the last bit of performance out of it if they >>>> want. >>> >>> The situation is slightly different for ZGC. This rule typically >>> kicks in when the allocation rate is very low. If the allocation >>> rate is very high, we typically collect a lot earlier than 95% full >>> heap anyway. >>> >>> Regarding a flag. We've discussed this within the team, and decided >>> to not have one for now. It can be added later if it turns out to be >>> a need for it. >>> >>> cheers, >>> Per >> From aph at redhat.com Tue May 21 14:48:46 2019 From: aph at redhat.com (Andrew Haley) Date: Tue, 21 May 2019 15:48:46 +0100 Subject: [aarch64-port-dev ] RFR: 8224187 Refactor arraycopy_prologue to allow ZGC read barriers on arraycopy In-Reply-To: References: <3ac5ea67-039b-cae8-f50d-5fe289255ae0@redhat.com> <1d9569d1-520e-2083-c7c6-acd1492732a3@oracle.com> <0ba5a6ac-40d5-7acf-a93a-44e8aaa690f3@oracle.com> Message-ID: On 5/21/19 11:22 AM, Stuart Monteith wrote: > I've reopened JDK-8224187, as 8223240 is another discussion. I > don't have committer's writes to push commit 8224187. Do you want me to do it? -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From stuart.monteith at linaro.org Tue May 21 14:54:42 2019 From: stuart.monteith at linaro.org (Stuart Monteith) Date: Tue, 21 May 2019 15:54:42 +0100 Subject: [aarch64-port-dev ] RFR: 8224187 Refactor arraycopy_prologue to allow ZGC read barriers on arraycopy In-Reply-To: References: <3ac5ea67-039b-cae8-f50d-5fe289255ae0@redhat.com> <1d9569d1-520e-2083-c7c6-acd1492732a3@oracle.com> <0ba5a6ac-40d5-7acf-a93a-44e8aaa690f3@oracle.com> Message-ID: If you don't mind, please. Is there anything in addition you need from my end? On Tue, 21 May 2019 at 15:48, Andrew Haley wrote: > > On 5/21/19 11:22 AM, Stuart Monteith wrote: > > I've reopened JDK-8224187, as 8223240 is another discussion. I > > don't have committer's writes to push commit 8224187. > > Do you want me to do it? > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > https://keybase.io/andrewhaley > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From zgu at redhat.com Tue May 21 16:45:33 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 21 May 2019 12:45:33 -0400 Subject: RFR(XXS) 8224495: Shenandoah: Do not rescan code roots in final mark pause if it is not degenerated GC Message-ID: <6d007baa-98c8-42c1-6cc2-109db197444d@redhat.com> As comments in code stated, it only guards against degenerated GC, that bypasses concurrent cycle. So, let's only re-scan code roots if degenerated GC is in progress. Bug: https://bugs.openjdk.java.net/browse/JDK-8224495 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224495/webrev.00/ Test: hotspot_gc_shenandoah (fastdebug and release) Thanks, -Zhengyu From shade at redhat.com Tue May 21 16:51:42 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 21 May 2019 18:51:42 +0200 Subject: RFR(XXS) 8224495: Shenandoah: Do not rescan code roots in final mark pause if it is not degenerated GC In-Reply-To: <6d007baa-98c8-42c1-6cc2-109db197444d@redhat.com> References: <6d007baa-98c8-42c1-6cc2-109db197444d@redhat.com> Message-ID: <59744575-b42d-c939-06a4-76c6f94e1654@redhat.com> On 5/21/19 6:45 PM, Zhengyu Gu wrote: > As comments in code stated, it only guards against degenerated GC, that bypasses concurrent cycle. > So, let's only re-scan code roots if degenerated GC is in progress. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8224495 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224495/webrev.00/ Local var "heap" is already available there. Looks okay otherwise. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From kim.barrett at oracle.com Tue May 21 18:02:43 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 21 May 2019 14:02:43 -0400 Subject: RFR: 8224167: Refactor PtrQueue completed buffer processing In-Reply-To: <24f7865e-64be-1676-718f-72fdbd5e150a@redhat.com> References: <219A1332-768F-4A7A-94A1-582F5BF10C8C@oracle.com> <24f7865e-64be-1676-718f-72fdbd5e150a@redhat.com> Message-ID: <70DC3050-D6F4-4AB2-BE84-24F567D4A8E7@oracle.com> > On May 21, 2019, at 4:35 AM, Aleksey Shipilev wrote: > > On 5/20/19 11:15 PM, Kim Barrett wrote: >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8224167/open.00/ > > Shenandoah parts look good. Thanks. > They do not seem to affect performance in adverse way. I expected this to be pretty much performance neutral; it avoids some wasted effort, but the benefit is pretty tiny. > Let me understand: SATBMarkQueue::handle_completed_buffer does reply either empty buffer (after full > filtering/enqueueing), or semi-full buffer if filters were applied and it still decided not to > enqueue it. Correct. I was hoping the description for the pure virtual in PtrQueue was sufficiently clear. From kim.barrett at oracle.com Tue May 21 18:03:04 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 21 May 2019 14:03:04 -0400 Subject: RFR: 8224167: Refactor PtrQueue completed buffer processing In-Reply-To: <43c83061a512dc99f4900b1fd4ac9250339e7a4e.camel@oracle.com> References: <219A1332-768F-4A7A-94A1-582F5BF10C8C@oracle.com> <43c83061a512dc99f4900b1fd4ac9250339e7a4e.camel@oracle.com> Message-ID: > On May 21, 2019, at 5:05 AM, Thomas Schatzl wrote: > > Hi, > > On Mon, 2019-05-20 at 17:15 -0400, Kim Barrett wrote: >> Please review this change to how PtrQueue handles completed (full) >> buffers. The code has been refactored to have a new virtual function >> [...] >> >> Shenandoah is affected by this refactoring, and has been updated >> accordingly, but I've only very lightly tested it. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8224167 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8224167/open.00/ >> >> Testing: >> mach5 tier1-5 >> Local (linux-x64) hs-tier1 with shenandoah included. >> > > thanks for this cleanup. Looks good. > > Thomas Thanks. From zgu at redhat.com Tue May 21 20:50:08 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 21 May 2019 16:50:08 -0400 Subject: RFR(XXS) 8224508: Shenandoah: Need to update thread roots in final mark for piggyback ref update cycle Message-ID: The thread roots may still contain forwarded oops. Bug: https://bugs.openjdk.java.net/browse/JDK-8224508 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224508/webrev.00/ Test: hotspot_gc_shenandoah (fastdebug and release) with fixed verifier Thanks, -Zhengyu From zgu at redhat.com Tue May 21 20:57:58 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 21 May 2019 16:57:58 -0400 Subject: RFR(XXS) 8224525: Shenandoah: Eliminate shenandoah verifier's side-effects Message-ID: <9a96d219-252b-e581-ae15-c4bad21c3dbb@redhat.com> Since switching to LRB, verifier has side-effects of evacuating oops, if it is ran during evacuation phase. The patch resets gc_state before LRB can be triggered by verifier, and restore gc_state when it is done. Bug: https://bugs.openjdk.java.net/browse/JDK-8224525 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224525/webrev.00/ Test: hotspot_gc_shenandoah with -XX:+ShenandoahVerify (fastdebug and release) Thanks, -Zhengyu From shade at redhat.com Tue May 21 21:10:02 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 21 May 2019 23:10:02 +0200 Subject: RFR(XXS) 8224508: Shenandoah: Need to update thread roots in final mark for piggyback ref update cycle In-Reply-To: References: Message-ID: <26ae4960-1f09-6965-e77e-56d76cadecdf@redhat.com> On 5/21/19 10:50 PM, Zhengyu Gu wrote: > The thread roots may still contain forwarded oops. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8224508 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224508/webrev.00/ Looks okay. Indent is not correct at L1509 here, please fix before pushing. 1508 if (has_forwarded_objects()) { 1509 // Degen may be caused by failed evacuation of roots 1510 if (is_degenerated_gc_in_progress()) { -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Tue May 21 21:23:05 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 21 May 2019 23:23:05 +0200 Subject: RFR(XXS) 8224525: Shenandoah: Eliminate shenandoah verifier's side-effects In-Reply-To: <9a96d219-252b-e581-ae15-c4bad21c3dbb@redhat.com> References: <9a96d219-252b-e581-ae15-c4bad21c3dbb@redhat.com> Message-ID: <1020a3c9-3e30-a7c8-a0e7-09807aaae5df@redhat.com> On 5/21/19 10:57 PM, Zhengyu Gu wrote: > Since switching to LRB, verifier has side-effects of evacuating oops, if it is ran during evacuation > phase. > > The patch resets gc_state before LRB can be triggered by verifier, and restore gc_state when it is > done. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8224525 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224525/webrev.00/ > > Test: > ? hotspot_gc_shenandoah with -XX:+ShenandoahVerify (fastdebug and release) The idea looks good. Let's put the comment like: 672 // Deactivate barriers temporarily: Verifier wants plain heap accesses 673 ShenandoahGCStateResetter resetter; You can common the use of ShenandoahHeap::heap() here (in fact you can probably just pull it into the ShenandoahGCStateResetter const field): 618 ~ShenandoahGCStateResetter() { 619 ShenandoahHeap::heap()->_gc_state.set(_gc_state); 620 assert(ShenandoahHeap::heap()->gc_state() == _gc_state, "Should be restored"); 621 } -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed May 22 07:25:51 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 22 May 2019 09:25:51 +0200 Subject: RFR(XXS) 8224525: Shenandoah: Eliminate shenandoah verifier's side-effects In-Reply-To: <9a96d219-252b-e581-ae15-c4bad21c3dbb@redhat.com> References: <9a96d219-252b-e581-ae15-c4bad21c3dbb@redhat.com> Message-ID: <804f9b18-8204-b1c0-5406-e96af34c2412@redhat.com> Where exactly does verification interfere with evacuation-in-progress? Because I don't see it. verify_before_evacuation() is done before activating the flag, and verify_after_evacuation() is done after clearing the flag. Roman > Since switching to LRB, verifier has side-effects of evacuating oops, if > it is ran during evacuation phase. > > The patch resets gc_state before LRB can be triggered by verifier, and > restore gc_state when it is done. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8224525 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224525/webrev.00/ > > Test: > ? hotspot_gc_shenandoah with -XX:+ShenandoahVerify (fastdebug and release) > > > Thanks, > > -Zhengyu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Wed May 22 07:29:50 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 22 May 2019 09:29:50 +0200 Subject: RFR(XXS) 8224525: Shenandoah: Eliminate shenandoah verifier's side-effects In-Reply-To: <804f9b18-8204-b1c0-5406-e96af34c2412@redhat.com> References: <9a96d219-252b-e581-ae15-c4bad21c3dbb@redhat.com> <804f9b18-8204-b1c0-5406-e96af34c2412@redhat.com> Message-ID: On 5/22/19 9:25 AM, Roman Kennke wrote: > Where exactly does verification interfere with evacuation-in-progress? > Because I don't see it. verify_before_evacuation() is done before > activating the flag, and verify_after_evacuation() is done after > clearing the flag. There is also ShenandoahVerifier::verify_during_evacuation(). But anyhow, we want to skip any barriers when doing verification, as we'd like to see the unaided view of the heap. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed May 22 07:37:37 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 22 May 2019 09:37:37 +0200 Subject: RFR(XXS) 8224525: Shenandoah: Eliminate shenandoah verifier's side-effects In-Reply-To: References: <9a96d219-252b-e581-ae15-c4bad21c3dbb@redhat.com> <804f9b18-8204-b1c0-5406-e96af34c2412@redhat.com> Message-ID: >> Where exactly does verification interfere with evacuation-in-progress? >> Because I don't see it. verify_before_evacuation() is done before >> activating the flag, and verify_after_evacuation() is done after >> clearing the flag. > > There is also ShenandoahVerifier::verify_during_evacuation(). But anyhow, we want to skip any > barriers when doing verification, as we'd like to see the unaided view of the heap. Ok, that makes sense. Good with me. Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed May 22 07:38:15 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 22 May 2019 09:38:15 +0200 Subject: RFR(XXS) 8224508: Shenandoah: Need to update thread roots in final mark for piggyback ref update cycle In-Reply-To: References: Message-ID: <45244187-136e-6ed2-9aab-a30561356599@redhat.com> Ok. Thanks, Roman > The thread roots may still contain forwarded oops. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8224508 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224508/webrev.00/ > > Test: > ? hotspot_gc_shenandoah (fastdebug and release) with fixed verifier > > Thanks, > > -Zhengyu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Wed May 22 11:31:05 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 22 May 2019 07:31:05 -0400 Subject: RFR(XXS) 8224525: Shenandoah: Eliminate shenandoah verifier's side-effects In-Reply-To: <1020a3c9-3e30-a7c8-a0e7-09807aaae5df@redhat.com> References: <9a96d219-252b-e581-ae15-c4bad21c3dbb@redhat.com> <1020a3c9-3e30-a7c8-a0e7-09807aaae5df@redhat.com> Message-ID: <13791027-0f87-2ee3-bc0c-93d2c71c2301@redhat.com> > The idea looks good. > > Let's put the comment like: > > 672 // Deactivate barriers temporarily: Verifier wants plain heap accesses > 673 ShenandoahGCStateResetter resetter; > > You can common the use of ShenandoahHeap::heap() here (in fact you can probably just pull it into > the ShenandoahGCStateResetter const field): > > 618 ~ShenandoahGCStateResetter() { > 619 ShenandoahHeap::heap()->_gc_state.set(_gc_state); > 620 assert(ShenandoahHeap::heap()->gc_state() == _gc_state, "Should be restored"); > 621 } Updated: http://cr.openjdk.java.net/~zgu/JDK-8224525/webrev.01/ Okay now? Thanks, -Zhengyu > > -Aleksey > From rkennke at redhat.com Wed May 22 11:35:54 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 22 May 2019 13:35:54 +0200 Subject: RFR: JDK-8224584: Shenandoah: Eliminate forwarding pointer word Message-ID: <2e90069e-94a5-a863-c80d-1441d3a55f8e@redhat.com> Until now, Shenandoah had the 'limitation' that it required an extra word per object to keep the forwarding pointer. It either points to itself (most of the time) or to the forwarded object when evacuated. The forwarding pointer is updated atomically using CAS. This is the heart of Shenandoah's evacuation protocol. With LRB, we have the opportunity to eliminate that extra word. The insight is that we really don't need the old copy of an object anymore and we can store the forwarding pointer in the mark word of it. We would distinguish a valid mark word from a forwarding pointer by checking the lowest 2 bits: if they are 0b11 then it's forwarded, otherwise it's not. 0b11 is not a valid combination of bits otherwise. This is the same protocol that is used by other GCs. The evacuation protocol changes slightly to CAS into the mark word instead of the extra word. But the basic mechanics are the same. The extra decoding does not (significantly) affect performance because it's only done in the mid/slow-path of the LRB. (As opposed to earlier attempts at it where it'd have to be in the read-barrier.) The full-GC needs some significant reworking because we used to store the forwarding ptr in the extra slot. Storing it in mark-word in full-gc is destructive because there are no from/to-space copies. Therfore we need to stash the mark-word when it's not trivial. This is pretty much what other GCs (G1, parallel, CMS) do, and we re-use the shared infrastructure for this. I've run SPECjvm2008. Performance benefits vary between 0 and ~+10%. SPECjbb2015 shows +~12% improvement in throughput and +~9% improvement in latency. Bug: https://bugs.openjdk.java.net/browse/JDK-8224584 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8224584/webrev.00/ Testing: The change has undergone intensive testing, reviews and bug-fixes in shenandoah/jdk repository. We have run several workloads on it, some of them many times. I've run the usual hotspot_gc_shenandoah on it many times, both on x86 and aarch64. There is one outstanding C2/matcher issue currently under review that causes (rare but fatal) failures with this change: https://bugs.openjdk.java.net/browse/JDK-8224580 https://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2019-May/033838.html Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Wed May 22 12:59:11 2019 From: per.liden at oracle.com (Per Liden) Date: Wed, 22 May 2019 14:59:11 +0200 Subject: RFR: 8224185: ZGC: Introduce "High Usage" rule In-Reply-To: <1128d53d-f1a7-d560-9bcb-184feb3608dc@oracle.com> References: <0e87a5ac-3a38-4301-5762-58d0d7fa1d94@oracle.com> <06c6cff49119376145dafc56bc2e493ee726b333.camel@oracle.com> <5790ee54-17e8-9699-c288-5c0c6184e845@oracle.com> <9f129198-29cd-5f5b-3bad-cbf39b0d6c2c@oracle.com> <1128d53d-f1a7-d560-9bcb-184feb3608dc@oracle.com> Message-ID: <1e616350-bb45-620d-6b77-42720e099a21@oracle.com> Thanks Erik! /Per On 05/21/2019 04:00 PM, Erik ?sterlund wrote: > Hi Per, > > Looks good. > > /Erik > > On 2019-05-21 15:48, Per Liden wrote: >> Tweaked the test slightly. >> >> Diff: http://cr.openjdk.java.net/~pliden/8224185/webrev.1-diff >> Full: http://cr.openjdk.java.net/~pliden/8224185/webrev.1 >> >> /Per >> >> On 5/20/19 11:59 AM, Erik ?sterlund wrote: >>> Hi Per and Thomas, >>> >>> Looks good to me. I also would not like to see a JVM flag for this. >>> We have a quite high bar before we introduce new JVM flags, to avoid >>> JVM flag hell. Especially for policies that are likely to change as >>> the algorithm evolves, such that the flag might become useless. And >>> this did not seem to warrant a flag yet. >>> >>> Thanks, >>> /Erik >>> >>> On 2019-05-20 11:44, Per Liden wrote: >>>> Hi Thomas, >>>> >>>> On 5/20/19 11:34 AM, Thomas Schatzl wrote: >>>>> Hi Per, >>>>> >>>>> On Mon, 2019-05-20 at 10:11 +0200, Per Liden wrote: >>>>>> Start a GC cycle if the amount of free memory is 5% or less. This is >>>>>> a preventive measure in the case where the application has a very >>>>>> low allocation rate, such that the allocation rate rule doesn't >>>>>> trigger, but the amount of free memory is still slowly but surely >>>>>> heading towards zero. In this situation, we should start a GC cycle >>>>>> to avoid a potential allocation stall later. >>>>>> >>>>>> Testing: New jtreg test added. Currently running tier 1-6. >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8224185 >>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8224185/webrev.0 >>>>>> >>>>> >>>>> just pointing out the obvious: this means that ZGC will potentially >>>>> never use 5-Epsilon% of the heap. >>>>> >>>>> I do not recommend such a threshold being a fixed, compiled in >>>>> constant >>>>> size: while I understand that you want to have as few options as >>>>> possible with ZGC, 5% of 100GB are still 5GB which may be better used >>>>> for use in the application in some cases. >>>>> >>>>> I.e. G1 has a similar (10%) threshold, and we've run in a few >>>>> situations where this has been too much and unneeded, particularly >>>>> with >>>>> large heaps. >>>>> Now I know that ZGC pauses are by far less intrusive than G1's, but >>>>> still the concurrent marking has its cost. Making it configurable >>>>> would >>>>> allow people getting the last bit of performance out of it if they >>>>> want. >>>> >>>> The situation is slightly different for ZGC. This rule typically >>>> kicks in when the allocation rate is very low. If the allocation >>>> rate is very high, we typically collect a lot earlier than 95% full >>>> heap anyway. >>>> >>>> Regarding a flag. We've discussed this within the team, and decided >>>> to not have one for now. It can be added later if it turns out to be >>>> a need for it. >>>> >>>> cheers, >>>> Per >>> > From shade at redhat.com Wed May 22 13:07:17 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 22 May 2019 15:07:17 +0200 Subject: RFR(XXS) 8224525: Shenandoah: Eliminate shenandoah verifier's side-effects In-Reply-To: <13791027-0f87-2ee3-bc0c-93d2c71c2301@redhat.com> References: <9a96d219-252b-e581-ae15-c4bad21c3dbb@redhat.com> <1020a3c9-3e30-a7c8-a0e7-09807aaae5df@redhat.com> <13791027-0f87-2ee3-bc0c-93d2c71c2301@redhat.com> Message-ID: <28b8b84d-1d2e-3daf-50fc-a07080124c9a@redhat.com> On 5/22/19 1:31 PM, Zhengyu Gu wrote: >> The idea looks good. >> >> Let's put the comment like: >> >> ? 672?? // Deactivate barriers temporarily: Verifier wants plain heap accesses >> ? 673?? ShenandoahGCStateResetter resetter; >> >> You can common the use of ShenandoahHeap::heap() here (in fact you can probably just pull it into >> the ShenandoahGCStateResetter const field): >> >> ? 618?? ~ShenandoahGCStateResetter() { >> ? 619???? ShenandoahHeap::heap()->_gc_state.set(_gc_state); >> ? 620???? assert(ShenandoahHeap::heap()->gc_state() == _gc_state, "Should be restored"); >> ? 621?? } > > Updated: http://cr.openjdk.java.net/~zgu/JDK-8224525/webrev.01/ > > Okay now? Yup, thumbs up. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Wed May 22 13:39:37 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 22 May 2019 15:39:37 +0200 Subject: RFR: JDK-8224584: Shenandoah: Eliminate forwarding pointer word In-Reply-To: <2e90069e-94a5-a863-c80d-1441d3a55f8e@redhat.com> References: <2e90069e-94a5-a863-c80d-1441d3a55f8e@redhat.com> Message-ID: On 5/22/19 1:35 PM, Roman Kennke wrote: > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8224584/webrev.00/ I looked at this patch multiple times over the last weeks, and it still looks good to me. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rwestrel at redhat.com Wed May 22 14:09:52 2019 From: rwestrel at redhat.com (Roland Westrelin) Date: Wed, 22 May 2019 16:09:52 +0200 Subject: RFR: JDK-8224584: Shenandoah: Eliminate forwarding pointer word In-Reply-To: <2e90069e-94a5-a863-c80d-1441d3a55f8e@redhat.com> References: <2e90069e-94a5-a863-c80d-1441d3a55f8e@redhat.com> Message-ID: <87v9y2k2bz.fsf@redhat.com> > http://cr.openjdk.java.net/~rkennke/JDK-8224584/webrev.00/ C2 changes look good. Roland. From per.liden at oracle.com Thu May 23 09:57:20 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 23 May 2019 11:57:20 +0200 Subject: RFR: JDK-8214527 AArch64: ZGC for Aarch64 In-Reply-To: References: <2abfb1d9-5235-e55f-6d90-ba8640cb197c@oracle.com> Message-ID: <1f9e4e96-26b3-8529-7025-6afd76d1ccc9@oracle.com> Hi Stuart, On 5/21/19 3:44 PM, Stuart Monteith wrote: > Hi Per, > Thanks for taking a look at that. > > I'll wait until ZGC on x86_64 isn't experimental. When that happens, > if aarch64 isn't ready for production use, I'd expect we'd make the > definition of "UseZGC" conditional, something like: > > ---- > #if defined(AARCH64) > #define USEZGC_COND(type, name, flag, descr) > experimental((type),(name),(flag),(descr)) > #else > #define USEZGC_COND(type, name, flag, descr) > product((type),(name),(flag),(descr)) > #endif > > USEZGC_COND(bool, UseZGC, false, \ > "Use the Z garbage collector") \ > ---- > > That'll need more thought, but I'd like a general solution that could > be applied more widely if, indeed, that were even necessary. Sounds good. > > I've done the refactoring of z_aarch64.ad as you suggested here: > http://cr.openjdk.java.net/~smonteith/8214527/webrev.2/ In z_aarch64.ad, you shouldn't need the "#if INCLUDE_ZGC" stuff, as this should never be compiled if ZGC is disabled, right?. Also it looks like the code inside ins_encode can be collapsed into a single line, like: z_load_barrier_slow_reg(_masm, $dst$$Register, $mem$$base$$Register, $mem$$index, $mem$$scale, $mem$$disp, false /* weak */); > > Apart from the system calls, I agree with you about the deduplication. > I presume there is no expectation for there to be a SPARC linux port, > or other architectures were there might be a desire for a different > implementation. We don't need to cater for that now. If/when that day comes, we can adjust what's shared and what's not, if there's a need to. A more general question. What kind of testing are you doing of this, and will you be continuously doing it going forward? Perhaps monitoring ZGC performance on aarch64 too? I don't expect a lot of aarch64-specific issues to pop up, but since aarch64 isn't part of the normal platforms Oracle tests/monitors we will likely not catch such issues. cheers, Per > > BR, > Stuart > > On Tue, 21 May 2019 at 12:05, Per Liden wrote: >> >> Hi Stuart, >> >> On 05/17/2019 04:28 PM, Stuart Monteith wrote: >>> Hello, >>> This patch implements ZGC for AArch64. Unlike the previous set of >>> patches, this duplicates the x86 multimapping scheme rather than using >>> the Aarch64 TBI feature, where the top 8 bits are ignored by the >>> hardware. I didn't measure any conclusive performance problems using >>> multimapping. It would necessitate changes to SA, JNI, and wasn't >>> going to be compatible with Graal or JVMCI in general until they were >>> also able to switch to 64-bit literals dynamically. In addition, there >>> are up and coming features in the AArch64 architecture that exploit >>> the TBI area that we should keep open, such as MTE (Memory Tagging >>> Extensions) that open the door to lightweight hardware ASAN. With MTE >>> using 4 of the 8 bits, and ZGC and MTE perhaps needing more than 4 >>> bits, future conflict looks likely. >>> >>> JTreg testing hasn't demonstrated any significant differences from x86 >>> ZGC in my testing. >>> >>> The behaviour should be the same as is expected from x86_64, including >>> for the behaviour with command line arguments. Currently ZGC requires >>> -XX:+UnlockExperimentalVMOptions, that should be the case for AArch64 >>> this release. If x86_64 were to do have ZGC designated as not >>> experimental, we must ensure it still applies for ZGC on AArch64. >> >> I don't think there's a good way to declare the a flag product on one >> platform, and experimental on another. But I guess >> zArguments::initialize_platform() on aarch64 could check for and >> warn/exit if UnlockExperimentalVMOptions is false. >> >>> >>> The associated bug is: >>> https://bugs.openjdk.java.net/browse/JDK-8214527 >>> The patchset is: >>> http://cr.openjdk.java.net/~smonteith/8214527/webrev.0/ >> >> src/hotspot/cpu/aarch64/gc/z/z_aarch64.ad >> ----------------------------------------- >> I'd suggest you break out the duplicate parts of these into a function, >> like we have done in z_x86.ad. >> >> >> src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingFile_linux_aarch64.cpp >> src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingFile_linux_aarch64.hpp >> src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingPath_linux_aarch64.cpp >> src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingPath_linux_aarch64.hpp >> src/hotspot/os_cpu/linux_aarch64/gc/z/zGlobals_linux_aarch64.cpp >> src/hotspot/os_cpu/linux_aarch64/gc/z/zGlobals_linux_aarch64.hpp >> src/hotspot/os_cpu/linux_aarch64/gc/z/zPhysicalMemoryBacking_linux_aarch64.cpp >> src/hotspot/os_cpu/linux_aarch64/gc/z/zPhysicalMemoryBacking_linux_aarch64.hpp >> >> -------------------------------------------------------------------- >> I'd would like to deduplicate most of these at some point, but I'm ok >> with doing that after this has been merged, as we might need/want to >> slice this slightly differently (to cater for the different syscall >> numbers etc). >> >> >> cheers, >> Per >> >>> >>> There is a patch contributed by Roland Westerlin required for fixing >>> memory barriers, >>> http://cr.openjdk.java.net/~smonteith/zgc-mm/20190430/membar/ >>> I'll follow up with a separate RFR patch to address that. >>> >>> Thanks, >>> Stuart >>> From martin.doerr at sap.com Thu May 23 10:21:55 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 23 May 2019 10:21:55 +0000 Subject: Assertion failure on PPC64 after 8200545: Improve filter for enqueued deferred cards Message-ID: Hi Thomas, we observe sporadically failing assertion on PPC64: assert(region_attr.needs_remset_update() == hr_obj->rem_set()->is_tracked()) failed: State flag indicating remset tracking disagrees (false) with actual remembered set (true) for region 62 with region: | 62|0x00000000f3e00000, 0x00000000f3f00000, 0x00000000f3f00000|100%| O| |TAMS 0x00000000f3f00000, 0x00000000f3e00000| Complete (This pattern has shown up twice, just with a different heap region, once on linuxppc64le and once on AIX.) Is the assertion too strict? I guess we could allow false positives of region_attr.needs_remset_update(), right? Or do you have an idea about a real problem like missing memory barriers? Best regards, Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: hs_err_pid13585.log Type: application/octet-stream Size: 123653 bytes Desc: hs_err_pid13585.log URL: From shade at redhat.com Thu May 23 10:40:46 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 23 May 2019 12:40:46 +0200 Subject: RFR(XS) 8224115: Shenandoah: Eliminate RWLock that protects recorded nmethod data array In-Reply-To: <25e4be93-8496-3c6b-127e-30d2ef9fc4e8@redhat.com> References: <3f736be4-bf6f-1559-f8fe-66902ce97c9f@redhat.com> <25e4be93-8496-3c6b-127e-30d2ef9fc4e8@redhat.com> Message-ID: On 5/20/19 1:26 PM, Zhengyu Gu wrote: > Split the original patch, moved root processing changes that deal with traversal GC problem into > JDK-8224179. > > Updated webrev: http://cr.openjdk.java.net/~zgu/JDK-8224115/webrev.01/ Looks fine, except: What is this removal? It does not seem related: 248 ShenandoahAllCodeRootsIterator ShenandoahCodeRoots::iterator() { 249 return ShenandoahAllCodeRootsIterator(); 250 } 251 252 ShenandoahCsetCodeRootsIterator ShenandoahCodeRoots::cset_iterator() { 253 return ShenandoahCsetCodeRootsIterator(); 254 } -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Thu May 23 12:17:08 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 23 May 2019 14:17:08 +0200 Subject: RFR: JDK-8224667: Shenandoah: Post-LRB cleanup Message-ID: <7e1bb712-cefa-877d-a21b-fd7b10c0c3c5@redhat.com> We've got some leftover mentions of write-barrier in our code. We better clean that up and change to corresponding load-reference-barrier mentions. Also, some arraycopy code that previously named things 'READ-*' and 'WRITE-' is now more apppropriately named 'RESOLVE*' and 'EVAC*'. Bug: https://bugs.openjdk.java.net/browse/JDK-8224667 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8224667/webrev.00/ Testing: hotspot_gc_shenandoah Ok? Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From thomas.schatzl at oracle.com Thu May 23 12:26:58 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 23 May 2019 14:26:58 +0200 Subject: Assertion failure on PPC64 after 8200545: Improve filter for enqueued deferred cards In-Reply-To: References: Message-ID: <345bd70dd4bbe4a97f618100fe4a0d742298cc36.camel@oracle.com> Hi, On Thu, 2019-05-23 at 10:21 +0000, Doerr, Martin wrote: > Hi Thomas, > > we observe sporadically failing assertion on PPC64: > assert(region_attr.needs_remset_update() == hr_obj->rem_set()- > >is_tracked()) failed: State flag indicating remset tracking > disagrees (false) with actual remembered set (true) for region 62 > with region: > > 62|0x00000000f3e00000, 0x00000000f3f00000, > > 0x00000000f3f00000|100%| O| |TAMS 0x00000000f3f00000, > > 0x00000000f3e00000| Complete > > (This pattern has shown up twice, just with a different heap region, > once on linuxppc64le and once on AIX.) Do you happen to have the other stacktrace and region information as above too? The situation we are in "should not happen" :(, see below why. > Is the assertion too strict? I guess we could allow false positives > of region_attr.needs_remset_update(), right? We could allow errorneous "true" values (just adds another card to scan by concurrent refinement), not wrong "false" ones though. These may cause missing remembered sets. > Or do you have an idea about a real problem like missing memory > barriers? Let me explain a bit why this particular situation is weird after (more than) a few minutes looking over the code. The needs_remembered_set_update region attribute table is set in one of two locations: - at the start of GC, before actual evacuation (in that stack trace we are already multiple parallel phases beyond that), just duplicating the values from HeapRegionRemSet::_state (in G1CollectedHeap::register_regions_with_region_attr()). We set the region_attr.needs_remembered_set_update() for all committed regions at that time there. I.e. impossible to get a wrong value due to that imho due to various full barriers from that time to the crashing time. - during copying, when allocating new regions. The remembered set state (HeapRegionRemSet::_state) and the corresponding region attribute table entry are set one after another during allocation of a new region in G1CollectedHeap::new_gc_alloc_region(), that is called by G1AllocRegion::new_alloc_region_and_allocate(). So they should correspond, although exactly the situation you describe may occur. (Note that this means that any memory visibility issue when setting these existed before 8200545, because previously G1 simply checked the value of HeapRegionRemSet::_state. The default value of needs_remembered_set_update of the region_attr elements is "false" though; I just saw that there is an implicit type coercion in the G1HeapRegionAttr constructor going on, but I assume that "false" is "0" anyway. That is something that could be tried, i.e. set the default to true and adapt the assert. That would mean we are not initializing some region attr correctly. Or something is setting wrong "false" values in there... probably not very useful...). However, by default, newly allocated old gen regions never get assigned a remembered set to update (in G1RemSetTrackingPolicy::update_at_allocate() we should use the r- >is_old() path, setting its new remembered set state to empty. Any newly allocated region is a "Free" region. "Free" region's remembered set state is "Empty" too (in HeapRegion::hr_clear() we call HeapRegionRemSet::clear_locked() which calls HeapRegionRemSet::set_state_empty() when freeing them). So if this were a newly allocated region, it must have had an "Empty" remembered set in the region information dump. ------------------ The situation is different for newly allocated survivor regions, remembered sets get allocated as "Complete". However I believe that is fine too because: - if that object we are scanning (i.e. "p") is in some survivor region: we do not call enqueue_card_if_tracked() from these regions at all as the code in G1ScanEvacuatedObjClosure::do_oop_work() shows. In this case the thread local _scanning_in_young == True. - that object we are scanning (i.e. "p") is in some old region (must be for the crashing case according to logs), i.e. we are scanning an object recently promoted into some old region; if the "obj" reference is in any kind of old region, the region attribute should have been stable (which it apparently is not) because either - that region was allocated during that gc. It's HeapRegionRemSet::_state (and region attribute) must have been "Empty" due to above. - that region was allocated before gc: it may have any HeapRegionRemSet::_state, but since we update that before the parallel phases, it must be stable. ------------------- So unfortunately I do not know right away how region_attr.needs_remembered_set_update() could get inconsistent with the corresponding HeapRegionRemSet::_state in this context. At least it failed before missing some remembered set entry.... Can you file a bug and assign it to me? I will think about it some more. Or maybe you can spot the problem(s) in my thinking? Thanks, Thomas From bob.vandette at oracle.com Thu May 23 13:15:44 2019 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 23 May 2019 09:15:44 -0400 Subject: RFR: 8222252 - Java ergonomics limits heap to 128GB with disabled compressed oops In-Reply-To: <3b410422-68b0-a143-78f5-dd0a141d1353@oracle.com> References: <46E18910-9D4E-4219-803C-A883F48AD43B@oracle.com> <772C07C5-F2E7-4878-A94C-A99927ED204A@oracle.com> <3b410422-68b0-a143-78f5-dd0a141d1353@oracle.com> Message-ID: <23C2B57E-A0CA-4E44-B83D-D3728F179740@oracle.com> > On May 16, 2019, at 11:34 PM, David Holmes wrote: > > Hi Bob, > > On 16/05/2019 12:52 am, Bob Vandette wrote: >> I?ve updated the webrev based on your comments. >> http://cr.openjdk.java.net/~bobv/8222252/webrev.02/ >> Changes: >> 1. MaxRAM is updated if fractional flags cause us to use OS memory size for heap selection. >> 2. If MaxRAM is specified with the other fractional based flags, we will use MaxRAM as >> upper limit instead of OS memory size. > > That seems fine, but here: > > 1799 if (FLAG_IS_ERGO(UseCompressedOops) && use_os_mem_limit) { > > Shouldn't this also be checking for FLAG_IS_DEFAULT(MaxRAM) ? Or better that check would fold into the use_os_mem_limit determination: > > bool use_os_mem_limit = (!FLAG_IS_DEFAULT(MaxRAMPercentage) || > !FLAG_IS_DEFAULT(MaxRAMFraction) || > !FLAG_IS_DEFAULT(MinRAMPercentage) || > !FLAG_IS_DEFAULT(MinRAMFraction) || > !FLAG_IS_DEFAULT(InitialRAMPercentage) || > !FLAG_IS_DEFAULT(InitialRAMFraction)) && > FLAG_IS_DEFAULT(MaxRAM); > if (use_os_mem_limit) { > phys_mem = os::physical_memory(); > FLAG_SET_ERGO(uint64_t, MaxRAM, (uint64_t)phys_mem); > } else { > phys_mem = FLAG_IS_DEFAULT(MaxRAM) ? MIN2(os::physical_memory(), (julong)MaxRAM) > : (julong)MaxRAM; > } > Yes, you are correct. I?ll add MaxRAM to the use_os_mem_limit check. This is the behavior I documented in the CSR. "Also, If any of these flags including MaxRAM are specified on the command line, the selected Java heap size will not be limited by the compressed oop limit.? I?ll update the patch provided in the CSR as well. Can you take a look and review the CSR for this change? Thanks, Bob. > ? > > Thanks, > David > ----- > >> Here?s a proposed CSR for this behavioral change. >> https://bugs.openjdk.java.net/browse/JDK-8223957 >> Bob. >>> On May 14, 2019, at 9:26 AM, Thomas Schatzl > wrote: >>> >>> Hi, >>> >>> On Mon, 2019-05-13 at 10:03 -0400, Bob Vandette wrote: >>>> Please review this suggested fix for correcting Heap size selection >>>> when -XX:{Max,Min,Initial)RAMPercentage and their Factional >>>> equivalent options are used. >>>> >>>> There are two bugs filed that are related to this issue (8222252 and >>>> 8213175) >>>> >>>> SUMMARY of FIX: >>>> The fix corrects what I believe is an incorrect implementation of the >>>> *Percentage/*Fraction options. These options should not have caused >>>> the heap size to be fractionally based on MaxRAM or limited by >>>> MaxRAM. They instead should be based on the host memory. This is >>>> what I assume was meant by ?real memory?. >>>> >>>> product(double, MaxRAMPercentage, >>>> 25.0, \ >>>> "Maximum percentage of real memory used for maximum heap >>>> size") \ >>>> range(0.0, 100.0) >>>> >>>> When these options are selected, they should take precedence over >>>> UseCompressedOops unless UseCompressedOops is also specified on the >>>> command line. >>>> >>>> >>>> WEBREV: >>>> http://cr.openjdk.java.net/~bobv/8222252/webrev.01 >>> >>> - this is a style preference: I would assign the result of the >>> condition in the if (arguments.cpp:1721+) to the initialization of >>> use_os_mem_limit directly instead of first setting it to false, and >>> then to true again if needed. Feel free to ignore. >>> >>> - as David pointed out, this change needs a CSR. >>> >>> - the interaction between MaxRAM and MaxRAMPercentage is a bit unclear; >>> if they are independent as implemented now, then the update of one >>> should probably update the other. Or the VM should fail if they >>> disagree? >>> >>> If they are dependent, if David suggests (i.e. MaxRAMPercentage uses >>> MaxRAM as maximum real memory if set; this is what I would tend to >>> prefer), then not of course. >>> >>> - there should be regressions tests verifying this in cases where >>> possible. >>> >>> I.e. it should be doable to get current os::physical_memory using e.g. >>> whitebox and then run a few tests verifying the results of MaxHeapSize >>> and coop mode. >>> >>> Particularly the interaction between MaxRAM, MaxRAMPercentage and coops >>> mode would be interesting. >>> >>> Thanks, >>> Thomas >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From zgu at redhat.com Thu May 23 13:24:28 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 23 May 2019 09:24:28 -0400 Subject: RFR(XS) 8224115: Shenandoah: Eliminate RWLock that protects recorded nmethod data array In-Reply-To: References: <3f736be4-bf6f-1559-f8fe-66902ce97c9f@redhat.com> <25e4be93-8496-3c6b-127e-30d2ef9fc4e8@redhat.com> Message-ID: <57e16c8c-a5d3-c6e8-d656-1a83016bfa91@redhat.com> On 5/23/19 6:40 AM, Aleksey Shipilev wrote: > On 5/20/19 1:26 PM, Zhengyu Gu wrote: >> Split the original patch, moved root processing changes that deal with traversal GC problem into >> JDK-8224179. >> >> Updated webrev: http://cr.openjdk.java.net/~zgu/JDK-8224115/webrev.01/ > > Looks fine, except: > > What is this removal? It does not seem related: > > 248 ShenandoahAllCodeRootsIterator ShenandoahCodeRoots::iterator() { > 249 return ShenandoahAllCodeRootsIterator(); > 250 } > 251 > 252 ShenandoahCsetCodeRootsIterator ShenandoahCodeRoots::cset_iterator() { > 253 return ShenandoahCsetCodeRootsIterator(); > 254 } Can make separate patch. Both methods seem to depend on inline behavior to avoid fatal error, where they may free ShenandoahParallelCodeHeapIterator inside ShenandoahParallelCodeCacheIterator twice due to copy constructors. Thanks, -Zhengyu > > -Aleksey > From thomas.schatzl at oracle.com Thu May 23 14:37:38 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 23 May 2019 16:37:38 +0200 Subject: RFR: 8222252 - Java ergonomics limits heap to 128GB with disabled compressed oops In-Reply-To: <772C07C5-F2E7-4878-A94C-A99927ED204A@oracle.com> References: <46E18910-9D4E-4219-803C-A883F48AD43B@oracle.com> <772C07C5-F2E7-4878-A94C-A99927ED204A@oracle.com> Message-ID: <4ff682442a6fc032b79944c607c075d3c954052a.camel@oracle.com> Hi, On Wed, 2019-05-15 at 10:52 -0400, Bob Vandette wrote: > I?ve updated the webrev based on your comments. > > http://cr.openjdk.java.net/~bobv/8222252/webrev.02/ > > Changes: > > 1. MaxRAM is updated if fractional flags cause us to use OS memory > size for heap selection. > 2. If MaxRAM is specified with the other fractional based flags, we > will use MaxRAM as > upper limit instead of OS memory size. > > > Here?s a proposed CSR for this behavioral change. > > https://bugs.openjdk.java.net/browse/JDK-8223957 Please provide a jtreg test checking resulting heap sizing in the various situations. Not all combinations are interesting of course, e.g. I think the *Fraction flags can be ignored in these tests as they are obsolete and others. The value of os::physical_memory() could be provided by an appropriate whitebox method if not already done. Thanks, Thomas From bob.vandette at oracle.com Thu May 23 14:57:00 2019 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 23 May 2019 10:57:00 -0400 Subject: RFR: 8222252 - Java ergonomics limits heap to 128GB with disabled compressed oops In-Reply-To: <4ff682442a6fc032b79944c607c075d3c954052a.camel@oracle.com> References: <46E18910-9D4E-4219-803C-A883F48AD43B@oracle.com> <772C07C5-F2E7-4878-A94C-A99927ED204A@oracle.com> <4ff682442a6fc032b79944c607c075d3c954052a.camel@oracle.com> Message-ID: Is the GC team ok with the new behavior? Bob. > On May 23, 2019, at 10:37 AM, Thomas Schatzl wrote: > > Hi, > > On Wed, 2019-05-15 at 10:52 -0400, Bob Vandette wrote: >> I?ve updated the webrev based on your comments. >> >> http://cr.openjdk.java.net/~bobv/8222252/webrev.02/ >> >> Changes: >> >> 1. MaxRAM is updated if fractional flags cause us to use OS memory >> size for heap selection. >> 2. If MaxRAM is specified with the other fractional based flags, we >> will use MaxRAM as >> upper limit instead of OS memory size. >> >> >> Here?s a proposed CSR for this behavioral change. >> >> https://bugs.openjdk.java.net/browse/JDK-8223957 > > Please provide a jtreg test checking resulting heap sizing in the > various situations. Not all combinations are interesting of course, > e.g. I think the *Fraction flags can be ignored in these tests as they > are obsolete and others. > > The value of os::physical_memory() could be provided by an appropriate > whitebox method if not already done. > > Thanks, > Thomas > > From zgu at redhat.com Thu May 23 15:01:35 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 23 May 2019 11:01:35 -0400 Subject: RFR(XS) 8224679: Shenandoah: Make ShenandoahParallelCodeCacheIterator noncopyable Message-ID: <911cd8f8-9a5f-4bbd-1e81-444568a3d587@redhat.com> Copies may cause its member '_iters' to be freed multiple times. Bug: https://bugs.openjdk.java.net/browse/JDK-8224679 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224679/webrev.00/ Test: hotspot_gc_shenandoah (fastdebug and release) Thanks, -Zhengyu From rkennke at redhat.com Thu May 23 15:03:30 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 23 May 2019 17:03:30 +0200 Subject: RFR: JDK-8224667: Shenandoah: Post-LRB cleanup In-Reply-To: <7e1bb712-cefa-877d-a21b-fd7b10c0c3c5@redhat.com> References: <7e1bb712-cefa-877d-a21b-fd7b10c0c3c5@redhat.com> Message-ID: <87e74561-ab6c-3438-6940-f0d21a3cf119@redhat.com> Somehow I managed to upload a totally unrelated webrev. Here's the correct one: http://cr.openjdk.java.net/~rkennke/JDK-8224667/webrev.01/ Roman > We've got some leftover mentions of write-barrier in our code. We > better clean that up and change to corresponding load-reference-barrier > mentions. Also, some arraycopy code that previously named things > 'READ-*' and 'WRITE-' is now more apppropriately named 'RESOLVE*' and > 'EVAC*'. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8224667 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8224667/webrev.00/ > > Testing: hotspot_gc_shenandoah > > Ok? > > Roman > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Thu May 23 15:10:24 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 23 May 2019 17:10:24 +0200 Subject: RFR(XS) 8224679: Shenandoah: Make ShenandoahParallelCodeCacheIterator noncopyable In-Reply-To: <911cd8f8-9a5f-4bbd-1e81-444568a3d587@redhat.com> References: <911cd8f8-9a5f-4bbd-1e81-444568a3d587@redhat.com> Message-ID: <4355ba79-ffb1-95c7-a4dd-9b1ca26638f4@redhat.com> Are there no users of iterator() and cset_iterator()? Or which part am I missing? Roman > Copies may cause its member '_iters' to be freed multiple times. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8224679 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224679/webrev.00/ > > Test: > ? hotspot_gc_shenandoah (fastdebug and release) > > Thanks, > > -Zhengyu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Thu May 23 15:18:31 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 23 May 2019 11:18:31 -0400 Subject: RFR(XS) 8224679: Shenandoah: Make ShenandoahParallelCodeCacheIterator noncopyable In-Reply-To: <4355ba79-ffb1-95c7-a4dd-9b1ca26638f4@redhat.com> References: <911cd8f8-9a5f-4bbd-1e81-444568a3d587@redhat.com> <4355ba79-ffb1-95c7-a4dd-9b1ca26638f4@redhat.com> Message-ID: <2c741499-e568-f476-5102-b9ec419b86bd@redhat.com> On 5/23/19 11:10 AM, Roman Kennke wrote: > Are there no users of iterator() and cset_iterator()? Or which part am I > missing? We no longer use them since SRP refactor. -Zhengyu > > Roman > > >> Copies may cause its member '_iters' to be freed multiple times. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8224679 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224679/webrev.00/ >> >> Test: >> ? hotspot_gc_shenandoah (fastdebug and release) >> >> Thanks, >> >> -Zhengyu > From rkennke at redhat.com Thu May 23 15:19:30 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 23 May 2019 17:19:30 +0200 Subject: RFR(XS) 8224679: Shenandoah: Make ShenandoahParallelCodeCacheIterator noncopyable In-Reply-To: <2c741499-e568-f476-5102-b9ec419b86bd@redhat.com> References: <911cd8f8-9a5f-4bbd-1e81-444568a3d587@redhat.com> <4355ba79-ffb1-95c7-a4dd-9b1ca26638f4@redhat.com> <2c741499-e568-f476-5102-b9ec419b86bd@redhat.com> Message-ID: >> Are there no users of iterator() and cset_iterator()? Or which part am I >> missing? > > We no longer use them since SRP refactor. If we don't use them, why bother? Why not remove it instead? Roman > > -Zhengyu > >> >> Roman >> >> >>> Copies may cause its member '_iters' to be freed multiple times. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8224679 >>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224679/webrev.00/ >>> >>> Test: >>> ?? hotspot_gc_shenandoah (fastdebug and release) >>> >>> Thanks, >>> >>> -Zhengyu >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Thu May 23 15:21:47 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 23 May 2019 11:21:47 -0400 Subject: RFR(XS) 8224679: Shenandoah: Make ShenandoahParallelCodeCacheIterator noncopyable In-Reply-To: References: <911cd8f8-9a5f-4bbd-1e81-444568a3d587@redhat.com> <4355ba79-ffb1-95c7-a4dd-9b1ca26638f4@redhat.com> <2c741499-e568-f476-5102-b9ec419b86bd@redhat.com> Message-ID: <3d298330-f860-2941-232c-5b662ce7fb22@redhat.com> On 5/23/19 11:19 AM, Roman Kennke wrote: >>> Are there no users of iterator() and cset_iterator()? Or which part am I >>> missing? >> >> We no longer use them since SRP refactor. > > If we don't use them, why bother? Why not remove it instead? Yes, we are removing them diff --git a/src/hotspot/share/gc/shenandoah/shenandoahCodeRoots.cpp b/src/hotspot/share/gc/shenandoah/shenandoahCodeRoots.cpp --- a/src/hotspot/share/gc/shenandoah/shenandoahCodeRoots.cpp +++ b/src/hotspot/share/gc/shenandoah/shenandoahCodeRoots.cpp @@ -245,14 +245,6 @@ } } -ShenandoahAllCodeRootsIterator ShenandoahCodeRoots::iterator() { - return ShenandoahAllCodeRootsIterator(); -} - -ShenandoahCsetCodeRootsIterator ShenandoahCodeRoots::cset_iterator() { - return ShenandoahCsetCodeRootsIterator(); -} - -Zhengyu > > Roman > >> >> -Zhengyu >> >>> >>> Roman >>> >>> >>>> Copies may cause its member '_iters' to be freed multiple times. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8224679 >>>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224679/webrev.00/ >>>> >>>> Test: >>>> ?? hotspot_gc_shenandoah (fastdebug and release) >>>> >>>> Thanks, >>>> >>>> -Zhengyu >>> > From zgu at redhat.com Thu May 23 15:29:16 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 23 May 2019 11:29:16 -0400 Subject: RFR(XS) 8224115: Shenandoah: Eliminate RWLock that protects recorded nmethod data array In-Reply-To: <57e16c8c-a5d3-c6e8-d656-1a83016bfa91@redhat.com> References: <3f736be4-bf6f-1559-f8fe-66902ce97c9f@redhat.com> <25e4be93-8496-3c6b-127e-30d2ef9fc4e8@redhat.com> <57e16c8c-a5d3-c6e8-d656-1a83016bfa91@redhat.com> Message-ID: <27523810-aaaf-1bd8-f1ba-9804463c503e@redhat.com> On 5/23/19 9:24 AM, Zhengyu Gu wrote: > > > On 5/23/19 6:40 AM, Aleksey Shipilev wrote: >> On 5/20/19 1:26 PM, Zhengyu Gu wrote: >>> Split the original patch, moved root processing changes that deal >>> with traversal GC problem into >>> JDK-8224179. >>> >>> Updated webrev: http://cr.openjdk.java.net/~zgu/JDK-8224115/webrev.01/ >> >> Looks fine, except: >> >> What is this removal? It does not seem related: >> >> ? 248 ShenandoahAllCodeRootsIterator ShenandoahCodeRoots::iterator() { >> ? 249?? return ShenandoahAllCodeRootsIterator(); >> ? 250 } >> ? 251 >> ? 252 ShenandoahCsetCodeRootsIterator >> ShenandoahCodeRoots::cset_iterator() { >> ? 253?? return ShenandoahCsetCodeRootsIterator(); >> ? 254 } > > Can make separate patch. Both methods seem to depend on inline behavior > to avoid fatal error, where they may free > ShenandoahParallelCodeHeapIterator inside > ShenandoahParallelCodeCacheIterator twice due to copy constructors. Addressed by JDK-8224679: Shenandoah: Make ShenandoahParallelCodeCacheIterator noncopyable Updated webrev: http://cr.openjdk.java.net/~zgu/JDK-8224115/webrev.02/ Reran hotspot_gc_shenandoah tests. Thanks, -Zhengyu > > Thanks, > > -Zhengyu > >> >> -Aleksey >> From rkennke at redhat.com Thu May 23 15:33:57 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 23 May 2019 17:33:57 +0200 Subject: RFR(XS) 8224679: Shenandoah: Make ShenandoahParallelCodeCacheIterator noncopyable In-Reply-To: <3d298330-f860-2941-232c-5b662ce7fb22@redhat.com> References: <911cd8f8-9a5f-4bbd-1e81-444568a3d587@redhat.com> <4355ba79-ffb1-95c7-a4dd-9b1ca26638f4@redhat.com> <2c741499-e568-f476-5102-b9ec419b86bd@redhat.com> <3d298330-f860-2941-232c-5b662ce7fb22@redhat.com> Message-ID: <720a23c2-6108-539c-e841-45ba87c676f8@redhat.com> Duh. Of course :-) Looks good. Thanks! Roman > On 5/23/19 11:19 AM, Roman Kennke wrote: >>>> Are there no users of iterator() and cset_iterator()? Or which part >>>> am I >>>> missing? >>> >>> We no longer use them since SRP refactor. >> >> If we don't use them, why bother? Why not remove it instead? > > Yes, we are removing them > > > diff --git a/src/hotspot/share/gc/shenandoah/shenandoahCodeRoots.cpp > b/src/hotspot/share/gc/shenandoah/shenandoahCodeRoots.cpp > --- a/src/hotspot/share/gc/shenandoah/shenandoahCodeRoots.cpp > +++ b/src/hotspot/share/gc/shenandoah/shenandoahCodeRoots.cpp > @@ -245,14 +245,6 @@ > ?? } > ?} > > -ShenandoahAllCodeRootsIterator ShenandoahCodeRoots::iterator() { > -? return ShenandoahAllCodeRootsIterator(); > -} > - > -ShenandoahCsetCodeRootsIterator ShenandoahCodeRoots::cset_iterator() { > -? return ShenandoahCsetCodeRootsIterator(); > -} > - > > -Zhengyu >> >> Roman >> >>> >>> -Zhengyu >>> >>>> >>>> Roman >>>> >>>> >>>>> Copies may cause its member '_iters' to be freed multiple times. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8224679 >>>>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224679/webrev.00/ >>>>> >>>>> Test: >>>>> ??? hotspot_gc_shenandoah (fastdebug and release) >>>>> >>>>> Thanks, >>>>> >>>>> -Zhengyu >>>> >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From martin.doerr at sap.com Thu May 23 17:13:46 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 23 May 2019 17:13:46 +0000 Subject: Assertion failure on PPC64 after 8200545: Improve filter for enqueued deferred cards In-Reply-To: <345bd70dd4bbe4a97f618100fe4a0d742298cc36.camel@oracle.com> References: <345bd70dd4bbe4a97f618100fe4a0d742298cc36.camel@oracle.com> Message-ID: Hi Thomas, thanks for your explanations. I have created JDK-8224681. Feel free to edit it. Thanks, Martin > -----Original Message----- > From: Thomas Schatzl > Sent: Donnerstag, 23. Mai 2019 14:27 > To: Doerr, Martin > Cc: hotspot-gc-dev at openjdk.java.net; Reingruber, Richard > > Subject: Re: Assertion failure on PPC64 after 8200545: Improve filter for > enqueued deferred cards > > Hi, > > On Thu, 2019-05-23 at 10:21 +0000, Doerr, Martin wrote: > > Hi Thomas, > > > > we observe sporadically failing assertion on PPC64: > > assert(region_attr.needs_remset_update() == hr_obj->rem_set()- > > >is_tracked()) failed: State flag indicating remset tracking > > disagrees (false) with actual remembered set (true) for region 62 > > with region: > > > 62|0x00000000f3e00000, 0x00000000f3f00000, > > > 0x00000000f3f00000|100%| O| |TAMS 0x00000000f3f00000, > > > 0x00000000f3e00000| Complete > > > > (This pattern has shown up twice, just with a different heap region, > > once on linuxppc64le and once on AIX.) > > Do you happen to have the other stacktrace and region information as > above too? The situation we are in "should not happen" :(, see below > why. > > > Is the assertion too strict? I guess we could allow false positives > > of region_attr.needs_remset_update(), right? > > We could allow errorneous "true" values (just adds another card to scan > by concurrent refinement), not wrong "false" ones though. These may > cause missing remembered sets. > > > Or do you have an idea about a real problem like missing memory > > barriers? > > Let me explain a bit why this particular situation is weird after (more > than) a few minutes looking over the code. > > The needs_remembered_set_update region attribute table is set in one of > two locations: > > - at the start of GC, before actual evacuation (in that stack trace we > are already multiple parallel phases beyond that), just duplicating the > values from HeapRegionRemSet::_state (in > G1CollectedHeap::register_regions_with_region_attr()). > > We set the region_attr.needs_remembered_set_update() for all committed > regions at that time there. > > I.e. impossible to get a wrong value due to that imho due to various > full barriers from that time to the crashing time. > > - during copying, when allocating new regions. > > The remembered set state (HeapRegionRemSet::_state) and the > corresponding region attribute table entry are set one after another > during allocation of a new region in > G1CollectedHeap::new_gc_alloc_region(), that is called by > G1AllocRegion::new_alloc_region_and_allocate(). So they should > correspond, although exactly the situation you describe may occur. > > (Note that this means that any memory visibility issue when setting > these existed before 8200545, because previously G1 simply checked the > value of HeapRegionRemSet::_state. > > The default value of needs_remembered_set_update of the region_attr > elements is "false" though; I just saw that there is an implicit type > coercion in the G1HeapRegionAttr constructor going on, but I assume > that "false" is "0" anyway. That is something that could be tried, i.e. > set the default to true and adapt the assert. That would mean we are > not initializing some region attr correctly. Or something is setting > wrong "false" values in there... probably not very useful...). > > However, by default, newly allocated old gen regions never get assigned > a remembered set to update (in > G1RemSetTrackingPolicy::update_at_allocate() we should use the r- > >is_old() path, setting its new remembered set state to empty. Any > newly allocated region is a "Free" region. > "Free" region's remembered set state is "Empty" too (in > HeapRegion::hr_clear() we call HeapRegionRemSet::clear_locked() which > calls HeapRegionRemSet::set_state_empty() when freeing them). > > So if this were a newly allocated region, it must have had an "Empty" > remembered set in the region information dump. > > ------------------ > > The situation is different for newly allocated survivor regions, > remembered sets get allocated as "Complete". However I believe that is > fine too because: > > - if that object we are scanning (i.e. "p") is in some survivor region: > we do not call enqueue_card_if_tracked() from these regions at all as > the code in G1ScanEvacuatedObjClosure::do_oop_work() shows. In this > case the thread local _scanning_in_young == True. > > - that object we are scanning (i.e. "p") is in some old region (must be > for the crashing case according to logs), i.e. we are scanning an > object recently promoted into some old region; if the "obj" reference > is in any kind of old region, the region attribute should have been > stable (which it apparently is not) because either > - that region was allocated during that gc. It's > HeapRegionRemSet::_state (and region attribute) must have been "Empty" > due to above. > - that region was allocated before gc: it may have any > HeapRegionRemSet::_state, but since we update that before the parallel > phases, it must be stable. > > ------------------- > > So unfortunately I do not know right away how > region_attr.needs_remembered_set_update() could get inconsistent with > the corresponding HeapRegionRemSet::_state in this context. > > At least it failed before missing some remembered set entry.... > > Can you file a bug and assign it to me? I will think about it some > more. Or maybe you can spot the problem(s) in my thinking? > > Thanks, > Thomas > From shade at redhat.com Thu May 23 19:17:00 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 23 May 2019 21:17:00 +0200 Subject: RFR: JDK-8224667: Shenandoah: Post-LRB cleanup In-Reply-To: <87e74561-ab6c-3438-6940-f0d21a3cf119@redhat.com> References: <7e1bb712-cefa-877d-a21b-fd7b10c0c3c5@redhat.com> <87e74561-ab6c-3438-6940-f0d21a3cf119@redhat.com> Message-ID: On 5/23/19 5:03 PM, Roman Kennke wrote: > Somehow I managed to upload a totally unrelated webrev. Here's the > correct one: > > http://cr.openjdk.java.net/~rkennke/JDK-8224667/webrev.01/ *) Missed the spot (read the comments), shenandoahBarrierSet.inline.hpp: 271 case RESOLVE_BARRIER: 272 case EVAC_BARRIER: 273 // The write-barrier case cannot really happen. It's traversal-only and traversal 274 // doesn't currently use SATB. And even if it did, it would not be fatal to just do the normal RB here. No other mentions of "RB", "WB", "read barrier", "read-barrier", "write barrier", "write-barrier" in shenandoah/ folders? Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Thu May 23 19:28:16 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 23 May 2019 21:28:16 +0200 Subject: RFR(XS) 8224679: Shenandoah: Make ShenandoahParallelCodeCacheIterator noncopyable In-Reply-To: <911cd8f8-9a5f-4bbd-1e81-444568a3d587@redhat.com> References: <911cd8f8-9a5f-4bbd-1e81-444568a3d587@redhat.com> Message-ID: <10f5877f-79dd-471f-68e3-4af39e617b40@redhat.com> On 5/23/19 5:01 PM, Zhengyu Gu wrote: > Copies may cause its member '_iters' to be freed multiple times. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8224679 Please link the issue that made these redundant. > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224679/webrev.00/ Please match what ShenandoahRegionIterator does, notably, declare variables? I remember the absence of those tripped some compilers. // No implicit copying: iterators should be passed by reference to capture the state ShenandoahRegionIterator(const ShenandoahRegionIterator& that); ShenandoahRegionIterator& operator=(const ShenandoahRegionIterator& o); Otherwise good. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Thu May 23 19:32:55 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 23 May 2019 21:32:55 +0200 Subject: RFR(XS) 8224115: Shenandoah: Eliminate RWLock that protects recorded nmethod data array In-Reply-To: <27523810-aaaf-1bd8-f1ba-9804463c503e@redhat.com> References: <3f736be4-bf6f-1559-f8fe-66902ce97c9f@redhat.com> <25e4be93-8496-3c6b-127e-30d2ef9fc4e8@redhat.com> <57e16c8c-a5d3-c6e8-d656-1a83016bfa91@redhat.com> <27523810-aaaf-1bd8-f1ba-9804463c503e@redhat.com> Message-ID: <5c3f5a8f-683c-a629-8234-ffa48d9f3d6e@redhat.com> On 5/23/19 5:29 PM, Zhengyu Gu wrote: > Updated webrev: http://cr.openjdk.java.net/~zgu/JDK-8224115/webrev.02/ > > Reran hotspot_gc_shenandoah tests. Okay, looks good! -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Thu May 23 19:50:45 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 23 May 2019 21:50:45 +0200 Subject: RFR: JDK-8224667: Shenandoah: Post-LRB cleanup In-Reply-To: References: <7e1bb712-cefa-877d-a21b-fd7b10c0c3c5@redhat.com> <87e74561-ab6c-3438-6940-f0d21a3cf119@redhat.com> Message-ID: Ooops. http://cr.openjdk.java.net/~rkennke/JDK-8224667/webrev.02/ I've grepped for various keywords again, including the ones you listed. No match. Ok? Roman > On 5/23/19 5:03 PM, Roman Kennke wrote: >> Somehow I managed to upload a totally unrelated webrev. Here's the >> correct one: >> >> http://cr.openjdk.java.net/~rkennke/JDK-8224667/webrev.01/ > > *) Missed the spot (read the comments), shenandoahBarrierSet.inline.hpp: > > 271 case RESOLVE_BARRIER: > 272 case EVAC_BARRIER: > 273 // The write-barrier case cannot really happen. It's traversal-only and traversal > 274 // doesn't currently use SATB. And even if it did, it would not be fatal to just do the > normal RB here. > > No other mentions of "RB", "WB", "read barrier", "read-barrier", "write barrier", "write-barrier" in > shenandoah/ folders? > > Thanks, > -Aleksey > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Thu May 23 19:53:05 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 23 May 2019 21:53:05 +0200 Subject: RFR: JDK-8224667: Shenandoah: Post-LRB cleanup In-Reply-To: References: <7e1bb712-cefa-877d-a21b-fd7b10c0c3c5@redhat.com> <87e74561-ab6c-3438-6940-f0d21a3cf119@redhat.com> Message-ID: On 5/23/19 9:50 PM, Roman Kennke wrote: > Ooops. > > http://cr.openjdk.java.net/~rkennke/JDK-8224667/webrev.02/ > > I've grepped for various keywords again, including the ones you listed. > No match. > > Ok? OK then! -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From chengjingwei1 at huawei.com Fri May 24 08:02:10 2019 From: chengjingwei1 at huawei.com (chengjingwei (A)) Date: Fri, 24 May 2019 08:02:10 +0000 Subject: PING: RFR: 8217432: MetaspaceGC::_capacity_until_GC exceeds MaxMetaspaceSize Message-ID: It seems that it's not intending to obtain 4GB memory, but forcing overflow of capacity_until_GC, according to the comment in the testcase. In the previous versions, this wouldn't get a false value returned from MetaspaceGC::inc_capacity_until_GC, so the test used to pass. But now false is returned and the test is broken. I guess WB_IncMetaspaceCapacityUntilGC in whitebox.cpp needs to be repaired too, but I'm not sure. Do you have any suggestions? Thanks -----????----- ???: Yasumasa Suenaga [mailto:yasuenag at gmail.com] ????: 2019?5?21? 10:41 ???: chengjingwei (A) ??: hotspot-gc-dev at openjdk.java.net ??: Re: PING: RFR: 8217432: MetaspaceGC::_capacity_until_GC exceeds MaxMetaspaceSize Hi, I think the result which you saw is correct because 32 bit platform cannot obtain 4GB memory. IMHO you need to remove or modify this testcase. Thanks, Yasumasa 2019?5?21?(?) 10:50 chengjingwei (A) : > > By the way, the link of the patch that caused problems was https://bugs.openjdk.java.net/browse/JDK-8217432. > > The test would fail with 32-bit jdk even on 64-bit platforms. > > > > > Hi, > > > > > > After this patch was applied (since jdk8u212 and jdk-11.0.3, to be specific), we started to get test failures with jtreg. > > > The failed test case was jdk8u/hotspot/test/gc/metaspace/TestCapacityUntilGCWrapAround.java. > > > > > > It always fails on windows-32bit platform, and fails on linux-32bit if -XX:MaxMetaspaceSize is set. > > > > > > The failure message was: > > > java.lang.IllegalStateException: WB_IncMetaspaceCapacityUntilGC: > > could not increase capacity until GC due to contention with another > > thread > > > at sun.hotspot.WhiteBox.incMetaspaceCapacityUntilGC(Native > > Method) > > > at > > TestCapacityUntilGCWrapAround.main(TestCapacityUntilGCWrapAround.jav > > a:51) > > > > > > I checked the code, and found that > > MetaspaceGC::inc_capacity_until_GC was passed a value of > > 4G-page_size-1, > > > which caused addition overflow when computing new_value, and new_value always set to "align_size_down(max_uintx, Metaspace::commit_alignment())". > > > So when MaxMetaspaceSize defaults to or is set to a value less than > > it, the function will return false to > > WB_IncMetaspaceCapacityUntilGC, > > > which will then throw an Exception, causing the test failure. > > > > > > On win-32 platform, the default value of MaxMetaspaceSize is 4G - > > allocation_granularity, > > > which is 4G - 64K (according to > > https://devblogs.microsoft.com/oldnewthing/20031008-00/?p=42223), > > > this is very likely to be less than "new_value" and cause test failure. > > > > > > Do you have any idea how to fix this? > > From martin.doerr at sap.com Fri May 24 11:02:02 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 24 May 2019 11:02:02 +0000 Subject: Assertion failure on PPC64 after 8200545: Improve filter for enqueued deferred cards In-Reply-To: References: <345bd70dd4bbe4a97f618100fe4a0d742298cc36.camel@oracle.com> Message-ID: Hi Thomas, I've taken a 2nd look at the hs_err files. Seems like the GCTaskThread which runs into the assertion has seen hr_obj->rem_set()->is_tracked() == true but another thread concurrently sets r->rem_set()->set_state_complete() When the hs_err file gets printed, the region shows up as "Complete" (which means untracked). Does this make sense? In which scenario can this happen? Best regards, Martin > -----Original Message----- > From: hotspot-gc-dev On Behalf > Of Doerr, Martin > Sent: Donnerstag, 23. Mai 2019 19:14 > To: Thomas Schatzl > Cc: hotspot-gc-dev at openjdk.java.net > Subject: [CAUTION] RE: Assertion failure on PPC64 after 8200545: Improve > filter for enqueued deferred cards > > Hi Thomas, > > thanks for your explanations. I have created JDK-8224681. Feel free to edit it. > > Thanks, > Martin > > > > -----Original Message----- > > From: Thomas Schatzl > > Sent: Donnerstag, 23. Mai 2019 14:27 > > To: Doerr, Martin > > Cc: hotspot-gc-dev at openjdk.java.net; Reingruber, Richard > > > > Subject: Re: Assertion failure on PPC64 after 8200545: Improve filter for > > enqueued deferred cards > > > > Hi, > > > > On Thu, 2019-05-23 at 10:21 +0000, Doerr, Martin wrote: > > > Hi Thomas, > > > > > > we observe sporadically failing assertion on PPC64: > > > assert(region_attr.needs_remset_update() == hr_obj->rem_set()- > > > >is_tracked()) failed: State flag indicating remset tracking > > > disagrees (false) with actual remembered set (true) for region 62 > > > with region: > > > > 62|0x00000000f3e00000, 0x00000000f3f00000, > > > > 0x00000000f3f00000|100%| O| |TAMS 0x00000000f3f00000, > > > > 0x00000000f3e00000| Complete > > > > > > (This pattern has shown up twice, just with a different heap region, > > > once on linuxppc64le and once on AIX.) > > > > Do you happen to have the other stacktrace and region information as > > above too? The situation we are in "should not happen" :(, see below > > why. > > > > > Is the assertion too strict? I guess we could allow false positives > > > of region_attr.needs_remset_update(), right? > > > > We could allow errorneous "true" values (just adds another card to scan > > by concurrent refinement), not wrong "false" ones though. These may > > cause missing remembered sets. > > > > > Or do you have an idea about a real problem like missing memory > > > barriers? > > > > Let me explain a bit why this particular situation is weird after (more > > than) a few minutes looking over the code. > > > > The needs_remembered_set_update region attribute table is set in one of > > two locations: > > > > - at the start of GC, before actual evacuation (in that stack trace we > > are already multiple parallel phases beyond that), just duplicating the > > values from HeapRegionRemSet::_state (in > > G1CollectedHeap::register_regions_with_region_attr()). > > > > We set the region_attr.needs_remembered_set_update() for all committed > > regions at that time there. > > > > I.e. impossible to get a wrong value due to that imho due to various > > full barriers from that time to the crashing time. > > > > - during copying, when allocating new regions. > > > > The remembered set state (HeapRegionRemSet::_state) and the > > corresponding region attribute table entry are set one after another > > during allocation of a new region in > > G1CollectedHeap::new_gc_alloc_region(), that is called by > > G1AllocRegion::new_alloc_region_and_allocate(). So they should > > correspond, although exactly the situation you describe may occur. > > > > (Note that this means that any memory visibility issue when setting > > these existed before 8200545, because previously G1 simply checked the > > value of HeapRegionRemSet::_state. > > > > The default value of needs_remembered_set_update of the region_attr > > elements is "false" though; I just saw that there is an implicit type > > coercion in the G1HeapRegionAttr constructor going on, but I assume > > that "false" is "0" anyway. That is something that could be tried, i.e. > > set the default to true and adapt the assert. That would mean we are > > not initializing some region attr correctly. Or something is setting > > wrong "false" values in there... probably not very useful...). > > > > However, by default, newly allocated old gen regions never get assigned > > a remembered set to update (in > > G1RemSetTrackingPolicy::update_at_allocate() we should use the r- > > >is_old() path, setting its new remembered set state to empty. Any > > newly allocated region is a "Free" region. > > "Free" region's remembered set state is "Empty" too (in > > HeapRegion::hr_clear() we call HeapRegionRemSet::clear_locked() which > > calls HeapRegionRemSet::set_state_empty() when freeing them). > > > > So if this were a newly allocated region, it must have had an "Empty" > > remembered set in the region information dump. > > > > ------------------ > > > > The situation is different for newly allocated survivor regions, > > remembered sets get allocated as "Complete". However I believe that is > > fine too because: > > > > - if that object we are scanning (i.e. "p") is in some survivor region: > > we do not call enqueue_card_if_tracked() from these regions at all as > > the code in G1ScanEvacuatedObjClosure::do_oop_work() shows. In this > > case the thread local _scanning_in_young == True. > > > > - that object we are scanning (i.e. "p") is in some old region (must be > > for the crashing case according to logs), i.e. we are scanning an > > object recently promoted into some old region; if the "obj" reference > > is in any kind of old region, the region attribute should have been > > stable (which it apparently is not) because either > > - that region was allocated during that gc. It's > > HeapRegionRemSet::_state (and region attribute) must have been "Empty" > > due to above. > > - that region was allocated before gc: it may have any > > HeapRegionRemSet::_state, but since we update that before the parallel > > phases, it must be stable. > > > > ------------------- > > > > So unfortunately I do not know right away how > > region_attr.needs_remembered_set_update() could get inconsistent with > > the corresponding HeapRegionRemSet::_state in this context. > > > > At least it failed before missing some remembered set entry.... > > > > Can you file a bug and assign it to me? I will think about it some > > more. Or maybe you can spot the problem(s) in my thinking? > > > > Thanks, > > Thomas > > From leo.korinth at oracle.com Fri May 24 11:25:47 2019 From: leo.korinth at oracle.com (Leo Korinth) Date: Fri, 24 May 2019 13:25:47 +0200 Subject: RFR: 8224659: Parallel GC: Use WorkGang (1: PCRefProcTask) Message-ID: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> Hi, Here is the first patch in a proposed patch series of eight that removes gcTaskManager and uses the WorkGang abstraction instead. This is primary a refactoring to cleanup code for future enhancements. After this change all GCs will use the WorkGang abstraction. # General Change 1-3 removes pcTask, 4-6 psTask, and then eventually, at nr 7, task manager is completely removed. Change 8 then both removes and obsoletes two product flags. The last enhancement has a CSR attached to it dealing with the flag removal. More than two thousands lines are removed, 4 JVM flags are removed or obsoleted. The old thread code is completely removed. In general I have tried to keep the old structure, and just done smaller improvements in code. Performance numbers seems to indicate that we go from roughly no improvements up to more than 40% improvements (!). Regressions seems rare. Performance improvements are primary in the young collections. In this first patch the WorkGang is added, and in the seventh patch, the gcTaskManager is removed. In between, both implementations will be used for different parts. In the end I will push everything together so that we will not see two sets of threads at the same time (although it does work). # This Patch This first patch does two things: 1) Adds work gang functionality + adds WorkGang named _workers to parallelScavengeHeap + sets active workers in psParallelCompact.cpp and psScavenge.cpp. The corresponding functionality in gcTaskManager is done by gc_task_manager()->set_active_gang() 2) Creates a PCRefProcTask that replaces RefProcTaskProxy and StealMarkingTask. The StealMarkingTask is removed in the next patch as it is still used by other parts. Enhancement: https://bugs.openjdk.java.net/browse/JDK-8224659 Webrev: http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224659-Parallel-GC-Use-WorkGang-1-PCRefProcTask/ http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ Testing (on the whole patch series): mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC gc test suite Thanks, Leo From thomas.schatzl at oracle.com Fri May 24 12:03:20 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 24 May 2019 14:03:20 +0200 Subject: Assertion failure on PPC64 after 8200545: Improve filter for enqueued deferred cards In-Reply-To: References: <345bd70dd4bbe4a97f618100fe4a0d742298cc36.camel@oracle.com> Message-ID: Hi, On Fri, 2019-05-24 at 11:02 +0000, Doerr, Martin wrote: > Hi Thomas, > > I've taken a 2nd look at the hs_err files. > > Seems like the GCTaskThread which runs into the assertion has seen > hr_obj->rem_set()->is_tracked() == true > but another thread concurrently sets > r->rem_set()->set_state_complete() > > When the hs_err file gets printed, the region shows up as "Complete" > (which means untracked). > > Does this make sense? In which scenario can this happen? > Is r == hr_obj? If so, that seems impossible at least in the way I believe the code is written. Random thoughts below, beware... :) The only way I could imagine such an error would be if the compiler did something weird with writing the fields of the heap attribute table? I.e. it loads a machine word (containing four of those RegionAttr entries), modifies just one of the bytes, and writes back the whole word. I.e. then some concurrent reader might see inconsistent values that flip back and forth. I really doubt this is the case though. Particularly I assume that at least on ppc64/linux you also use gcc. Old regions' remembered set should never be changed during GC, and if they are set, they are set to Empty (from Empty)(HeapRegionRemSet::_state)/false(region_attr._needs_remembered_se t). Since the default value of region_attr._needs_remembered_set is also 0 (i.e. false), this is also a value-preserving write. Can you dump the remembered set states/heap region attributes after setting them in the G1CollectedHeap::register_regions_with_region_attr() method and comparing them with the information from the assert? You might want to check that the default value written to _needs_remembered_set in G1HeapRegionAttr in its constructor really ends up as 0/false. As mentioned there is a bool->uint8_t implicit type coercion. (Now I'm in paranoia mode :)) The only remembered sets that are changed from Empty to Complete are Survivor regions, as originally they were Free regions (which are "Empty" initially). Thanks, Thomas From stuart.monteith at linaro.org Fri May 24 12:06:27 2019 From: stuart.monteith at linaro.org (Stuart Monteith) Date: Fri, 24 May 2019 13:06:27 +0100 Subject: RFR: JDK-8214527 AArch64: ZGC for Aarch64 In-Reply-To: <1f9e4e96-26b3-8529-7025-6afd76d1ccc9@oracle.com> References: <2abfb1d9-5235-e55f-6d90-ba8640cb197c@oracle.com> <1f9e4e96-26b3-8529-7025-6afd76d1ccc9@oracle.com> Message-ID: Hello, Thanks again Per, I've clean that up: http://cr.openjdk.java.net/~smonteith/8214527/webrev.3/ My intention is to add a test loop for ZGC aarch64 once it has been merged. The performance monitoring would fit in with what we have internally just now. I appreciate the necessity for this. I've been testing with JTReg, JCStress, specJBB and I've been also been using Lucene, which has been reliable at finding issues. Of course, the more testing there is, the better. BR, Stuart On Thu, 23 May 2019 at 10:57, Per Liden wrote: > > Hi Stuart, > > On 5/21/19 3:44 PM, Stuart Monteith wrote: > > Hi Per, > > Thanks for taking a look at that. > > > > I'll wait until ZGC on x86_64 isn't experimental. When that happens, > > if aarch64 isn't ready for production use, I'd expect we'd make the > > definition of "UseZGC" conditional, something like: > > > > ---- > > #if defined(AARCH64) > > #define USEZGC_COND(type, name, flag, descr) > > experimental((type),(name),(flag),(descr)) > > #else > > #define USEZGC_COND(type, name, flag, descr) > > product((type),(name),(flag),(descr)) > > #endif > > > > USEZGC_COND(bool, UseZGC, false, \ > > "Use the Z garbage collector") \ > > ---- > > > > That'll need more thought, but I'd like a general solution that could > > be applied more widely if, indeed, that were even necessary. > > Sounds good. > > > > > I've done the refactoring of z_aarch64.ad as you suggested here: > > http://cr.openjdk.java.net/~smonteith/8214527/webrev.2/ > > In z_aarch64.ad, you shouldn't need the "#if INCLUDE_ZGC" stuff, as this > should never be compiled if ZGC is disabled, right?. Also it looks like > the code inside ins_encode can be collapsed into a single line, like: > > z_load_barrier_slow_reg(_masm, $dst$$Register, $mem$$base$$Register, > $mem$$index, $mem$$scale, $mem$$disp, false /* weak */); > > > > > Apart from the system calls, I agree with you about the deduplication. > > I presume there is no expectation for there to be a SPARC linux port, > > or other architectures were there might be a desire for a different > > implementation. > > We don't need to cater for that now. If/when that day comes, we can > adjust what's shared and what's not, if there's a need to. > > A more general question. What kind of testing are you doing of this, and > will you be continuously doing it going forward? Perhaps monitoring ZGC > performance on aarch64 too? I don't expect a lot of aarch64-specific > issues to pop up, but since aarch64 isn't part of the normal platforms > Oracle tests/monitors we will likely not catch such issues. > > cheers, > Per > > > > > BR, > > Stuart > > > > On Tue, 21 May 2019 at 12:05, Per Liden wrote: > >> > >> Hi Stuart, > >> > >> On 05/17/2019 04:28 PM, Stuart Monteith wrote: > >>> Hello, > >>> This patch implements ZGC for AArch64. Unlike the previous set of > >>> patches, this duplicates the x86 multimapping scheme rather than using > >>> the Aarch64 TBI feature, where the top 8 bits are ignored by the > >>> hardware. I didn't measure any conclusive performance problems using > >>> multimapping. It would necessitate changes to SA, JNI, and wasn't > >>> going to be compatible with Graal or JVMCI in general until they were > >>> also able to switch to 64-bit literals dynamically. In addition, there > >>> are up and coming features in the AArch64 architecture that exploit > >>> the TBI area that we should keep open, such as MTE (Memory Tagging > >>> Extensions) that open the door to lightweight hardware ASAN. With MTE > >>> using 4 of the 8 bits, and ZGC and MTE perhaps needing more than 4 > >>> bits, future conflict looks likely. > >>> > >>> JTreg testing hasn't demonstrated any significant differences from x86 > >>> ZGC in my testing. > >>> > >>> The behaviour should be the same as is expected from x86_64, including > >>> for the behaviour with command line arguments. Currently ZGC requires > >>> -XX:+UnlockExperimentalVMOptions, that should be the case for AArch64 > >>> this release. If x86_64 were to do have ZGC designated as not > >>> experimental, we must ensure it still applies for ZGC on AArch64. > >> > >> I don't think there's a good way to declare the a flag product on one > >> platform, and experimental on another. But I guess > >> zArguments::initialize_platform() on aarch64 could check for and > >> warn/exit if UnlockExperimentalVMOptions is false. > >> > >>> > >>> The associated bug is: > >>> https://bugs.openjdk.java.net/browse/JDK-8214527 > >>> The patchset is: > >>> http://cr.openjdk.java.net/~smonteith/8214527/webrev.0/ > >> > >> src/hotspot/cpu/aarch64/gc/z/z_aarch64.ad > >> ----------------------------------------- > >> I'd suggest you break out the duplicate parts of these into a function, > >> like we have done in z_x86.ad. > >> > >> > >> src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingFile_linux_aarch64.cpp > >> src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingFile_linux_aarch64.hpp > >> src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingPath_linux_aarch64.cpp > >> src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingPath_linux_aarch64.hpp > >> src/hotspot/os_cpu/linux_aarch64/gc/z/zGlobals_linux_aarch64.cpp > >> src/hotspot/os_cpu/linux_aarch64/gc/z/zGlobals_linux_aarch64.hpp > >> src/hotspot/os_cpu/linux_aarch64/gc/z/zPhysicalMemoryBacking_linux_aarch64.cpp > >> src/hotspot/os_cpu/linux_aarch64/gc/z/zPhysicalMemoryBacking_linux_aarch64.hpp > >> > >> -------------------------------------------------------------------- > >> I'd would like to deduplicate most of these at some point, but I'm ok > >> with doing that after this has been merged, as we might need/want to > >> slice this slightly differently (to cater for the different syscall > >> numbers etc). > >> > >> > >> cheers, > >> Per > >> > >>> > >>> There is a patch contributed by Roland Westerlin required for fixing > >>> memory barriers, > >>> http://cr.openjdk.java.net/~smonteith/zgc-mm/20190430/membar/ > >>> I'll follow up with a separate RFR patch to address that. > >>> > >>> Thanks, > >>> Stuart > >>> From thomas.schatzl at oracle.com Fri May 24 12:25:55 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 24 May 2019 14:25:55 +0200 Subject: RFR (S): 8223693: Memory wastage in size of per-region type buffers in GC Message-ID: <797ef3a1fbc70a84b66713d7ca272a7cb131bf72.camel@oracle.com> Hi all, can I have reviews for this small change in the constants for G1HeapRegionAttr::_type field, eliminating some confusion and minor memory wastage? Perf testing showed no measurable impact of the change. CR: https://bugs.openjdk.java.net/browse/JDK-8223693 Webrev: http://cr.openjdk.java.net/~tschatzl/8223693/webrev/ Testing: hs-tier1-5 in conjunction with other changes; perf testing showed no measurable impact Thanks, Thomas From thomas.schatzl at oracle.com Fri May 24 12:58:22 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 24 May 2019 14:58:22 +0200 Subject: RFR (XXL): 8213108: Improve work distribution during remembered set scan Message-ID: <18b673f2bccabfb2ef3fe4f1aa3bc0e94f078ce2.camel@oracle.com> Hi all, can I have reviews for this change to the way we scan cards from the various heap roots (remembered sets, log buffers, hcc) during garbage collection to improve performance? Instead of iterating over remembered sets of all regions in the current collection set increment, the threads trying to claim batches of cards, and only just before scanning a card try to detect whether we already scanned that card, first create a combined set of heap roots (i.e. cards to scan), and do work distribution/iteration over them by claiming areas within this combined set of heap roots. This is similar to other collectors that just scan the old gen card table. However G1 can do a little better, as we also collect approximate location of cards to scan (like in a second-level card table), and skip large areas guaranteed to not contain any interesting cards. Further, this allows G1 to more easily ignore them further on. Implementation wise, this combined heap roots set is materialized on the card table at this moment. This has the advantage that the card table is already allocated anyway, it is easy to modify concurrently, and the change itself does not add any overhead in clearing it later too. I.e. the number of regions to clean at the end is the same as before. This speeds up garbage collection in most cases significantly: - specjbb2015 critical-jops are improved by >12% in our testing in some setups - on the BigRAMTester microbenchmark[0], stressing remembered set scanning, maximum pause times are reduced by 40%+ for mixed gcs (i.e. almost halving these pause times), and 20% for normal young gcs. There are unfortunately some situations where the added, required heap roots merging phase will cause some significant regressions by just being there. This are the cases when total pause time is already very low (3-5ms), and there is not much to do at all during that root merging, still taking 0.1-0.3ms mainly to spin up and tear down worker threads. There are plans to fix this by e.g. doing pre-merging of parts of the heap roots for young-only collection. However the current remembered set implementation inhibits this; I plan to fix this shortcoming later in follow-ups of a remembered set data structure rewrite ([1]). I believe the change is worth this small (in absolute terms) regression for cases that other collectors might handle better anyway at this time, because as soon as pause times are in the range of 10ms (or even lower if there are *some* heap roots), and there are actually some heap roots to merge, the additional phase quickly amortizes itself. The change not only improves if you run the VM with many threads, but also some low-thread count runs showed similar improvements vs. without the change. This change significantly alters the log output for the garbage collection; there is a comment in the CR [2] that describes it in detail. For reviewing, I recommend reading the comment at the start of g1RemSet.cpp [3] to get acquainted with the algorithm and the new terminology I used. I will prepare a release note about this. There will be some minor follow-up changes that micro-optimize the code a bit (e.g. [4]). They were separated to keep complexity down a bit. I would really like to get this into jdk13 if possible. :) CR: https://bugs.openjdk.java.net/browse/JDK-8213108 Webrev: http://cr.openjdk.java.net/~tschatzl/8213108/webrev/ Testing: hs-tier1-5(many times), hs-tier6-8, many perf test runs Thanks, Thomas [0] https://bugs.openjdk.java.net/browse/JDK-8152438 [1] https://bugs.openjdk.java.net/browse/JDK-8017163 [2] https://bugs.openjdk.java.net/browse/JDK-8213108?focusedCommentId=14266190&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14266190 [3] http://cr.openjdk.java.net/~tschatzl/8213108/webrev/src/hotspot/share/gc/g1/g1RemSet.cpp.frames.html [4] https://bugs.openjdk.java.net/browse/JDK-8224741 From thomas.schatzl at oracle.com Fri May 24 13:24:32 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 24 May 2019 15:24:32 +0200 Subject: RFR (S): 8223162: Increase ergonomics for Sparse PRT entry sizing Message-ID: <5a7f7eb20525ee5b2ff775dc69ffa7b18820d870.camel@oracle.com> Hi all, can I have reviews for this small tweak for the ergonomic sizing of Sparse PRT entries? This change, instead of linearly increasing the number of entries per sparse prt, increases it logarithmically like the region sizes do. In effect, it increases the usage of sparse PRTs ("the data structure holding remembered set entries for a particular region", "Per-Region- Table"), and so decreases the usage of fine and coarse PRTs. Which is more aligned with the intent of this first-level PRT. The impact is that we decrease overall memory usage significantly for heavy users of remembered sets (numbers follow), decrease the number of coarsenings (and fine PRTs) that need to be traversed during GC, increasing performance significantly as well. In BigRAMTester, which somewhat resembles some of the large big data applications, I have seen reductions of maximum remembered set size to 1/3rd of previous value. E.g. BigRAMTester 20G heap: Region Avg Pause Max Pause Avg RS size Max RS size Size [MB] [ms] [ms] [kB] [kB] baseline 32M 727.8 1077 914029 1966080 changes 32M 694.5 1082 621725 1186816 I.e. with "tuned" region size of 32M (to only get 640 regions total) to decrease remembered set overhead, this change decreases remembered set size by ~35%. No impact on pause times though (A drop from ~10% of java heap to 6%). There is no remembered set coarsening. The situation changes if you let G1 ergonomically determine number of regions. In this case there are 2560 regions, more than the coarsening threshold. Region Avg Pause Max Pause Avg RS size Max RS size Size [MB] [ms] [ms] [kB] [kB] baseline default 1045 1714 1532460 3028992 changes default 731.2 1152 516574 926720 G1 max pause time drops by 33%, and remembered set size by 66% :) (using 8M regions, i.e. 32 entries instead of 16 in a sparse PRT). (These improvements are in addition to the changes in JDK-8213108, but should also apply without) Note that in both cases the optimized value for the number of sparse prt entries for this particular application would probably be a bit higher. However I chose this heuristic (and the drop from 16 to 8 initial size of the hash table) to keep the impact on applications that do not use many remembered sets low (i.e. in practice zero). This heuristic could certainly be optimized (e.g. by sampling of actual remembered sets which is easy in conjunction with JDK-8213108; make the number of sparse prt entries dependent on heap size/#regions), but given the plan to re-implement the remembered sets (JDK-8017163) where we would redo that work, I think this heuristic is a good tradeoff between effort and results. CR: https://bugs.openjdk.java.net/browse/JDK-8223162 Webrev: http://cr.openjdk.java.net/~tschatzl/8223162/webrev/ Testing: lots of remembered set size measurements on lots of applications, some hs-tier1-5 runs together with other changes. Thanks, Thomas From thomas.schatzl at oracle.com Fri May 24 13:28:11 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 24 May 2019 15:28:11 +0200 Subject: RFR (S): 8223162: Improve ergonomics for Sparse PRT entry sizing In-Reply-To: <5a7f7eb20525ee5b2ff775dc69ffa7b18820d870.camel@oracle.com> References: <5a7f7eb20525ee5b2ff775dc69ffa7b18820d870.camel@oracle.com> Message-ID: <5fc78faad0cf437ac74547dcb02c810e9e2e76f7.camel@oracle.com> Fixed CR and mail subject :) Thanks, Thomas On Fri, 2019-05-24 at 15:24 +0200, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this small tweak for the ergonomic sizing of > Sparse PRT entries? > > This change, instead of linearly increasing the number of entries per > sparse prt, increases it logarithmically like the region sizes do. > > In effect, it increases the usage of sparse PRTs ("the data structure > holding remembered set entries for a particular region", "Per-Region- > Table"), and so decreases the usage of fine and coarse PRTs. Which is > more aligned with the intent of this first-level PRT. > > The impact is that we decrease overall memory usage significantly for > heavy users of remembered sets (numbers follow), decrease the number > of coarsenings (and fine PRTs) that need to be traversed during GC, > increasing performance significantly as well. > > In BigRAMTester, which somewhat resembles some of the large big data > applications, I have seen reductions of maximum remembered set size > to 1/3rd of previous value. > > E.g. BigRAMTester 20G heap: > > Region Avg Pause Max Pause Avg RS size Max RS size > Size [MB] [ms] [ms] [kB] [kB] > baseline 32M 727.8 1077 914029 1966080 > changes 32M 694.5 1082 621725 1186816 > > I.e. with "tuned" region size of 32M (to only get 640 regions total) > to decrease remembered set overhead, this change decreases remembered > set size by ~35%. No impact on pause times though (A drop from ~10% > of java heap to 6%). There is no remembered set coarsening. > > The situation changes if you let G1 ergonomically determine number of > regions. In this case there are 2560 regions, more than the > coarsening threshold. > > Region Avg Pause Max Pause Avg RS size Max RS size > Size [MB] [ms] [ms] [kB] [kB] > baseline default 1045 1714 1532460 3028992 > changes default 731.2 1152 516574 926720 > > G1 max pause time drops by 33%, and remembered set size by 66% :) > (using 8M regions, i.e. 32 entries instead of 16 in a sparse PRT). > > (These improvements are in addition to the changes in JDK-8213108, > but should also apply without) > > Note that in both cases the optimized value for the number of sparse > prt entries for this particular application would probably be a bit > higher. However I chose this heuristic (and the drop from 16 to 8 > initial size of the hash table) to keep the impact on applications > that do not use many remembered sets low (i.e. in practice zero). > > This heuristic could certainly be optimized (e.g. by sampling of > actual remembered sets which is easy in conjunction with JDK-8213108; > make the number of sparse prt entries dependent on heap > size/#regions), but given the plan to re-implement the remembered > sets (JDK-8017163) where we would redo that work, I think this > heuristic is a good tradeoff between effort and results. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8223162 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8223162/webrev/ > Testing: > lots of remembered set size measurements on lots of applications, > some > hs-tier1-5 runs together with other changes. > > Thanks, > Thomas > From martin.doerr at sap.com Fri May 24 13:48:31 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 24 May 2019 13:48:31 +0000 Subject: Assertion failure on PPC64 after 8200545: Improve filter for enqueued deferred cards In-Reply-To: References: <345bd70dd4bbe4a97f618100fe4a0d742298cc36.camel@oracle.com> Message-ID: Hi Thomas, > The only way I could imagine such an error would be if the compiler did > something weird with writing the fields of the heap attribute table? > I.e. it loads a machine word (containing four of those RegionAttr > entries), modifies just one of the bytes, and writes back the whole > word. I.e. then some concurrent reader might see inconsistent values > that flip back and forth. > I really doubt this is the case though. Particularly I assume that at > least on ppc64/linux you also use gcc. Correct. It has been built by gcc 7.3.1. AIX version has been built by clang IBM XL C/C++ for AIX, Version 16.1.0.2. I wonder why only SPARC needs a workaround. If Solaris Studio compilers perform such kind of optimizations, other compilers may do that too. If not now, the next compiler release may do it. Right? "#ifdef SPARC" doesn't look appropriate to me. Maybe the fields need to be volatile to prevent such optimizations? As long as they are not volatile, we allow compilers to reload, reorder, combine loads or stores, etc. > Can you dump the remembered set states/heap region attributes after > setting them in the > G1CollectedHeap::register_regions_with_region_attr() method and > comparing them with the information from the assert? I only have the hs_err files at the moment. But I could add tracing and wait for the next occurrence. > You might want to check that the default value written to > _needs_remembered_set in G1HeapRegionAttr in its constructor really > ends up as 0/false. As mentioned there is a bool->uint8_t implicit type > coercion. (Now I'm in paranoia mode :)) I don't think C++ allows any other encoding as 0/1. And if it did, I think the code would still work because we're only comparing to 0. And we see the same issue with 2 different compilers (gcc and clang). Thanks and best regards, Martin From leo.korinth at oracle.com Fri May 24 13:54:42 2019 From: leo.korinth at oracle.com (Leo Korinth) Date: Fri, 24 May 2019 15:54:42 +0200 Subject: RFR: 8224660: Parallel GC: Use WorkGang (2: MarksFromRootsTask) In-Reply-To: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> Message-ID: <90c03f6c-f5b1-bb39-f35b-e1de643ba652@oracle.com> Hi, Here is the second patch in a proposed patch series of eight that removes gcTaskManager and uses the WorkGang abstraction instead. I will try to put these mails in the same mail thread. This patch moves the MarkFromRootsTask, ThreadRootsMarkingTask and the StealMarkingTask (the last one we already moved in the last patch). They are all called from the new MarkFromRootsTask. The RootType enum as well as EnumClaimer, a class that can claim enumeration values atomically is put in an inline header file. It will be used in later patches outside psParallelCompact if you wonder about the header file placement. Enhancement: https://bugs.openjdk.java.net/browse/JDK-8224660 Webrev: http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224660-Parallel-GC-Use-WorkGang-2-MarksFromRootsTask/ http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ Testing (on the whole patch series): mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC gc test suite Thanks, Leo From thomas.schatzl at oracle.com Fri May 24 14:54:08 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 24 May 2019 16:54:08 +0200 Subject: Assertion failure on PPC64 after 8200545: Improve filter for enqueued deferred cards In-Reply-To: References: <345bd70dd4bbe4a97f618100fe4a0d742298cc36.camel@oracle.com> Message-ID: <4331245b8140d2808d38aaf335b004b5d8c7833a.camel@oracle.com> Hi, On Fri, 2019-05-24 at 13:48 +0000, Doerr, Martin wrote: > Hi Thomas, > > > The only way I could imagine such an error would be if the compiler > > did something weird with writing the fields of the heap attribute > > table? I.e. it loads a machine word (containing four of those > > RegionAttr entries), modifies just one of the bytes, and writes > > back the whole word. I.e. then some concurrent reader might see > > inconsistent values that flip back and forth. > > I really doubt this is the case though. Particularly I assume that > > at least on ppc64/linux you also use gcc. > > Correct. It has been built by gcc 7.3.1. > AIX version has been built by clang IBM XL C/C++ for AIX, Version > 16.1.0.2. > > I wonder why only SPARC needs a workaround. > If Solaris Studio compilers perform such kind of optimizations, other > compilers may do that too. The SPARC workaround has another background: SPARC does not like (small) structs. See http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2019-May/025790.html referring to http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2014-December/011557.html . Some fiddling on godbolt did not show strange code generation on a very much shortened variant of that code with the available compilers (did not have gcc 7.3.1). I do not know if this optimization would be allowed (if it is an optimization), but I could imagine so... I can prepare some change for you to test, making the fields accessed "atomically" (via e.g. our Atomic::load/store methods, or making them word sized) but I can't really test if it fixes the problem as we've never reproduced this case afaict in our CI. Would that help you? Next week I intend to dedicate some time to think through the code again. It seems worth thinking through the code with memory ordering problems in mind again. We could go backout this change, but this will make JDK-8213108 out for review a bit more complicated at least. But if it is a memory visibility problem, I fear that as explained in an earlier email, this would just make the assert go away but not fix the issue... Thanks, Thomas From martin.doerr at sap.com Fri May 24 15:03:59 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 24 May 2019 15:03:59 +0000 Subject: Assertion failure on PPC64 after 8200545: Improve filter for enqueued deferred cards In-Reply-To: <4331245b8140d2808d38aaf335b004b5d8c7833a.camel@oracle.com> References: <345bd70dd4bbe4a97f618100fe4a0d742298cc36.camel@oracle.com> <4331245b8140d2808d38aaf335b004b5d8c7833a.camel@oracle.com> Message-ID: Hi Thomas, > The SPARC workaround has another background: SPARC does not like > (small) structs. > > See > http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2019- > May/025790.html > referring to > http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2014- > December/011557.html Thanks for the pointers. > I can prepare some change for you to test, making the fields accessed > "atomically" (via e.g. our Atomic::load/store methods, or making them > word sized) but I can't really test if it fixes the problem as we've > never reproduced this case afaict in our CI. Would that help you? I could add a change for our nightly tests and run it for several weeks. I don't have a good feeling about how often this problems occurs. Maybe we should run the test in a loop for a long time. > Next week I intend to dedicate some time to think through the code > again. It seems worth thinking through the code with memory ordering > problems in mind again. That would be great. Thanks. Yeah, it may be a memory ordering issue. > We could go backout this change, but this will make JDK-8213108 out for > review a bit more complicated at least. But if it is a memory > visibility problem, I fear that as explained in an earlier email, this > would just make the assert go away but not fix the issue... I agree. Best regards, Martin From thomas.schatzl at oracle.com Fri May 24 15:31:08 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 24 May 2019 17:31:08 +0200 Subject: RFR (M): 8224741: Optimize the scan area during the Scan Heap Roots phase Message-ID: <7b434e2bc4dbe649a3795689073e1b9909abbdd0.camel@oracle.com> Hi all, can I have reviews for this small change based on JDK-8213108 (out for review right now) that optimizes the scanning of blocks within a chunk (see comment of g1RemSet.cpp introduced in JDK-8213108) a bit? It tracks up to which extent (i.e. heap address) G1 already walked the heap for references during scanning a card block. This can be different for blocks where the last object is a non-ObjArray that extends beyond the given block. If that object is large enough, it might actually cover the next block(s), allowing G1 to skip some duplicate work completely. In my testing this did not happen too often, as such large objects seem pretty seldom (needs to cover at least three cards, and have multiple dirty cards on it). As another added bonus it makes G1 handle dead humongous objects with multiple dirty cards (may happen due to stale remembered set entries) on them a bit more quickly as well. Overall I could not measure any performance impact, but it seems useful. Other collectors implement a variant of that too. The only reason this is a separate CR is because I thought that the change in JDK-8213108 is large enough already as is, and easier to understand without. CR: https://bugs.openjdk.java.net/browse/JDK-8224741 Webrev: http://cr.openjdk.java.net/~tschatzl/8224741/webrev/ Testing: hs-tier1-5, local jtreg gc and gc/g1 testing, perf testing Thanks, Thomas From bob.vandette at oracle.com Fri May 24 16:02:53 2019 From: bob.vandette at oracle.com (Bob Vandette) Date: Fri, 24 May 2019 12:02:53 -0400 Subject: RFR: 8222252 - Java ergonomics limits heap to 128GB with disabled compressed oops In-Reply-To: <23C2B57E-A0CA-4E44-B83D-D3728F179740@oracle.com> References: <46E18910-9D4E-4219-803C-A883F48AD43B@oracle.com> <772C07C5-F2E7-4878-A94C-A99927ED204A@oracle.com> <3b410422-68b0-a143-78f5-dd0a141d1353@oracle.com> <23C2B57E-A0CA-4E44-B83D-D3728F179740@oracle.com> Message-ID: Here?s the updated webrev. http://cr.openjdk.java.net/~bobv/8222252/webrev.03 I?ve also updated the CSR (https://bugs.openjdk.java.net/browse/JDK-8223957 ). I?ve filed a bug to create tests for this change (https://bugs.openjdk.java.net/browse/JDK-8224764 ) which I will start working on in parallel with the integration of this change. Here are the latest results showing the old and new MaxHeapSize and UseCompressedOops values before and after this change. ./java -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep MaxHeapSize size_t MaxHeapSize [OLD] = 32178700288 {product} {ergonomic} size_t MaxHeapSize [NEW] = 113883742208 {product} {ergonomic} /java -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep UseCompressedOops bool UseCompressedOops [OLD] = true {lp64_product} {ergonomic} bool UseCompressedOops [NEW] = false {lp64_product} {ergonomic} ./java -XX:MaxRAM=192g -XX:+PrintFlagsFinal -version | grep MaxHeapSize size_t MaxHeapSize [OLD] = 32178700288 {product} {ergonomic} size_t MaxHeapSize [NEW] = 51539607552 {product} {ergonomic} ./java -XX:MaxRAM=192g -XX:+PrintFlagsFinal -version | grep UseCompressedOops bool UseCompressedOops [OLD] = true {lp64_product} {ergonomic} bool UseCompressedOops [NEW] = false {lp64_product} {ergonomic} ./java -XX:MaxRAM=192g -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep MaxHeapSize size_t MaxHeapSize [OLD] = 32178700288 {product} {ergonomic} size_t MaxHeapSize [NEW] = 154618822656 {product} {ergonomic} ./java -XX:MaxRAM=192g -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep UseCompressedOops bool UseCompressedOops [OLD] = true {lp64_product} {ergonomic} bool UseCompressedOops [NEW] = false {lp64_product} {ergonomic} ./java -XX:MaxRAMPercentage=10 -XX:+PrintFlagsFinal -version | grep MaxHeapSize size_t MaxHeapSize [OLD] = 13744734208 {product} {ergonomic} size_t MaxHeapSize [NEW] = 15183380480 {product} {ergonomic} ./java -XX:MaxRAMPercentage=10 -XX:+PrintFlagsFinal -version | grep UseCompressedOops bool UseCompressedOops = true {lp64_product} {ergonomic} ./java -XX:+PrintFlagsFinal -version | grep MaxHeapSize size_t MaxHeapSize = 32178700288 {product} {ergonomic} ./java -XX:+PrintFlagsFinal -version | grep UseCompressedOops bool UseCompressedOops = true {lp64_product} {ergonomic} ./java -XX:-UseCompressedOops -XX:+PrintFlagsFinal -version | grep MaxHeapSize size_t MaxHeapSize = 34359738368 {product} {ergonomic} ./java -XX:-UseCompressedOops -XX:+PrintFlagsFinal -version | grep UseCompressedOops bool UseCompressedOops = false {lp64_product} {command line} ./java -XX:+UseCompressedOops -XX:MaxRAM=128g -XX:+PrintFlagsFinal -version | grep MaxHeapSize size_t MaxHeapSize = 32178700288 {product} {ergonomic} ./java -XX:+UseCompressedOops -XX:MaxRAM=128g -XX:+PrintFlagsFinal -version | grep UseCompressedOops bool UseCompressedOops = true {lp64_product} {command line} ./java -XX:+UseCompressedOops -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep MaxHeapSize size_t MaxHeapSize = 32178700288 {product} {ergonomic} ./java -XX:+UseCompressedOops -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep UseCompressedOops bool UseCompressedOops = true {lp64_product} {command line} Bob. > On May 23, 2019, at 9:15 AM, Bob Vandette wrote: > > >> On May 16, 2019, at 11:34 PM, David Holmes wrote: >> >> Hi Bob, >> >> On 16/05/2019 12:52 am, Bob Vandette wrote: >>> I?ve updated the webrev based on your comments. >>> http://cr.openjdk.java.net/~bobv/8222252/webrev.02/ >>> Changes: >>> 1. MaxRAM is updated if fractional flags cause us to use OS memory size for heap selection. >>> 2. If MaxRAM is specified with the other fractional based flags, we will use MaxRAM as >>> upper limit instead of OS memory size. >> >> That seems fine, but here: >> >> 1799 if (FLAG_IS_ERGO(UseCompressedOops) && use_os_mem_limit) { >> >> Shouldn't this also be checking for FLAG_IS_DEFAULT(MaxRAM) ? Or better that check would fold into the use_os_mem_limit determination: >> >> bool use_os_mem_limit = (!FLAG_IS_DEFAULT(MaxRAMPercentage) || >> !FLAG_IS_DEFAULT(MaxRAMFraction) || >> !FLAG_IS_DEFAULT(MinRAMPercentage) || >> !FLAG_IS_DEFAULT(MinRAMFraction) || >> !FLAG_IS_DEFAULT(InitialRAMPercentage) || >> !FLAG_IS_DEFAULT(InitialRAMFraction)) && >> FLAG_IS_DEFAULT(MaxRAM); >> if (use_os_mem_limit) { >> phys_mem = os::physical_memory(); >> FLAG_SET_ERGO(uint64_t, MaxRAM, (uint64_t)phys_mem); >> } else { >> phys_mem = FLAG_IS_DEFAULT(MaxRAM) ? MIN2(os::physical_memory(), (julong)MaxRAM) >> : (julong)MaxRAM; >> } >> > > Yes, you are correct. I?ll add MaxRAM to the use_os_mem_limit check. This is the behavior > I documented in the CSR. > > "Also, If any of these flags including MaxRAM are specified on the command line, the selected Java heap size will not be limited by the compressed oop limit.? > > I?ll update the patch provided in the CSR as well. > > > Can you take a look and review the CSR for this change? > > Thanks, > Bob. > > >> ? >> >> Thanks, >> David >> ----- >> >>> Here?s a proposed CSR for this behavioral change. >>> https://bugs.openjdk.java.net/browse/JDK-8223957 >>> Bob. >>>> On May 14, 2019, at 9:26 AM, Thomas Schatzl > wrote: >>>> >>>> Hi, >>>> >>>> On Mon, 2019-05-13 at 10:03 -0400, Bob Vandette wrote: >>>>> Please review this suggested fix for correcting Heap size selection >>>>> when -XX:{Max,Min,Initial)RAMPercentage and their Factional >>>>> equivalent options are used. >>>>> >>>>> There are two bugs filed that are related to this issue (8222252 and >>>>> 8213175) >>>>> >>>>> SUMMARY of FIX: >>>>> The fix corrects what I believe is an incorrect implementation of the >>>>> *Percentage/*Fraction options. These options should not have caused >>>>> the heap size to be fractionally based on MaxRAM or limited by >>>>> MaxRAM. They instead should be based on the host memory. This is >>>>> what I assume was meant by ?real memory?. >>>>> >>>>> product(double, MaxRAMPercentage, >>>>> 25.0, \ >>>>> "Maximum percentage of real memory used for maximum heap >>>>> size") \ >>>>> range(0.0, 100.0) >>>>> >>>>> When these options are selected, they should take precedence over >>>>> UseCompressedOops unless UseCompressedOops is also specified on the >>>>> command line. >>>>> >>>>> >>>>> WEBREV: >>>>> http://cr.openjdk.java.net/~bobv/8222252/webrev.01 >>>> >>>> - this is a style preference: I would assign the result of the >>>> condition in the if (arguments.cpp:1721+) to the initialization of >>>> use_os_mem_limit directly instead of first setting it to false, and >>>> then to true again if needed. Feel free to ignore. >>>> >>>> - as David pointed out, this change needs a CSR. >>>> >>>> - the interaction between MaxRAM and MaxRAMPercentage is a bit unclear; >>>> if they are independent as implemented now, then the update of one >>>> should probably update the other. Or the VM should fail if they >>>> disagree? >>>> >>>> If they are dependent, if David suggests (i.e. MaxRAMPercentage uses >>>> MaxRAM as maximum real memory if set; this is what I would tend to >>>> prefer), then not of course. >>>> >>>> - there should be regressions tests verifying this in cases where >>>> possible. >>>> >>>> I.e. it should be doable to get current os::physical_memory using e.g. >>>> whitebox and then run a few tests verifying the results of MaxHeapSize >>>> and coop mode. >>>> >>>> Particularly the interaction between MaxRAM, MaxRAMPercentage and coops >>>> mode would be interesting. >>>> >>>> Thanks, >>>> Thomas >>>> >>>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Fri May 24 21:13:21 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 24 May 2019 17:13:21 -0400 Subject: RFR (S): 8223693: Memory wastage in size of per-region type buffers in GC In-Reply-To: <797ef3a1fbc70a84b66713d7ca272a7cb131bf72.camel@oracle.com> References: <797ef3a1fbc70a84b66713d7ca272a7cb131bf72.camel@oracle.com> Message-ID: <8EF22203-2193-4CF0-9DD1-D4AA781765E6@oracle.com> > On May 24, 2019, at 8:25 AM, Thomas Schatzl wrote: > > Hi all, > > can I have reviews for this small change in the constants for > G1HeapRegionAttr::_type field, eliminating some confusion and minor > memory wastage? > > Perf testing showed no measurable impact of the change. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8223693 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8223693/webrev/ > Testing: > hs-tier1-5 in conjunction with other changes; perf testing showed no > measurable impact > > Thanks, > Thomas Looks good. From kim.barrett at oracle.com Fri May 24 21:26:03 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 24 May 2019 17:26:03 -0400 Subject: RFR (S): 8223162: Increase ergonomics for Sparse PRT entry sizing In-Reply-To: <5a7f7eb20525ee5b2ff775dc69ffa7b18820d870.camel@oracle.com> References: <5a7f7eb20525ee5b2ff775dc69ffa7b18820d870.camel@oracle.com> Message-ID: <5ABE328D-C653-4A7B-8B5E-04F59D1843BD@oracle.com> > On May 24, 2019, at 9:24 AM, Thomas Schatzl wrote: > [?]CR: > https://bugs.openjdk.java.net/browse/JDK-8223162 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8223162/webrev/ > Testing: > lots of remembered set size measurements on lots of applications, some > hs-tier1-5 runs together with other changes. > > Thanks, > Thomas src/hotspot/share/gc/g1/sparsePRT.hpp 241 enum SomeAdditionalPrivateConstants { 242 InitialCapacity = 8 243 }; Since you are touching it, consider making InitialCapacity a static const size_t rather than an enumerator, so it has a consistent type across platforms. Other than that, looks good. I don't need a new review if you change how InitialCapacity is defined. From 1948638989 at qq.com Sat May 25 12:13:17 2019 From: 1948638989 at qq.com (=?ISO-8859-1?B?a2VsdGh1emFkeA==?=) Date: Sat, 25 May 2019 20:13:17 +0800 Subject: Why serial gc evacuates the whole heap rather than just To+Old spaces? Message-ID: Hi I'm reading the hotspot serial gc source code recently, but I've had a puzzling problem. As I understand, FastScanClosure would copy objects from Eden+From spaces or promote to old generation when age was reached to threshold. After this phase, only To+Old spaces has object reference, which means we can evacuate their followers from these spaces. Why [FastEvacuationFollowersClosure](http://hg.openjdk.java.net/jdk/jdk12/file/06222165c35f/src/hotspot/share/gc/serial/serialHeap.inline.hpp#l33) evacuates the whole heap rather than just To+Old space? -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Sat May 25 18:25:16 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Sat, 25 May 2019 20:25:16 +0200 Subject: Why serial gc evacuates the whole heap rather than just To+Old spaces? In-Reply-To: References: Message-ID: <89999f26cd12271b0cf3526afcc2e5c7f5b78b3d.camel@oracle.com> Hi, On Sat, 2019-05-25 at 20:13 +0800, kelthuzadx wrote: > Hi I'm reading the hotspot serial gc source code recently, but I've > had a puzzling problem. As I understand, FastScanClosure would copy > objects from Eden+From spaces or promote to old generation when age > was reached to threshold. After this phase, only To+Old spaces has > object reference, which means we can evacuate their followers from > these spaces. Why [FastEvacuationFollowersClosure]( > http://hg.openjdk.java.net/jdk/jdk12/file/06222165c35f/src/hotspot/share/gc/serial/serialHeap.inline.hpp#l33 > ) evacuates the whole heap rather than just To+Old space? can you clarify your question? The code you referenced seems to do exactly what you suggest it should do above. It iterates over the objects pulled from the roots into survivor/old to potentially pull the objects they reference. If you look at the caller e.g. in DefNewGeneration, it is called until no more evacuations occur. Thanks, Thomas From thomas.schatzl at oracle.com Sun May 26 07:33:27 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Sun, 26 May 2019 09:33:27 +0200 Subject: Why serial gc evacuates the whole heap rather than just To+Old spaces? In-Reply-To: References: <89999f26cd12271b0cf3526afcc2e5c7f5b78b3d.camel@oracle.com> Message-ID: (cc'ed the list again, as this question has been started there and others might have the same questions) Hi, On Sun, 2019-05-26 at 13:59 +0800, kelthuzadx wrote: > Hi Thomas, > I means, fast evacuation followers closure only need to evacuate > To+Old spaces. Current implementation I referenced applies this > closure on the whole young generation, but evacuation on Eden+From > spaces is unnecessary. as far as I understand your statements, both are incorrect. Let me try to explain what Serial does below. The serial gc code roughly does the following (from memory and looking at the code for a few minutes): - iterate the roots, copy those referenced objects to to-space or old gen. - these copied objects may contain references to eden/from space too. So you need to iterate over those (only those!) objects as well. - the code you mentioned does exactly that: at line 35 Serial GC iterates over the objects it recently copied into to-space; at line 36 it iterates over recently copied objects in old-space. - this process may add more objects to to-/old-space. So repeat, until no more objects were copied into to/old space. In no case Serial GC iterates over eden or from space (or the whole old space). Maybe the "young_gen()->oop_since_save_marks_iterate(cur);" line mentioning "young_gen" is confusing? However the called method explicitly only iterates over objects that were allocated *since* the save marks were set... those are set at the start of every such iteration to examine copied objects afair (There are other options to achieve the same effect, but I think this is the case for Serial). I would like to ask you to rephrase your question if that does not answer your concerns, or let me understand your point better e.g. by giving a more detailed rundown of Serial GC execution and pointing out the location you are worried about a bit better. Thanks, Thomas From 1948638989 at qq.com Sun May 26 08:10:18 2019 From: 1948638989 at qq.com (=?gb18030?B?a2VsdGh1emFkeA==?=) Date: Sun, 26 May 2019 16:10:18 +0800 Subject: =?gb18030?B?u9i4tKO6IFdoeSBzZXJpYWwgZ2MgZXZhY3VhdGVz?= =?gb18030?B?IHRoZSB3aG9sZSBoZWFwIHJhdGhlciB0aGFuIGp1?= =?gb18030?B?c3QgVG8rT2xkc3BhY2VzPw==?= In-Reply-To: References: <89999f26cd12271b0cf3526afcc2e5c7f5b78b3d.camel@oracle.com> Message-ID: Hi Thomas, I thought only To+Old spaces exist GC Root-directly-reachable objects since FastScanClosure copied them to To+Old spaces, and FastEvacuationFollowersClosure followed theirs member fields, so we could reach all live objects if we started searching from To+Old spaces. --------------------------------------------------------------- | Eden+From | To + Old | |---x------x------x-----x----|xxxx-------------|------------| | ^ ^ ^ ^ | ^saved_mark | |____|_____|____|____| --------------------------------------------------------------- (x means marked) Thanks for your explanation! It seems that I misunderstand source code. ------------------ ???? ------------------ ???: "Thomas Schatzl"; ????: 2019?5?26?(???) ??3:33 ???: "kelthuzadx"<1948638989 at qq.com>; ??: "hotspot-gc-dev"; ??: Re: Why serial gc evacuates the whole heap rather than just To+Oldspaces? (cc'ed the list again, as this question has been started there and others might have the same questions) Hi, On Sun, 2019-05-26 at 13:59 +0800, kelthuzadx wrote: > Hi Thomas, > I means, fast evacuation followers closure only need to evacuate > To+Old spaces. Current implementation I referenced applies this > closure on the whole young generation, but evacuation on Eden+From > spaces is unnecessary. as far as I understand your statements, both are incorrect. Let me try to explain what Serial does below. The serial gc code roughly does the following (from memory and looking at the code for a few minutes): - iterate the roots, copy those referenced objects to to-space or old gen. - these copied objects may contain references to eden/from space too. So you need to iterate over those (only those!) objects as well. - the code you mentioned does exactly that: at line 35 Serial GC iterates over the objects it recently copied into to-space; at line 36 it iterates over recently copied objects in old-space. - this process may add more objects to to-/old-space. So repeat, until no more objects were copied into to/old space. In no case Serial GC iterates over eden or from space (or the whole old space). Maybe the "young_gen()->oop_since_save_marks_iterate(cur);" line mentioning "young_gen" is confusing? However the called method explicitly only iterates over objects that were allocated *since* the save marks were set... those are set at the start of every such iteration to examine copied objects afair (There are other options to achieve the same effect, but I think this is the case for Serial). I would like to ask you to rephrase your question if that does not answer your concerns, or let me understand your point better e.g. by giving a more detailed rundown of Serial GC execution and pointing out the location you are worried about a bit better. Thanks, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan.karlsson at oracle.com Mon May 27 07:53:36 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 27 May 2019 09:53:36 +0200 Subject: RFR: 8223837: Add -XX:MinHeapSize flag to set the minimum heap size Message-ID: <914c862a-2fab-e97f-6ba2-6edcddee9e9e@oracle.com> Hi all, Please review this patch to add -XX:MinHeapSize flag to set the minimum heap size. http://cr.openjdk.java.net/~stefank/8223837/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8223837 Currently the -Xms flag is documented to set the initial heap size, but it also sets the minimum heap size. The initial heap size can be overridden with the -XX:InitialHeapSize flag, but there is no such flag for the minimum heap size. To set the minimum heap size to a value different than the initial heap size the user must specify -Xms before -XX:InitialHeapSize. For example: -Xms4g -XX:InitialHeapSize=8g sets the minimum size to commit for the heap to 4g and the initially committed heap size to 8g. On the other hand, the following command line: -XX:InitialHeapSize=8g -Xms4g sets both the minimum and initial heap size to 4g. This RFE proposes that we introduce a -XX:MinHeapSize flag to make it easier to independently set the minimum heap size and initial heap size. The user will get the expected behavior from either of the following command lines: -XX:MinHeapSize=4g -XX:InitialHeapSize=8g -XX:InitialHeapSize=8g -XX:MinHeapSize=4g The -Xms flag will keep it's old behavior and set both minimum and initial heap size, but when setting the minimum heap size it will change the MinHeapSize flag and not a JVM internal global variable. -Xms will behave as if both -XX:MinHeapSize and -XX:InitialHeapSize were set on the command line. The VM will enforce the relationship: MinHeapSize <= InitialHeapSize <= MaxHeapSize I've created a CSR for this (JDK-8223852), but have not moved it out of draft yet. I'm also thinking about making MinHeapSize manageable, to allow the users to change the value the JVM is running, using our management APIs. Either I fold that into the same CSR, or I simply create a new CSR to cover that change. Tested with updated version of TestMaxHeapSizeTools.java, to only cover cases where MinHeapSize is set. That tool is used by the tests in TestHeapSizeFlags.java. Other suggestions of testing for this would be appreciated. Thanks, StefanK From leo.korinth at oracle.com Mon May 27 08:21:39 2019 From: leo.korinth at oracle.com (Leo Korinth) Date: Mon, 27 May 2019 10:21:39 +0200 Subject: RFR: 8224661: Parallel GC: Use WorkGang (3: UpdateDensePrefixAndCompactionTask) In-Reply-To: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> Message-ID: <1c1eff7f-5b13-5020-df2a-bb2f43813ba3@oracle.com> Hi, Here is the third patch in a proposed patch series of eight that removes gcTaskManager and uses the WorkGang abstraction instead. I will try to put these mails in the same mail thread. This patch converts the last old tasks in pcTasks.?pp. The old code divides the workload into sub tasks by over partitioning the workload. I do not want to change this scheme so I created a simple TaskQueue in the compaction manager so that we can use the more-tasks-than-threads behaviour of the old code (lock free this time). The new queue is populated with a new UpdateDensePrefixTask holding three values and no behaviour. Otherwise I mostly moved the code in pcTask.cpp to psParallelCompact.cpp and converted it to a gang task. Enhancement: https://bugs.openjdk.java.net/browse/JDK-8224661 Webrev: http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224661-Parallel-GC-Use-WorkGang-3-UpdateDensePrefixAndCompactionTask/ http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ Testing (on the whole patch series): mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC gc test suite Thanks, Leo From thomas.schatzl at oracle.com Mon May 27 08:44:40 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 27 May 2019 10:44:40 +0200 Subject: RFR (S): 8223162: Increase ergonomics for Sparse PRT entry sizing In-Reply-To: <5ABE328D-C653-4A7B-8B5E-04F59D1843BD@oracle.com> References: <5a7f7eb20525ee5b2ff775dc69ffa7b18820d870.camel@oracle.com> <5ABE328D-C653-4A7B-8B5E-04F59D1843BD@oracle.com> Message-ID: <49a808121451ce6805c61b05d31d471b209f72b2.camel@oracle.com> Hi Kim, On Fri, 2019-05-24 at 17:26 -0400, Kim Barrett wrote: > > On May 24, 2019, at 9:24 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > [?]CR: > > https://bugs.openjdk.java.net/browse/JDK-8223162 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8223162/webrev/ > > Testing: > > lots of remembered set size measurements on lots of applications, > > some > > hs-tier1-5 runs together with other changes. > > > > Thanks, > > Thomas > > src/hotspot/share/gc/g1/sparsePRT.hpp > 241 enum SomeAdditionalPrivateConstants { > 242 InitialCapacity = 8 > 243 }; > > Since you are touching it, consider making InitialCapacity a static > const size_t rather than an enumerator, so it has a consistent type > across platforms. > > Other than that, looks good. I don't need a new review if you change > how InitialCapacity is defined. > thanks for your review. I changed InitialCapacity to a static const as suggested and regenerated the webrev. Thanks, Thomas From thomas.schatzl at oracle.com Mon May 27 08:45:16 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 27 May 2019 10:45:16 +0200 Subject: RFR (S): 8223693: Memory wastage in size of per-region type buffers in GC In-Reply-To: <8EF22203-2193-4CF0-9DD1-D4AA781765E6@oracle.com> References: <797ef3a1fbc70a84b66713d7ca272a7cb131bf72.camel@oracle.com> <8EF22203-2193-4CF0-9DD1-D4AA781765E6@oracle.com> Message-ID: Hi, On Fri, 2019-05-24 at 17:13 -0400, Kim Barrett wrote: > > On May 24, 2019, at 8:25 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > Hi all, > > > > can I have reviews for this small change in the constants for > > G1HeapRegionAttr::_type field, eliminating some confusion and minor > > memory wastage? > > > > Perf testing showed no measurable impact of the change. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8223693 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8223693/webrev/ > > Testing: > > hs-tier1-5 in conjunction with other changes; perf testing showed > > no > > measurable impact > > > > Thanks, > > Thomas > > Looks good. > thanks for your review. Thomas From leo.korinth at oracle.com Mon May 27 08:56:46 2019 From: leo.korinth at oracle.com (Leo Korinth) Date: Mon, 27 May 2019 10:56:46 +0200 Subject: RFR: 8224662: Parallel GC: Use WorkGang (4: SharedRestorePreservedMarksTaskExecutor) In-Reply-To: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> Message-ID: <226f085b-6767-5ca8-a32e-76c0eea2de63@oracle.com> Hi, Here is the fourth patch in a proposed patch series of eight that removes gcTaskManager and uses the WorkGang abstraction instead. I will try to put these mails in the same mail thread. This patch is short. it uses the already existing SharedRestorePreservedMarksTaskExecutor. No new code was needed. In the future it would be nice to remove this "executor" closure-inside-closure model. I did a quick try to do this, and failed due to some arguments to the task is applied too late. Improving this will have to be done later. Enhancement: https://bugs.openjdk.java.net/browse/JDK-8224662 Webrev: http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224662-Parallel-GC-Use-WorkGang-4-SharedRestorePreservedMarksTaskExecutor/ http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ Testing (on the whole patch series): mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC gc test suite Thanks, Leo From thomas.schatzl at oracle.com Mon May 27 10:48:10 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 27 May 2019 12:48:10 +0200 Subject: Assertion failure on PPC64 after 8200545: Improve filter for enqueued deferred cards In-Reply-To: <4331245b8140d2808d38aaf335b004b5d8c7833a.camel@oracle.com> References: <345bd70dd4bbe4a97f618100fe4a0d742298cc36.camel@oracle.com> <4331245b8140d2808d38aaf335b004b5d8c7833a.camel@oracle.com> Message-ID: Hi Martin, On Fri, 2019-05-24 at 16:54 +0200, Thomas Schatzl wrote: > Hi, > > On Fri, 2019-05-24 at 13:48 +0000, Doerr, Martin wrote: > > Hi Thomas, > > > > > The only way I could imagine such an error would be if the > > > compiler did something weird with writing the fields of the heap > > > attribute table? I.e. it loads a machine word (containing four of > > > those RegionAttr entries), modifies just one of the bytes, and > > > writes back the whole word. I.e. then some concurrent reader > > > might see inconsistent values that flip back and forth. > > > I really doubt this is the case though. Particularly I assume > > > that at least on ppc64/linux you also use gcc. > > > > [...] > I can prepare some change for you to test, making the fields accessed > "atomically" (via e.g. our Atomic::load/store methods, or making > them word sized) but I can't really test if it fixes the problem as > we've never reproduced this case afaict in our CI. Would that help > you? Changes: http://cr.openjdk.java.net/~tschatzl/8224681/webrev.test1/ uses Atomic::load/store to access the G1HeapRegionAttr members only. http://cr.openjdk.java.net/~tschatzl/8224681/webrev.test2/ test1 + using 32 bit words for the members (64 bit struct). So above problem should not happen. I will follow up with some memory visibility analysis in the next days. Thanks, Thomas From martin.doerr at sap.com Mon May 27 12:25:35 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 27 May 2019 12:25:35 +0000 Subject: Assertion failure on PPC64 after 8200545: Improve filter for enqueued deferred cards In-Reply-To: References: <345bd70dd4bbe4a97f618100fe4a0d742298cc36.camel@oracle.com> <4331245b8140d2808d38aaf335b004b5d8c7833a.camel@oracle.com> Message-ID: Hi Thomas, thanks for the webrevs. As discussed offline, I don't think that PPC64 uses word instead of byte accesses. I can test webrev.test1. This looks like a good idea. Maybe it would also resolve the issue on SPARC when using 8 bit types? My main concern is that the issue seems to be hard to reproduce, but I'll try to run tests for some time. Thanks and best regards, Martin > -----Original Message----- > From: Thomas Schatzl > Sent: Montag, 27. Mai 2019 12:48 > To: Doerr, Martin > Cc: hotspot-gc-dev at openjdk.java.net > Subject: Re: Assertion failure on PPC64 after 8200545: Improve filter for > enqueued deferred cards > > Hi Martin, > > On Fri, 2019-05-24 at 16:54 +0200, Thomas Schatzl wrote: > > Hi, > > > > On Fri, 2019-05-24 at 13:48 +0000, Doerr, Martin wrote: > > > Hi Thomas, > > > > > > > The only way I could imagine such an error would be if the > > > > compiler did something weird with writing the fields of the heap > > > > attribute table? I.e. it loads a machine word (containing four of > > > > those RegionAttr entries), modifies just one of the bytes, and > > > > writes back the whole word. I.e. then some concurrent reader > > > > might see inconsistent values that flip back and forth. > > > > I really doubt this is the case though. Particularly I assume > > > > that at least on ppc64/linux you also use gcc. > > > > > > > [...] > > I can prepare some change for you to test, making the fields accessed > > "atomically" (via e.g. our Atomic::load/store methods, or making > > them word sized) but I can't really test if it fixes the problem as > > we've never reproduced this case afaict in our CI. Would that help > > you? > > Changes: > http://cr.openjdk.java.net/~tschatzl/8224681/webrev.test1/ uses > Atomic::load/store to access the G1HeapRegionAttr members only. > > http://cr.openjdk.java.net/~tschatzl/8224681/webrev.test2/ test1 + > using 32 bit words for the members (64 bit struct). So above problem > should not happen. > > I will follow up with some memory visibility analysis in the next days. > > Thanks, > Thomas From rkennke at redhat.com Mon May 27 12:51:48 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 27 May 2019 14:51:48 +0200 Subject: RFR: JDK-8224836: Backout: JDK-8224626: Shenandoah: Elide barriers on uncommon traps Message-ID: This is a clean backout of "JDK-8224626: Shenandoah: Elide barriers on uncommon traps". This seems to cause failures in a ctw test. It seems a bit odd that this would cause problems, but life's too short to debug this ;-) http://cr.openjdk.java.net/~rkennke/JDK-8224836/webrev.00/ Testing: the failing ctw test, hotspot_gc_shenandoah Ok? Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Mon May 27 12:53:44 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 27 May 2019 14:53:44 +0200 Subject: RFR: JDK-8224836: Backout: JDK-8224626: Shenandoah: Elide barriers on uncommon traps In-Reply-To: References: Message-ID: <80854181-a599-5ea5-cfaa-a43a19268b46@redhat.com> On 5/27/19 2:51 PM, Roman Kennke wrote: > This is a clean backout of "JDK-8224626: Shenandoah: Elide barriers on > uncommon traps". This seems to cause failures in a ctw test. It seems a > bit odd that this would cause problems, but life's too short to debug > this ;-) True. > http://cr.openjdk.java.net/~rkennke/JDK-8224836/webrev.00/ Looks good. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From martin.doerr at sap.com Mon May 27 13:11:17 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 27 May 2019 13:11:17 +0000 Subject: Assertion failure on PPC64 after 8200545: Improve filter for enqueued deferred cards In-Reply-To: References: <345bd70dd4bbe4a97f618100fe4a0d742298cc36.camel@oracle.com> <4331245b8140d2808d38aaf335b004b5d8c7833a.camel@oracle.com> Message-ID: Hi Thomas, there's an important update: The problem showed up on linuxx64_64, too. Attached hs_err_pid67024.log to the bug. Test was api/java_text/NumberFormat/UnicodeExtensions from jck suite. Best regards, Martin > -----Original Message----- > From: Doerr, Martin > Sent: Montag, 27. Mai 2019 14:26 > To: Thomas Schatzl > Cc: hotspot-gc-dev at openjdk.java.net > Subject: RE: Assertion failure on PPC64 after 8200545: Improve filter for > enqueued deferred cards > > Hi Thomas, > > thanks for the webrevs. As discussed offline, I don't think that PPC64 uses > word instead of byte accesses. > > I can test webrev.test1. This looks like a good idea. Maybe it would also > resolve the issue on SPARC when using 8 bit types? > My main concern is that the issue seems to be hard to reproduce, but I'll try > to run tests for some time. > > Thanks and best regards, > Martin > > > > -----Original Message----- > > From: Thomas Schatzl > > Sent: Montag, 27. Mai 2019 12:48 > > To: Doerr, Martin > > Cc: hotspot-gc-dev at openjdk.java.net > > Subject: Re: Assertion failure on PPC64 after 8200545: Improve filter for > > enqueued deferred cards > > > > Hi Martin, > > > > On Fri, 2019-05-24 at 16:54 +0200, Thomas Schatzl wrote: > > > Hi, > > > > > > On Fri, 2019-05-24 at 13:48 +0000, Doerr, Martin wrote: > > > > Hi Thomas, > > > > > > > > > The only way I could imagine such an error would be if the > > > > > compiler did something weird with writing the fields of the heap > > > > > attribute table? I.e. it loads a machine word (containing four of > > > > > those RegionAttr entries), modifies just one of the bytes, and > > > > > writes back the whole word. I.e. then some concurrent reader > > > > > might see inconsistent values that flip back and forth. > > > > > I really doubt this is the case though. Particularly I assume > > > > > that at least on ppc64/linux you also use gcc. > > > > > > > > > > [...] > > > I can prepare some change for you to test, making the fields accessed > > > "atomically" (via e.g. our Atomic::load/store methods, or making > > > them word sized) but I can't really test if it fixes the problem as > > > we've never reproduced this case afaict in our CI. Would that help > > > you? > > > > Changes: > > http://cr.openjdk.java.net/~tschatzl/8224681/webrev.test1/ uses > > Atomic::load/store to access the G1HeapRegionAttr members only. > > > > http://cr.openjdk.java.net/~tschatzl/8224681/webrev.test2/ test1 + > > using 32 bit words for the members (64 bit struct). So above problem > > should not happen. > > > > I will follow up with some memory visibility analysis in the next days. > > > > Thanks, > > Thomas From zgu at redhat.com Mon May 27 13:58:50 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 27 May 2019 09:58:50 -0400 Subject: RFR (S) 8224751: Shenandoah: Shenandoah Verifier should select proper roots according to current GC cycle Message-ID: <5d41da11-59b5-f469-d6d9-72e43ac9efcf@redhat.com> Verifier should seed the marking with the same roots as current GC cycle. Bug: https://bugs.openjdk.java.net/browse/JDK-8224751 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224751/webrev.00/ Test: hotspot_gc_shenandoah with ShenandoahVerify (fastdebug and release) Thanks, -Zhengyu From rkennke at redhat.com Mon May 27 14:31:41 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 27 May 2019 16:31:41 +0200 Subject: RFR (S) 8224751: Shenandoah: Shenandoah Verifier should select proper roots according to current GC cycle In-Reply-To: <5d41da11-59b5-f469-d6d9-72e43ac9efcf@redhat.com> References: <5d41da11-59b5-f469-d6d9-72e43ac9efcf@redhat.com> Message-ID: <76c5e0cf-0be9-09ab-a1e4-c66079803a3c@redhat.com> Looks good. Thanks! Roman > Verifier should seed the marking with the same roots as current GC cycle. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8224751 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224751/webrev.00/ > > Test: > ? hotspot_gc_shenandoah with ShenandoahVerify (fastdebug and release) > > Thanks, > > -Zhengyu > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From leo.korinth at oracle.com Mon May 27 16:30:42 2019 From: leo.korinth at oracle.com (Leo Korinth) Date: Mon, 27 May 2019 18:30:42 +0200 Subject: RFR: 8224663: Parallel GC: Use WorkGang (5: ScavengeRootsTask) In-Reply-To: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> Message-ID: <3ea47da5-4b1e-0301-461a-ede9047cb638@oracle.com> Hi, Here is the fifth patch in a proposed patch series of eight that removes gcTaskManager and uses the WorkGang abstraction instead. ScavengeRootsTask, ThreadRootsTask and OldToYoungRootsTask is replaced with ScavengeRootsTask. Code is basically the same, the major difference is that roots are visited using EnumClaimer and Threads::possibly_parallel_threads_do. Here we can reuse the RootType and EnumClaimer from patch number two. The reason "case threads:" was removed is that the code is dead. That part is confusing as the code is done (in parallel from the calling function). Enhancement: https://bugs.openjdk.java.net/browse/JDK-8224663 Webrev: http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224663-Parallel-GC-Use-WorkGang-5-ScavengeRootsTask/ http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ Testing (on the whole patch series): mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC gc test suite Thanks, Leo From kim.barrett at oracle.com Mon May 27 17:18:14 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 27 May 2019 13:18:14 -0400 Subject: RFR: 8224659: Parallel GC: Use WorkGang (1: PCRefProcTask) In-Reply-To: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> Message-ID: <54AD082B-6261-466B-A8A1-F5B7C455633A@oracle.com> > On May 24, 2019, at 7:25 AM, Leo Korinth wrote: > > Hi, > > Here is the first patch in a proposed patch series of eight that > removes gcTaskManager and uses the WorkGang abstraction instead. This > is primary a refactoring to cleanup code for future > enhancements. After this change all GCs will use the WorkGang > abstraction. Thanks for splitting this up into sections. It?s definitely making reviewing easier. > [?] > # This Patch > > This first patch does two things: > 1) Adds work gang functionality > + adds WorkGang named _workers to parallelScavengeHeap > + sets active workers in psParallelCompact.cpp and > psScavenge.cpp. The corresponding functionality in gcTaskManager > is done by gc_task_manager()->set_active_gang() > > 2) Creates a PCRefProcTask that replaces RefProcTaskProxy and > StealMarkingTask. The StealMarkingTask is removed in the next > patch as it is still used by other parts. > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8224659 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224659-Parallel-GC-Use-WorkGang-1-PCRefProcTask/ > http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ > > Testing (on the whole patch series): > mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC > gc test suite > > Thanks, > Leo ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psParallelCompact.cpp 2100 void steal_marking_task(ParallelTaskTerminator& terminator, uint worker_id) { Should be static (file-scoped). That might also obviate the initial assertion that it is called during a GC. ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psParallelCompact.cpp 2108 do { 2109 while (ParCompactionManager::steal_objarray(worker_id, task)) { 2110 cm->follow_array((objArrayOop)task.obj(), task.index()); 2111 cm->follow_marking_stacks(); 2112 } 2113 while (ParCompactionManager::steal(worker_id, obj)) { 2114 cm->follow_contents(obj); 2115 cm->follow_marking_stacks(); 2116 } 2117 } while (!terminator.offer_termination()); This might offer termination earlier than it should, e.g. even if some objarrays have become available while other objects were being processed. Consider instead something like while (true) { if (ParCompactionManager::steal_objarray(worker_id, task)) { cm->follow_array((objArrayOop)task.obj(), task.index()); cm->follow_marking_stacks(); } else if (ParCompactionManager::steal(worker_id, obj)) { cm->follow_contents(obj); cm->follow_marking_stacks(); } else if (terminator.offer_termination()) { break; } } or maybe make the second clause } else if (ParCompactionManager::steal(worker_id, obj)) { do { cm->follow_contents(obj); cm->follow_marking_stacks(); } while (ParCompactionManager::steal(worker_id, obj)) { } to only check for arrays again when run out of normal objects (as done in the original proposed change). ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psParallelCompact.cpp 2103 ParCompactionManager* cm = 2104 ParCompactionManager::gc_thread_compaction_manager(worker_id); Rather than making a new manager here, why not use the one from the caller, having it passed to this helper function as an additional argument? ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psParallelCompact.cpp 1793 ParallelScavengeHeap::heap()->workers().update_active_workers(WorkerPolicy::calc_active_workers( 1794 ParallelScavengeHeap::heap()->workers().total_workers(), 1795 ParallelScavengeHeap::heap()->workers().active_workers(), 1796 Threads::number_of_non_daemon_threads())); This appears again in src/hotspot/share/gc/parallel/psScavenge.cpp (in PSScavenge::invoke_no_policy()): 342 uint active_workers = ParallelScavengeHeap::heap()->workers().update_active_workers(WorkerPolicy::calc_active_workers( 343 ParallelScavengeHeap::heap()->workers().total_workers(), 344 ParallelScavengeHeap::heap()->workers().active_workers(), 345 Threads::number_of_non_daemon_threads())); Perhaps there should be a helper for this? Or do things change later in the sequence of changesets? ------------------------------------------------------------------------------ From leo.korinth at oracle.com Mon May 27 17:34:24 2019 From: leo.korinth at oracle.com (Leo Korinth) Date: Mon, 27 May 2019 19:34:24 +0200 Subject: RFR: 8224664: Parallel GC: Use WorkGang (6: PSRefProcTaskProxy) In-Reply-To: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> Message-ID: Hi, Here is the sixth patch in a proposed patch series of eight that removes gcTaskManager and uses the WorkGang abstraction instead. Here, the new PSRefProcTask is composed of PSRefProcTaskProxy and the old StealTask (that was partially moved before). Now both psTasks.* and pcTasks.* are removed. Enhancement: https://bugs.openjdk.java.net/browse/JDK-8224664 Webrev: http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224664-Parallel-GC-Use-WorkGang-6-PSRefProcTaskProxy/ http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ Testing (on the whole patch series): mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC gc test suite Thanks, Leo From leo.korinth at oracle.com Mon May 27 17:44:52 2019 From: leo.korinth at oracle.com (Leo Korinth) Date: Mon, 27 May 2019 19:44:52 +0200 Subject: RFR: 8224665: Parallel GC: Use WorkGang (7: remove task manager) In-Reply-To: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> Message-ID: <5bc88bed-7c2c-b4e7-fef2-2d457d7ba4f1@oracle.com> Hi, Here is the seventh patch in a proposed patch series of eight that removes gcTaskManager and uses the WorkGang abstraction instead. We try to remove everything task manager and task thread related. Some utility methods (gc_threads_do, print_gc_threads_on) are changed to use the WorkGang versions. Some worker counts are fetched from WorkGang as well. Most of the change is just code deletion. Enhancement: https://bugs.openjdk.java.net/browse/JDK-8224665 Webrev: http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224665-Parallel-GC-Use-WorkGang-7-remove-task-manager/ http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ Testing (on the whole patch series): mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC gc test suite Thanks, Leo From Peter.Kessler at ACM.ORG Mon May 27 19:01:03 2019 From: Peter.Kessler at ACM.ORG (Peter B. Kessler) Date: Mon, 27 May 2019 12:01:03 -0700 Subject: RFR: 8224659: Parallel GC: Use WorkGang (1: PCRefProcTask) In-Reply-To: References: Message-ID: On Fri, 24 May 2019 13:25:47 +0200 Leo Korinth wrote > Subject: RFR: 8224659: Parallel GC: Use WorkGang (1: PCRefProcTask) > > Here is the first patch in a proposed patch series of eight that > removes gcTaskManager and uses the WorkGang abstraction instead. > .... > Performance numbers seems to indicate > that we go from roughly no improvements up to more than > 40% improvements (!). Regressions seems rare. Performance improvements > are primary in the young collections. > .... I assume the performance you cite is in the collection times, not overall application performance improvement. Do you know if the improvement is in the actual parceling out of the work, or better balancing of the parceled out work among the workers, or in the termination algorithm, or whatever? What causes the rare regressions? E.g., is there something adversarial about the object graphs, or something else we should be on the lookout for? ... peter From david.holmes at oracle.com Mon May 27 22:54:19 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 28 May 2019 08:54:19 +1000 Subject: RFR: 8222252 - Java ergonomics limits heap to 128GB with disabled compressed oops In-Reply-To: References: <46E18910-9D4E-4219-803C-A883F48AD43B@oracle.com> <772C07C5-F2E7-4878-A94C-A99927ED204A@oracle.com> <3b410422-68b0-a143-78f5-dd0a141d1353@oracle.com> <23C2B57E-A0CA-4E44-B83D-D3728F179740@oracle.com> Message-ID: <0c4150c9-91a6-d64f-880e-1173a21a03c5@oracle.com> Hi Bob, On 25/05/2019 2:02 am, Bob Vandette wrote: > Here?s the updated webrev. > > http://cr.openjdk.java.net/~bobv/8222252/webrev.03 > > ?I?ve also updated the CSR > (https://bugs.openjdk.java.net/browse/JDK-8223957). Sorry but I'm having some trouble reconciling the code with the general description. I would have expected to see something like this: bool override_coop_limit = ((!FLAG_IS_DEFAULT(MaxRAMPercentage) || !FLAG_IS_DEFAULT(MaxRAMFraction) || !FLAG_IS_DEFAULT(MinRAMPercentage) || !FLAG_IS_DEFAULT(MinRAMFraction) || !FLAG_IS_DEFAULT(InitialRAMPercentage) || ! !FLAG_IS_DEFAULT(InitialRAMFraction)) && ! FLAG_IS_DEFAULT(MaxRAM)); where override_coop_limit might also be called use_physical_memory, or ignore_maxram, if I understand correctly. Then things would simplify to: if (override_coop_limit) { // use physical, ignore MaxRAM phys_mem = os::physical_memory(); FLAG_SET_ERGO(MaxRAM, (uint64_t)phys_mem); } else { // use MaxRAM phys_mem = FLAG_IS_DEFAULT(MaxRAM) ? MIN2(os::physical_memory(), (julong)MaxRAM) : (julong)MaxRAM; } > I?ve filed a bug to create tests for this change > (https://bugs.openjdk.java.net/browse/JDK-8224764) which > I will start working on in parallel with the integration of this change. Given the change is taking somewhat longer than you may have hoped, I hope the test has now progressed. I think it would be good to get everything in together rather than break it up. Thanks, David > Here are the latest results showing the old and new MaxHeapSize and > UseCompressedOops values before > and after this change. > > ./java -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep > MaxHeapSize > ?? size_t MaxHeapSize ? ? ? ? ? ? ? ? ? ? [OLD]? ? = 32178700288 > ? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} > ?? size_t MaxHeapSize ? ? ? ? ? ? ? ? ? ? [NEW]? ? = 113883742208 > ? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} > /java -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep > UseCompressedOops > ?? ? bool UseCompressedOops ? ? ? ? ? ? ? [OLD]? ? = true > ? ? ? ? ? ? ? ? ? {lp64_product} {ergonomic} > ?? ? bool UseCompressedOops ? ? ? ? ? ? ? [NEW]? ? = false > ? ? ? ? ? ? ? ? ? {lp64_product} {ergonomic} > ./java -XX:MaxRAM=192g -XX:+PrintFlagsFinal -version | grep MaxHeapSize > ?? size_t MaxHeapSize ? ? ? ? ? ? ? ? ? ? [OLD]? ? = 32178700288 > ? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} > ?? size_t MaxHeapSize ? ? ? ? ? ? ? ? ? ? [NEW]? ? = 51539607552 > ? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} > ./java -XX:MaxRAM=192g -XX:+PrintFlagsFinal -version | grep > UseCompressedOops > ?? ? bool UseCompressedOops ? ? ? ? ? ? ? [OLD]? ? = true > ? ? ? ? ? ? ? ? ? {lp64_product} {ergonomic} > ?? ? bool UseCompressedOops ? ? ? ? ? ? ? [NEW]? ? = false > ? ? ? ? ? ? ? ? ? {lp64_product} {ergonomic} > ./java -XX:MaxRAM=192g -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal > -version | grep MaxHeapSize > ?? size_t MaxHeapSize ? ? ? ? ? ? ? ? ? ? [OLD]? ? = 32178700288 > ? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} > ?? size_t MaxHeapSize ? ? ? ? ? ? ? ? ? ? [NEW]? ? = 154618822656 > ? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} > ./java -XX:MaxRAM=192g -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal > -version | grep UseCompressedOops > ?? ? bool UseCompressedOops ? ? ? ? ? ? ? [OLD]? ? = true > ? ? ? ? ? ? ? ? ? {lp64_product} {ergonomic} > ?? ? bool UseCompressedOops ? ? ? ? ? ? ? [NEW]? ? = false > ? ? ? ? ? ? ? ? ? {lp64_product} {ergonomic} > ./java -XX:MaxRAMPercentage=10 -XX:+PrintFlagsFinal -version | grep > MaxHeapSize > ?? size_t MaxHeapSize ? ? ? ? ? ? ? ? ? ? [OLD]? ? = 13744734208 > ? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} > ?? size_t MaxHeapSize ? ? ? ? ? ? ? ? ? ? [NEW]? ? = 15183380480 > ? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} > ./java -XX:MaxRAMPercentage=10 -XX:+PrintFlagsFinal -version | grep > UseCompressedOops > ?? ? bool UseCompressedOops? ? ? ? ? ? ? ? ? ? ? ? = true > ? ? ? ? ? ? ? ? ? {lp64_product} {ergonomic} > ./java -XX:+PrintFlagsFinal -version | grep MaxHeapSize > ?? size_t MaxHeapSize? ? ? ? ? ? ? ? ? ? ? ? ? ? ? = 32178700288 > ? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} > ./java -XX:+PrintFlagsFinal -version | grep UseCompressedOops > ?? ? bool UseCompressedOops? ? ? ? ? ? ? ? ? ? ? ? = true > ? ? ? ? ? ? ? ? ? {lp64_product} {ergonomic} > ./java -XX:-UseCompressedOops -XX:+PrintFlagsFinal -version | grep > MaxHeapSize > ?? size_t MaxHeapSize? ? ? ? ? ? ? ? ? ? ? ? ? ? ? = 34359738368 > ? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} > ./java -XX:-UseCompressedOops -XX:+PrintFlagsFinal -version | grep > UseCompressedOops > ?? ? bool UseCompressedOops? ? ? ? ? ? ? ? ? ? ? ? = false > ? ? ? ? ? ? ? ? ? {lp64_product} {command line} > ./java -XX:+UseCompressedOops -XX:MaxRAM=128g -XX:+PrintFlagsFinal > -version | grep MaxHeapSize > ?? size_t MaxHeapSize? ? ? ? ? ? ? ? ? ? ? ? ? ? ? = 32178700288 > ? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} > ./java -XX:+UseCompressedOops -XX:MaxRAM=128g -XX:+PrintFlagsFinal > -version | grep UseCompressedOops > ?? ? bool UseCompressedOops? ? ? ? ? ? ? ? ? ? ? ? = true > ? ? ? ? ? ? ? ? ? {lp64_product} {command line} > ./java -XX:+UseCompressedOops -XX:MaxRAMPercentage=75 > -XX:+PrintFlagsFinal -version | grep MaxHeapSize > ?? size_t MaxHeapSize? ? ? ? ? ? ? ? ? ? ? ? ? ? ? = 32178700288 > ? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} > ./java -XX:+UseCompressedOops -XX:MaxRAMPercentage=75 > -XX:+PrintFlagsFinal -version | grep UseCompressedOops > ?? ? bool UseCompressedOops? ? ? ? ? ? ? ? ? ? ? ? = true > ? ? ? ? ? ? ? ? ? {lp64_product} {command line} > > Bob. > >> On May 23, 2019, at 9:15 AM, Bob Vandette > > wrote: >> >> >>> On May 16, 2019, at 11:34 PM, David Holmes >> > wrote: >>> >>> Hi Bob, >>> >>> On 16/05/2019 12:52 am, Bob Vandette wrote: >>>> I?ve updated the webrev based on your comments. >>>> http://cr.openjdk.java.net/~bobv/8222252/webrev.02/ >>>> Changes: >>>> 1. MaxRAM is updated if fractional flags cause us to use OS memory >>>> size for heap selection. >>>> 2. If MaxRAM is specified with the other fractional based flags, we >>>> will use MaxRAM as >>>> ??upper limit instead of OS memory size. >>> >>> That seems fine, but here: >>> >>> 1799 ????????if (FLAG_IS_ERGO(UseCompressedOops) && use_os_mem_limit) { >>> >>> Shouldn't this also be checking for FLAG_IS_DEFAULT(MaxRAM) ? Or >>> better that check would fold into the use_os_mem_limit determination: >>> >>> bool use_os_mem_limit = (!FLAG_IS_DEFAULT(MaxRAMPercentage) || >>> ?????????????????????????!FLAG_IS_DEFAULT(MaxRAMFraction) || >>> ?????????????????????????!FLAG_IS_DEFAULT(MinRAMPercentage) || >>> ?????????????????????????!FLAG_IS_DEFAULT(MinRAMFraction) || >>> ?????????????????????????!FLAG_IS_DEFAULT(InitialRAMPercentage) || >>> ?????????????????????????!FLAG_IS_DEFAULT(InitialRAMFraction)) && >>> ?????????????????????????FLAG_IS_DEFAULT(MaxRAM); >>> if (use_os_mem_limit) { >>> ??phys_mem = os::physical_memory(); >>> ??FLAG_SET_ERGO(uint64_t, MaxRAM, (uint64_t)phys_mem); >>> } else { >>> ??phys_mem = FLAG_IS_DEFAULT(MaxRAM) ? MIN2(os::physical_memory(), >>> (julong)MaxRAM) >>> ?????????????????????????????????????: (julong)MaxRAM; >>> } >>> >> >> Yes, you are correct. ?I?ll add MaxRAM to the use_os_mem_limit check. >> ?This is the behavior >> I documented in the CSR. >> >> ?"Also, If any of these flags including MaxRAM are specified on the >> command line, the selected Java heap size will not be limited by the >> compressed oop limit.? >> >> I?ll update the patch provided in the CSR as well. >> >> >> Can you take a look and review the CSR for this change? >> >> Thanks, >> Bob. >> >> >>> ? >>> >>> Thanks, >>> David >>> ----- >>> >>>> Here?s a proposed CSR for this behavioral change. >>>> https://bugs.openjdk.java.net/browse/JDK-8223957 >>>> Bob. >>>>> On May 14, 2019, at 9:26 AM, Thomas Schatzl >>>>> > wrote: >>>>> >>>>> Hi, >>>>> >>>>> On Mon, 2019-05-13 at 10:03 -0400, Bob Vandette wrote: >>>>>> Please review this suggested fix for correcting Heap size selection >>>>>> when -XX:{Max,Min,Initial)RAMPercentage and their Factional >>>>>> equivalent options are used. >>>>>> >>>>>> There are two bugs filed that are related to this issue (8222252 and >>>>>> 8213175) >>>>>> >>>>>> SUMMARY of FIX: >>>>>> The fix corrects what I believe is an incorrect implementation of the >>>>>> *Percentage/*Fraction options. ?These options should not have caused >>>>>> the heap size to be fractionally based on MaxRAM or limited by >>>>>> MaxRAM. ?They instead should be based on the host memory. This is >>>>>> what I assume was meant by ?real memory?. >>>>>> >>>>>> product(double, MaxRAMPercentage, >>>>>> 25.0, ??????????????????????????????????\ >>>>>> ???????"Maximum percentage of real memory used for maximum heap >>>>>> size") \ >>>>>> ???????range(0.0, 100.0) >>>>>> >>>>>> When these options are selected, they should take precedence over >>>>>> UseCompressedOops unless UseCompressedOops is also specified on the >>>>>> command line. >>>>>> >>>>>> >>>>>> WEBREV: >>>>>> http://cr.openjdk.java.net/~bobv/8222252/webrev.01 >>>>> >>>>> - this is a style preference: I would assign the result of the >>>>> condition in the if (arguments.cpp:1721+) to the initialization of >>>>> use_os_mem_limit directly instead of first setting it to false, and >>>>> then to true again if needed. Feel free to ignore. >>>>> >>>>> - as David pointed out, this change needs a CSR. >>>>> >>>>> - the interaction between MaxRAM and MaxRAMPercentage is a bit unclear; >>>>> if they are independent as implemented now, then the update of one >>>>> should probably update the other. Or the VM should fail if they >>>>> disagree? >>>>> >>>>> If they are dependent, if David suggests (i.e. MaxRAMPercentage uses >>>>> MaxRAM as maximum real memory if set; this is what I would tend to >>>>> prefer), then not of course. >>>>> >>>>> - there should be regressions tests verifying this in cases where >>>>> possible. >>>>> >>>>> I.e. it should be doable to get current os::physical_memory ?using e.g. >>>>> whitebox and then run a few tests verifying the results of MaxHeapSize >>>>> and coop mode. >>>>> >>>>> Particularly the interaction between MaxRAM, MaxRAMPercentage and coops >>>>> mode would be interesting. >>>>> >>>>> Thanks, >>>>> Thomas >>>>> >>>>> >> > From kim.barrett at oracle.com Tue May 28 03:56:12 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 27 May 2019 23:56:12 -0400 Subject: RFR: 8224660: Parallel GC: Use WorkGang (2: MarksFromRootsTask) In-Reply-To: <90c03f6c-f5b1-bb39-f35b-e1de643ba652@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <90c03f6c-f5b1-bb39-f35b-e1de643ba652@oracle.com> Message-ID: > On May 24, 2019, at 9:54 AM, Leo Korinth wrote: > > Hi, > > Here is the second patch in a proposed patch series of eight that > removes gcTaskManager and uses the WorkGang abstraction instead. I will try to put these mails in the same mail thread. > > This patch moves the MarkFromRootsTask, ThreadRootsMarkingTask and > the StealMarkingTask (the last one we already moved in the last patch). > They are all called from the new MarkFromRootsTask. > > The RootType enum as well as EnumClaimer, a class that can claim > enumeration values atomically is put in an inline header file. It will > be used in later patches outside psParallelCompact if you wonder about > the header file placement. > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8224660 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224660-Parallel-GC-Use-WorkGang-2-MarksFromRootsTask/ > http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ > > Testing (on the whole patch series): > mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC > gc test suite > > Thanks, > Leo ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psRootType.inline.hpp 32 namespace Parallel { namespace RootType { We mostly don't use namespaces in HotSpot code (the style guide forbids them, probably at least in part for obsolete reasons). I don't see any reason for violating that dictum here, rather than just using AllStatic classes. ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psRootType.inline.hpp + enum_reference = reinterpret_cast(claimed); We have PrimitiveConversions::cast for this. (Until C++11+ that will require registering the enum; see metaprogramming/isRegisteredEnum.hpp.) ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psRootType.inline.hpp 53 EnumClaimer(T to_exclusive) : _counter(0), _to_exclusive(to_exclusive) { Using PrimitiveConversions::cast to explicitly convert to_exclusive for initialization of _to_exclusive is future-proof against making the enum a C++11 enum class. Again here, requires registering the enum until C++11+. ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psRootType.inline.hpp 53 EnumClaimer(T to_exclusive) : _counter(0), _to_exclusive(to_exclusive) { Should be "explicit" to disable implicit conversion. ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psRootType.inline.hpp Is EnumClaimer enough better than the existing mechanism of an if-sequence on SubTasksDone::try_claim_task to be worth having? One thing I dislike is the need for a sentinal enumerator. Of course, I don't much like the SubTasksDone::all_tasks_completed protocol either; I have some notes about it that should be turned into an RFE. Assuming we're going to use it (and replace existing SubTasksDone usages with it?), the class really should be in gc/shared, probably in workgroup.hpp (having the class declaration in a .inline.hpp prevents embedding an instance in some other class that is declared in a .hpp), with the implementation in (new) workgroup.inline.hpp. And the Parallel::RootType::Value enum should be in psRootType.hpp. ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psRootType.inline.hpp 58 assert(_counter >= _to_exclusive, "not all enumerations were claimed"); s/enumerations/enumerators/ ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psParallelCompact.cpp 2203 MarkFromRootsTask(uint active_workers) 2204 : AbstractGangTask("MarkFromRootsTask"), Typical HotSpot style puts the colon at the end of the parameter list, and indents the initializer list either 2 spaces (or 4 spaces in some recent code, mostly in ZGC). ------------------------------------------------------------------------------ src/hotspot/share/gc/parallel/psParallelCompact.cpp 2211 for (Parallel::RootType::Value root_type; _enum_claimer.try_claim(root_type); /* empty */) { 2212 mark_from_roots_task(root_type, worker_id); and 2218 if (_active_workers > 1) { 2219 steal_marking_task(*_terminator.terminator(), worker_id); Unusual indentation. ------------------------------------------------------------------------------ From leo.korinth at oracle.com Tue May 28 07:29:28 2019 From: leo.korinth at oracle.com (Leo Korinth) Date: Tue, 28 May 2019 09:29:28 +0200 Subject: RFR: 8224659: Parallel GC: Use WorkGang (1: PCRefProcTask) In-Reply-To: References: Message-ID: Hi! Yes, improvements are mostly on young collection times. The improvements really differ between benchmarks and machines (OS and number of threads). It is not unusual to see a 10% improvement in young collections. The reason seems to be that the work gang implementation is better than the task manager one. Also when taking items, no lock is used. Looking at a run, the time between actual gc work tasks is shorter, and it also seems like start and finis of the whole collection is improved. The few small regressions I saw hopefully is mostly jitter. No part of the actual algorithm has been changed (on purpose at least). The performance improvements are just a small bonus, the usage of work gangs in all GCs is the real improvement. Thanks, Leo On 27/05/2019 21:01, Peter B. Kessler wrote: > On Fri, 24 May 2019 13:25:47 +0200 Leo Korinth > wrote >> Subject: RFR: 8224659: Parallel GC: Use WorkGang (1: PCRefProcTask) >> >> Here is the first patch in a proposed patch series of eight that >> removes gcTaskManager and uses the WorkGang abstraction instead. >> .... >> Performance numbers seems to indicate >> that we go from roughly no improvements up to more than >> 40% improvements (!). Regressions seems rare. Performance improvements >> are primary in the young collections. >> .... > > I assume the performance you cite is in the collection times, not > overall application performance improvement.? Do you know if the > improvement is in the actual parceling out of the work, or better > balancing of the parceled out work among the workers, or in the > termination algorithm, or whatever?? What causes the rare regressions? > E.g., is there something adversarial about the object graphs, or > something else we should be on the lookout for? > > ??????????? ... peter From bob.vandette at oracle.com Tue May 28 13:23:19 2019 From: bob.vandette at oracle.com (Bob Vandette) Date: Tue, 28 May 2019 09:23:19 -0400 Subject: RFR: 8222252 - Java ergonomics limits heap to 128GB with disabled compressed oops In-Reply-To: <0c4150c9-91a6-d64f-880e-1173a21a03c5@oracle.com> References: <46E18910-9D4E-4219-803C-A883F48AD43B@oracle.com> <772C07C5-F2E7-4878-A94C-A99927ED204A@oracle.com> <3b410422-68b0-a143-78f5-dd0a141d1353@oracle.com> <23C2B57E-A0CA-4E44-B83D-D3728F179740@oracle.com> <0c4150c9-91a6-d64f-880e-1173a21a03c5@oracle.com> Message-ID: <548A45B9-0A25-40A1-983B-377B14520BA0@oracle.com> > On May 27, 2019, at 6:54 PM, David Holmes wrote: > > Hi Bob, > > On 25/05/2019 2:02 am, Bob Vandette wrote: >> Here?s the updated webrev. >> http://cr.openjdk.java.net/~bobv/8222252/webrev.03 >> I?ve also updated the CSR (https://bugs.openjdk.java.net/browse/JDK-8223957). > > Sorry but I'm having some trouble reconciling the code with the general description. I would have expected to see something like this: > > bool override_coop_limit = ((!FLAG_IS_DEFAULT(MaxRAMPercentage) || > !FLAG_IS_DEFAULT(MaxRAMFraction) || > !FLAG_IS_DEFAULT(MinRAMPercentage) || > !FLAG_IS_DEFAULT(MinRAMFraction) || > !FLAG_IS_DEFAULT(InitialRAMPercentage) || > ! !FLAG_IS_DEFAULT(InitialRAMFraction)) && > ! FLAG_IS_DEFAULT(MaxRAM)); > > where override_coop_limit might also be called use_physical_memory, or ignore_maxram, if I understand correctly. Then things would simplify to: > > if (override_coop_limit) { // use physical, ignore MaxRAM > phys_mem = os::physical_memory(); > FLAG_SET_ERGO(MaxRAM, (uint64_t)phys_mem); > } else { // use MaxRAM > phys_mem = FLAG_IS_DEFAULT(MaxRAM) ? MIN2(os::physical_memory(), (julong)MaxRAM) > : (julong)MaxRAM; > } The problem with your implementation is that if the user specifies a MaxRAM value that?s higher than the coop_limit, the heap size will be reduced in favor of UseCompressedOops. My CSR states "Also, If any of these flags including MaxRAM are specified on the command line, the selected Java heap size will NOT be limited by the compressed oop limit.? My rationale is that if someone goes to the effort of specifying a large MaxRAM value, they don?t want to end up with a different value without knowing it?s happening. If this is not an acceptable behavior, I can simplify my changes to match yours. > >> I?ve filed a bug to create tests for this change (https://bugs.openjdk.java.net/browse/JDK-8224764) which >> I will start working on in parallel with the integration of this change. > > Given the change is taking somewhat longer than you may have hoped, I hope the test has now progressed. I think it would be good to get everything in together rather than break it up. I split out the test since it can be integrated later in the process. I?m out of the office next week and might not have time to get that done before I return. Bob. > > Thanks, > David > >> Here are the latest results showing the old and new MaxHeapSize and UseCompressedOops values before >> and after this change. >> ./java -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep MaxHeapSize >> size_t MaxHeapSize [OLD] = 32178700288 {product} {ergonomic} >> size_t MaxHeapSize [NEW] = 113883742208 {product} {ergonomic} >> /java -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep UseCompressedOops >> bool UseCompressedOops [OLD] = true {lp64_product} {ergonomic} >> bool UseCompressedOops [NEW] = false {lp64_product} {ergonomic} >> ./java -XX:MaxRAM=192g -XX:+PrintFlagsFinal -version | grep MaxHeapSize >> size_t MaxHeapSize [OLD] = 32178700288 {product} {ergonomic} >> size_t MaxHeapSize [NEW] = 51539607552 {product} {ergonomic} >> ./java -XX:MaxRAM=192g -XX:+PrintFlagsFinal -version | grep UseCompressedOops >> bool UseCompressedOops [OLD] = true {lp64_product} {ergonomic} >> bool UseCompressedOops [NEW] = false {lp64_product} {ergonomic} >> ./java -XX:MaxRAM=192g -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep MaxHeapSize >> size_t MaxHeapSize [OLD] = 32178700288 {product} {ergonomic} >> size_t MaxHeapSize [NEW] = 154618822656 {product} {ergonomic} >> ./java -XX:MaxRAM=192g -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep UseCompressedOops >> bool UseCompressedOops [OLD] = true {lp64_product} {ergonomic} >> bool UseCompressedOops [NEW] = false {lp64_product} {ergonomic} >> ./java -XX:MaxRAMPercentage=10 -XX:+PrintFlagsFinal -version | grep MaxHeapSize >> size_t MaxHeapSize [OLD] = 13744734208 {product} {ergonomic} >> size_t MaxHeapSize [NEW] = 15183380480 {product} {ergonomic} >> ./java -XX:MaxRAMPercentage=10 -XX:+PrintFlagsFinal -version | grep UseCompressedOops >> bool UseCompressedOops = true {lp64_product} {ergonomic} >> ./java -XX:+PrintFlagsFinal -version | grep MaxHeapSize >> size_t MaxHeapSize = 32178700288 {product} {ergonomic} >> ./java -XX:+PrintFlagsFinal -version | grep UseCompressedOops >> bool UseCompressedOops = true {lp64_product} {ergonomic} >> ./java -XX:-UseCompressedOops -XX:+PrintFlagsFinal -version | grep MaxHeapSize >> size_t MaxHeapSize = 34359738368 {product} {ergonomic} >> ./java -XX:-UseCompressedOops -XX:+PrintFlagsFinal -version | grep UseCompressedOops >> bool UseCompressedOops = false {lp64_product} {command line} >> ./java -XX:+UseCompressedOops -XX:MaxRAM=128g -XX:+PrintFlagsFinal -version | grep MaxHeapSize >> size_t MaxHeapSize = 32178700288 {product} {ergonomic} >> ./java -XX:+UseCompressedOops -XX:MaxRAM=128g -XX:+PrintFlagsFinal -version | grep UseCompressedOops >> bool UseCompressedOops = true {lp64_product} {command line} >> ./java -XX:+UseCompressedOops -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep MaxHeapSize >> size_t MaxHeapSize = 32178700288 {product} {ergonomic} >> ./java -XX:+UseCompressedOops -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep UseCompressedOops >> bool UseCompressedOops = true {lp64_product} {command line} >> Bob. >>> On May 23, 2019, at 9:15 AM, Bob Vandette > wrote: >>> >>> >>>> On May 16, 2019, at 11:34 PM, David Holmes > wrote: >>>> >>>> Hi Bob, >>>> >>>> On 16/05/2019 12:52 am, Bob Vandette wrote: >>>>> I?ve updated the webrev based on your comments. >>>>> http://cr.openjdk.java.net/~bobv/8222252/webrev.02/ >>>>> Changes: >>>>> 1. MaxRAM is updated if fractional flags cause us to use OS memory size for heap selection. >>>>> 2. If MaxRAM is specified with the other fractional based flags, we will use MaxRAM as >>>>> upper limit instead of OS memory size. >>>> >>>> That seems fine, but here: >>>> >>>> 1799 if (FLAG_IS_ERGO(UseCompressedOops) && use_os_mem_limit) { >>>> >>>> Shouldn't this also be checking for FLAG_IS_DEFAULT(MaxRAM) ? Or better that check would fold into the use_os_mem_limit determination: >>>> >>>> bool use_os_mem_limit = (!FLAG_IS_DEFAULT(MaxRAMPercentage) || >>>> !FLAG_IS_DEFAULT(MaxRAMFraction) || >>>> !FLAG_IS_DEFAULT(MinRAMPercentage) || >>>> !FLAG_IS_DEFAULT(MinRAMFraction) || >>>> !FLAG_IS_DEFAULT(InitialRAMPercentage) || >>>> !FLAG_IS_DEFAULT(InitialRAMFraction)) && >>>> FLAG_IS_DEFAULT(MaxRAM); >>>> if (use_os_mem_limit) { >>>> phys_mem = os::physical_memory(); >>>> FLAG_SET_ERGO(uint64_t, MaxRAM, (uint64_t)phys_mem); >>>> } else { >>>> phys_mem = FLAG_IS_DEFAULT(MaxRAM) ? MIN2(os::physical_memory(), (julong)MaxRAM) >>>> : (julong)MaxRAM; >>>> } >>>> >>> >>> Yes, you are correct. I?ll add MaxRAM to the use_os_mem_limit check. This is the behavior >>> I documented in the CSR. >>> >>> "Also, If any of these flags including MaxRAM are specified on the command line, the selected Java heap size will not be limited by the compressed oop limit.? >>> >>> I?ll update the patch provided in the CSR as well. >>> >>> >>> Can you take a look and review the CSR for this change? >>> >>> Thanks, >>> Bob. >>> >>> >>>> ? >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> Here?s a proposed CSR for this behavioral change. >>>>> https://bugs.openjdk.java.net/browse/JDK-8223957 >>>>> Bob. >>>>>> On May 14, 2019, at 9:26 AM, Thomas Schatzl > wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> On Mon, 2019-05-13 at 10:03 -0400, Bob Vandette wrote: >>>>>>> Please review this suggested fix for correcting Heap size selection >>>>>>> when -XX:{Max,Min,Initial)RAMPercentage and their Factional >>>>>>> equivalent options are used. >>>>>>> >>>>>>> There are two bugs filed that are related to this issue (8222252 and >>>>>>> 8213175) >>>>>>> >>>>>>> SUMMARY of FIX: >>>>>>> The fix corrects what I believe is an incorrect implementation of the >>>>>>> *Percentage/*Fraction options. These options should not have caused >>>>>>> the heap size to be fractionally based on MaxRAM or limited by >>>>>>> MaxRAM. They instead should be based on the host memory. This is >>>>>>> what I assume was meant by ?real memory?. >>>>>>> >>>>>>> product(double, MaxRAMPercentage, >>>>>>> 25.0, \ >>>>>>> "Maximum percentage of real memory used for maximum heap >>>>>>> size") \ >>>>>>> range(0.0, 100.0) >>>>>>> >>>>>>> When these options are selected, they should take precedence over >>>>>>> UseCompressedOops unless UseCompressedOops is also specified on the >>>>>>> command line. >>>>>>> >>>>>>> >>>>>>> WEBREV: >>>>>>> http://cr.openjdk.java.net/~bobv/8222252/webrev.01 >>>>>> >>>>>> - this is a style preference: I would assign the result of the >>>>>> condition in the if (arguments.cpp:1721+) to the initialization of >>>>>> use_os_mem_limit directly instead of first setting it to false, and >>>>>> then to true again if needed. Feel free to ignore. >>>>>> >>>>>> - as David pointed out, this change needs a CSR. >>>>>> >>>>>> - the interaction between MaxRAM and MaxRAMPercentage is a bit unclear; >>>>>> if they are independent as implemented now, then the update of one >>>>>> should probably update the other. Or the VM should fail if they >>>>>> disagree? >>>>>> >>>>>> If they are dependent, if David suggests (i.e. MaxRAMPercentage uses >>>>>> MaxRAM as maximum real memory if set; this is what I would tend to >>>>>> prefer), then not of course. >>>>>> >>>>>> - there should be regressions tests verifying this in cases where >>>>>> possible. >>>>>> >>>>>> I.e. it should be doable to get current os::physical_memory using e.g. >>>>>> whitebox and then run a few tests verifying the results of MaxHeapSize >>>>>> and coop mode. >>>>>> >>>>>> Particularly the interaction between MaxRAM, MaxRAMPercentage and coops >>>>>> mode would be interesting. >>>>>> >>>>>> Thanks, >>>>>> Thomas >>>>>> >>>>>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From kishor.kharbas at intel.com Tue May 28 15:40:28 2019 From: kishor.kharbas at intel.com (Kharbas, Kishor) Date: Tue, 28 May 2019 15:40:28 +0000 Subject: RFR(XS) 8224247: AllocateOldGenAt fires assertion failure Message-ID: Hi, Requesting review for, CR : https://bugs.openjdk.java.net/browse/JDK-8224247 Webrev : http://cr.openjdk.java.net/~kkharbas/8224247/webrev.00 Changeset 54678 for https://bugs.openjdk.java.net/browse/JDK-8198505 replaced the two methods in CollectorPolicy - max_heap_byte_size() and heap_reserved_size_bytes(), with a single method heap_reserved_size_bytes() in [g1/parallel]Arguments. The second method was introduced in https://bugs.openjdk.java.net/browse/JDK-8202286 to differentiate between the size of reserved memory (heap_reserved_size_bytes(), which might be 2x of MaxHeapSize) and the max size heap can grow (heap_reserved_size_bytes(), which is always MaxHeapSize). When 'AllocateOldGenAt' flag is used, during heap initialization we need the first size and later on the second size. Adding the second method back fixes the issue. The fix is just adding a new method and using it during initialization of G1 and PS heap, but the existing method's name is more appropriate for the new method. So I renamed existing method. This would also avoid confusion about these methods down the line. Please suggest alternated approach if you see one. Thanks, Kishor -------------- next part -------------- An HTML attachment was scrubbed... URL: From zgu at redhat.com Tue May 28 16:17:55 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 28 May 2019 12:17:55 -0400 Subject: RFR 8224875: Shenandoah: ParallelCleaning code unloading should take lock to protect shared code roots array Message-ID: <7da54d3f-7818-3128-34ba-1adcbb2bb7c5@redhat.com> The patch changes ShenandoahHeapLock to general purpose spin lock and uses it to protect concurrent access to shared array during parallel cleaning at safepoints. Bug: https://bugs.openjdk.java.net/browse/JDK-8224875 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224875/webrev.00/ Test: hotspot_gc_shenandoah (fastdebug and release) Thanks, -Zhengyu From shade at redhat.com Tue May 28 16:28:31 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 28 May 2019 18:28:31 +0200 Subject: RFR 8224875: Shenandoah: ParallelCleaning code unloading should take lock to protect shared code roots array In-Reply-To: <7da54d3f-7818-3128-34ba-1adcbb2bb7c5@redhat.com> References: <7da54d3f-7818-3128-34ba-1adcbb2bb7c5@redhat.com> Message-ID: <8de74d12-43cc-bc0f-ef65-19a0821a5816@redhat.com> On 5/28/19 6:17 PM, Zhengyu Gu wrote: > The patch changes ShenandoahHeapLock to general purpose spin lock and uses it to protect concurrent > access to shared array during parallel cleaning at safepoints. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8224875 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224875/webrev.00/ *) Name it ShenandoahLock, maybe? The fact it is a spin lock is the implementation detail. Maybe we actually want to split out the rename into a separate changeset, so we can backport it ahead of time. *) These two typedefs deserve to be in shenandoahHeap.hpp? 96 typedef ShenandoahSpinLock ShenandoahHeapLock; 97 typedef ShenandoahSpinLocker ShenandoahHeapLocker; *) You don't need a comment here, I think it is clear what that lock protects 136 // Lock to protect recorded nms array 137 static ShenandoahSpinLock _recorded_nms_lock; *) Since you moved the assert in ShenandoahCodeRoots::remove_nmethod, is it worth it to move it in ShenandoahCodeRoots::add_nmethod too? -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From kim.barrett at oracle.com Tue May 28 17:26:02 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 28 May 2019 13:26:02 -0400 Subject: RFR: 8224659: Parallel GC: Use WorkGang (1: PCRefProcTask) In-Reply-To: <54AD082B-6261-466B-A8A1-F5B7C455633A@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <54AD082B-6261-466B-A8A1-F5B7C455633A@oracle.com> Message-ID: <80E888DF-983E-46A0-A9B9-3BDB9AC24C6B@oracle.com> On May 27, 2019, at 1:18 PM, Kim Barrett wrote: > >> On May 24, 2019, at 7:25 AM, Leo Korinth wrote: >> [?] >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8224659 >> >> Webrev: >> http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224659-Parallel-GC-Use-WorkGang-1-PCRefProcTask/ >> http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ >> >> [?] > > src/hotspot/share/gc/parallel/psParallelCompact.cpp > 2108 do { > 2109 while (ParCompactionManager::steal_objarray(worker_id, task)) { > 2110 cm->follow_array((objArrayOop)task.obj(), task.index()); > 2111 cm->follow_marking_stacks(); > 2112 } > 2113 while (ParCompactionManager::steal(worker_id, obj)) { > 2114 cm->follow_contents(obj); > 2115 cm->follow_marking_stacks(); > 2116 } > 2117 } while (!terminator.offer_termination()); > > This might offer termination earlier than it should, e.g. even if some > objarrays have become available while other objects were being > processed. Consider instead something like > > while (true) { > if (ParCompactionManager::steal_objarray(worker_id, task)) { > cm->follow_array((objArrayOop)task.obj(), task.index()); > cm->follow_marking_stacks(); > } else if (ParCompactionManager::steal(worker_id, obj)) { > cm->follow_contents(obj); > cm->follow_marking_stacks(); > } else if (terminator.offer_termination()) { > break; > } > } > > or maybe make the second clause > > } else if (ParCompactionManager::steal(worker_id, obj)) { > do { > cm->follow_contents(obj); > cm->follow_marking_stacks(); > } while (ParCompactionManager::steal(worker_id, obj)) { > } > > to only check for arrays again when run out of normal objects (as done > in the original proposed change). Leo pointed out to me that the webrev code was cribbed from the pre-existing StealMarkingTask::do_it(). So I?m okay with leaving the webrev code as-is, and have asked Leo to create a followup RFE to look into changing this (and any other similar places). From david.holmes at oracle.com Wed May 29 07:21:53 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 29 May 2019 17:21:53 +1000 Subject: RFR: 8222252 - Java ergonomics limits heap to 128GB with disabled compressed oops In-Reply-To: <548A45B9-0A25-40A1-983B-377B14520BA0@oracle.com> References: <46E18910-9D4E-4219-803C-A883F48AD43B@oracle.com> <772C07C5-F2E7-4878-A94C-A99927ED204A@oracle.com> <3b410422-68b0-a143-78f5-dd0a141d1353@oracle.com> <23C2B57E-A0CA-4E44-B83D-D3728F179740@oracle.com> <0c4150c9-91a6-d64f-880e-1173a21a03c5@oracle.com> <548A45B9-0A25-40A1-983B-377B14520BA0@oracle.com> Message-ID: <7d71f24a-39a2-0ed1-71bb-26fbe488c28b@oracle.com> Hi Bob, On 28/05/2019 11:23 pm, Bob Vandette wrote: > >> On May 27, 2019, at 6:54 PM, David Holmes > > wrote: >> >> Hi Bob, >> >> On 25/05/2019 2:02 am, Bob Vandette wrote: >>> Here?s the updated webrev. >>> http://cr.openjdk.java.net/~bobv/8222252/webrev.03 >>> ?I?ve also updated the CSR >>> (https://bugs.openjdk.java.net/browse/JDK-8223957). >> >> Sorry but I'm having some trouble reconciling the code with the >> general description. I would have expected to see something like this: >> >> bool override_coop_limit = ((!FLAG_IS_DEFAULT(MaxRAMPercentage) || >> ?????????????????????????????!FLAG_IS_DEFAULT(MaxRAMFraction) || >> ?????????????????????????????!FLAG_IS_DEFAULT(MinRAMPercentage) || >> ?????????????????????????????!FLAG_IS_DEFAULT(MinRAMFraction) || >> ?????????????????????????????!FLAG_IS_DEFAULT(InitialRAMPercentage) || >> ! ????????????????????????????!FLAG_IS_DEFAULT(InitialRAMFraction)) && >> ! ????????????????????????????FLAG_IS_DEFAULT(MaxRAM)); >> >> where override_coop_limit might also be called use_physical_memory, or >> ignore_maxram, if I understand correctly. Then things would simplify to: >> >> ?if (override_coop_limit) { // use physical, ignore MaxRAM >> ???phys_mem = os::physical_memory(); >> ???FLAG_SET_ERGO(MaxRAM, (uint64_t)phys_mem); >> ?} else { // use MaxRAM >> ???phys_mem = FLAG_IS_DEFAULT(MaxRAM) ? MIN2(os::physical_memory(), >> (julong)MaxRAM) >> ??????????????????????????????????????: (julong)MaxRAM; >> ?} > > The problem with your implementation is that if the user specifies a > MaxRAM value that?s higher than the coop_limit, > the heap size will be reduced in favor of UseCompressedOops. > > My CSR states "Also, If any of these flags including MaxRAM are > specified on the command line, the selected Java heap size will NOT be > limited by the compressed oop limit.? > > My rationale is that if someone goes to the effort of specifying a large > MaxRAM value, ?they > don?t want to end up with a different value without knowing it?s happening. So I'm having trouble establishing that the two problems: - when do I use MaxRAM over os::physical_memory(); and - when do I not allow coop_limit to set the limit are actually satisfied by the same condition that is being captured in override_coop_limit. I admit I focussed on the code needed to address the first problem, assuming it would also address the second. If that's not the case then perhaps a more explicit formulation of when to override coop_limit is needed? David ----- > If this is not an acceptable behavior, I can simplify my changes to > match yours. > >> >>> I?ve filed a bug to create tests for this change >>> (https://bugs.openjdk.java.net/browse/JDK-8224764) which >>> I will start working on in parallel with the integration of this change. >> >> Given the change is taking somewhat longer than you may have hoped, I >> hope the test has now progressed. I think it would be good to get >> everything in together rather than break it up. > > I split out the test since it can be integrated later in the process. > ?I?m out of the office next week and might not have time to get > that done before I return. > > Bob. > >> >> Thanks, >> David >> >>> Here are the latest results showing the old and new MaxHeapSize and >>> UseCompressedOops values before >>> and after this change. >>> ./java -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep >>> MaxHeapSize >>> ?? size_t MaxHeapSize ? ? ? ? ? ? ? ? ? ? [OLD]? ? = 32178700288 >>> ???????? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} >>> ?? size_t MaxHeapSize ? ? ? ? ? ? ? ? ? ? [NEW]? ? = 113883742208 >>> ??????? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} >>> /java -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep >>> UseCompressedOops >>> ?? ? bool UseCompressedOops ? ? ? ? ? ? ? [OLD]? ? = true >>> ???????????????? ? ? ? ? ? ? ? ? {lp64_product} {ergonomic} >>> ?? ? bool UseCompressedOops ? ? ? ? ? ? ? [NEW]? ? = false >>> ??????????????? ? ? ? ? ? ? ? ? {lp64_product} {ergonomic} >>> ./java -XX:MaxRAM=192g -XX:+PrintFlagsFinal -version | grep MaxHeapSize >>> ?? size_t MaxHeapSize ? ? ? ? ? ? ? ? ? ? [OLD]? ? = 32178700288 >>> ???????? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} >>> ?? size_t MaxHeapSize ? ? ? ? ? ? ? ? ? ? [NEW]? ? = 51539607552 >>> ???????? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} >>> ./java -XX:MaxRAM=192g -XX:+PrintFlagsFinal -version | grep >>> UseCompressedOops >>> ?? ? bool UseCompressedOops ? ? ? ? ? ? ? [OLD]? ? = true >>> ???????????????? ? ? ? ? ? ? ? ? {lp64_product} {ergonomic} >>> ?? ? bool UseCompressedOops ? ? ? ? ? ? ? [NEW]? ? = false >>> ??????????????? ? ? ? ? ? ? ? ? {lp64_product} {ergonomic} >>> ./java -XX:MaxRAM=192g -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal >>> -version | grep MaxHeapSize >>> ?? size_t MaxHeapSize ? ? ? ? ? ? ? ? ? ? [OLD]? ? = 32178700288 >>> ???????? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} >>> ?? size_t MaxHeapSize ? ? ? ? ? ? ? ? ? ? [NEW]? ? = 154618822656 >>> ??????? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} >>> ./java -XX:MaxRAM=192g -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal >>> -version | grep UseCompressedOops >>> ?? ? bool UseCompressedOops ? ? ? ? ? ? ? [OLD]? ? = true >>> ???????????????? ? ? ? ? ? ? ? ? {lp64_product} {ergonomic} >>> ?? ? bool UseCompressedOops ? ? ? ? ? ? ? [NEW]? ? = false >>> ??????????????? ? ? ? ? ? ? ? ? {lp64_product} {ergonomic} >>> ./java -XX:MaxRAMPercentage=10 -XX:+PrintFlagsFinal -version | grep >>> MaxHeapSize >>> ?? size_t MaxHeapSize ? ? ? ? ? ? ? ? ? ? [OLD]? ? = 13744734208 >>> ???????? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} >>> ?? size_t MaxHeapSize ? ? ? ? ? ? ? ? ? ? [NEW]? ? = 15183380480 >>> ???????? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} >>> ./java -XX:MaxRAMPercentage=10 -XX:+PrintFlagsFinal -version | grep >>> UseCompressedOops >>> ?? ? bool UseCompressedOops? ? ? ? ? ? ? ? ? ? ? ? = true >>> ???????????????? ? ? ? ? ? ? ? ? {lp64_product} {ergonomic} >>> ./java -XX:+PrintFlagsFinal -version | grep MaxHeapSize >>> ?? size_t MaxHeapSize? ? ? ? ? ? ? ? ? ? ? ? ? ? ? = 32178700288 >>> ???????? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} >>> ./java -XX:+PrintFlagsFinal -version | grep UseCompressedOops >>> ?? ? bool UseCompressedOops? ? ? ? ? ? ? ? ? ? ? ? = true >>> ???????????????? ? ? ? ? ? ? ? ? {lp64_product} {ergonomic} >>> ./java -XX:-UseCompressedOops -XX:+PrintFlagsFinal -version | grep >>> MaxHeapSize >>> ?? size_t MaxHeapSize? ? ? ? ? ? ? ? ? ? ? ? ? ? ? = 34359738368 >>> ???????? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} >>> ./java -XX:-UseCompressedOops -XX:+PrintFlagsFinal -version | grep >>> UseCompressedOops >>> ?? ? bool UseCompressedOops? ? ? ? ? ? ? ? ? ? ? ? = false >>> ??????????????? ? ? ? ? ? ? ? ? {lp64_product} {command line} >>> ./java -XX:+UseCompressedOops -XX:MaxRAM=128g -XX:+PrintFlagsFinal >>> -version | grep MaxHeapSize >>> ?? size_t MaxHeapSize? ? ? ? ? ? ? ? ? ? ? ? ? ? ? = 32178700288 >>> ???????? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} >>> ./java -XX:+UseCompressedOops -XX:MaxRAM=128g -XX:+PrintFlagsFinal >>> -version | grep UseCompressedOops >>> ?? ? bool UseCompressedOops? ? ? ? ? ? ? ? ? ? ? ? = true >>> ???????????????? ? ? ? ? ? ? ? ? {lp64_product} {command line} >>> ./java -XX:+UseCompressedOops -XX:MaxRAMPercentage=75 >>> -XX:+PrintFlagsFinal -version | grep MaxHeapSize >>> ?? size_t MaxHeapSize? ? ? ? ? ? ? ? ? ? ? ? ? ? ? = 32178700288 >>> ???????? ? ? ? ? ? ? ? ? ? ? ? {product} {ergonomic} >>> ./java -XX:+UseCompressedOops -XX:MaxRAMPercentage=75 >>> -XX:+PrintFlagsFinal -version | grep UseCompressedOops >>> ?? ? bool UseCompressedOops? ? ? ? ? ? ? ? ? ? ? ? = true >>> ???????????????? ? ? ? ? ? ? ? ? {lp64_product} {command line} >>> Bob. >>>> On May 23, 2019, at 9:15 AM, Bob Vandette >>> > >>>> wrote: >>>> >>>> >>>>> On May 16, 2019, at 11:34 PM, David Holmes >>>> > >>>>> wrote: >>>>> >>>>> Hi Bob, >>>>> >>>>> On 16/05/2019 12:52 am, Bob Vandette wrote: >>>>>> I?ve updated the webrev based on your comments. >>>>>> http://cr.openjdk.java.net/~bobv/8222252/webrev.02/ >>>>>> Changes: >>>>>> 1. MaxRAM is updated if fractional flags cause us to use OS memory >>>>>> size for heap selection. >>>>>> 2. If MaxRAM is specified with the other fractional based flags, >>>>>> we will use MaxRAM as >>>>>> ??upper limit instead of OS memory size. >>>>> >>>>> That seems fine, but here: >>>>> >>>>> 1799 ????????if (FLAG_IS_ERGO(UseCompressedOops) && use_os_mem_limit) { >>>>> >>>>> Shouldn't this also be checking for FLAG_IS_DEFAULT(MaxRAM) ? Or >>>>> better that check would fold into the use_os_mem_limit determination: >>>>> >>>>> bool use_os_mem_limit = (!FLAG_IS_DEFAULT(MaxRAMPercentage) || >>>>> ?????????????????????????!FLAG_IS_DEFAULT(MaxRAMFraction) || >>>>> ?????????????????????????!FLAG_IS_DEFAULT(MinRAMPercentage) || >>>>> ?????????????????????????!FLAG_IS_DEFAULT(MinRAMFraction) || >>>>> ?????????????????????????!FLAG_IS_DEFAULT(InitialRAMPercentage) || >>>>> ?????????????????????????!FLAG_IS_DEFAULT(InitialRAMFraction)) && >>>>> ?????????????????????????FLAG_IS_DEFAULT(MaxRAM); >>>>> if (use_os_mem_limit) { >>>>> ??phys_mem = os::physical_memory(); >>>>> ??FLAG_SET_ERGO(uint64_t, MaxRAM, (uint64_t)phys_mem); >>>>> } else { >>>>> ??phys_mem = FLAG_IS_DEFAULT(MaxRAM) ? MIN2(os::physical_memory(), >>>>> (julong)MaxRAM) >>>>> ?????????????????????????????????????: (julong)MaxRAM; >>>>> } >>>>> >>>> >>>> Yes, you are correct. ?I?ll add MaxRAM to the use_os_mem_limit >>>> check. ?This is the behavior >>>> I documented in the CSR. >>>> >>>> ?"Also, If any of these flags including MaxRAM are specified on the >>>> command line, the selected Java heap size will not be limited by the >>>> compressed oop limit.? >>>> >>>> I?ll update the patch provided in the CSR as well. >>>> >>>> >>>> Can you take a look and review the CSR for this change? >>>> >>>> Thanks, >>>> Bob. >>>> >>>> >>>>> ? >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> Here?s a proposed CSR for this behavioral change. >>>>>> https://bugs.openjdk.java.net/browse/JDK-8223957 >>>>>> Bob. >>>>>>> On May 14, 2019, at 9:26 AM, Thomas Schatzl >>>>>>> > wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> On Mon, 2019-05-13 at 10:03 -0400, Bob Vandette wrote: >>>>>>>> Please review this suggested fix for correcting Heap size selection >>>>>>>> when -XX:{Max,Min,Initial)RAMPercentage and their Factional >>>>>>>> equivalent options are used. >>>>>>>> >>>>>>>> There are two bugs filed that are related to this issue (8222252 and >>>>>>>> 8213175) >>>>>>>> >>>>>>>> SUMMARY of FIX: >>>>>>>> The fix corrects what I believe is an incorrect implementation >>>>>>>> of the >>>>>>>> *Percentage/*Fraction options. ?These options should not have caused >>>>>>>> the heap size to be fractionally based on MaxRAM or limited by >>>>>>>> MaxRAM. ?They instead should be based on the host memory. This is >>>>>>>> what I assume was meant by ?real memory?. >>>>>>>> >>>>>>>> product(double, MaxRAMPercentage, >>>>>>>> 25.0, ??????????????????????????????????\ >>>>>>>> ???????"Maximum percentage of real memory used for maximum heap >>>>>>>> size") \ >>>>>>>> ???????range(0.0, 100.0) >>>>>>>> >>>>>>>> When these options are selected, they should take precedence over >>>>>>>> UseCompressedOops unless UseCompressedOops is also specified on the >>>>>>>> command line. >>>>>>>> >>>>>>>> >>>>>>>> WEBREV: >>>>>>>> http://cr.openjdk.java.net/~bobv/8222252/webrev.01 >>>>>>> >>>>>>> - this is a style preference: I would assign the result of the >>>>>>> condition in the if (arguments.cpp:1721+) to the initialization of >>>>>>> use_os_mem_limit directly instead of first setting it to false, and >>>>>>> then to true again if needed. Feel free to ignore. >>>>>>> >>>>>>> - as David pointed out, this change needs a CSR. >>>>>>> >>>>>>> - the interaction between MaxRAM and MaxRAMPercentage is a bit >>>>>>> unclear; >>>>>>> if they are independent as implemented now, then the update of one >>>>>>> should probably update the other. Or the VM should fail if they >>>>>>> disagree? >>>>>>> >>>>>>> If they are dependent, if David suggests (i.e. MaxRAMPercentage uses >>>>>>> MaxRAM as maximum real memory if set; this is what I would tend to >>>>>>> prefer), then not of course. >>>>>>> >>>>>>> - there should be regressions tests verifying this in cases where >>>>>>> possible. >>>>>>> >>>>>>> I.e. it should be doable to get current os::physical_memory >>>>>>> ?using e.g. >>>>>>> whitebox and then run a few tests verifying the results of >>>>>>> MaxHeapSize >>>>>>> and coop mode. >>>>>>> >>>>>>> Particularly the interaction between MaxRAM, MaxRAMPercentage and >>>>>>> coops >>>>>>> mode would be interesting. >>>>>>> >>>>>>> Thanks, >>>>>>> Thomas >>>>>>> >>>>>>> >>>> > From per.liden at oracle.com Wed May 29 09:26:23 2019 From: per.liden at oracle.com (Per Liden) Date: Wed, 29 May 2019 11:26:23 +0200 Subject: RFR: 8224962: ZGC: Strengthen ZHeap::is_oop() Message-ID: <4e23c79b-2acf-2339-c634-91d8c6392c80@oracle.com> ZHeap::is_oop() currently only verifies that the metadata bits are good. However, we should also check that non-offset and non-metadata bits are not set. Bug: https://bugs.openjdk.java.net/browse/JDK-8224962 Webrev: http://cr.openjdk.java.net/~pliden/8224962/webrev.0 /Per From per.liden at oracle.com Wed May 29 09:26:55 2019 From: per.liden at oracle.com (Per Liden) Date: Wed, 29 May 2019 11:26:55 +0200 Subject: RFR: 8224965: ZGC: Strengthen ZHeap::is_in() Message-ID: In preparation for a world where we don't reserve the "third" heap view, we need to strengthen ZHeap::is_in() to take that into account. Bug: https://bugs.openjdk.java.net/browse/JDK-8224965 Webrev: http://cr.openjdk.java.net/~pliden/8224965/webrev.0 /Per From per.liden at oracle.com Wed May 29 09:27:58 2019 From: per.liden at oracle.com (Per Liden) Date: Wed, 29 May 2019 11:27:58 +0200 Subject: RFR: 8224966: ZGC: Don't skip oop verification in OopMapSet::all_do() Message-ID: <935a76c4-2aaa-0612-3e9f-18e7ebb4be87@oracle.com> When ZGC is enabled, we skip oop verification in OopMapSet::all_do(). There's no good reason to skip that. The comment suggesting this is skipped because a safepoint can happen between a load and the barrier is incorrect. We can still do the verification as is_in_or_null() doesn't require the address to have a "good" color. Also, once JDK-8224675 is integrated, this comment will make even less sense. Bug: https://bugs.openjdk.java.net/browse/JDK-8224966 Webrev: http://cr.openjdk.java.net/~pliden/8224966/webrev.0 /Per From leo.korinth at oracle.com Wed May 29 10:20:36 2019 From: leo.korinth at oracle.com (Leo Korinth) Date: Wed, 29 May 2019 12:20:36 +0200 Subject: RFR: 8224660: Parallel GC: Use WorkGang (2: MarksFromRootsTask) In-Reply-To: References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <90c03f6c-f5b1-bb39-f35b-e1de643ba652@oracle.com> Message-ID: Hi, comments inlined below... > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psRootType.inline.hpp > 32 namespace Parallel { namespace RootType { > > We mostly don't use namespaces in HotSpot code (the style guide > forbids them, probably at least in part for obsolete reasons). I > don't see any reason for violating that dictum here, rather than just > using AllStatic classes. One good reason is that it is hard to put stuff in a huge "Parallel" AllStatic class. I am attempting to change the style guide regarding suggestions of not using namespaces. I will put it in an AllStatic and name it ParallelRootType if you feel it is /better/, do you think that? > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psRootType.inline.hpp > + enum_reference = reinterpret_cast(claimed); > > We have PrimitiveConversions::cast for this. (Until C++11+ that will > require registering the enum; see metaprogramming/isRegisteredEnum.hpp.) > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psRootType.inline.hpp > 53 EnumClaimer(T to_exclusive) : _counter(0), _to_exclusive(to_exclusive) { > > Using PrimitiveConversions::cast to explicitly convert to_exclusive > for initialization of _to_exclusive is future-proof against making the > enum a C++11 enum class. > > Again here, requires registering the enum until C++11+. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psRootType.inline.hpp > 53 EnumClaimer(T to_exclusive) : _counter(0), _to_exclusive(to_exclusive) { Thanks, I will take a look at PrimitiveConversions::cast. > Should be "explicit" to disable implicit conversion. I agree (and I will fix), but it does force you to use a sentinel as you can not do enum_value_last_inclusive + 1 (see your next point). > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psRootType.inline.hpp > > Is EnumClaimer enough better than the existing mechanism of an > if-sequence on SubTasksDone::try_claim_task to be worth having? > One thing I dislike is the need for a sentinal enumerator. Of course, > I don't much like the SubTasksDone::all_tasks_completed protocol > either; I have some notes about it that should be turned into an RFE. I prefer it because it tries to be typesafe and does not have all_tasks_completed which in this case is not needed. I would prefer having a separate barrier for when it is needed. > Assuming we're going to use it (and replace existing SubTasksDone > usages with it?), the class really should be in gc/shared, probably in > workgroup.hpp (having the class declaration in a .inline.hpp prevents > embedding an instance in some other class that is declared in a .hpp), > with the implementation in (new) workgroup.inline.hpp > > And the Parallel::RootType::Value enum should be in psRootType.hpp. I did put it in workgroup to begin with and moved it out (after suggestions from colleagues) as I were the only user of it. Do you think it would be okay to leave it in the .inline.hpp file (as is) and if we find more users of the class in the future maybe move it to some shared class (and then separate it into hpp and cpp file)? I feel splitting it is overkill and hurts readability when the interface is never needed without the implementation. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psRootType.inline.hpp > 58 assert(_counter >= _to_exclusive, "not all enumerations were claimed"); > > s/enumerations/enumerators/ Will fix! > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psParallelCompact.cpp > 2203 MarkFromRootsTask(uint active_workers) > 2204 : AbstractGangTask("MarkFromRootsTask"), > > Typical HotSpot style puts the colon at the end of the parameter list, > and indents the initializer list either 2 spaces (or 4 spaces in some > recent code, mostly in ZGC). Will fix. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psParallelCompact.cpp > 2211 for (Parallel::RootType::Value root_type; _enum_claimer.try_claim(root_type); /* empty */) { > 2212 mark_from_roots_task(root_type, worker_id); > > and > > 2218 if (_active_workers > 1) { > 2219 steal_marking_task(*_terminator.terminator(), worker_id); > > Unusual indentation. will fix. > > ------------------------------------------------------------------------------ > Thanks, Leo From zgu at redhat.com Wed May 29 11:13:03 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 29 May 2019 07:13:03 -0400 Subject: RFR(S) 8224932: Shenandoah: Rename ShenandoahHeapLock, make it general purpose lock Message-ID: <44155f7e-4b04-71e1-8a4d-999c42190507@redhat.com> Please review this refactor ShenandoahHeapLock to general purpose spin lock (I kept ShenandoahSpinLock, cause ShenandoahLock was taken in another changeset. Any other suggestions for the name?) Bug: https://bugs.openjdk.java.net/browse/JDK-8224932 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224932/webrev.00/ Test: hotspot_gc_shenandoah (fastdebug and release) Thanks, -Zhengyu From shade at redhat.com Wed May 29 11:19:58 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 29 May 2019 13:19:58 +0200 Subject: RFR (XS) 8224880: ShenandoahRootScanner::roots_do assert is too strong Message-ID: <1a025fa8-c21c-d631-189b-8b0683125c14@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8224970 It reliably happens with x86_32 and gc/shenandoah/oom tests that enter the heap inspection on OOM. That operation is done under its own VM op. So, the fix is to relax the assert to accept non-Shenandoah safepoints. diff -r 785a12e0f89b src/hotspot/share/gc/shenandoah/shenandoahRootProcessor.inline.hpp --- a/src/hotspot/share/gc/shenandoah/shenandoahRootProcessor.inline.hpp Wed May 29 12:01:21 2019 +0200 +++ b/src/hotspot/share/gc/shenandoah/shenandoahRootProcessor.inline.hpp Wed May 29 12:57:03 2019 +0200 @@ -28,4 +28,5 @@ #include "gc/shenandoah/shenandoahRootProcessor.hpp" #include "gc/shenandoah/shenandoahTimingTracker.hpp" +#include "gc/shenandoah/shenandoahUtils.hpp" #include "memory/resourceArea.hpp" @@ -91,7 +92,8 @@ template void ShenandoahRootScanner::roots_do(uint worker_id, OopClosure* oops, CLDClosure* clds, CodeBlobClosure* code, ThreadClosure *tc) { - assert(!ShenandoahHeap::heap()->unload_classes() || + assert(!ShenandoahSafepoint::is_at_shenandoah_safepoint() || + !ShenandoahHeap::heap()->unload_classes() || ShenandoahHeap::heap()->heuristics()->can_do_traversal_gc(), - "No class unloading or traversal GC"); + "Expect class unloading or traversal when Shenandoah cycle is running"); ShenandoahParallelOopsDoThreadClosure tc_cl(oops, code, tc); ResourceMark rm; Testing: hotspot_gc_shenandoah + failing test -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Wed May 29 11:26:01 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 29 May 2019 13:26:01 +0200 Subject: RFR(S) 8224932: Shenandoah: Rename ShenandoahHeapLock, make it general purpose lock In-Reply-To: <44155f7e-4b04-71e1-8a4d-999c42190507@redhat.com> References: <44155f7e-4b04-71e1-8a4d-999c42190507@redhat.com> Message-ID: On 5/29/19 1:13 PM, Zhengyu Gu wrote: > Please review this refactor ShenandoahHeapLock to general purpose spin lock (I kept > ShenandoahSpinLock, cause ShenandoahLock was taken in another changeset. Any other suggestions for > the name?) I don't understand. What does it mean "cause ShenandoahLock was taken in another changeset"? There are no conflicts in current jdk/jdk I can see. > Bug: https://bugs.openjdk.java.net/browse/JDK-8224932 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224932/webrev.00/ Otherwise looks good. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Wed May 29 11:29:11 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 29 May 2019 07:29:11 -0400 Subject: RFR 8224875: Shenandoah: ParallelCleaning code unloading should take lock to protect shared code roots array In-Reply-To: <8de74d12-43cc-bc0f-ef65-19a0821a5816@redhat.com> References: <7da54d3f-7818-3128-34ba-1adcbb2bb7c5@redhat.com> <8de74d12-43cc-bc0f-ef65-19a0821a5816@redhat.com> Message-ID: <8c43b056-601b-8ed9-a625-b2ce4f66f30c@redhat.com> On 5/28/19 12:28 PM, Aleksey Shipilev wrote: > On 5/28/19 6:17 PM, Zhengyu Gu wrote: >> The patch changes ShenandoahHeapLock to general purpose spin lock and uses it to protect concurrent >> access to shared array during parallel cleaning at safepoints. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8224875 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224875/webrev.00/ > > *) Name it ShenandoahLock, maybe? The fact it is a spin lock is the implementation detail. Maybe we > actually want to split out the rename into a separate changeset, so we can backport it ahead of time. Okay, split into JDK-8224932 > > *) These two typedefs deserve to be in shenandoahHeap.hpp? > > 96 typedef ShenandoahSpinLock ShenandoahHeapLock; > 97 typedef ShenandoahSpinLocker ShenandoahHeapLocker; Done in JDK-8224932 > > *) You don't need a comment here, I think it is clear what that lock protects > > 136 // Lock to protect recorded nms array > 137 static ShenandoahSpinLock _recorded_nms_lock; Okay. > > *) Since you moved the assert in ShenandoahCodeRoots::remove_nmethod, is it worth it to move it in > ShenandoahCodeRoots::add_nmethod too? > Sure. Updated webrev: http://cr.openjdk.java.net/~zgu/JDK-8224875/webrev.01/ Reran hotspot_gc_shenandoah test. Thanks, -Zhengyu From shade at redhat.com Wed May 29 11:31:49 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 29 May 2019 13:31:49 +0200 Subject: RFR 8224875: Shenandoah: ParallelCleaning code unloading should take lock to protect shared code roots array In-Reply-To: <8c43b056-601b-8ed9-a625-b2ce4f66f30c@redhat.com> References: <7da54d3f-7818-3128-34ba-1adcbb2bb7c5@redhat.com> <8de74d12-43cc-bc0f-ef65-19a0821a5816@redhat.com> <8c43b056-601b-8ed9-a625-b2ce4f66f30c@redhat.com> Message-ID: <3aec2f45-4013-510c-2b63-883f580296f8@redhat.com> On 5/29/19 1:29 PM, Zhengyu Gu wrote: >> *) Since you moved the assert in ShenandoahCodeRoots::remove_nmethod, is it worth it to move it in >> ShenandoahCodeRoots::add_nmethod too? >> > Sure. So... should we make (add|remove)_nmethod symmetrical? If remove_* can acquire the lock, then add_* should do that too? > Updated webrev: http://cr.openjdk.java.net/~zgu/JDK-8224875/webrev.01/ Otherwise good. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Wed May 29 11:40:58 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 29 May 2019 07:40:58 -0400 Subject: RFR 8224875: Shenandoah: ParallelCleaning code unloading should take lock to protect shared code roots array In-Reply-To: <3aec2f45-4013-510c-2b63-883f580296f8@redhat.com> References: <7da54d3f-7818-3128-34ba-1adcbb2bb7c5@redhat.com> <8de74d12-43cc-bc0f-ef65-19a0821a5816@redhat.com> <8c43b056-601b-8ed9-a625-b2ce4f66f30c@redhat.com> <3aec2f45-4013-510c-2b63-883f580296f8@redhat.com> Message-ID: On 5/29/19 7:31 AM, Aleksey Shipilev wrote: > On 5/29/19 1:29 PM, Zhengyu Gu wrote: >>> *) Since you moved the assert in ShenandoahCodeRoots::remove_nmethod, is it worth it to move it in >>> ShenandoahCodeRoots::add_nmethod too? >>> >> Sure. > > So... should we make (add|remove)_nmethod symmetrical? If remove_* can acquire the lock, then add_* > should do that too? I don't think add_nmethod needs additional lock, cause it is always called with CodeCache_lock outside safepoints. The new lock only protects concurrent workers when unregister nmethods during safepoint cleanup. > >> Updated webrev: http://cr.openjdk.java.net/~zgu/JDK-8224875/webrev.01/ > > Otherwise good. Thanks, -Zhengyu > > -Aleksey > From zgu at redhat.com Wed May 29 11:43:34 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 29 May 2019 07:43:34 -0400 Subject: RFR(S) 8224932: Shenandoah: Rename ShenandoahHeapLock, make it general purpose lock In-Reply-To: References: <44155f7e-4b04-71e1-8a4d-999c42190507@redhat.com> Message-ID: <4088fa15-c496-987f-c566-78f968d942ac@redhat.com> On 5/29/19 7:26 AM, Aleksey Shipilev wrote: > On 5/29/19 1:13 PM, Zhengyu Gu wrote: >> Please review this refactor ShenandoahHeapLock to general purpose spin lock (I kept >> ShenandoahSpinLock, cause ShenandoahLock was taken in another changeset. Any other suggestions for >> the name?) > > I don't understand. What does it mean "cause ShenandoahLock was taken in another changeset"? There > are no conflicts in current jdk/jdk I can see. Right, it is in concurrent class unloading changeset, and not yet here. Thanks, -Zhengyu > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8224932 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224932/webrev.00/ > > Otherwise looks good. > > -Aleksey > From bob.vandette at oracle.com Wed May 29 11:42:57 2019 From: bob.vandette at oracle.com (Bob Vandette) Date: Wed, 29 May 2019 07:42:57 -0400 Subject: RFR: 8222252 - Java ergonomics limits heap to 128GB with disabled compressed oops In-Reply-To: <7d71f24a-39a2-0ed1-71bb-26fbe488c28b@oracle.com> References: <46E18910-9D4E-4219-803C-A883F48AD43B@oracle.com> <772C07C5-F2E7-4878-A94C-A99927ED204A@oracle.com> <3b410422-68b0-a143-78f5-dd0a141d1353@oracle.com> <23C2B57E-A0CA-4E44-B83D-D3728F179740@oracle.com> <0c4150c9-91a6-d64f-880e-1173a21a03c5@oracle.com> <548A45B9-0A25-40A1-983B-377B14520BA0@oracle.com> <7d71f24a-39a2-0ed1-71bb-26fbe488c28b@oracle.com> Message-ID: > On May 29, 2019, at 3:21 AM, David Holmes wrote: > > Hi Bob, > > On 28/05/2019 11:23 pm, Bob Vandette wrote: >>> On May 27, 2019, at 6:54 PM, David Holmes > wrote: >>> >>> Hi Bob, >>> >>> On 25/05/2019 2:02 am, Bob Vandette wrote: >>>> Here?s the updated webrev. >>>> http://cr.openjdk.java.net/~bobv/8222252/webrev.03 >>>> I?ve also updated the CSR (https://bugs.openjdk.java.net/browse/JDK-8223957). >>> >>> Sorry but I'm having some trouble reconciling the code with the general description. I would have expected to see something like this: >>> >>> bool override_coop_limit = ((!FLAG_IS_DEFAULT(MaxRAMPercentage) || >>> !FLAG_IS_DEFAULT(MaxRAMFraction) || >>> !FLAG_IS_DEFAULT(MinRAMPercentage) || >>> !FLAG_IS_DEFAULT(MinRAMFraction) || >>> !FLAG_IS_DEFAULT(InitialRAMPercentage) || >>> ! !FLAG_IS_DEFAULT(InitialRAMFraction)) && >>> ! FLAG_IS_DEFAULT(MaxRAM)); >>> >>> where override_coop_limit might also be called use_physical_memory, or ignore_maxram, if I understand correctly. Then things would simplify to: >>> >>> if (override_coop_limit) { // use physical, ignore MaxRAM >>> phys_mem = os::physical_memory(); >>> FLAG_SET_ERGO(MaxRAM, (uint64_t)phys_mem); >>> } else { // use MaxRAM >>> phys_mem = FLAG_IS_DEFAULT(MaxRAM) ? MIN2(os::physical_memory(), (julong)MaxRAM) >>> : (julong)MaxRAM; >>> } >> The problem with your implementation is that if the user specifies a MaxRAM value that?s higher than the coop_limit, >> the heap size will be reduced in favor of UseCompressedOops. >> My CSR states "Also, If any of these flags including MaxRAM are specified on the command line, the selected Java heap size will NOT be limited by the compressed oop limit.? >> My rationale is that if someone goes to the effort of specifying a large MaxRAM value, they >> don?t want to end up with a different value without knowing it?s happening. > > So I'm having trouble establishing that the two problems: > - when do I use MaxRAM over os::physical_memory(); and In all cases where MaxRAM is set on the command line. > - when do I not allow coop_limit to set the limit When any of the RAM Percentage or MaxRAM values are set. > are actually satisfied by the same condition that is being captured in override_coop_limit. I admit I focussed on the code needed to address the first problem, assuming it would also address the second. If that's not the case then perhaps a more explicit formulation of when to override coop_limit is needed? Both problems are satisfied with the changes I?ve implemented in the new webrev. Bob. > > David > ----- > >> If this is not an acceptable behavior, I can simplify my changes to match yours. >>> >>>> I?ve filed a bug to create tests for this change (https://bugs.openjdk.java.net/browse/JDK-8224764) which >>>> I will start working on in parallel with the integration of this change. >>> >>> Given the change is taking somewhat longer than you may have hoped, I hope the test has now progressed. I think it would be good to get everything in together rather than break it up. >> I split out the test since it can be integrated later in the process. I?m out of the office next week and might not have time to get >> that done before I return. >> Bob. >>> >>> Thanks, >>> David >>> >>>> Here are the latest results showing the old and new MaxHeapSize and UseCompressedOops values before >>>> and after this change. >>>> ./java -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep MaxHeapSize >>>> size_t MaxHeapSize [OLD] = 32178700288 {product} {ergonomic} >>>> size_t MaxHeapSize [NEW] = 113883742208 {product} {ergonomic} >>>> /java -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep UseCompressedOops >>>> bool UseCompressedOops [OLD] = true {lp64_product} {ergonomic} >>>> bool UseCompressedOops [NEW] = false {lp64_product} {ergonomic} >>>> ./java -XX:MaxRAM=192g -XX:+PrintFlagsFinal -version | grep MaxHeapSize >>>> size_t MaxHeapSize [OLD] = 32178700288 {product} {ergonomic} >>>> size_t MaxHeapSize [NEW] = 51539607552 {product} {ergonomic} >>>> ./java -XX:MaxRAM=192g -XX:+PrintFlagsFinal -version | grep UseCompressedOops >>>> bool UseCompressedOops [OLD] = true {lp64_product} {ergonomic} >>>> bool UseCompressedOops [NEW] = false {lp64_product} {ergonomic} >>>> ./java -XX:MaxRAM=192g -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep MaxHeapSize >>>> size_t MaxHeapSize [OLD] = 32178700288 {product} {ergonomic} >>>> size_t MaxHeapSize [NEW] = 154618822656 {product} {ergonomic} >>>> ./java -XX:MaxRAM=192g -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep UseCompressedOops >>>> bool UseCompressedOops [OLD] = true {lp64_product} {ergonomic} >>>> bool UseCompressedOops [NEW] = false {lp64_product} {ergonomic} >>>> ./java -XX:MaxRAMPercentage=10 -XX:+PrintFlagsFinal -version | grep MaxHeapSize >>>> size_t MaxHeapSize [OLD] = 13744734208 {product} {ergonomic} >>>> size_t MaxHeapSize [NEW] = 15183380480 {product} {ergonomic} >>>> ./java -XX:MaxRAMPercentage=10 -XX:+PrintFlagsFinal -version | grep UseCompressedOops >>>> bool UseCompressedOops = true {lp64_product} {ergonomic} >>>> ./java -XX:+PrintFlagsFinal -version | grep MaxHeapSize >>>> size_t MaxHeapSize = 32178700288 {product} {ergonomic} >>>> ./java -XX:+PrintFlagsFinal -version | grep UseCompressedOops >>>> bool UseCompressedOops = true {lp64_product} {ergonomic} >>>> ./java -XX:-UseCompressedOops -XX:+PrintFlagsFinal -version | grep MaxHeapSize >>>> size_t MaxHeapSize = 34359738368 {product} {ergonomic} >>>> ./java -XX:-UseCompressedOops -XX:+PrintFlagsFinal -version | grep UseCompressedOops >>>> bool UseCompressedOops = false {lp64_product} {command line} >>>> ./java -XX:+UseCompressedOops -XX:MaxRAM=128g -XX:+PrintFlagsFinal -version | grep MaxHeapSize >>>> size_t MaxHeapSize = 32178700288 {product} {ergonomic} >>>> ./java -XX:+UseCompressedOops -XX:MaxRAM=128g -XX:+PrintFlagsFinal -version | grep UseCompressedOops >>>> bool UseCompressedOops = true {lp64_product} {command line} >>>> ./java -XX:+UseCompressedOops -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep MaxHeapSize >>>> size_t MaxHeapSize = 32178700288 {product} {ergonomic} >>>> ./java -XX:+UseCompressedOops -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep UseCompressedOops >>>> bool UseCompressedOops = true {lp64_product} {command line} >>>> Bob. >>>>> On May 23, 2019, at 9:15 AM, Bob Vandette > wrote: >>>>> >>>>> >>>>>> On May 16, 2019, at 11:34 PM, David Holmes > wrote: >>>>>> >>>>>> Hi Bob, >>>>>> >>>>>> On 16/05/2019 12:52 am, Bob Vandette wrote: >>>>>>> I?ve updated the webrev based on your comments. >>>>>>> http://cr.openjdk.java.net/~bobv/8222252/webrev.02/ >>>>>>> Changes: >>>>>>> 1. MaxRAM is updated if fractional flags cause us to use OS memory size for heap selection. >>>>>>> 2. If MaxRAM is specified with the other fractional based flags, we will use MaxRAM as >>>>>>> upper limit instead of OS memory size. >>>>>> >>>>>> That seems fine, but here: >>>>>> >>>>>> 1799 if (FLAG_IS_ERGO(UseCompressedOops) && use_os_mem_limit) { >>>>>> >>>>>> Shouldn't this also be checking for FLAG_IS_DEFAULT(MaxRAM) ? Or better that check would fold into the use_os_mem_limit determination: >>>>>> >>>>>> bool use_os_mem_limit = (!FLAG_IS_DEFAULT(MaxRAMPercentage) || >>>>>> !FLAG_IS_DEFAULT(MaxRAMFraction) || >>>>>> !FLAG_IS_DEFAULT(MinRAMPercentage) || >>>>>> !FLAG_IS_DEFAULT(MinRAMFraction) || >>>>>> !FLAG_IS_DEFAULT(InitialRAMPercentage) || >>>>>> !FLAG_IS_DEFAULT(InitialRAMFraction)) && >>>>>> FLAG_IS_DEFAULT(MaxRAM); >>>>>> if (use_os_mem_limit) { >>>>>> phys_mem = os::physical_memory(); >>>>>> FLAG_SET_ERGO(uint64_t, MaxRAM, (uint64_t)phys_mem); >>>>>> } else { >>>>>> phys_mem = FLAG_IS_DEFAULT(MaxRAM) ? MIN2(os::physical_memory(), (julong)MaxRAM) >>>>>> : (julong)MaxRAM; >>>>>> } >>>>>> >>>>> >>>>> Yes, you are correct. I?ll add MaxRAM to the use_os_mem_limit check. This is the behavior >>>>> I documented in the CSR. >>>>> >>>>> "Also, If any of these flags including MaxRAM are specified on the command line, the selected Java heap size will not be limited by the compressed oop limit.? >>>>> >>>>> I?ll update the patch provided in the CSR as well. >>>>> >>>>> >>>>> Can you take a look and review the CSR for this change? >>>>> >>>>> Thanks, >>>>> Bob. >>>>> >>>>> >>>>>> ? >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> Here?s a proposed CSR for this behavioral change. >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8223957 >>>>>>> Bob. >>>>>>>> On May 14, 2019, at 9:26 AM, Thomas Schatzl > wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> On Mon, 2019-05-13 at 10:03 -0400, Bob Vandette wrote: >>>>>>>>> Please review this suggested fix for correcting Heap size selection >>>>>>>>> when -XX:{Max,Min,Initial)RAMPercentage and their Factional >>>>>>>>> equivalent options are used. >>>>>>>>> >>>>>>>>> There are two bugs filed that are related to this issue (8222252 and >>>>>>>>> 8213175) >>>>>>>>> >>>>>>>>> SUMMARY of FIX: >>>>>>>>> The fix corrects what I believe is an incorrect implementation of the >>>>>>>>> *Percentage/*Fraction options. These options should not have caused >>>>>>>>> the heap size to be fractionally based on MaxRAM or limited by >>>>>>>>> MaxRAM. They instead should be based on the host memory. This is >>>>>>>>> what I assume was meant by ?real memory?. >>>>>>>>> >>>>>>>>> product(double, MaxRAMPercentage, >>>>>>>>> 25.0, \ >>>>>>>>> "Maximum percentage of real memory used for maximum heap >>>>>>>>> size") \ >>>>>>>>> range(0.0, 100.0) >>>>>>>>> >>>>>>>>> When these options are selected, they should take precedence over >>>>>>>>> UseCompressedOops unless UseCompressedOops is also specified on the >>>>>>>>> command line. >>>>>>>>> >>>>>>>>> >>>>>>>>> WEBREV: >>>>>>>>> http://cr.openjdk.java.net/~bobv/8222252/webrev.01 >>>>>>>> >>>>>>>> - this is a style preference: I would assign the result of the >>>>>>>> condition in the if (arguments.cpp:1721+) to the initialization of >>>>>>>> use_os_mem_limit directly instead of first setting it to false, and >>>>>>>> then to true again if needed. Feel free to ignore. >>>>>>>> >>>>>>>> - as David pointed out, this change needs a CSR. >>>>>>>> >>>>>>>> - the interaction between MaxRAM and MaxRAMPercentage is a bit unclear; >>>>>>>> if they are independent as implemented now, then the update of one >>>>>>>> should probably update the other. Or the VM should fail if they >>>>>>>> disagree? >>>>>>>> >>>>>>>> If they are dependent, if David suggests (i.e. MaxRAMPercentage uses >>>>>>>> MaxRAM as maximum real memory if set; this is what I would tend to >>>>>>>> prefer), then not of course. >>>>>>>> >>>>>>>> - there should be regressions tests verifying this in cases where >>>>>>>> possible. >>>>>>>> >>>>>>>> I.e. it should be doable to get current os::physical_memory using e.g. >>>>>>>> whitebox and then run a few tests verifying the results of MaxHeapSize >>>>>>>> and coop mode. >>>>>>>> >>>>>>>> Particularly the interaction between MaxRAM, MaxRAMPercentage and coops >>>>>>>> mode would be interesting. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Thomas >>>>>>>> >>>>>>>> >>>>> From leo.korinth at oracle.com Wed May 29 12:12:04 2019 From: leo.korinth at oracle.com (Leo Korinth) Date: Wed, 29 May 2019 14:12:04 +0200 Subject: RFR: 8224659: Parallel GC: Use WorkGang (1: PCRefProcTask) In-Reply-To: <54AD082B-6261-466B-A8A1-F5B7C455633A@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <54AD082B-6261-466B-A8A1-F5B7C455633A@oracle.com> Message-ID: Hi, comments inline! On 27/05/2019 19:18, Kim Barrett wrote: >> On May 24, 2019, at 7:25 AM, Leo Korinth wrote: >> >> Hi, >> >> Here is the first patch in a proposed patch series of eight that >> removes gcTaskManager and uses the WorkGang abstraction instead. This >> is primary a refactoring to cleanup code for future >> enhancements. After this change all GCs will use the WorkGang >> abstraction. > > Thanks for splitting this up into sections. It?s definitely making reviewing easier. > >> [?] >> # This Patch >> >> This first patch does two things: >> 1) Adds work gang functionality >> + adds WorkGang named _workers to parallelScavengeHeap >> + sets active workers in psParallelCompact.cpp and >> psScavenge.cpp. The corresponding functionality in gcTaskManager >> is done by gc_task_manager()->set_active_gang() >> >> 2) Creates a PCRefProcTask that replaces RefProcTaskProxy and >> StealMarkingTask. The StealMarkingTask is removed in the next >> patch as it is still used by other parts. >> >> Enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8224659 >> >> Webrev: >> http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224659-Parallel-GC-Use-WorkGang-1-PCRefProcTask/ >> http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ >> >> Testing (on the whole patch series): >> mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC >> gc test suite >> >> Thanks, >> Leo > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psParallelCompact.cpp > 2100 void steal_marking_task(ParallelTaskTerminator& terminator, uint worker_id) { > > Should be static (file-scoped). That might also obviate the initial > assertion that it is called during a GC. Yes, I will make it static as well as all other *_task methods I have created in later patches. I got off-list feedback on this, and I do not know why I missed it. Sorry. The assertion is old, sometimes people get angry when asserts are removed. I have no strong opinion on my own here. I have kept all old asserts (except multiple redundant asserts when I have merged functions). Do you want it to go away? > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psParallelCompact.cpp > 2108 do { > 2109 while (ParCompactionManager::steal_objarray(worker_id, task)) { > 2110 cm->follow_array((objArrayOop)task.obj(), task.index()); > 2111 cm->follow_marking_stacks(); > 2112 } > 2113 while (ParCompactionManager::steal(worker_id, obj)) { > 2114 cm->follow_contents(obj); > 2115 cm->follow_marking_stacks(); > 2116 }()this > 2117 } while (!terminator.offer_termination()); > > This might offer termination earlier than it should, e.g. even if some > objarrays have become available while other objects were being > processed. Consider instead something like > > while (true) { > if (ParCompactionManager::steal_objarray(worker_id, task)) { > cm->follow_array((objArrayOop)task.obj(), task.index()); > cm->follow_marking_stacks(); > } else if (ParCompactionManager::steal(worker_id, obj)) { > cm->follow_contents(obj); > cm->follow_marking_stacks(); > } else if (terminator.offer_termination()) { > break; > } > } > > or maybe make the second clause > > } else if (ParCompactionManager::steal(worker_id, obj)) { > do { > cm->follow_contents(obj); > cm->follow_marking_stacks(); > } while (ParCompactionManager::steal(worker_id, obj)) { > } > > to only check for arrays again when run out of normal objects (as done > in the original proposed change). I have created https://bugs.openjdk.java.net/browse/JDK-8224976 > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psParallelCompact.cpp > 2103 ParCompactionManager* cm = > 2104 ParCompactionManager::gc_thread_compaction_manager(worker_id); > > Rather than making a new manager here, why not use the one from the > caller, having it passed to this helper function as an additional > argument? I am not making a new manager, just fetching an old one. I could pass it as a parameter (I kind of agree that the code would look better), but the code would look less like before. I would like to keep it, but I have no problem changing it if you feel a parameter makes a difference. > ------------------------------------------------------------------------------ > src/hotspot/share/gc/parallel/psParallelCompact.cpp > 1793 ParallelScavengeHeap::heap()->workers().update_active_workers(WorkerPolicy::calc_active_workers( > 1794 ParallelScavengeHeap::heap()->workers().total_workers(), > 1795 ParallelScavengeHeap::heap()->workers().active_workers(), > 1796 Threads::number_of_non_daemon_threads())); > > This appears again in src/hotspot/share/gc/parallel/psScavenge.cpp (in > PSScavenge::invoke_no_policy()): > 342 uint active_workers = ParallelScavengeHeap::heap()->workers().update_active_workers(WorkerPolicy::calc_active_workers( > 343 ParallelScavengeHeap::heap()->workers().total_workers(), > 344 ParallelScavengeHeap::heap()->workers().active_workers(), > 345 Threads::number_of_non_daemon_threads())); > > Perhaps there should be a helper for this? Or do things change later > in the sequence of changesets? I did a helper in WorkGang before, but removed it because I got some feedback that the work gang should not be involved with the worker policy, which I agree with. I think you are right in that an abstraction would be good though, maybe a method in the worker policy code that takes a WorkGang as argument? However I think this should not be fixed here, and instead be added in a separate patch as it is not specific to parallel. The same behaviour is used in G1 (a bit different in the full collector) and CMS as well. > > ------------------------------------------------------------------------------ > Thanks, Leo From zgu at redhat.com Wed May 29 12:22:26 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 29 May 2019 08:22:26 -0400 Subject: RFR (XS) 8224880: ShenandoahRootScanner::roots_do assert is too strong In-Reply-To: <1a025fa8-c21c-d631-189b-8b0683125c14@redhat.com> References: <1a025fa8-c21c-d631-189b-8b0683125c14@redhat.com> Message-ID: <957b36cf-f373-7ab5-4de6-c515a09b12a6@redhat.com> On 5/29/19 7:19 AM, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8224970 > > It reliably happens with x86_32 and gc/shenandoah/oom tests that enter the heap inspection on OOM. > That operation is done under its own VM op. So, the fix is to relax the assert to accept > non-Shenandoah safepoints. > > diff -r 785a12e0f89b src/hotspot/share/gc/shenandoah/shenandoahRootProcessor.inline.hpp > --- a/src/hotspot/share/gc/shenandoah/shenandoahRootProcessor.inline.hpp Wed May 29 12:01:21 > 2019 +0200 > +++ b/src/hotspot/share/gc/shenandoah/shenandoahRootProcessor.inline.hpp Wed May 29 12:57:03 > 2019 +0200 > @@ -28,4 +28,5 @@ > #include "gc/shenandoah/shenandoahRootProcessor.hpp" > #include "gc/shenandoah/shenandoahTimingTracker.hpp" > +#include "gc/shenandoah/shenandoahUtils.hpp" > #include "memory/resourceArea.hpp" > > @@ -91,7 +92,8 @@ > template > void ShenandoahRootScanner::roots_do(uint worker_id, OopClosure* oops, CLDClosure* clds, > CodeBlobClosure* code, ThreadClosure *tc) { > - assert(!ShenandoahHeap::heap()->unload_classes() || > + assert(!ShenandoahSafepoint::is_at_shenandoah_safepoint() || > + !ShenandoahHeap::heap()->unload_classes() || > ShenandoahHeap::heap()->heuristics()->can_do_traversal_gc(), > - "No class unloading or traversal GC"); > + "Expect class unloading or traversal when Shenandoah cycle is running"); > ShenandoahParallelOopsDoThreadClosure tc_cl(oops, code, tc); > ResourceMark rm; Okay. -Zhengyu > > > Testing: hotspot_gc_shenandoah + failing test > From leo.korinth at oracle.com Wed May 29 12:55:58 2019 From: leo.korinth at oracle.com (Leo Korinth) Date: Wed, 29 May 2019 14:55:58 +0200 Subject: RFR: 8224666: Parallel GC: Use WorkGang (8: obsolete and remove flags) In-Reply-To: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> Message-ID: Hi, Here is the eighth and last patch that removes gcTaskManager and uses the WorkGang abstraction instead. This patch removes and obsoletes two product flags. BindGCTaskThreadsToCPUs is used to run threads pinned on specific CPUs. Only implemented on Solaris and seems mostly unused. UseGCTaskAffinity is used to take tasks with a certain thread affinity; it does not work and should have been obsoleted since (at least) JDK-8138707. The code tries to obsolete BindGCTaskThreadsToCPUs and UseGCTaskAffinity in JDK 13. Push the removal _before_ next LTS(17); proposed removal is thus 16. Enhancement: https://bugs.openjdk.java.net/browse/JDK-8224666 Webrev: http://cr.openjdk.java.net/~lkorinth/workgang/0/_8224666-Parallel-GC-Use-WorkGang-8-obsolete-and-remove-flags/ http://cr.openjdk.java.net/~lkorinth/workgang/0/all/ Testing (on the whole patch series): mach5 remote-build-and-test --build-profiles linux-x64,linux-x64-debug,macosx-x64,solaris-sparcv9,windows-x64 --test open/test/hotspot/jtreg/:hotspot_gc -a -XX:+UseParallelGC gc test suite Thanks, Leo From david.holmes at oracle.com Wed May 29 13:05:36 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 29 May 2019 23:05:36 +1000 Subject: RFR: 8222252 - Java ergonomics limits heap to 128GB with disabled compressed oops In-Reply-To: References: <46E18910-9D4E-4219-803C-A883F48AD43B@oracle.com> <772C07C5-F2E7-4878-A94C-A99927ED204A@oracle.com> <3b410422-68b0-a143-78f5-dd0a141d1353@oracle.com> <23C2B57E-A0CA-4E44-B83D-D3728F179740@oracle.com> <0c4150c9-91a6-d64f-880e-1173a21a03c5@oracle.com> <548A45B9-0A25-40A1-983B-377B14520BA0@oracle.com> <7d71f24a-39a2-0ed1-71bb-26fbe488c28b@oracle.com> Message-ID: <11762b8f-32d4-d9de-29a2-74ef91bac219@oracle.com> On 29/05/2019 9:42 pm, Bob Vandette wrote: > >> On May 29, 2019, at 3:21 AM, David Holmes wrote: >> >> Hi Bob, >> >> On 28/05/2019 11:23 pm, Bob Vandette wrote: >>>> On May 27, 2019, at 6:54 PM, David Holmes > wrote: >>>> >>>> Hi Bob, >>>> >>>> On 25/05/2019 2:02 am, Bob Vandette wrote: >>>>> Here?s the updated webrev. >>>>> http://cr.openjdk.java.net/~bobv/8222252/webrev.03 >>>>> I?ve also updated the CSR (https://bugs.openjdk.java.net/browse/JDK-8223957). >>>> >>>> Sorry but I'm having some trouble reconciling the code with the general description. I would have expected to see something like this: >>>> >>>> bool override_coop_limit = ((!FLAG_IS_DEFAULT(MaxRAMPercentage) || >>>> !FLAG_IS_DEFAULT(MaxRAMFraction) || >>>> !FLAG_IS_DEFAULT(MinRAMPercentage) || >>>> !FLAG_IS_DEFAULT(MinRAMFraction) || >>>> !FLAG_IS_DEFAULT(InitialRAMPercentage) || >>>> ! !FLAG_IS_DEFAULT(InitialRAMFraction)) && >>>> ! FLAG_IS_DEFAULT(MaxRAM)); >>>> >>>> where override_coop_limit might also be called use_physical_memory, or ignore_maxram, if I understand correctly. Then things would simplify to: >>>> >>>> if (override_coop_limit) { // use physical, ignore MaxRAM >>>> phys_mem = os::physical_memory(); >>>> FLAG_SET_ERGO(MaxRAM, (uint64_t)phys_mem); >>>> } else { // use MaxRAM >>>> phys_mem = FLAG_IS_DEFAULT(MaxRAM) ? MIN2(os::physical_memory(), (julong)MaxRAM) >>>> : (julong)MaxRAM; >>>> } >>> The problem with your implementation is that if the user specifies a MaxRAM value that?s higher than the coop_limit, >>> the heap size will be reduced in favor of UseCompressedOops. >>> My CSR states "Also, If any of these flags including MaxRAM are specified on the command line, the selected Java heap size will NOT be limited by the compressed oop limit.? >>> My rationale is that if someone goes to the effort of specifying a large MaxRAM value, they >>> don?t want to end up with a different value without knowing it?s happening. >> >> So I'm having trouble establishing that the two problems: >> - when do I use MaxRAM over os::physical_memory(); and > In all cases where MaxRAM is set on the command line. >> - when do I not allow coop_limit to set the limit > When any of the RAM Percentage or MaxRAM values are set. >> are actually satisfied by the same condition that is being captured in override_coop_limit. I admit I focussed on the code needed to address the first problem, assuming it would also address the second. If that's not the case then perhaps a more explicit formulation of when to override coop_limit is needed? > > Both problems are satisfied with the changes I?ve implemented in the new webrev. Okay I see it now. Thanks. I still think there should be some way to simplify the code further but what you have does what you specified. Thanks for bearing with me. CSR also reviewed. David ----- > > Bob. > > >> >> David >> ----- >> >>> If this is not an acceptable behavior, I can simplify my changes to match yours. >>>> >>>>> I?ve filed a bug to create tests for this change (https://bugs.openjdk.java.net/browse/JDK-8224764) which >>>>> I will start working on in parallel with the integration of this change. >>>> >>>> Given the change is taking somewhat longer than you may have hoped, I hope the test has now progressed. I think it would be good to get everything in together rather than break it up. >>> I split out the test since it can be integrated later in the process. I?m out of the office next week and might not have time to get >>> that done before I return. >>> Bob. >>>> >>>> Thanks, >>>> David >>>> >>>>> Here are the latest results showing the old and new MaxHeapSize and UseCompressedOops values before >>>>> and after this change. >>>>> ./java -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep MaxHeapSize >>>>> size_t MaxHeapSize [OLD] = 32178700288 {product} {ergonomic} >>>>> size_t MaxHeapSize [NEW] = 113883742208 {product} {ergonomic} >>>>> /java -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep UseCompressedOops >>>>> bool UseCompressedOops [OLD] = true {lp64_product} {ergonomic} >>>>> bool UseCompressedOops [NEW] = false {lp64_product} {ergonomic} >>>>> ./java -XX:MaxRAM=192g -XX:+PrintFlagsFinal -version | grep MaxHeapSize >>>>> size_t MaxHeapSize [OLD] = 32178700288 {product} {ergonomic} >>>>> size_t MaxHeapSize [NEW] = 51539607552 {product} {ergonomic} >>>>> ./java -XX:MaxRAM=192g -XX:+PrintFlagsFinal -version | grep UseCompressedOops >>>>> bool UseCompressedOops [OLD] = true {lp64_product} {ergonomic} >>>>> bool UseCompressedOops [NEW] = false {lp64_product} {ergonomic} >>>>> ./java -XX:MaxRAM=192g -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep MaxHeapSize >>>>> size_t MaxHeapSize [OLD] = 32178700288 {product} {ergonomic} >>>>> size_t MaxHeapSize [NEW] = 154618822656 {product} {ergonomic} >>>>> ./java -XX:MaxRAM=192g -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep UseCompressedOops >>>>> bool UseCompressedOops [OLD] = true {lp64_product} {ergonomic} >>>>> bool UseCompressedOops [NEW] = false {lp64_product} {ergonomic} >>>>> ./java -XX:MaxRAMPercentage=10 -XX:+PrintFlagsFinal -version | grep MaxHeapSize >>>>> size_t MaxHeapSize [OLD] = 13744734208 {product} {ergonomic} >>>>> size_t MaxHeapSize [NEW] = 15183380480 {product} {ergonomic} >>>>> ./java -XX:MaxRAMPercentage=10 -XX:+PrintFlagsFinal -version | grep UseCompressedOops >>>>> bool UseCompressedOops = true {lp64_product} {ergonomic} >>>>> ./java -XX:+PrintFlagsFinal -version | grep MaxHeapSize >>>>> size_t MaxHeapSize = 32178700288 {product} {ergonomic} >>>>> ./java -XX:+PrintFlagsFinal -version | grep UseCompressedOops >>>>> bool UseCompressedOops = true {lp64_product} {ergonomic} >>>>> ./java -XX:-UseCompressedOops -XX:+PrintFlagsFinal -version | grep MaxHeapSize >>>>> size_t MaxHeapSize = 34359738368 {product} {ergonomic} >>>>> ./java -XX:-UseCompressedOops -XX:+PrintFlagsFinal -version | grep UseCompressedOops >>>>> bool UseCompressedOops = false {lp64_product} {command line} >>>>> ./java -XX:+UseCompressedOops -XX:MaxRAM=128g -XX:+PrintFlagsFinal -version | grep MaxHeapSize >>>>> size_t MaxHeapSize = 32178700288 {product} {ergonomic} >>>>> ./java -XX:+UseCompressedOops -XX:MaxRAM=128g -XX:+PrintFlagsFinal -version | grep UseCompressedOops >>>>> bool UseCompressedOops = true {lp64_product} {command line} >>>>> ./java -XX:+UseCompressedOops -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep MaxHeapSize >>>>> size_t MaxHeapSize = 32178700288 {product} {ergonomic} >>>>> ./java -XX:+UseCompressedOops -XX:MaxRAMPercentage=75 -XX:+PrintFlagsFinal -version | grep UseCompressedOops >>>>> bool UseCompressedOops = true {lp64_product} {command line} >>>>> Bob. >>>>>> On May 23, 2019, at 9:15 AM, Bob Vandette > wrote: >>>>>> >>>>>> >>>>>>> On May 16, 2019, at 11:34 PM, David Holmes > wrote: >>>>>>> >>>>>>> Hi Bob, >>>>>>> >>>>>>> On 16/05/2019 12:52 am, Bob Vandette wrote: >>>>>>>> I?ve updated the webrev based on your comments. >>>>>>>> http://cr.openjdk.java.net/~bobv/8222252/webrev.02/ >>>>>>>> Changes: >>>>>>>> 1. MaxRAM is updated if fractional flags cause us to use OS memory size for heap selection. >>>>>>>> 2. If MaxRAM is specified with the other fractional based flags, we will use MaxRAM as >>>>>>>> upper limit instead of OS memory size. >>>>>>> >>>>>>> That seems fine, but here: >>>>>>> >>>>>>> 1799 if (FLAG_IS_ERGO(UseCompressedOops) && use_os_mem_limit) { >>>>>>> >>>>>>> Shouldn't this also be checking for FLAG_IS_DEFAULT(MaxRAM) ? Or better that check would fold into the use_os_mem_limit determination: >>>>>>> >>>>>>> bool use_os_mem_limit = (!FLAG_IS_DEFAULT(MaxRAMPercentage) || >>>>>>> !FLAG_IS_DEFAULT(MaxRAMFraction) || >>>>>>> !FLAG_IS_DEFAULT(MinRAMPercentage) || >>>>>>> !FLAG_IS_DEFAULT(MinRAMFraction) || >>>>>>> !FLAG_IS_DEFAULT(InitialRAMPercentage) || >>>>>>> !FLAG_IS_DEFAULT(InitialRAMFraction)) && >>>>>>> FLAG_IS_DEFAULT(MaxRAM); >>>>>>> if (use_os_mem_limit) { >>>>>>> phys_mem = os::physical_memory(); >>>>>>> FLAG_SET_ERGO(uint64_t, MaxRAM, (uint64_t)phys_mem); >>>>>>> } else { >>>>>>> phys_mem = FLAG_IS_DEFAULT(MaxRAM) ? MIN2(os::physical_memory(), (julong)MaxRAM) >>>>>>> : (julong)MaxRAM; >>>>>>> } >>>>>>> >>>>>> >>>>>> Yes, you are correct. I?ll add MaxRAM to the use_os_mem_limit check. This is the behavior >>>>>> I documented in the CSR. >>>>>> >>>>>> "Also, If any of these flags including MaxRAM are specified on the command line, the selected Java heap size will not be limited by the compressed oop limit.? >>>>>> >>>>>> I?ll update the patch provided in the CSR as well. >>>>>> >>>>>> >>>>>> Can you take a look and review the CSR for this change? >>>>>> >>>>>> Thanks, >>>>>> Bob. >>>>>> >>>>>> >>>>>>> ? >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> Here?s a proposed CSR for this behavioral change. >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8223957 >>>>>>>> Bob. >>>>>>>>> On May 14, 2019, at 9:26 AM, Thomas Schatzl > wrote: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> On Mon, 2019-05-13 at 10:03 -0400, Bob Vandette wrote: >>>>>>>>>> Please review this suggested fix for correcting Heap size selection >>>>>>>>>> when -XX:{Max,Min,Initial)RAMPercentage and their Factional >>>>>>>>>> equivalent options are used. >>>>>>>>>> >>>>>>>>>> There are two bugs filed that are related to this issue (8222252 and >>>>>>>>>> 8213175) >>>>>>>>>> >>>>>>>>>> SUMMARY of FIX: >>>>>>>>>> The fix corrects what I believe is an incorrect implementation of the >>>>>>>>>> *Percentage/*Fraction options. These options should not have caused >>>>>>>>>> the heap size to be fractionally based on MaxRAM or limited by >>>>>>>>>> MaxRAM. They instead should be based on the host memory. This is >>>>>>>>>> what I assume was meant by ?real memory?. >>>>>>>>>> >>>>>>>>>> product(double, MaxRAMPercentage, >>>>>>>>>> 25.0, \ >>>>>>>>>> "Maximum percentage of real memory used for maximum heap >>>>>>>>>> size") \ >>>>>>>>>> range(0.0, 100.0) >>>>>>>>>> >>>>>>>>>> When these options are selected, they should take precedence over >>>>>>>>>> UseCompressedOops unless UseCompressedOops is also specified on the >>>>>>>>>> command line. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> WEBREV: >>>>>>>>>> http://cr.openjdk.java.net/~bobv/8222252/webrev.01 >>>>>>>>> >>>>>>>>> - this is a style preference: I would assign the result of the >>>>>>>>> condition in the if (arguments.cpp:1721+) to the initialization of >>>>>>>>> use_os_mem_limit directly instead of first setting it to false, and >>>>>>>>> then to true again if needed. Feel free to ignore. >>>>>>>>> >>>>>>>>> - as David pointed out, this change needs a CSR. >>>>>>>>> >>>>>>>>> - the interaction between MaxRAM and MaxRAMPercentage is a bit unclear; >>>>>>>>> if they are independent as implemented now, then the update of one >>>>>>>>> should probably update the other. Or the VM should fail if they >>>>>>>>> disagree? >>>>>>>>> >>>>>>>>> If they are dependent, if David suggests (i.e. MaxRAMPercentage uses >>>>>>>>> MaxRAM as maximum real memory if set; this is what I would tend to >>>>>>>>> prefer), then not of course. >>>>>>>>> >>>>>>>>> - there should be regressions tests verifying this in cases where >>>>>>>>> possible. >>>>>>>>> >>>>>>>>> I.e. it should be doable to get current os::physical_memory using e.g. >>>>>>>>> whitebox and then run a few tests verifying the results of MaxHeapSize >>>>>>>>> and coop mode. >>>>>>>>> >>>>>>>>> Particularly the interaction between MaxRAM, MaxRAMPercentage and coops >>>>>>>>> mode would be interesting. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Thomas >>>>>>>>> >>>>>>>>> >>>>>> > From shade at redhat.com Wed May 29 13:07:43 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 29 May 2019 15:07:43 +0200 Subject: RFR(S) 8224932: Shenandoah: Rename ShenandoahHeapLock, make it general purpose lock In-Reply-To: <4088fa15-c496-987f-c566-78f968d942ac@redhat.com> References: <44155f7e-4b04-71e1-8a4d-999c42190507@redhat.com> <4088fa15-c496-987f-c566-78f968d942ac@redhat.com> Message-ID: <99dbf2fa-eae9-5a21-2e77-49d26a23dbb9@redhat.com> On 5/29/19 1:43 PM, Zhengyu Gu wrote: > On 5/29/19 7:26 AM, Aleksey Shipilev wrote: >> On 5/29/19 1:13 PM, Zhengyu Gu wrote: >>> Please review this refactor ShenandoahHeapLock to general purpose spin lock (I kept >>> ShenandoahSpinLock, cause ShenandoahLock was taken in another changeset. Any other suggestions for >>> the name?) >> >> I don't understand. What does it mean "cause ShenandoahLock was taken in another changeset"? There >> are no conflicts in current jdk/jdk I can see. > > Right, it is in concurrent class unloading changeset, and not yet here. Well, that means *that* changeset should use another identifier. It has no business hoarding the generic "ShenandoahLock", IMO. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Wed May 29 13:37:56 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 29 May 2019 15:37:56 +0200 Subject: RFR 8224875: Shenandoah: ParallelCleaning code unloading should take lock to protect shared code roots array In-Reply-To: References: <7da54d3f-7818-3128-34ba-1adcbb2bb7c5@redhat.com> <8de74d12-43cc-bc0f-ef65-19a0821a5816@redhat.com> <8c43b056-601b-8ed9-a625-b2ce4f66f30c@redhat.com> <3aec2f45-4013-510c-2b63-883f580296f8@redhat.com> Message-ID: On 5/29/19 1:40 PM, Zhengyu Gu wrote: > On 5/29/19 7:31 AM, Aleksey Shipilev wrote: >> On 5/29/19 1:29 PM, Zhengyu Gu wrote: >>>> *) Since you moved the assert in ShenandoahCodeRoots::remove_nmethod, is it worth it to move it in >>>> ShenandoahCodeRoots::add_nmethod too? >>>> >>> Sure. >> >> So... should we make (add|remove)_nmethod symmetrical? If remove_* can acquire the lock, then add_* >> should do that too? > I don't think add_nmethod needs additional lock, cause it is always called with CodeCache_lock > outside safepoints. The new lock only protects concurrent workers when unregister nmethods during > safepoint cleanup. Okay, this asymmetry still irks me. I see G1CollectedHeap::register_nmethod eventually calls into HeapRegion::add_strong_code_root_locked, which does assert_locked_or_safepoint(CodeCache_lock), and then does the complicated check like this downstream: assert((CodeCache_lock->owned_by_self() || (SafepointSynchronize::is_at_safepoint() && (_m.owned_by_self() || Thread::current()->is_VM_thread()))), How much does it really hurt for us to be on a safe side and grab the lock on add_* path too? We would normally enter there with CodeCache lock held, and this might just protect us from some rare corner case. Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Wed May 29 13:44:53 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 29 May 2019 09:44:53 -0400 Subject: RFR(S) 8224932: Shenandoah: Rename ShenandoahHeapLock, make it general purpose lock In-Reply-To: <99dbf2fa-eae9-5a21-2e77-49d26a23dbb9@redhat.com> References: <44155f7e-4b04-71e1-8a4d-999c42190507@redhat.com> <4088fa15-c496-987f-c566-78f968d942ac@redhat.com> <99dbf2fa-eae9-5a21-2e77-49d26a23dbb9@redhat.com> Message-ID: <7d164b80-b166-de63-4873-3833418e751b@redhat.com> Okay, renamed and pushed. Thanks, -Zhengyu On 5/29/19 9:07 AM, Aleksey Shipilev wrote: > On 5/29/19 1:43 PM, Zhengyu Gu wrote: >> On 5/29/19 7:26 AM, Aleksey Shipilev wrote: >>> On 5/29/19 1:13 PM, Zhengyu Gu wrote: >>>> Please review this refactor ShenandoahHeapLock to general purpose spin lock (I kept >>>> ShenandoahSpinLock, cause ShenandoahLock was taken in another changeset. Any other suggestions for >>>> the name?) >>> >>> I don't understand. What does it mean "cause ShenandoahLock was taken in another changeset"? There >>> are no conflicts in current jdk/jdk I can see. >> >> Right, it is in concurrent class unloading changeset, and not yet here. > > Well, that means *that* changeset should use another identifier. It has no business hoarding the > generic "ShenandoahLock", IMO. > > -Aleksey > From zgu at redhat.com Wed May 29 14:30:24 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 29 May 2019 10:30:24 -0400 Subject: RFR 8224875: Shenandoah: ParallelCleaning code unloading should take lock to protect shared code roots array In-Reply-To: References: <7da54d3f-7818-3128-34ba-1adcbb2bb7c5@redhat.com> <8de74d12-43cc-bc0f-ef65-19a0821a5816@redhat.com> <8c43b056-601b-8ed9-a625-b2ce4f66f30c@redhat.com> <3aec2f45-4013-510c-2b63-883f580296f8@redhat.com> Message-ID: On 5/29/19 9:37 AM, Aleksey Shipilev wrote: > On 5/29/19 1:40 PM, Zhengyu Gu wrote: >> On 5/29/19 7:31 AM, Aleksey Shipilev wrote: >>> On 5/29/19 1:29 PM, Zhengyu Gu wrote: >>>>> *) Since you moved the assert in ShenandoahCodeRoots::remove_nmethod, is it worth it to move it in >>>>> ShenandoahCodeRoots::add_nmethod too? >>>>> >>>> Sure. >>> >>> So... should we make (add|remove)_nmethod symmetrical? If remove_* can acquire the lock, then add_* >>> should do that too? >> I don't think add_nmethod needs additional lock, cause it is always called with CodeCache_lock >> outside safepoints. The new lock only protects concurrent workers when unregister nmethods during >> safepoint cleanup. > > Okay, this asymmetry still irks me. > > I see G1CollectedHeap::register_nmethod eventually calls into > HeapRegion::add_strong_code_root_locked, which does assert_locked_or_safepoint(CodeCache_lock), and > then does the complicated check like this downstream: > > assert((CodeCache_lock->owned_by_self() || > (SafepointSynchronize::is_at_safepoint() && > (_m.owned_by_self() || Thread::current()->is_VM_thread()))), > > How much does it really hurt for us to be on a safe side and grab the lock on add_* path too? We > would normally enter there with CodeCache lock held, and this might just protect us from some rare > corner case. We can make it symmetric, but it does nothing :-) In add_nmethod(), We assert: assert(CodeCache_lock->owned_by_self(), "Must own CodeCache_lock"); if we put lock there as we do for remove_nmethod() ShenandoahLocker locker(CodeCache_lock->owned_by_self() ? NULL : &_recorded_nms_lock); It should always pass NULL to locker. How about enhancing assertions and comments, clearly states the purpose of the lock. http://cr.openjdk.java.net/~zgu/JDK-8224875/webrev.02/ Thanks, -Zhengyu > > Thanks, > -Aleksey > From shade at redhat.com Wed May 29 14:42:09 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 29 May 2019 16:42:09 +0200 Subject: RFR 8224875: Shenandoah: ParallelCleaning code unloading should take lock to protect shared code roots array In-Reply-To: References: <7da54d3f-7818-3128-34ba-1adcbb2bb7c5@redhat.com> <8de74d12-43cc-bc0f-ef65-19a0821a5816@redhat.com> <8c43b056-601b-8ed9-a625-b2ce4f66f30c@redhat.com> <3aec2f45-4013-510c-2b63-883f580296f8@redhat.com> Message-ID: <449d30c2-bd06-1999-af46-ad0b5f7f4b77@redhat.com> On 5/29/19 4:30 PM, Zhengyu Gu wrote: >> How much does it really hurt for us to be on a safe side and grab the lock on add_* path too? We >> would normally enter there with CodeCache lock held, and this might just protect us from some rare >> corner case. > > We can make it symmetric, but it does nothing :-) Please do. We *think* it does nothing now, because we don't see how we can enter add_nmethod without holding the CodeCache_lock. But do understand that is a flaky temporary understanding that can change at any moment. The code should fail safe when that happens. > ShenandoahLocker locker(CodeCache_lock->owned_by_self() ? NULL : &_recorded_nms_lock); > It should always pass NULL to locker. ...and that is a problem how? The same happens in remove_nmethod. > How about enhancing assertions and comments, clearly states the purpose of the lock. > http://cr.openjdk.java.net/~zgu/JDK-8224875/webrev.02/ No, that is somehow worse. What prevents us to put this to both methods, and be done? assert_locked_or_safepoint(CodeCache_lock); ShenandoahSpinLocker locker(CodeCache_lock->owned_by_self() ? NULL : &_recorded_nms_lock); Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Wed May 29 14:50:53 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 29 May 2019 10:50:53 -0400 Subject: RFR 8224875: Shenandoah: ParallelCleaning code unloading should take lock to protect shared code roots array In-Reply-To: <449d30c2-bd06-1999-af46-ad0b5f7f4b77@redhat.com> References: <7da54d3f-7818-3128-34ba-1adcbb2bb7c5@redhat.com> <8de74d12-43cc-bc0f-ef65-19a0821a5816@redhat.com> <8c43b056-601b-8ed9-a625-b2ce4f66f30c@redhat.com> <3aec2f45-4013-510c-2b63-883f580296f8@redhat.com> <449d30c2-bd06-1999-af46-ad0b5f7f4b77@redhat.com> Message-ID: <6e02977b-caf1-f0f8-296e-cc65320a5b2b@redhat.com> > > assert_locked_or_safepoint(CodeCache_lock); > ShenandoahSpinLocker locker(CodeCache_lock->owned_by_self() ? NULL : &_recorded_nms_lock); Okay. http://cr.openjdk.java.net/~zgu/JDK-8224875/webrev.03/ -Zhengyu > > > Thanks, > -Aleksey > From shade at redhat.com Wed May 29 14:55:16 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 29 May 2019 16:55:16 +0200 Subject: RFR 8224875: Shenandoah: ParallelCleaning code unloading should take lock to protect shared code roots array In-Reply-To: <6e02977b-caf1-f0f8-296e-cc65320a5b2b@redhat.com> References: <7da54d3f-7818-3128-34ba-1adcbb2bb7c5@redhat.com> <8de74d12-43cc-bc0f-ef65-19a0821a5816@redhat.com> <8c43b056-601b-8ed9-a625-b2ce4f66f30c@redhat.com> <3aec2f45-4013-510c-2b63-883f580296f8@redhat.com> <449d30c2-bd06-1999-af46-ad0b5f7f4b77@redhat.com> <6e02977b-caf1-f0f8-296e-cc65320a5b2b@redhat.com> Message-ID: <69274d88-1325-1a6c-816b-c3cc6084fbb0@redhat.com> On 5/29/19 4:50 PM, Zhengyu Gu wrote: >> >> ?? assert_locked_or_safepoint(CodeCache_lock); >> ?? ShenandoahSpinLocker locker(CodeCache_lock->owned_by_self() ? NULL : &_recorded_nms_lock); > > Okay. http://cr.openjdk.java.net/~zgu/JDK-8224875/webrev.03/ Add a new line here: 137 assert_locked_or_safepoint(CodeCache_lock); 138 ShenandoahLocker locker(CodeCache_lock->owned_by_self() ? NULL : &_recorded_nms_lock); 139 ShenandoahNMethodOopDetector detector; ...and the whole thing is good to go in. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Wed May 29 16:01:13 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 29 May 2019 12:01:13 -0400 Subject: RFR(S) 8224978: Shenandoah: Allows root verifier to verify some roots outside safepoints with proper locks Message-ID: Please review this patch that allows root verifier to verify CLDG and code roots outside safepoints, with proper locks. Bug: https://bugs.openjdk.java.net/browse/JDK-8224978 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224978/webrev.00/ Test: hotspot_gc_shenandoah + ShenandoahVerify (fastdebug and release) Thanks, -Zhengyu From kim.barrett at oracle.com Wed May 29 17:34:05 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 29 May 2019 13:34:05 -0400 Subject: RFR: 8224659: Parallel GC: Use WorkGang (1: PCRefProcTask) In-Reply-To: References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <54AD082B-6261-466B-A8A1-F5B7C455633A@oracle.com> Message-ID: <1D3A46A8-E690-4412-86D7-0FF122850F48@oracle.com> > On May 29, 2019, at 8:12 AM, Leo Korinth wrote: > On 27/05/2019 19:18, Kim Barrett wrote: >> src/hotspot/share/gc/parallel/psParallelCompact.cpp >> 2100 void steal_marking_task(ParallelTaskTerminator& terminator, uint worker_id) { >> Should be static (file-scoped). That might also obviate the initial >> assertion that it is called during a GC. > > Yes, I will make it static as well as all other *_task methods I have created in later patches. I got off-list feedback on this, and I do not know why I missed it. Sorry. > > The assertion is old, sometimes people get angry when asserts are removed. I have no strong opinion on my own here. I have kept all old asserts (except multiple redundant asserts when I have merged functions). Do you want it to go away? No particular preference. It just seems a bit pointless, given the sole caller is right there just a few lines later. > I have created https://bugs.openjdk.java.net/browse/JDK-8224976 Thanks. >> src/hotspot/share/gc/parallel/psParallelCompact.cpp >> 2103 ParCompactionManager* cm = >> 2104 ParCompactionManager::gc_thread_compaction_manager(worker_id); >> Rather than making a new manager here, why not use the one from the >> caller, having it passed to this helper function as an additional >> argument? > > I am not making a new manager, just fetching an old one. I could pass it as a parameter (I kind of agree that the code would look better), but the code would look less like before. I would like to keep it, but I have no problem changing it if you feel a parameter makes a difference. Oops, I missed that is's a pointer. In that case I have no strong preference. >> src/hotspot/share/gc/parallel/psParallelCompact.cpp >> 1793 ParallelScavengeHeap::heap()->workers().update_active_workers(WorkerPolicy::calc_active_workers( >> 1794 ParallelScavengeHeap::heap()->workers().total_workers(), >> 1795 ParallelScavengeHeap::heap()->workers().active_workers(), >> 1796 Threads::number_of_non_daemon_threads())); >> This appears again in src/hotspot/share/gc/parallel/psScavenge.cpp (in >> PSScavenge::invoke_no_policy()): >> 342 uint active_workers = ParallelScavengeHeap::heap()->workers().update_active_workers(WorkerPolicy::calc_active_workers( >> 343 ParallelScavengeHeap::heap()->workers().total_workers(), >> 344 ParallelScavengeHeap::heap()->workers().active_workers(), >> 345 Threads::number_of_non_daemon_threads())); >> Perhaps there should be a helper for this? Or do things change later >> in the sequence of changesets? > > I did a helper in WorkGang before, but removed it because I got some feedback that the work gang should not be involved with the worker policy, which I agree with. I think you are right in that an abstraction would be good though, maybe a method in the worker policy code that takes a WorkGang as argument? > > However I think this should not be fixed here, and instead be added in a separate patch as it is not specific to parallel. The same behaviour is used in G1 (a bit different in the full collector) and CMS as well. OK. From kim.barrett at oracle.com Wed May 29 18:08:33 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 29 May 2019 14:08:33 -0400 Subject: RFR: 8224660: Parallel GC: Use WorkGang (2: MarksFromRootsTask) In-Reply-To: References: <68496c5c-2b3b-37e5-4d02-69fed10f172e@oracle.com> <90c03f6c-f5b1-bb39-f35b-e1de643ba652@oracle.com> Message-ID: <3FA47236-0E45-4B83-B92A-E822341F75C7@oracle.com> > On May 29, 2019, at 6:20 AM, Leo Korinth wrote: >> src/hotspot/share/gc/parallel/psRootType.inline.hpp >> 32 namespace Parallel { namespace RootType { >> We mostly don't use namespaces in HotSpot code (the style guide >> forbids them, probably at least in part for obsolete reasons). I >> don't see any reason for violating that dictum here, rather than just >> using AllStatic classes. > > One good reason is that it is hard to put stuff in a huge "Parallel" AllStatic class. I am attempting to change the style guide regarding suggestions of not using namespaces. I will put it in an AllStatic and name it ParallelRootType if you feel it is /better/, do you think that? But why would we need a huge "Parallel" AllStatic class? We can just continue to use prefix naming conventions, like generations of C ;-) programmers before us. If we were starting HotSpot today, I'd be in favor of using namespaces. I just don't see the benefit of introducing them now, except in a few specific cases (like os, or if actively wanting ADL). ParallelRootType looks good to me. >> src/hotspot/share/gc/parallel/psRootType.inline.hpp >> 53 EnumClaimer(T to_exclusive) : _counter(0), _to_exclusive(to_exclusive) { > > Thanks, I will take a look at PrimitiveConversions::cast. I just noticed that the comment describing PrimitiveConversions::cast forgot to include enum in the list. (enums are not integral types.) That's a bug. >> src/hotspot/share/gc/parallel/psRootType.inline.hpp >> Is EnumClaimer enough better than the existing mechanism of an >> if-sequence on SubTasksDone::try_claim_task to be worth having? >> One thing I dislike is the need for a sentinal enumerator. Of course, >> I don't much like the SubTasksDone::all_tasks_completed protocol >> either; I have some notes about it that should be turned into an RFE. > > I prefer it because it tries to be typesafe and does not have all_tasks_completed which in this case is not needed. I would prefer having a separate barrier for when it is needed. all_tasks_completed exists to support reuse of a SubTasksDone object. It would be better to just disallow such reuse; it's so not worth the added usage complexity. My notes say the only place with a reused SubTasksDone is GenCollectedHeap, which I think therefore means CMS (since Serial doesn't have parallelism, but is code-sharing with CMS here). I haven't looked at how hard it would be to kill that re-use. >> Assuming we're going to use it (and replace existing SubTasksDone >> usages with it?), the class really should be in gc/shared, probably in >> workgroup.hpp (having the class declaration in a .inline.hpp prevents >> embedding an instance in some other class that is declared in a .hpp), >> with the implementation in (new) workgroup.inline.hpp > >> And the Parallel::RootType::Value enum should be in psRootType.hpp. > > I did put it in workgroup to begin with and moved it out (after suggestions from colleagues) as I were the only user of it. Do you think it would be okay to leave it in the .inline.hpp file (as is) and if we find more users of the class in the future maybe move it to some shared class (and then separate it into hpp and cpp file)? I feel splitting it is overkill and hurts readability when the interface is never needed without the implementation. What I really want to avoid is having two facilities for solving the same problem. That's why I suggested a different location. If the plan is to replace SubTasksDone with EnumClaimer, it should be in shared code. If that's not the plan, then I don't want EnumClaimer at all. From sangheon.kim at oracle.com Wed May 29 18:33:42 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Wed, 29 May 2019 11:33:42 -0700 Subject: RFR(XS) 8224247: AllocateOldGenAt fires assertion failure In-Reply-To: References: Message-ID: <69bb92cf-bb1d-f436-99d1-71f1abe60c81@oracle.com> Hi Kishor, On 5/28/19 8:40 AM, Kharbas, Kishor wrote: > > Hi, > > Requesting review for, > > CR : https://bugs.openjdk.java.net/browse/JDK-8224247 > > > Webrev : http://cr.openjdk.java.net/~kkharbas/8224247/webrev.00 > > Looks good. The patch passed hs-tier 1, 2 and 3. It would be helpful if you mention what kind of tests were passed when send RFR. Thanks, Sangheon > Changeset 54678 for https://bugs.openjdk.java.net/browse/JDK-8198505 > replaced the two > methods in CollectorPolicy - max_heap_byte_size() and > heap_reserved_size_bytes(), with a single method > heap_reserved_size_bytes() in [g1/parallel]Arguments. > > The second method was introduced in > https://bugs.openjdk.java.net/browse/JDK-8202286 > to differentiate > between the size of reserved memory (heap_reserved_size_bytes(), which > might be 2x of MaxHeapSize) and the max size heap can grow > (heap_reserved_size_bytes(), which is always MaxHeapSize). When > 'AllocateOldGenAt' flag is used, during heap initialization we need > the first size and later on the second size. Adding the second method > back fixes the issue. > > The fix is just adding a new method and using it during initialization > of G1 and PS heap, but the existing method's name is more appropriate > for the new method. So I renamed existing method. This would also > avoid confusion about these methods down the line. > > Please suggest alternated approach if you see one. > > Thanks, > > Kishor > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Wed May 29 19:10:40 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 29 May 2019 21:10:40 +0200 Subject: RFR(XS) 8224247: AllocateOldGenAt fires assertion failure In-Reply-To: References: Message-ID: Hi, On Tue, 2019-05-28 at 15:40 +0000, Kharbas, Kishor wrote: > Hi, > > Requesting review for, > CR : https://bugs.openjdk.java.net/browse/JDK-8224247 > Webrev : http://cr.openjdk.java.net/~kkharbas/8224247/webrev.00 > > [...] > The fix is just adding a new method and using it during > initialization of G1 and PS heap, but the existing method's name is > more appropriate for the new method. So I renamed existing method. > This would also avoid confusion about these methods down the line. > > Please suggest alternated approach if you see one. > looks good. Thomas From shade at redhat.com Wed May 29 20:13:42 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 29 May 2019 22:13:42 +0200 Subject: RFR (XS) 8225017: [TESTBUG] gc/shenandoah/oom/TestThreadFailure.java takes too long Message-ID: <1e80edef-f27e-c885-002a-5cfa6519402b@redhat.com> RFE: https://bugs.openjdk.java.net/browse/JDK-8225017 Change is to drop one of the configs and trim down the other one: diff -r b7ac60035a28 test/hotspot/jtreg/gc/shenandoah/oom/TestThreadFailure.java --- a/test/hotspot/jtreg/gc/shenandoah/oom/TestThreadFailure.java Thu May 16 15:58:15 2019 -0700 +++ b/test/hotspot/jtreg/gc/shenandoah/oom/TestThreadFailure.java Wed May 29 22:12:38 2019 +0200 @@ -64,19 +64,5 @@ { ProcessBuilder pb = ProcessTools.createJavaProcessBuilder( - "-Xmx16m", - "-XX:+UnlockExperimentalVMOptions", - "-XX:+UseShenandoahGC", - TestThreadFailure.class.getName(), - "test"); - - OutputAnalyzer analyzer = new OutputAnalyzer(pb.start()); - analyzer.shouldHaveExitValue(0); - analyzer.shouldContain("java.lang.OutOfMemoryError"); - analyzer.shouldContain("All good"); - } - - { - ProcessBuilder pb = ProcessTools.createJavaProcessBuilder( - "-Xmx128m", + "-Xmx64m", "-XX:+UnlockExperimentalVMOptions", "-XX:+UseShenandoahGC", Testing: hotspot_gc_shenandoah {x86_64, x86_32} -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Wed May 29 20:33:22 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 29 May 2019 22:33:22 +0200 Subject: RFR (XS) 8225017: [TESTBUG] gc/shenandoah/oom/TestThreadFailure.java takes too long In-Reply-To: <1e80edef-f27e-c885-002a-5cfa6519402b@redhat.com> References: <1e80edef-f27e-c885-002a-5cfa6519402b@redhat.com> Message-ID: <09f399ad-11d7-9a9d-7b7d-a849b4e0c3b9@redhat.com> Seems ok. Thanks, Roman > RFE: > https://bugs.openjdk.java.net/browse/JDK-8225017 > > Change is to drop one of the configs and trim down the other one: > > diff -r b7ac60035a28 test/hotspot/jtreg/gc/shenandoah/oom/TestThreadFailure.java > --- a/test/hotspot/jtreg/gc/shenandoah/oom/TestThreadFailure.java Thu May 16 15:58:15 2019 -0700 > +++ b/test/hotspot/jtreg/gc/shenandoah/oom/TestThreadFailure.java Wed May 29 22:12:38 2019 +0200 > @@ -64,19 +64,5 @@ > { > ProcessBuilder pb = ProcessTools.createJavaProcessBuilder( > - "-Xmx16m", > - "-XX:+UnlockExperimentalVMOptions", > - "-XX:+UseShenandoahGC", > - TestThreadFailure.class.getName(), > - "test"); > - > - OutputAnalyzer analyzer = new OutputAnalyzer(pb.start()); > - analyzer.shouldHaveExitValue(0); > - analyzer.shouldContain("java.lang.OutOfMemoryError"); > - analyzer.shouldContain("All good"); > - } > - > - { > - ProcessBuilder pb = ProcessTools.createJavaProcessBuilder( > - "-Xmx128m", > + "-Xmx64m", > "-XX:+UnlockExperimentalVMOptions", > "-XX:+UseShenandoahGC", > > Testing: hotspot_gc_shenandoah {x86_64, x86_32} > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Thu May 30 00:52:59 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 29 May 2019 20:52:59 -0400 Subject: RFR 8225014: Separate ShenandoahRootScanner method for object_iterate Message-ID: <64e54f3d-d0b2-d15e-9a04-64ae89e37ae6@redhat.com> Please review this patch that allows SH::object_iterate() invokes ShenandoahRootScanner method without assertion. A regression test is also added. Bug: https://bugs.openjdk.java.net/browse/JDK-8225014 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8225014/webrev.00/ Test: hotspot_gc_shenandoah with new regression test (fastdebug and release) Thanks, -Zhengyu From shade at redhat.com Thu May 30 09:16:17 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 30 May 2019 11:16:17 +0200 Subject: RFR 8225014: Separate ShenandoahRootScanner method for object_iterate In-Reply-To: <64e54f3d-d0b2-d15e-9a04-64ae89e37ae6@redhat.com> References: <64e54f3d-d0b2-d15e-9a04-64ae89e37ae6@redhat.com> Message-ID: <3b3cb71d-8d5d-b480-c654-06fcadc3bfef@redhat.com> On 5/30/19 2:52 AM, Zhengyu Gu wrote: > Please review this patch that allows SH::object_iterate() invokes ShenandoahRootScanner method > without assertion. A regression test is also added. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8225014 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8225014/webrev.00/ *) Suggestion for the name "roots_do_no_assert" -> "roots_do_unchecked" *) I was thinking that object_iterate should visit the code cache too. Current shared method would do that optionally depending on ShenandoahConcurrentScanCodeRoots. Maybe it is cleaner to just do the separate "roots_do_all" specifically for heap iteration? Regression test looks nice. -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Thu May 30 09:17:16 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 30 May 2019 11:17:16 +0200 Subject: RFR (XS) 8225017: [TESTBUG] gc/shenandoah/oom/TestThreadFailure.java takes too long In-Reply-To: <09f399ad-11d7-9a9d-7b7d-a849b4e0c3b9@redhat.com> References: <1e80edef-f27e-c885-002a-5cfa6519402b@redhat.com> <09f399ad-11d7-9a9d-7b7d-a849b4e0c3b9@redhat.com> Message-ID: <8bc3a938-c900-5089-55db-575690c4923d@redhat.com> Thanks, pushed. -Aleksey On 5/29/19 10:33 PM, Roman Kennke wrote: > Seems ok. > Thanks, > Roman > > >> RFE: >> https://bugs.openjdk.java.net/browse/JDK-8225017 >> >> Change is to drop one of the configs and trim down the other one: >> >> diff -r b7ac60035a28 test/hotspot/jtreg/gc/shenandoah/oom/TestThreadFailure.java >> --- a/test/hotspot/jtreg/gc/shenandoah/oom/TestThreadFailure.java Thu May 16 15:58:15 2019 -0700 >> +++ b/test/hotspot/jtreg/gc/shenandoah/oom/TestThreadFailure.java Wed May 29 22:12:38 2019 +0200 >> @@ -64,19 +64,5 @@ >> { >> ProcessBuilder pb = ProcessTools.createJavaProcessBuilder( >> - "-Xmx16m", >> - "-XX:+UnlockExperimentalVMOptions", >> - "-XX:+UseShenandoahGC", >> - TestThreadFailure.class.getName(), >> - "test"); >> - >> - OutputAnalyzer analyzer = new OutputAnalyzer(pb.start()); >> - analyzer.shouldHaveExitValue(0); >> - analyzer.shouldContain("java.lang.OutOfMemoryError"); >> - analyzer.shouldContain("All good"); >> - } >> - >> - { >> - ProcessBuilder pb = ProcessTools.createJavaProcessBuilder( >> - "-Xmx128m", >> + "-Xmx64m", >> "-XX:+UnlockExperimentalVMOptions", >> "-XX:+UseShenandoahGC", >> >> Testing: hotspot_gc_shenandoah {x86_64, x86_32} >> > -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Thu May 30 10:16:19 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 30 May 2019 12:16:19 +0200 Subject: RFR(S) 8224978: Shenandoah: Allows root verifier to verify some roots outside safepoints with proper locks In-Reply-To: References: Message-ID: <2e2de49d-782a-7966-e6d4-13ec8508b24f@redhat.com> On 5/29/19 6:01 PM, Zhengyu Gu wrote: > Please review this patch that allows root verifier to verify CLDG and code roots outside safepoints, > with proper locks. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8224978 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224978/webrev.00/ *) Don't we want to demote those checks to "asserts"? Is there a reason to have these checks in release bits (and make it pay on tiny root sets)? *) Maybe it should be moved to shenandoahAsserts. I think these new definitions: shenandoah_assert_safepoint() shenandoah_assert_locked_or_safepoint(lock) -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Thu May 30 11:44:37 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 30 May 2019 13:44:37 +0200 Subject: RFR (M) 8225048: Shenandoah x86_32 support Message-ID: <4fb7a59f-1ceb-af71-2038-a48ef6366087@redhat.com> http://cr.openjdk.java.net/~shade/8225048/webrev.01/ Some history: Shenandoah used to support x86_32 in "passive" mode long time ago. This mode relies only on stop-the-world GC to avoid implementing barriers (basically, runs Degenerated GC all the time). It was an interesting mode to see the footprint numbers you can get with uncommits and slimmer native pointers with really small microservice-size VMs. This mode was dropped before integration upstream, because many Shenandoah tests expect all heuristics/modes to work properly, and having the rudimentary x86_32 support was breaking tier1 tests. So we disabled it. Today, we have significantly simplified runtime interface thanks to LRB and elimination of separate forwarding pointer slot, and we can build the fully concurrent x86_32 with ease. This allows us to maintain 32-bit cleanness in Shenandoah code, plus serves as the proof of concept that Shenandoah can be implemented on 32-bit platform. I am planning to backport this all the way to 8u, once other improvements are backported, so keeping the patch simple is paramount. A brief tour of changes: *) One minor change in build config to accept both x86_32 and x86_64 (therefore, cc'ing build-dev); this is the same check we had before reversal in jdk12; *) C1 changes need to disambiguate for single/double-cpu slots in CAS, this is the same other BarrierSets do; *) C2 changes need shenandoah_x86_32.ad to carry adapters for CAS barriers, plus accept StoreIConditional for raw ptr stores on some paths. *) SBSAssembler change reshuffles _LP64 checks to enable x86_32 paths. It needs to deal with UseCompressedOops blocks, getting threads into a separate register, etc.; *) Test changes enable running them on x86_32, and generally 32-bit platforms -- I can split them out, but they make little sense on their own, without having the product code that actually uses them; Builds: {x86_32, x86_64} with/without PCH; other platform builds in CI Testing: hotspot_gc_shenandoah, CTW tests, jcstress, ad-hoc footprint experiments IIRC, Oracle does not build either x86_32 or Shenandoah, so I did not run it through jdk-submit. -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Thu May 30 12:14:14 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 30 May 2019 14:14:14 +0200 Subject: RFR (M) 8225048: Shenandoah x86_32 support In-Reply-To: <4fb7a59f-1ceb-af71-2038-a48ef6366087@redhat.com> References: <4fb7a59f-1ceb-af71-2038-a48ef6366087@redhat.com> Message-ID: The changes look good to me. Thanks!! Roman > http://cr.openjdk.java.net/~shade/8225048/webrev.01/ > > Some history: Shenandoah used to support x86_32 in "passive" mode long time ago. This mode relies > only on stop-the-world GC to avoid implementing barriers (basically, runs Degenerated GC all the > time). It was an interesting mode to see the footprint numbers you can get with uncommits and > slimmer native pointers with really small microservice-size VMs. This mode was dropped before > integration upstream, because many Shenandoah tests expect all heuristics/modes to work properly, > and having the rudimentary x86_32 support was breaking tier1 tests. So we disabled it. > > Today, we have significantly simplified runtime interface thanks to LRB and elimination of separate > forwarding pointer slot, and we can build the fully concurrent x86_32 with ease. This allows us to > maintain 32-bit cleanness in Shenandoah code, plus serves as the proof of concept that Shenandoah > can be implemented on 32-bit platform. > > I am planning to backport this all the way to 8u, once other improvements are backported, so keeping > the patch simple is paramount. > > A brief tour of changes: > > *) One minor change in build config to accept both x86_32 and x86_64 (therefore, cc'ing build-dev); > this is the same check we had before reversal in jdk12; > > *) C1 changes need to disambiguate for single/double-cpu slots in CAS, this is the same other > BarrierSets do; > > *) C2 changes need shenandoah_x86_32.ad to carry adapters for CAS barriers, plus accept > StoreIConditional for raw ptr stores on some paths. > > *) SBSAssembler change reshuffles _LP64 checks to enable x86_32 paths. It needs to deal with > UseCompressedOops blocks, getting threads into a separate register, etc.; > > *) Test changes enable running them on x86_32, and generally 32-bit platforms -- I can split them > out, but they make little sense on their own, without having the product code that actually uses them; > > Builds: {x86_32, x86_64} with/without PCH; other platform builds in CI > Testing: hotspot_gc_shenandoah, CTW tests, jcstress, ad-hoc footprint experiments > > IIRC, Oracle does not build either x86_32 or Shenandoah, so I did not run it through jdk-submit. > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From zgu at redhat.com Thu May 30 12:54:04 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 30 May 2019 08:54:04 -0400 Subject: RFR 8225014: Separate ShenandoahRootScanner method for object_iterate In-Reply-To: <3b3cb71d-8d5d-b480-c654-06fcadc3bfef@redhat.com> References: <64e54f3d-d0b2-d15e-9a04-64ae89e37ae6@redhat.com> <3b3cb71d-8d5d-b480-c654-06fcadc3bfef@redhat.com> Message-ID: <9c6247ae-3580-877d-362f-f499c89c10e4@redhat.com> On 5/30/19 5:16 AM, Aleksey Shipilev wrote: > On 5/30/19 2:52 AM, Zhengyu Gu wrote: >> Please review this patch that allows SH::object_iterate() invokes ShenandoahRootScanner method >> without assertion. A regression test is also added. >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8225014 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8225014/webrev.00/ > > *) Suggestion for the name "roots_do_no_assert" -> "roots_do_unchecked" > > *) I was thinking that object_iterate should visit the code cache too. Current shared method would > do that optionally depending on ShenandoahConcurrentScanCodeRoots. Maybe it is cleaner to just do > the separate "roots_do_all" specifically for heap iteration? Right. Updated: http://cr.openjdk.java.net/~zgu/JDK-8225014/webrev.01/index.html Thanks, -Zhengyu > > Regression test looks nice. > From erik.joelsson at oracle.com Thu May 30 13:29:19 2019 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 30 May 2019 06:29:19 -0700 Subject: RFR (M) 8225048: Shenandoah x86_32 support In-Reply-To: <4fb7a59f-1ceb-af71-2038-a48ef6366087@redhat.com> References: <4fb7a59f-1ceb-af71-2038-a48ef6366087@redhat.com> Message-ID: <52336e65-ba7d-891e-921b-eab2cb54bab1@oracle.com> Build change looks good. /Erik On 2019-05-30 04:44, Aleksey Shipilev wrote: > http://cr.openjdk.java.net/~shade/8225048/webrev.01/ > > Some history: Shenandoah used to support x86_32 in "passive" mode long time ago. This mode relies > only on stop-the-world GC to avoid implementing barriers (basically, runs Degenerated GC all the > time). It was an interesting mode to see the footprint numbers you can get with uncommits and > slimmer native pointers with really small microservice-size VMs. This mode was dropped before > integration upstream, because many Shenandoah tests expect all heuristics/modes to work properly, > and having the rudimentary x86_32 support was breaking tier1 tests. So we disabled it. > > Today, we have significantly simplified runtime interface thanks to LRB and elimination of separate > forwarding pointer slot, and we can build the fully concurrent x86_32 with ease. This allows us to > maintain 32-bit cleanness in Shenandoah code, plus serves as the proof of concept that Shenandoah > can be implemented on 32-bit platform. > > I am planning to backport this all the way to 8u, once other improvements are backported, so keeping > the patch simple is paramount. > > A brief tour of changes: > > *) One minor change in build config to accept both x86_32 and x86_64 (therefore, cc'ing build-dev); > this is the same check we had before reversal in jdk12; > > *) C1 changes need to disambiguate for single/double-cpu slots in CAS, this is the same other > BarrierSets do; > > *) C2 changes need shenandoah_x86_32.ad to carry adapters for CAS barriers, plus accept > StoreIConditional for raw ptr stores on some paths. > > *) SBSAssembler change reshuffles _LP64 checks to enable x86_32 paths. It needs to deal with > UseCompressedOops blocks, getting threads into a separate register, etc.; > > *) Test changes enable running them on x86_32, and generally 32-bit platforms -- I can split them > out, but they make little sense on their own, without having the product code that actually uses them; > > Builds: {x86_32, x86_64} with/without PCH; other platform builds in CI > Testing: hotspot_gc_shenandoah, CTW tests, jcstress, ad-hoc footprint experiments > > IIRC, Oracle does not build either x86_32 or Shenandoah, so I did not run it through jdk-submit. > From zgu at redhat.com Thu May 30 15:10:23 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 30 May 2019 11:10:23 -0400 Subject: RFR(S) 8224978: Shenandoah: Allows root verifier to verify some roots outside safepoints with proper locks In-Reply-To: <2e2de49d-782a-7966-e6d4-13ec8508b24f@redhat.com> References: <2e2de49d-782a-7966-e6d4-13ec8508b24f@redhat.com> Message-ID: <55d59de5-3b45-9abf-e6c3-21389dcbecce@redhat.com> On 5/30/19 6:16 AM, Aleksey Shipilev wrote: > On 5/29/19 6:01 PM, Zhengyu Gu wrote: >> Please review this patch that allows root verifier to verify CLDG and code roots outside safepoints, >> with proper locks. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8224978 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8224978/webrev.00/ > > *) Don't we want to demote those checks to "asserts"? Is there a reason to have these checks in > release bits (and make it pay on tiny root sets)? > > *) Maybe it should be moved to shenandoahAsserts. I think these new definitions: > > shenandoah_assert_safepoint() > shenandoah_assert_locked_or_safepoint(lock) Updated and retested: http://cr.openjdk.java.net/~zgu/JDK-8224978/webrev.01/ Thanks, -Zhengyu > > -Aleksey > From shade at redhat.com Thu May 30 17:00:37 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 30 May 2019 19:00:37 +0200 Subject: RFR 8225014: Separate ShenandoahRootScanner method for object_iterate In-Reply-To: <9c6247ae-3580-877d-362f-f499c89c10e4@redhat.com> References: <64e54f3d-d0b2-d15e-9a04-64ae89e37ae6@redhat.com> <3b3cb71d-8d5d-b480-c654-06fcadc3bfef@redhat.com> <9c6247ae-3580-877d-362f-f499c89c10e4@redhat.com> Message-ID: On 5/30/19 2:54 PM, Zhengyu Gu wrote: > Updated: http://cr.openjdk.java.net/~zgu/JDK-8225014/webrev.01/index.html Looks good. Maybe you can drop "worker_id" argument that is always 0. Your call. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From stuart.monteith at linaro.org Thu May 30 17:05:59 2019 From: stuart.monteith at linaro.org (Stuart Monteith) Date: Thu, 30 May 2019 18:05:59 +0100 Subject: RFR: JDK-8214527 AArch64: ZGC for Aarch64 In-Reply-To: References: <2abfb1d9-5235-e55f-6d90-ba8640cb197c@oracle.com> <1f9e4e96-26b3-8529-7025-6afd76d1ccc9@oracle.com> Message-ID: I'd like to have this merged (once everyone is happy) after Nils' patch has been merged, the thread is here: https://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2019-May/033893.html The bug: https://bugs.openjdk.java.net/browse/JDK-8224675 This necessitates some changes to the AArch64 code - I intend on wrapping the changes necessary for JDK-8224675 into the patch I've asked for reviews here, otherwise my patch will break the build. BR, Stuart On Fri, 24 May 2019 at 13:06, Stuart Monteith wrote: > > Hello, > Thanks again Per, I've clean that up: > http://cr.openjdk.java.net/~smonteith/8214527/webrev.3/ > > My intention is to add a test loop for ZGC aarch64 once it has been > merged. The performance monitoring would fit in with what we have > internally just now. > I appreciate the necessity for this. > > I've been testing with JTReg, JCStress, specJBB and I've been also > been using Lucene, which has been reliable at finding issues. Of > course, the more testing there is, the better. > > BR, > Stuart > > > On Thu, 23 May 2019 at 10:57, Per Liden wrote: > > > > Hi Stuart, > > > > On 5/21/19 3:44 PM, Stuart Monteith wrote: > > > Hi Per, > > > Thanks for taking a look at that. > > > > > > I'll wait until ZGC on x86_64 isn't experimental. When that happens, > > > if aarch64 isn't ready for production use, I'd expect we'd make the > > > definition of "UseZGC" conditional, something like: > > > > > > ---- > > > #if defined(AARCH64) > > > #define USEZGC_COND(type, name, flag, descr) > > > experimental((type),(name),(flag),(descr)) > > > #else > > > #define USEZGC_COND(type, name, flag, descr) > > > product((type),(name),(flag),(descr)) > > > #endif > > > > > > USEZGC_COND(bool, UseZGC, false, \ > > > "Use the Z garbage collector") \ > > > ---- > > > > > > That'll need more thought, but I'd like a general solution that could > > > be applied more widely if, indeed, that were even necessary. > > > > Sounds good. > > > > > > > > I've done the refactoring of z_aarch64.ad as you suggested here: > > > http://cr.openjdk.java.net/~smonteith/8214527/webrev.2/ > > > > In z_aarch64.ad, you shouldn't need the "#if INCLUDE_ZGC" stuff, as this > > should never be compiled if ZGC is disabled, right?. Also it looks like > > the code inside ins_encode can be collapsed into a single line, like: > > > > z_load_barrier_slow_reg(_masm, $dst$$Register, $mem$$base$$Register, > > $mem$$index, $mem$$scale, $mem$$disp, false /* weak */); > > > > > > > > Apart from the system calls, I agree with you about the deduplication. > > > I presume there is no expectation for there to be a SPARC linux port, > > > or other architectures were there might be a desire for a different > > > implementation. > > > > We don't need to cater for that now. If/when that day comes, we can > > adjust what's shared and what's not, if there's a need to. > > > > A more general question. What kind of testing are you doing of this, and > > will you be continuously doing it going forward? Perhaps monitoring ZGC > > performance on aarch64 too? I don't expect a lot of aarch64-specific > > issues to pop up, but since aarch64 isn't part of the normal platforms > > Oracle tests/monitors we will likely not catch such issues. > > > > cheers, > > Per > > > > > > > > BR, > > > Stuart > > > > > > On Tue, 21 May 2019 at 12:05, Per Liden wrote: > > >> > > >> Hi Stuart, > > >> > > >> On 05/17/2019 04:28 PM, Stuart Monteith wrote: > > >>> Hello, > > >>> This patch implements ZGC for AArch64. Unlike the previous set of > > >>> patches, this duplicates the x86 multimapping scheme rather than using > > >>> the Aarch64 TBI feature, where the top 8 bits are ignored by the > > >>> hardware. I didn't measure any conclusive performance problems using > > >>> multimapping. It would necessitate changes to SA, JNI, and wasn't > > >>> going to be compatible with Graal or JVMCI in general until they were > > >>> also able to switch to 64-bit literals dynamically. In addition, there > > >>> are up and coming features in the AArch64 architecture that exploit > > >>> the TBI area that we should keep open, such as MTE (Memory Tagging > > >>> Extensions) that open the door to lightweight hardware ASAN. With MTE > > >>> using 4 of the 8 bits, and ZGC and MTE perhaps needing more than 4 > > >>> bits, future conflict looks likely. > > >>> > > >>> JTreg testing hasn't demonstrated any significant differences from x86 > > >>> ZGC in my testing. > > >>> > > >>> The behaviour should be the same as is expected from x86_64, including > > >>> for the behaviour with command line arguments. Currently ZGC requires > > >>> -XX:+UnlockExperimentalVMOptions, that should be the case for AArch64 > > >>> this release. If x86_64 were to do have ZGC designated as not > > >>> experimental, we must ensure it still applies for ZGC on AArch64. > > >> > > >> I don't think there's a good way to declare the a flag product on one > > >> platform, and experimental on another. But I guess > > >> zArguments::initialize_platform() on aarch64 could check for and > > >> warn/exit if UnlockExperimentalVMOptions is false. > > >> > > >>> > > >>> The associated bug is: > > >>> https://bugs.openjdk.java.net/browse/JDK-8214527 > > >>> The patchset is: > > >>> http://cr.openjdk.java.net/~smonteith/8214527/webrev.0/ > > >> > > >> src/hotspot/cpu/aarch64/gc/z/z_aarch64.ad > > >> ----------------------------------------- > > >> I'd suggest you break out the duplicate parts of these into a function, > > >> like we have done in z_x86.ad. > > >> > > >> > > >> src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingFile_linux_aarch64.cpp > > >> src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingFile_linux_aarch64.hpp > > >> src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingPath_linux_aarch64.cpp > > >> src/hotspot/os_cpu/linux_aarch64/gc/z/zBackingPath_linux_aarch64.hpp > > >> src/hotspot/os_cpu/linux_aarch64/gc/z/zGlobals_linux_aarch64.cpp > > >> src/hotspot/os_cpu/linux_aarch64/gc/z/zGlobals_linux_aarch64.hpp > > >> src/hotspot/os_cpu/linux_aarch64/gc/z/zPhysicalMemoryBacking_linux_aarch64.cpp > > >> src/hotspot/os_cpu/linux_aarch64/gc/z/zPhysicalMemoryBacking_linux_aarch64.hpp > > >> > > >> -------------------------------------------------------------------- > > >> I'd would like to deduplicate most of these at some point, but I'm ok > > >> with doing that after this has been merged, as we might need/want to > > >> slice this slightly differently (to cater for the different syscall > > >> numbers etc). > > >> > > >> > > >> cheers, > > >> Per > > >> > > >>> > > >>> There is a patch contributed by Roland Westerlin required for fixing > > >>> memory barriers, > > >>> http://cr.openjdk.java.net/~smonteith/zgc-mm/20190430/membar/ > > >>> I'll follow up with a separate RFR patch to address that. > > >>> > > >>> Thanks, > > >>> Stuart > > >>> From shade at redhat.com Thu May 30 17:06:36 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 30 May 2019 19:06:36 +0200 Subject: RFR(S) 8224978: Shenandoah: Allows root verifier to verify some roots outside safepoints with proper locks In-Reply-To: <55d59de5-3b45-9abf-e6c3-21389dcbecce@redhat.com> References: <2e2de49d-782a-7966-e6d4-13ec8508b24f@redhat.com> <55d59de5-3b45-9abf-e6c3-21389dcbecce@redhat.com> Message-ID: <7315a095-b85d-f801-fd61-5c727eda3e81@redhat.com> On 5/30/19 5:10 PM, Zhengyu Gu wrote: > Updated and retested: > ? http://cr.openjdk.java.net/~zgu/JDK-8224978/webrev.01/ *) Synopsis is missing "S": "8224978: henandoah: Allows root verifier to verify some roots outside safepoints with proper locks" *) This include in shenandoahRootVerifier.cpp is not needed anymore, right? 36 #include "gc/shenandoah/shenandoahUtils.hpp" *) Excess capitalization. Suggestion for rewording: "Should be at Shenandoah safepoint": 132 assert(ShenandoahSafepoint::is_at_shenandoah_safepoint(), "Only at Shenandoah Safepoints"); *) Also here: "Must be at Shenandoah safepoint or hold %s lock": 375 ShenandoahMessageBuffer msg("Must at a Shenandoah safepoint or held %s lock", lock->name()); Fix it, and the patch is good to go. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From kim.barrett at oracle.com Thu May 30 19:29:49 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 30 May 2019 15:29:49 -0400 Subject: RFR: 8212630: jtreg/gc/logging/TestMetaSpaceLog.java failed with Agent timed out Message-ID: <41D37ACB-5E64-483D-9042-206C9BCE135A@oracle.com> Please review this change to a test to attempt to eliminate very intermittent timeouts in Oracle's regression testing. (Note that more recent failures don't involve agent timeouts, as the test has been modified to no longer use agent mode.) The test currently consists of one process which launches child processes to test each of a set of GCs. We suspect that approach may result in the single main process exceeding the normal timeout period when under high load or on a slow machine (or both). If so, the situation will only get worse if more GCs are added to the current (incomplete) set. We attempt to address this by instead having a separate main process for each tested GC, with the GC to test specified by a command line argument. [It's possible there is some other underlying problem that has nothing to do with the specific test, in which case this change might actually result in more frequent timeouts of this test. If that happens, we'll deal with it as a new problem, with more information than we have now, possibly including extra timing information from JDK-8219149.] CR: https://bugs.openjdk.java.net/browse/JDK-8212630 Webrev: http://cr.openjdk.java.net/~kbarrett/8212630/open.00/ Testing: A few dozen local executions of the test. That's not enough and probably not in a configuration to encounter the problem -- failures have been occurring on the order of 0.5% of the time under high load. From thomas.schatzl at oracle.com Thu May 30 20:13:22 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 30 May 2019 22:13:22 +0200 Subject: RFR: 8212630: jtreg/gc/logging/TestMetaSpaceLog.java failed with Agent timed out In-Reply-To: <41D37ACB-5E64-483D-9042-206C9BCE135A@oracle.com> References: <41D37ACB-5E64-483D-9042-206C9BCE135A@oracle.com> Message-ID: <9a8cf46de310b49f1112a4789fcc65991c76a604.camel@oracle.com> Hi, On Thu, 2019-05-30 at 15:29 -0400, Kim Barrett wrote: > Please review this change to a test to attempt to eliminate very > intermittent timeouts in Oracle's regression testing. (Note that > more recent failures don't involve agent timeouts, as the test has > been modified to no longer use agent mode.) > [...] > CR: > https://bugs.openjdk.java.net/browse/JDK-8212630 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8212630/open.00/ Looks good. Thomas From sangheon.kim at oracle.com Thu May 30 20:38:16 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Thu, 30 May 2019 13:38:16 -0700 Subject: RFR (S): 8223162: Increase ergonomics for Sparse PRT entry sizing In-Reply-To: <49a808121451ce6805c61b05d31d471b209f72b2.camel@oracle.com> References: <5a7f7eb20525ee5b2ff775dc69ffa7b18820d870.camel@oracle.com> <5ABE328D-C653-4A7B-8B5E-04F59D1843BD@oracle.com> <49a808121451ce6805c61b05d31d471b209f72b2.camel@oracle.com> Message-ID: Hi Thomas, On 5/27/19 1:44 AM, Thomas Schatzl wrote: > Hi Kim, > > On Fri, 2019-05-24 at 17:26 -0400, Kim Barrett wrote: >>> On May 24, 2019, at 9:24 AM, Thomas Schatzl < >>> thomas.schatzl at oracle.com> wrote: >>> [?]CR: >>> https://bugs.openjdk.java.net/browse/JDK-8223162 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8223162/webrev/ >>> Testing: >>> lots of remembered set size measurements on lots of applications, >>> some >>> hs-tier1-5 runs together with other changes. >>> >>> Thanks, >>> Thomas >> src/hotspot/share/gc/g1/sparsePRT.hpp >> 241 enum SomeAdditionalPrivateConstants { >> 242 InitialCapacity = 8 >> 243 }; >> >> Since you are touching it, consider making InitialCapacity a static >> const size_t rather than an enumerator, so it has a consistent type >> across platforms. >> >> Other than that, looks good. I don't need a new review if you change >> how InitialCapacity is defined. >> > thanks for your review. I changed InitialCapacity to a static const > as suggested and regenerated the webrev. Looks good to me too. Thanks, Sangheon > > Thanks, > Thomas > > From sangheon.kim at oracle.com Thu May 30 20:40:15 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Thu, 30 May 2019 13:40:15 -0700 Subject: RFR (S): 8223693: Memory wastage in size of per-region type buffers in GC In-Reply-To: <797ef3a1fbc70a84b66713d7ca272a7cb131bf72.camel@oracle.com> References: <797ef3a1fbc70a84b66713d7ca272a7cb131bf72.camel@oracle.com> Message-ID: <7ee8b98e-e049-80e3-25ec-089cd44a6145@oracle.com> Hi Thomas, On 5/24/19 5:25 AM, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this small change in the constants for > G1HeapRegionAttr::_type field, eliminating some confusion and minor > memory wastage? > > Perf testing showed no measurable impact of the change. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8223693 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8223693/webrev/ Looks good. Thanks, Sangheon > Testing: > hs-tier1-5 in conjunction with other changes; perf testing showed no > measurable impact > > Thanks, > Thomas > > From kim.barrett at oracle.com Thu May 30 21:08:17 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 30 May 2019 17:08:17 -0400 Subject: RFR: 8212630: jtreg/gc/logging/TestMetaSpaceLog.java failed with Agent timed out In-Reply-To: <9a8cf46de310b49f1112a4789fcc65991c76a604.camel@oracle.com> References: <41D37ACB-5E64-483D-9042-206C9BCE135A@oracle.com> <9a8cf46de310b49f1112a4789fcc65991c76a604.camel@oracle.com> Message-ID: <4479B6F7-8DA4-45EB-BD41-E0667C694D20@oracle.com> > On May 30, 2019, at 4:13 PM, Thomas Schatzl wrote: > > Hi, > > On Thu, 2019-05-30 at 15:29 -0400, Kim Barrett wrote: >> Please review this change to a test to attempt to eliminate very >> intermittent timeouts in Oracle's regression testing. (Note that >> more recent failures don't involve agent timeouts, as the test has >> been modified to no longer use agent mode.) >> > [...] >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8212630 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8212630/open.00/ > > Looks good. > > Thomas Thanks. From leonid.mesnik at oracle.com Thu May 30 22:22:03 2019 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Thu, 30 May 2019 15:22:03 -0700 Subject: RFR: 8212630: jtreg/gc/logging/TestMetaSpaceLog.java failed with Agent timed out In-Reply-To: <41D37ACB-5E64-483D-9042-206C9BCE135A@oracle.com> References: <41D37ACB-5E64-483D-9042-206C9BCE135A@oracle.com> Message-ID: <8D90A90A-0105-42B4-AE68-B6AF14A484B6@oracle.com> Hi I think that '@run main/othervm -Xbootclasspath/a:. -XX:+UnlockDiagnosticVMOptions -XX:+WhiteBoxAPI' could be replaced with @run main/driver because test spawns other process and don't use any options and WB. Also I think you need to have several separate headers for each test instead of multiple runs. There is only one result per test (not for every run) and test is is skipped if any of GC is not supported. Additionally these tests might be guarded with require instead of throwing SkipException. So your tests is not going to be executed saying if ZGC is selected by jtreg CLI. Here is the example of fix which update test with different GCs. It contains a lot of examples of similar tests. http://hg.openjdk.java.net/jdk/jdk/rev/9b8926bf85c1 And in the case of new timeouts/other failures we will know exactly which GC was used right in the report. Leonid > On May 30, 2019, at 12:29 PM, Kim Barrett wrote: > > Please review this change to a test to attempt to eliminate very > intermittent timeouts in Oracle's regression testing. (Note that more > recent failures don't involve agent timeouts, as the test has been > modified to no longer use agent mode.) > > The test currently consists of one process which launches child > processes to test each of a set of GCs. We suspect that approach may > result in the single main process exceeding the normal timeout period > when under high load or on a slow machine (or both). If so, the > situation will only get worse if more GCs are added to the current > (incomplete) set. > > We attempt to address this by instead having a separate main process > for each tested GC, with the GC to test specified by a command line > argument. > > [It's possible there is some other underlying problem that has nothing > to do with the specific test, in which case this change might actually > result in more frequent timeouts of this test. If that happens, we'll > deal with it as a new problem, with more information than we have now, > possibly including extra timing information from JDK-8219149.] > > CR: > https://bugs.openjdk.java.net/browse/JDK-8212630 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8212630/open.00/ > > Testing: > A few dozen local executions of the test. That's not enough and > probably not in a configuration to encounter the problem -- failures > have been occurring on the order of 0.5% of the time under high load. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Fri May 31 00:04:44 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 30 May 2019 20:04:44 -0400 Subject: RFR: 8212630: jtreg/gc/logging/TestMetaSpaceLog.java failed with Agent timed out In-Reply-To: <8D90A90A-0105-42B4-AE68-B6AF14A484B6@oracle.com> References: <41D37ACB-5E64-483D-9042-206C9BCE135A@oracle.com> <8D90A90A-0105-42B4-AE68-B6AF14A484B6@oracle.com> Message-ID: > On May 30, 2019, at 6:22 PM, Leonid Mesnik wrote: > > Hi > > I think that '@run main/othervm -Xbootclasspath/a:. -XX:+UnlockDiagnosticVMOptions -XX:+WhiteBoxAPI' could be replaced with @run main/driver because > test spawns other process and don't use any options and WB. The main tests are using sun.hotspot.gc.GC, which uses WB. > Also I think you need to have several separate headers for each test instead of multiple runs. There is only one result per test (not for every run) and test is is skipped if any of GC is not supported. Additionally these tests might be guarded with require instead of throwing SkipException. So your tests is not going to be executed saying if ZGC is selected by jtreg CLI. Here is the example of fix which update test with different GCs. It contains a lot of examples of similar tests. > http://hg.openjdk.java.net/jdk/jdk/rev/9b8926bf85c1 > > And in the case of new timeouts/other failures we will know exactly which GC was used right in the report. I thought about using multiple @tests rather than multiple @runs, but decided not to take that path, as doing so is much more verbose, for not much (if any) benefit. The verbosity comes from duplicating most of the @test block N times (including build and install WB support, since the child process uses it too), rather than just having N @run lines plus a little bit of code in main to look up the requested GC object and dispatch on whether it's supported. I don't think getting N results rather than 1 and avoiding the SkippedException case are worth that "code" duplication. (The multiple @test use-case really wants some sort of template mechanism.) Note that I tested it with an additional @run line for Shenandoah (with it configured out as usual for our builds, so not supported), and that worked fine, giving the expected "passed because skipped" result. This test wouldn't have been executed with ZGC or Shenandoah previously either. If we want it to test those we should now add new @run lines for them. (Or new @test blocks, but that just makes my verbosity complaint even worse.) (The old version required adding "if-supported then test" blocks for them.) (And note that this test fails as expected for Epsilon.) I see several other tests in that changeset that could be restructured like I'm doing here (any using GC.isSupported() seem like candidates for this same treatment), perhaps avoiding similar timeouts for similar reasons. > > Leonid > >> On May 30, 2019, at 12:29 PM, Kim Barrett wrote: >> >> [?] >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8212630 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8212630/open.00/ >> >> Testing: >> A few dozen local executions of the test. That's not enough and >> probably not in a configuration to encounter the problem -- failures >> have been occurring on the order of 0.5% of the time under high load. From leonid.mesnik at oracle.com Fri May 31 05:04:54 2019 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Thu, 30 May 2019 22:04:54 -0700 Subject: RFR: 8212630: jtreg/gc/logging/TestMetaSpaceLog.java failed with Agent timed out In-Reply-To: References: <41D37ACB-5E64-483D-9042-206C9BCE135A@oracle.com> <8D90A90A-0105-42B4-AE68-B6AF14A484B6@oracle.com> Message-ID: > On May 30, 2019, at 5:04 PM, Kim Barrett wrote: > >> On May 30, 2019, at 6:22 PM, Leonid Mesnik wrote: >> >> Hi >> >> I think that '@run main/othervm -Xbootclasspath/a:. -XX:+UnlockDiagnosticVMOptions -XX:+WhiteBoxAPI' could be replaced with @run main/driver because >> test spawns other process and don't use any options and WB. > > The main tests are using sun.hotspot.gc.GC, which uses WB. > Sorry, I added this comment after realized how tests could be rewritten. Currently using of WB is needed. >> Also I think you need to have several separate headers for each test instead of multiple runs. There is only one result per test (not for every run) and test is is skipped if any of GC is not supported. Additionally these tests might be guarded with require instead of throwing SkipException. So your tests is not going to be executed saying if ZGC is selected by jtreg CLI. Here is the example of fix which update test with different GCs. It contains a lot of examples of similar tests. >> http://hg.openjdk.java.net/jdk/jdk/rev/9b8926bf85c1 >> >> And in the case of new timeouts/other failures we will know exactly which GC was used right in the report. > > I thought about using multiple @tests rather than multiple @runs, but > decided not to take that path, as doing so is much more verbose, for > not much (if any) benefit. The verbosity comes from duplicating most > of the @test block N times (including build and install WB support, > since the child process uses it too), rather than just having N @run > lines plus a little bit of code in main to look up the requested GC > object and dispatch on whether it's supported. I don't think getting N > results rather than 1 and avoiding the SkippedException case are > worth that "code" duplication. (The multiple @test use-case really > wants some sort of template mechanism.) > > Note that I tested it with an additional @run line for Shenandoah > (with it configured out as usual for our builds, so not supported), > and that worked fine, giving the expected "passed because skipped" > result. As I understand in this case test is not going to execute any @run after skipped one. So if anyone decide to add Shenandoah testing as first @run then test is going to test nothing and always pass as skipped in our testing. Please note that this test is always executed with all GCs specified even if specific GC is set using jtreg options. It doesn't harm for single test but increase wasting time if many tests use this approach. BTW If test is not GC specific then it might just pick GC from jtreg command-line. It might be no sense to run it with all GCs explicitly each time. Leonid > > This test wouldn't have been executed with ZGC or Shenandoah > previously either. If we want it to test those we should now add new > @run lines for them. (Or new @test blocks, but that just makes my > verbosity complaint even worse.) (The old version required adding > "if-supported then test" blocks for them.) (And note that this test > fails as expected for Epsilon.) > > I see several other tests in that changeset that could be restructured > like I'm doing here (any using GC.isSupported() seem like candidates > for this same treatment), perhaps avoiding similar timeouts for > similar reasons. > >> >> Leonid >> >>> On May 30, 2019, at 12:29 PM, Kim Barrett wrote: >>> >>> [?] >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8212630 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8212630/open.00/ >>> >>> Testing: >>> A few dozen local executions of the test. That's not enough and >>> probably not in a configuration to encounter the problem -- failures >>> have been occurring on the order of 0.5% of the time under high load. > > From per.liden at oracle.com Fri May 31 07:46:42 2019 From: per.liden at oracle.com (Per Liden) Date: Fri, 31 May 2019 09:46:42 +0200 Subject: RFR: 8225084: gc/z/TestHighUsage.java fails with unexpected allocation stall Message-ID: <24d9d8c9-8117-a6ae-199f-852787e8e24c@oracle.com> TestHighUsage might fail in case the test system is starved on CPU. This patch adds a delay to allow for a GC cycle to complete even in those situations. Also adding gc+start logging to allow for easier debugging of test failures. Bug: https://bugs.openjdk.java.net/browse/JDK-8225084 Webrev: http://cr.openjdk.java.net/~pliden/8225084/webrev.0 /Per From aph at redhat.com Fri May 31 09:02:25 2019 From: aph at redhat.com (Andrew Haley) Date: Fri, 31 May 2019 10:02:25 +0100 Subject: RFR: JDK-8214527 AArch64: ZGC for Aarch64 In-Reply-To: References: <2abfb1d9-5235-e55f-6d90-ba8640cb197c@oracle.com> <1f9e4e96-26b3-8529-7025-6afd76d1ccc9@oracle.com> Message-ID: <0a71b763-c01e-0dbb-1837-ec5a9641df1a@redhat.com> On 5/30/19 6:05 PM, Stuart Monteith wrote: > I'd like to have this merged (once everyone is happy) after Nils' > patch has been merged, the thread is here: > https://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2019-May/033893.html > The bug: https://bugs.openjdk.java.net/browse/JDK-8224675 > > This necessitates some changes to the AArch64 code - I intend on > wrapping the changes necessary for JDK-8224675 into the patch I've > asked for reviews here, otherwise my patch will break the build. I've not been following all of this. When you have something final, let me know. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From shade at redhat.com Fri May 31 11:02:46 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 31 May 2019 13:02:46 +0200 Subject: RFR (S) 8225111: Make Shenandoah tests work with 32-bit VMs Message-ID: <5f8d0549-bcad-8c3f-631f-7109b9e3f4e5@redhat.com> RFE: https://bugs.openjdk.java.net/browse/JDK-8225111 On the second thought, I want to split these test changes from the x86_32 support patch. This allows other 32-bit targets to work as well, and it would only require to backport this smaller test bunch, without dragging x86_32 in. This only touches Shenandoah tests: http://cr.openjdk.java.net/~shade/8225111/webrev.01/ Testing: hotspot_gc_shenandoah {fastdebug,release} -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Fri May 31 11:05:44 2019 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 31 May 2019 13:05:44 +0200 Subject: RFR (S) 8225111: Make Shenandoah tests work with 32-bit VMs In-Reply-To: <5f8d0549-bcad-8c3f-631f-7109b9e3f4e5@redhat.com> References: <5f8d0549-bcad-8c3f-631f-7109b9e3f4e5@redhat.com> Message-ID: Ok. Thanks! Roman > RFE: > https://bugs.openjdk.java.net/browse/JDK-8225111 > > On the second thought, I want to split these test changes from the x86_32 support patch. This allows > other 32-bit targets to work as well, and it would only require to backport this smaller test bunch, > without dragging x86_32 in. > > This only touches Shenandoah tests: > http://cr.openjdk.java.net/~shade/8225111/webrev.01/ > > Testing: hotspot_gc_shenandoah {fastdebug,release} > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Fri May 31 11:06:03 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 31 May 2019 13:06:03 +0200 Subject: RFR (M) 8225048: Shenandoah x86_32 support In-Reply-To: <52336e65-ba7d-891e-921b-eab2cb54bab1@oracle.com> References: <4fb7a59f-1ceb-af71-2038-a48ef6366087@redhat.com> <52336e65-ba7d-891e-921b-eab2cb54bab1@oracle.com> Message-ID: <73c875ae-0198-1a23-be64-db8cfa081cc0@redhat.com> On 5/30/19 3:29 PM, Erik Joelsson wrote: > Build change looks good. Thanks, Erik. Not expecting any more build changes, I am going to drop build-dev@ from the following replies on this thread. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Fri May 31 11:07:19 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 31 May 2019 13:07:19 +0200 Subject: RFR (M) 8225048: Shenandoah x86_32 support In-Reply-To: References: <4fb7a59f-1ceb-af71-2038-a48ef6366087@redhat.com> Message-ID: <5e2083ef-7ac0-d2d4-b578-074fbcfce3c3@redhat.com> On 5/30/19 2:14 PM, Roman Kennke wrote: > The changes look good to me. Thanks!! Thanks! Split out test changes as JDK-8225111, so this patch does only x86_32 parts: http://cr.openjdk.java.net/~shade/8225048/webrev.01/ Still passes hotspot_gc_shenandoah. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Fri May 31 11:09:03 2019 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 31 May 2019 13:09:03 +0200 Subject: RFR (M) 8225048: Shenandoah x86_32 support In-Reply-To: <5e2083ef-7ac0-d2d4-b578-074fbcfce3c3@redhat.com> References: <4fb7a59f-1ceb-af71-2038-a48ef6366087@redhat.com> <5e2083ef-7ac0-d2d4-b578-074fbcfce3c3@redhat.com> Message-ID: <4b260309-b483-4d47-8f71-9e1befa13988@redhat.com> >> The changes look good to me. Thanks!! > > Thanks! Split out test changes as JDK-8225111, so this patch does only x86_32 parts: > http://cr.openjdk.java.net/~shade/8225048/webrev.01/ > > Still passes hotspot_gc_shenandoah. It's still got plenty of test changes in it. Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Fri May 31 11:09:57 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 31 May 2019 13:09:57 +0200 Subject: RFR (M) 8225048: Shenandoah x86_32 support In-Reply-To: <4b260309-b483-4d47-8f71-9e1befa13988@redhat.com> References: <4fb7a59f-1ceb-af71-2038-a48ef6366087@redhat.com> <5e2083ef-7ac0-d2d4-b578-074fbcfce3c3@redhat.com> <4b260309-b483-4d47-8f71-9e1befa13988@redhat.com> Message-ID: <5af505b0-cd85-bfd0-fc9e-007ab2680431@redhat.com> On 5/31/19 1:09 PM, Roman Kennke wrote: >>> The changes look good to me. Thanks!! >> >> Thanks! Split out test changes as JDK-8225111, so this patch does only x86_32 parts: >> http://cr.openjdk.java.net/~shade/8225048/webrev.01/ >> >> Still passes hotspot_gc_shenandoah. > > It's still got plenty of test changes in it. Dang, wrong link! Here: http://cr.openjdk.java.net/~shade/8225048/webrev.02/ -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Fri May 31 11:18:17 2019 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 31 May 2019 13:18:17 +0200 Subject: RFR (M) 8225048: Shenandoah x86_32 support In-Reply-To: <5af505b0-cd85-bfd0-fc9e-007ab2680431@redhat.com> References: <4fb7a59f-1ceb-af71-2038-a48ef6366087@redhat.com> <5e2083ef-7ac0-d2d4-b578-074fbcfce3c3@redhat.com> <4b260309-b483-4d47-8f71-9e1befa13988@redhat.com> <5af505b0-cd85-bfd0-fc9e-007ab2680431@redhat.com> Message-ID: <1ee76bda-f346-397f-495e-e345edd60cd8@redhat.com> >>>> The changes look good to me. Thanks!! >>> >>> Thanks! Split out test changes as JDK-8225111, so this patch does only x86_32 parts: >>> http://cr.openjdk.java.net/~shade/8225048/webrev.01/ >>> >>> Still passes hotspot_gc_shenandoah. >> >> It's still got plenty of test changes in it. > > Dang, wrong link! Here: > http://cr.openjdk.java.net/~shade/8225048/webrev.02/ This is much cuter indeed :-) Good! Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: