From ayang at openjdk.org Wed Nov 1 09:29:08 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 1 Nov 2023 09:29:08 GMT Subject: RFR: 8319203: Parallel: Rename addr_is_marked_imprecise Message-ID: Simple renaming. ------------- Commit messages: - pgc-rename Changes: https://git.openjdk.org/jdk/pull/16445/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16445&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319203 Stats: 4 lines in 2 files changed: 0 ins; 0 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/16445.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16445/head:pull/16445 PR: https://git.openjdk.org/jdk/pull/16445 From ayang at openjdk.org Wed Nov 1 09:34:29 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 1 Nov 2023 09:34:29 GMT Subject: RFR: 8319204: G1: Change G1CMTask::_termination_time_ms to wallclock time Message-ID: <-XZQmjAaASQW0XC_6PfYBXhmVKXTsTO1BIw2FKk_YpM=.9935a72d-9b76-4c03-ab5a-0c63b8a1e255@github.com> Simple switching from thread-cpu to wall-clock time for termination duration metric. ------------- Commit messages: - g1-termination-time Changes: https://git.openjdk.org/jdk/pull/16446/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16446&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319204 Stats: 8 lines in 2 files changed: 0 ins; 6 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16446.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16446/head:pull/16446 PR: https://git.openjdk.org/jdk/pull/16446 From sjohanss at openjdk.org Wed Nov 1 09:37:08 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Wed, 1 Nov 2023 09:37:08 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v35] In-Reply-To: References: Message-ID: <6tngC-Jwyx8e25LGT8dAwKbaPb9qb_w5ONctnFieH3o=.61b013b2-beb9-4053-8c06-86a700208d77@github.com> On Tue, 31 Oct 2023 04:23:13 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Replace NULL with nullptr Sorry for being a bit late to this PR. I think the addition of CPU time tracking is good, but I wonder if we could do it in a way that is a bit more general. A more general way of tracking CPU time for a set of threads and we could then have different consumers of this data. In addition to hsperf counters I think having logging and JFR events for this could be interesting as well. Have you had any thought along those lines, any obvious problems with such approach? ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1788665727 From iwalulya at openjdk.org Wed Nov 1 09:42:02 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Wed, 1 Nov 2023 09:42:02 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v5] In-Reply-To: References: Message-ID: On Tue, 31 Oct 2023 19:14:13 GMT, Thomas Schatzl wrote: >> The JEP covers the idea very well, so I'm only covering some implementation details here: >> >> * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. >> >> * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: >> >> * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. >> >> * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). >> >> * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. >> >> * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) >> >> The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. >> >> I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. >> >> * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in a... > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > Fix compilation src/hotspot/share/gc/g1/g1CollectedHeap.inline.hpp line 266: > 264: > 265: inline void G1CollectedHeap::pin_object(JavaThread* thread, oop obj) { > 266: assert(obj != NULL, "obj must not be null"); please update the `NULL`s to `nullptr` ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1378580763 From shade at openjdk.org Wed Nov 1 09:54:28 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 1 Nov 2023 09:54:28 GMT Subject: RFR: 8309622: Re-examine the cache mechanism in BaseLocale In-Reply-To: References: <5MqmN4P1TLKDCfXPhKWrH1b2FTgOXmjtKgd1bVXDd_M=.2000f20d-365d-48dc-bc37-45d8bcb9447f@github.com> Message-ID: On Mon, 12 Jun 2023 17:28:44 GMT, Naoto Sato wrote: >> One other thing to consider... >> If the BaseLocale entries are being GC'd sooner, then perhaps the value of the String.intern() is reduced. >> Eliminating `.intern90` would make creating a normalized BaseLocale faster. > >> One other thing to consider... If the BaseLocale entries are being GC'd sooner, then perhaps the value of the String.intern() is reduced. Eliminating `.intern90` would make creating a normalized BaseLocale faster. > > Some code inside the Locale still relies on `intern` and do `==` comparison. Yes, it would be faster but I think that would be out of the scope of this fix @naotoj, we are still seeing the CI failures due to this. Are you planning to finish this work? Thanks! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14404#issuecomment-1788686569 From iwalulya at openjdk.org Wed Nov 1 10:45:59 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Wed, 1 Nov 2023 10:45:59 GMT Subject: RFR: 8319203: Parallel: Rename addr_is_marked_imprecise In-Reply-To: References: Message-ID: On Wed, 1 Nov 2023 09:24:34 GMT, Albert Mingkun Yang wrote: > Simple renaming. Marked as reviewed by iwalulya (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16445#pullrequestreview-1707960026 From iwalulya at openjdk.org Wed Nov 1 11:04:03 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Wed, 1 Nov 2023 11:04:03 GMT Subject: RFR: 8318647: Serial: Refactor BlockOffsetTable [v4] In-Reply-To: References: Message-ID: <3D7BbNf6LxPbzs9H9AcYz5omKgj6mY4clPJrgRmkpyo=.84c4dc5e-adbb-4c37-a251-f0bb3a52b065@github.com> On Mon, 30 Oct 2023 10:57:59 GMT, Albert Mingkun Yang wrote: >> The diff is too large; maybe it's better to read the new impl directly, which I believe is much easier to follow. >> >> There is some duplication with G1's implementation, which should probably be dealt with in its own PR. > > Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision: > > - uint8 > - s1-rename > - s1-rename > - Merge branch 'master' into s1-bot > - review > - review > - s1-bot Marked as reviewed by iwalulya (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16304#pullrequestreview-1707983776 From ayang at openjdk.org Wed Nov 1 11:50:04 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 1 Nov 2023 11:50:04 GMT Subject: RFR: 8318647: Serial: Refactor BlockOffsetTable [v4] In-Reply-To: References: Message-ID: On Mon, 30 Oct 2023 10:57:59 GMT, Albert Mingkun Yang wrote: >> The diff is too large; maybe it's better to read the new impl directly, which I believe is much easier to follow. >> >> There is some duplication with G1's implementation, which should probably be dealt with in its own PR. > > Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision: > > - uint8 > - s1-rename > - s1-rename > - Merge branch 'master' into s1-bot > - review > - review > - s1-bot Thanks for the review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16304#issuecomment-1788821725 From ayang at openjdk.org Wed Nov 1 11:55:09 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 1 Nov 2023 11:55:09 GMT Subject: Integrated: 8318647: Serial: Refactor BlockOffsetTable In-Reply-To: References: Message-ID: On Mon, 23 Oct 2023 08:14:46 GMT, Albert Mingkun Yang wrote: > The diff is too large; maybe it's better to read the new impl directly, which I believe is much easier to follow. > > There is some duplication with G1's implementation, which should probably be dealt with in its own PR. This pull request has now been integrated. Changeset: ab193484 Author: Albert Mingkun Yang URL: https://git.openjdk.org/jdk/commit/ab1934848b2680aff86631e7a68e5ef22857742f Stats: 849 lines in 9 files changed: 23 ins; 644 del; 182 mod 8318647: Serial: Refactor BlockOffsetTable Reviewed-by: tschatzl, iwalulya ------------- PR: https://git.openjdk.org/jdk/pull/16304 From ayang at openjdk.org Wed Nov 1 11:55:32 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 1 Nov 2023 11:55:32 GMT Subject: RFR: 8319205: Parallel: Reenable work stealing after JDK-8310031 Message-ID: No regression in young-gc for DelayInducer any more. ------------- Commit messages: - pgc-work-stealing Changes: https://git.openjdk.org/jdk/pull/16448/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16448&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319205 Stats: 8 lines in 1 file changed: 0 ins; 5 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/16448.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16448/head:pull/16448 PR: https://git.openjdk.org/jdk/pull/16448 From jjoo at openjdk.org Wed Nov 1 19:50:45 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Wed, 1 Nov 2023 19:50:45 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v36] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 36 additional commits since the last revision: - Merge branch 'openjdk:master' into master - Replace NULL with nullptr - Implement hsperf counter for G1ServiceThread - Remove StringDedup from GC thread list - Use 64-bit atomic add for incrementing counters - Merge branch 'openjdk:master' into master - Add call to publish in parallel gc and update counter names - Add Copyright header to test and formatting changes - Fix test - add comment and change if defined to ifdef - ... and 26 more: https://git.openjdk.org/jdk/compare/d748dccb...2446149c ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/be104e16..2446149c Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=35 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=34-35 Stats: 27809 lines in 1182 files changed: 15264 ins; 4162 del; 8383 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Wed Nov 1 23:56:25 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Wed, 1 Nov 2023 23:56:25 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v37] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: - revert gitignore change - Attempt to fix broken test ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/2446149c..9fb36a9e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=36 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=35-36 Stats: 3 lines in 1 file changed: 3 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Thu Nov 2 01:24:08 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Thu, 2 Nov 2023 01:24:08 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v35] In-Reply-To: <6tngC-Jwyx8e25LGT8dAwKbaPb9qb_w5ONctnFieH3o=.61b013b2-beb9-4053-8c06-86a700208d77@github.com> References: <6tngC-Jwyx8e25LGT8dAwKbaPb9qb_w5ONctnFieH3o=.61b013b2-beb9-4053-8c06-86a700208d77@github.com> Message-ID: On Wed, 1 Nov 2023 09:34:01 GMT, Stefan Johansson wrote: >> Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: >> >> Replace NULL with nullptr > > Sorry for being a bit late to this PR. I think the addition of CPU time tracking is good, but I wonder if we could do it in a way that is a bit more general. A more general way of tracking CPU time for a set of threads and we could then have different consumers of this data. In addition to hsperf counters I think having logging and JFR events for this could be interesting as well. > > Have you had any thought along those lines, any obvious problems with such approach? @kstefanj thank you for taking a look! caoman@ and I discussed your comment and our thoughts are below: > A more general way of tracking CPU time for a set of threads and we could then have different consumers of this data. Could you elaborate a bit on what you were thinking of here? If we are assuming something like a thread that updates all other threads, I think this implementation could get a bit complicated. There are two main issues that we can see with a generic thread approach: 1. We would have to figure out how often to pull metrics from the various gc threads from the central thread, and possibly determine this frequency separately for every thread. Instead with our current implementation, we can manually trigger publishes based on when the GC thread is done doing work. 2. We would still need to tag each thread we want to track somewhere, and keep track of a mapping from thread to its counter name, etc. which doesn't seem to simplify things too much. (I imagine we will still need to touch numerous files to "tag" each thread with whether we want to track it or not?) The existing `sun.management:type=HotspotThreading` MBean approach (discussed [here]( https://mail.openjdk.org/pipermail/core-libs-dev/2023-September/111397.html)) could be another general way to track CPU. However, the discussion concludes that it is an internal API, and discourages users from using it. > In addition to hsperf counters I think having logging and JFR events for this could be interesting as well. There's actually already an existing implementation that covers process CPU from GC pauses, but it doesn't handle concurrent work: https://bugs.openjdk.org/browse/JDK-8291753. However, for our implementation of AHS, we decided against a JFR-based approach, as it has a bit more overhead and the hsperf-based implementation seemed simpler. So while a JFR-based approach to track these counters might be feasible, we believe it would be considerably more work, and could be done in a separate PR later if there is sufficient user interest. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1789914128 From tschatzl at openjdk.org Thu Nov 2 08:25:00 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 2 Nov 2023 08:25:00 GMT Subject: RFR: 8319203: Parallel: Rename addr_is_marked_imprecise In-Reply-To: References: Message-ID: <2TnYra8eKgmXXTUaY0hbaS25TssOAJ2o_qkW1cuvKPE=.fd1c2961-edc4-445d-98b5-7ab05217a1c3@github.com> On Wed, 1 Nov 2023 09:24:34 GMT, Albert Mingkun Yang wrote: > Simple renaming. Lgtm. ------------- Marked as reviewed by tschatzl (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16445#pullrequestreview-1709677087 From tschatzl at openjdk.org Thu Nov 2 08:33:46 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 2 Nov 2023 08:33:46 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v6] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: NULL -> nullptr ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/e5dfbb73..fb1deac4 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=04-05 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From ayang at openjdk.org Thu Nov 2 09:42:09 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Thu, 2 Nov 2023 09:42:09 GMT Subject: RFR: 8319203: Parallel: Rename addr_is_marked_imprecise In-Reply-To: References: Message-ID: <_SFUztx1mUlZOZENByM_FyAZct2xWxZJmdk1ZahSo3c=.87ef02bd-e4df-4c93-9727-3afd467d06e3@github.com> On Wed, 1 Nov 2023 09:24:34 GMT, Albert Mingkun Yang wrote: > Simple renaming. Thanks for the review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16445#issuecomment-1790381829 From ayang at openjdk.org Thu Nov 2 09:42:09 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Thu, 2 Nov 2023 09:42:09 GMT Subject: Integrated: 8319203: Parallel: Rename addr_is_marked_imprecise In-Reply-To: References: Message-ID: On Wed, 1 Nov 2023 09:24:34 GMT, Albert Mingkun Yang wrote: > Simple renaming. This pull request has now been integrated. Changeset: 7f31a059 Author: Albert Mingkun Yang URL: https://git.openjdk.org/jdk/commit/7f31a0591c94983cae96b8b5f381cb6a7c0f8fcd Stats: 4 lines in 2 files changed: 0 ins; 0 del; 4 mod 8319203: Parallel: Rename addr_is_marked_imprecise Reviewed-by: iwalulya, tschatzl ------------- PR: https://git.openjdk.org/jdk/pull/16445 From tschatzl at openjdk.org Thu Nov 2 10:15:04 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 2 Nov 2023 10:15:04 GMT Subject: RFR: 8319205: Parallel: Reenable work stealing after JDK-8310031 In-Reply-To: References: Message-ID: On Wed, 1 Nov 2023 11:46:39 GMT, Albert Mingkun Yang wrote: > No regression in young-gc for DelayInducer any more. Lgtm. Thanks. ------------- PR Review: https://git.openjdk.org/jdk/pull/16448#pullrequestreview-1709886880 From ayang at openjdk.org Thu Nov 2 10:15:14 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Thu, 2 Nov 2023 10:15:14 GMT Subject: RFR: 8319306: Serial: Remove TenuredSpace::verify Message-ID: I didn't find the BOT verification logic much useful while working on JDK-8318647, so I propose removing it to simplify the code. ------------- Commit messages: - s1-remove-verify Changes: https://git.openjdk.org/jdk/pull/16471/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16471&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319306 Stats: 48 lines in 2 files changed: 0 ins; 47 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16471.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16471/head:pull/16471 PR: https://git.openjdk.org/jdk/pull/16471 From iwalulya at openjdk.org Thu Nov 2 10:23:02 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Thu, 2 Nov 2023 10:23:02 GMT Subject: RFR: 8318894: G1: Use uint for age in G1SurvRateGroup In-Reply-To: References: Message-ID: On Thu, 26 Oct 2023 11:23:57 GMT, Albert Mingkun Yang wrote: > Simple refactoring to use unsigned type for region-age. Marked as reviewed by iwalulya (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16376#pullrequestreview-1709900813 From iwalulya at openjdk.org Thu Nov 2 10:25:02 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Thu, 2 Nov 2023 10:25:02 GMT Subject: RFR: 8319204: G1: Change G1CMTask::_termination_time_ms to wallclock time In-Reply-To: <-XZQmjAaASQW0XC_6PfYBXhmVKXTsTO1BIw2FKk_YpM=.9935a72d-9b76-4c03-ab5a-0c63b8a1e255@github.com> References: <-XZQmjAaASQW0XC_6PfYBXhmVKXTsTO1BIw2FKk_YpM=.9935a72d-9b76-4c03-ab5a-0c63b8a1e255@github.com> Message-ID: <5T_Kif2gpxjpgUJD-Von_ReT_1_I5NoMt6rVsi1hmTo=.de9c2048-5f53-4a16-b032-067ec5e6c9c8@github.com> On Wed, 1 Nov 2023 09:27:41 GMT, Albert Mingkun Yang wrote: > Simple switching from thread-cpu to wall-clock time for termination duration metric. Marked as reviewed by iwalulya (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16446#pullrequestreview-1709905214 From iwalulya at openjdk.org Thu Nov 2 10:27:02 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Thu, 2 Nov 2023 10:27:02 GMT Subject: RFR: 8319205: Parallel: Reenable work stealing after JDK-8310031 In-Reply-To: References: Message-ID: On Wed, 1 Nov 2023 11:46:39 GMT, Albert Mingkun Yang wrote: > No regression in young-gc for DelayInducer any more. Marked as reviewed by iwalulya (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16448#pullrequestreview-1709908457 From tschatzl at openjdk.org Thu Nov 2 10:39:01 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 2 Nov 2023 10:39:01 GMT Subject: RFR: 8319205: Parallel: Reenable work stealing after JDK-8310031 In-Reply-To: References: Message-ID: On Wed, 1 Nov 2023 11:46:39 GMT, Albert Mingkun Yang wrote: > No regression in young-gc for DelayInducer any more. Marked as reviewed by tschatzl (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16448#pullrequestreview-1709929306 From simonis at openjdk.org Thu Nov 2 10:46:11 2023 From: simonis at openjdk.org (Volker Simonis) Date: Thu, 2 Nov 2023 10:46:11 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v37] In-Reply-To: References: Message-ID: On Wed, 1 Nov 2023 23:56:25 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: > > - revert gitignore change > - Attempt to fix broken test @kstefanj , it is a pity that the `sun.management:type=HotspotThreading` MBean is not exported any more. If we move that or a similar functionality to a new MBean under `com.sun.management` (as proposed in the [cited discussion](https://mail.openjdk.org/pipermail/core-libs-dev/2023-September/111397.html)) then we might reuse these new hsperf counters in the same way this is already done by some other MBeans which already use hsperf counters as their information source. I think logging or JFR functionality could also easily be implemented on top of the new hsperf counters. As for @albertnetymk's proposal to simple use the `/proc` file system to retrieve thread CPU time information, that is not only inconvenient for the reasons detailed by @caoman. It will also not work for compiler threads (which I think should have similar counters in the future) because compiler threads can be created and and destroyed dynamically (due to [`-XX:+UseDynamicNumberOfCompilerThreads`](https://bugs.openjdk.org/browse/JDK-8198756)). The current PR still looks good to me. ------------- Marked as reviewed by simonis (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/15082#pullrequestreview-1709943605 From ayang at openjdk.org Thu Nov 2 11:30:17 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Thu, 2 Nov 2023 11:30:17 GMT Subject: Integrated: 8319205: Parallel: Reenable work stealing after JDK-8310031 In-Reply-To: References: Message-ID: On Wed, 1 Nov 2023 11:46:39 GMT, Albert Mingkun Yang wrote: > No regression in young-gc for DelayInducer any more. This pull request has now been integrated. Changeset: 792d8293 Author: Albert Mingkun Yang URL: https://git.openjdk.org/jdk/commit/792d82932860b6fac347928284e3c6fba17dfa2d Stats: 8 lines in 1 file changed: 0 ins; 5 del; 3 mod 8319205: Parallel: Reenable work stealing after JDK-8310031 Reviewed-by: tschatzl, iwalulya ------------- PR: https://git.openjdk.org/jdk/pull/16448 From ayang at openjdk.org Thu Nov 2 11:30:17 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Thu, 2 Nov 2023 11:30:17 GMT Subject: Integrated: 8318894: G1: Use uint for age in G1SurvRateGroup In-Reply-To: References: Message-ID: On Thu, 26 Oct 2023 11:23:57 GMT, Albert Mingkun Yang wrote: > Simple refactoring to use unsigned type for region-age. This pull request has now been integrated. Changeset: 23a96bf3 Author: Albert Mingkun Yang URL: https://git.openjdk.org/jdk/commit/23a96bf31223aaaf6dc129a51c0dd161062caaf8 Stats: 41 lines in 5 files changed: 6 ins; 10 del; 25 mod 8318894: G1: Use uint for age in G1SurvRateGroup Reviewed-by: tschatzl, iwalulya ------------- PR: https://git.openjdk.org/jdk/pull/16376 From ayang at openjdk.org Thu Nov 2 11:30:16 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Thu, 2 Nov 2023 11:30:16 GMT Subject: RFR: 8318894: G1: Use uint for age in G1SurvRateGroup In-Reply-To: References: Message-ID: On Thu, 26 Oct 2023 11:23:57 GMT, Albert Mingkun Yang wrote: > Simple refactoring to use unsigned type for region-age. Thanks for the review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16376#issuecomment-1790546201 From ayang at openjdk.org Thu Nov 2 11:30:16 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Thu, 2 Nov 2023 11:30:16 GMT Subject: RFR: 8319205: Parallel: Reenable work stealing after JDK-8310031 In-Reply-To: References: Message-ID: On Wed, 1 Nov 2023 11:46:39 GMT, Albert Mingkun Yang wrote: > No regression in young-gc for DelayInducer any more. Thanks for the review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16448#issuecomment-1790546329 From tschatzl at openjdk.org Thu Nov 2 11:42:54 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 2 Nov 2023 11:42:54 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v7] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: Renamings to (almost) consistently use the following nomenclature for evacuation failure and types of it: * evacuation failure is the general concept. It includes * pinned regions * allocation failure One region can both be pinned and experience an allocation failure. G1 GC messages use tags "(Pinned)" and "(Allocation Failure)" now instead of "(Evacuation Failure)" Did not rename the G1EvacFailureInjector since this adds a lot of noise. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/fb1deac4..73f61da9 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=05-06 Stats: 180 lines in 18 files changed: 54 ins; 28 del; 98 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From tschatzl at openjdk.org Thu Nov 2 11:46:02 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 2 Nov 2023 11:46:02 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v4] In-Reply-To: References: <8BH2UtnHn-DYz3c80Su4v9BF_v0w-N4fHkASCXP_E2c=.70c7ff8f-32e2-4970-87e3-fe22f7b08e6b@github.com> Message-ID: On Tue, 31 Oct 2023 18:54:26 GMT, Thomas Schatzl wrote: > Had a discussion with @albertnetymk and we came to the following agreement about naming: > >"allocation failure" - allocation failed in the to-space due to memory exhaustion >"pinned" - the region/object has been pinned >"evacuation failure" - either pinned or allocation failure > >I will apply this new naming asap. Done. I left out the `G1EvacFailureInjector` (it only injects allocation failures, not evacuation failures) related renamings as this adds lots of noise (including the debug options). I'll file a follow-up and assign it to me. Tier1 seems to pass, will redo upper tiers again. The only noteworthy externally visible change is that the `(Evacuation Failure)` tag in log messages is now `(Allocation Failure)`. I did not want combinations of `(Evacuation Failure)` and additionally `(Pinned) (Allocation Failure)`, but maybe it is fine, or just fine to keep only `(Evacuation Failure)` as before and assume that users enable higher level logging to find out details. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16342#issuecomment-1790570048 From tschatzl at openjdk.org Thu Nov 2 15:51:35 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 2 Nov 2023 15:51:35 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v8] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: Add documentation about why and how we handle pinned regions in the young/old generation. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/73f61da9..5ae05e4c Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=07 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=06-07 Stats: 18 lines in 2 files changed: 16 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From tschatzl at openjdk.org Thu Nov 2 15:55:05 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 2 Nov 2023 15:55:05 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v8] In-Reply-To: References: Message-ID: On Mon, 30 Oct 2023 17:12:19 GMT, Thomas Schatzl wrote: >> I do not think so. I will do some more testing about this. > > I (still) do not think it is possible after some more re-testing. There are the following situations I can think of: > > * string deduplication is a need-to-be-supported case where only the C code may have a reference to a pinned object: thread A critical sections a string, gets the char array address, locking the region containing the char array. Then string dedup goes ahead and replaces the original char array with something else. Now the C code has the only reference to that char array. > There is no API to convert a raw array pointer back to a Java object so destroying the header is fine; unpinning does not need the header. > > * there is some other case I can think of that could be problematic, but is actually a spec violation: the array is critical-locked by thread A, then shared with other C code (not critical-unlocked), resumes with Java code that forgets that reference. At some point other C code accesses that locked memory and (hopefully) critically-unlocks it. > Again, there is no API to convert a raw array pointer back to a Java object so destroying the header is fine. > > In all other cases I can think of there is always a reference to the encapsulating java object either from the stack frame (when passing in the object into the JNI function they are part of the oop maps) or if you create a new array object (via `NewArray` and lock it, the VM will add a handle to it. > > There is also no API to inspect the array header using the raw pointer (e.g. passing the raw pointer to `GetArrayLength` - doesn't compile as it expects a `jarray`, and in debug VMs there is actually a check that the passed argument is something that resembles a handle), so modifications are already invalid, and the change is fine imo. > > hth, > Thomas Here is some example (pseudo-) code for the first case mentioned above that should be valid JNI code: Java code: String x = ...; native_f1(x); [ some java code, x.chars gets deduplicated, its char array pointing to somewhere else now. Now native code is the only one having a reference to the old char array ] native_f2(); ----------- sample native code: void native_f1(jobject jstring) { global_string = NewGlobalRef(jstring); global_raw_chars = GetStringChars(global_string); } void native_f2() { ReleaseStringChars(global_string, global_raw_chars); DeleteGlobalRef(global_string); } ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1380370821 From ayang at openjdk.org Fri Nov 3 10:19:10 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 3 Nov 2023 10:19:10 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v8] In-Reply-To: References: Message-ID: On Thu, 2 Nov 2023 15:51:35 GMT, Thomas Schatzl wrote: >> The JEP covers the idea very well, so I'm only covering some implementation details here: >> >> * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. >> >> * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: >> >> * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. >> >> * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). >> >> * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. >> >> * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) >> >> The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. >> >> I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. >> >> * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in a... > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > Add documentation about why and how we handle pinned regions in the young/old generation. src/hotspot/share/gc/g1/g1FullGCPrepareTask.inline.hpp line 76: > 74: } > 75: > 76: inline bool G1DetermineCompactionQueueClosure::has_pinned_objects(HeapRegion* hr) const { Could this be a static-local function so that it doesn't appear in the header file? (Its name is the same as the public API in heap-region.) src/hotspot/share/gc/g1/g1GCPhaseTimes.cpp line 482: > 480: } > 481: > 482: double G1GCPhaseTimes::print_post_evacuate_collection_set(bool evacuation_retained) const { Why the renaming here? src/hotspot/share/gc/g1/g1GCPhaseTimes.hpp line 150: > 148: > 149: enum RestoreRetainedRegionsWorkItems { > 150: RestoreRetainedRegionsEvacFailedNum, // How many regions experienced an evacuation failure (pinned or allocation failure) Kind of a preexisting issue. "retained" here seems to mean evac-fail, not "kept in retain-list". src/hotspot/share/gc/g1/g1HeapRegionAttr.hpp line 43: > 41: remset_is_tracked_t _remset_is_tracked; > 42: region_type_t _type; > 43: bool _is_pinned; Maybe `uint8_t` as documented above? src/hotspot/share/gc/g1/g1Policy.cpp line 547: > 545: } > 546: > 547: log_trace(gc, ergo, heap)("Selected %u of %u retained candidates (unreclaimable %u) taking %1.3fms additional time", I actually think calling it "pinned", instead of "unreclaimable", is more informative (to users/dev). (And other places when it is shown in logs.) src/hotspot/share/gc/g1/g1YoungCollector.cpp line 1102: > 1100: jtm.report_pause_type(collector_state()->young_gc_pause_type(_concurrent_operation_is_full_mark)); > 1101: > 1102: policy()->record_young_collection_end(_concurrent_operation_is_full_mark, evacuation_alloc_failed()); The arg name (where this method is defined) should be updated to sth like `evac_alloc_failed` from `evacuation_failure`. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381415671 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381418678 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381440184 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381419525 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381423626 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381433739 From ayang at openjdk.org Fri Nov 3 10:19:12 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 3 Nov 2023 10:19:12 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v8] In-Reply-To: References: Message-ID: <-W9sGrhwC6dMCJZrvR8cYU_YzEoE23CMbsU9WGXSocs=.4771371a-5cf9-4694-b32e-f173a14dcb7c@github.com> On Thu, 2 Nov 2023 15:52:09 GMT, Thomas Schatzl wrote: >> I (still) do not think it is possible after some more re-testing. There are the following situations I can think of: >> >> * string deduplication is a need-to-be-supported case where only the C code may have a reference to a pinned object: thread A critical sections a string, gets the char array address, locking the region containing the char array. Then string dedup goes ahead and replaces the original char array with something else. Now the C code has the only reference to that char array. >> There is no API to convert a raw array pointer back to a Java object so destroying the header is fine; unpinning does not need the header. >> >> * there is some other case I can think of that could be problematic, but is actually a spec violation: the array is critical-locked by thread A, then shared with other C code (not critical-unlocked), resumes with Java code that forgets that reference. At some point other C code accesses that locked memory and (hopefully) critically-unlocks it. >> Again, there is no API to convert a raw array pointer back to a Java object so destroying the header is fine. >> >> In all other cases I can think of there is always a reference to the encapsulating java object either from the stack frame (when passing in the object into the JNI function they are part of the oop maps) or if you create a new array object (via `NewArray` and lock it, the VM will add a handle to it. >> >> There is also no API to inspect the array header using the raw pointer (e.g. passing the raw pointer to `GetArrayLength` - doesn't compile as it expects a `jarray`, and in debug VMs there is actually a check that the passed argument is something that resembles a handle), so modifications are already invalid, and the change is fine imo. >> >> hth, >> Thomas > > Here is some example (pseudo-) code for the first case mentioned above that should be valid JNI code: > > > Java code: > > String x = ...; > native_f1(x); > [ some java code, x.chars gets deduplicated, its char array pointing to somewhere else now. Now native code is the only one having a reference to the old char array ] > native_f2(); > > ----------- sample native code: > > void native_f1(jobject jstring) { > global_string = NewGlobalRef(jstring); > global_raw_chars = GetStringChars(global_string); > } > > void native_f2() { > ReleaseStringChars(global_string, global_raw_chars); > DeleteGlobalRef(global_string); > } > string deduplication is a need-to-be-supported case... OK, so this is the only valid scenario where a type-array should be kept live even though it's not reachable from GC's perspective. Could you describe it in the comment? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381436094 From tschatzl at openjdk.org Fri Nov 3 10:59:06 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 3 Nov 2023 10:59:06 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v8] In-Reply-To: References: Message-ID: On Fri, 3 Nov 2023 09:56:43 GMT, Albert Mingkun Yang wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> Add documentation about why and how we handle pinned regions in the young/old generation. > > src/hotspot/share/gc/g1/g1GCPhaseTimes.cpp line 482: > >> 480: } >> 481: >> 482: double G1GCPhaseTimes::print_post_evacuate_collection_set(bool evacuation_retained) const { > > Why the renaming here? Probably forgot to undo the rename. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381501087 From tschatzl at openjdk.org Fri Nov 3 12:32:02 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 3 Nov 2023 12:32:02 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v9] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: ayang review - renamings + documentation ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/5ae05e4c..8342b80b Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=08 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=07-08 Stats: 79 lines in 17 files changed: 8 ins; 3 del; 68 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From tschatzl at openjdk.org Fri Nov 3 12:32:06 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 3 Nov 2023 12:32:06 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v8] In-Reply-To: References: Message-ID: On Thu, 2 Nov 2023 15:51:35 GMT, Thomas Schatzl wrote: >> The JEP covers the idea very well, so I'm only covering some implementation details here: >> >> * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. >> >> * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: >> >> * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. >> >> * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). >> >> * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. >> >> * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) >> >> The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. >> >> I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. >> >> * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in a... > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > Add documentation about why and how we handle pinned regions in the young/old generation. Fwiw, recent changes (without the most recent renamings) passed tier1-5 ------------- PR Comment: https://git.openjdk.org/jdk/pull/16342#issuecomment-1792352540 From ayang at openjdk.org Fri Nov 3 12:43:19 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 3 Nov 2023 12:43:19 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC Message-ID: Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. ------------- Commit messages: - s1-young-gc Changes: https://git.openjdk.org/jdk/pull/16492/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319373 Stats: 470 lines in 4 files changed: 187 ins; 270 del; 13 mod Patch: https://git.openjdk.org/jdk/pull/16492.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16492/head:pull/16492 PR: https://git.openjdk.org/jdk/pull/16492 From ayang at openjdk.org Fri Nov 3 12:53:08 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 3 Nov 2023 12:53:08 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v9] In-Reply-To: References: Message-ID: On Fri, 3 Nov 2023 12:32:02 GMT, Thomas Schatzl wrote: >> The JEP covers the idea very well, so I'm only covering some implementation details here: >> >> * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. >> >> * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: >> >> * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. >> >> * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). >> >> * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. >> >> * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) >> >> The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. >> >> I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. >> >> * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in a... > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > ayang review - renamings + documentation src/hotspot/share/gc/g1/g1Policy.hpp line 275: > 273: double start, > 274: double end, > 275: bool alloocation_failure = false); Typo. src/hotspot/share/gc/g1/g1Policy.hpp line 314: > 312: // Record the start and end of the actual collection part of the evacuation pause. > 313: void record_young_collection_start(); > 314: void record_young_collection_end(bool concurrent_operation_is_full_mark, bool alllocation_failure); Typo. src/hotspot/share/gc/g1/g1YoungCollector.cpp line 87: > 85: GCCause::to_string(_pause_cause), > 86: _collector->evacuation_pinned() ? " (Pinned)" : "", > 87: _collector->evacuation_alloc_failed() ? " (Allocation Failure)" : ""); > GC(6) Pause Young (Normal) (Pinned) (Evacuation Failure) I wonder if the last two can be merged into one `()`, sth like `(Pinned / ...)`, because they are on the same abstraction level. src/hotspot/share/gc/g1/g1_globals.hpp line 327: > 325: range(1, 256) \ > 326: \ > 327: product(uint, G1NumCollectionsKeepPinned, 8, DIAGNOSTIC, \ Any particular reason this is not `EXPERIMENTAL`? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381627320 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381627700 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381628625 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381624492 From ayang at openjdk.org Fri Nov 3 13:39:12 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 3 Nov 2023 13:39:12 GMT Subject: RFR: 8319379: G1: gc/logging/TestUnifiedLoggingSwitchStress.java crashes after JDK-8318894 Message-ID: Can reproduce the failure using `TEST=gc/logging/TestUnifiedLoggingSwitchStress.java JTREG="REPEAT_COUNT=10"`, but not any more after the fix. (The MIN2 was removed as part of JDK-8318894; added back now.) ------------- Commit messages: - g1-age-fix Changes: https://git.openjdk.org/jdk/pull/16495/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16495&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319379 Stats: 3 lines in 1 file changed: 3 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16495.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16495/head:pull/16495 PR: https://git.openjdk.org/jdk/pull/16495 From tschatzl at openjdk.org Fri Nov 3 13:46:51 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 3 Nov 2023 13:46:51 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v10] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: typos ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/8342b80b..f9735539 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=09 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=08-09 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From tschatzl at openjdk.org Fri Nov 3 13:50:09 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 3 Nov 2023 13:50:09 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v9] In-Reply-To: References: Message-ID: <4uKST6lml9Okm18TVjp2hgBUQHBPH0FP_Uv13Pr7CLE=.dab77bbb-eec6-41d7-820d-3ed89779feb0@github.com> On Fri, 3 Nov 2023 12:41:05 GMT, Albert Mingkun Yang wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> ayang review - renamings + documentation > > src/hotspot/share/gc/g1/g1_globals.hpp line 327: > >> 325: range(1, 256) \ >> 326: \ >> 327: product(uint, G1NumCollectionsKeepPinned, 8, DIAGNOSTIC, \ > > Any particular reason this is not `EXPERIMENTAL`? Changing this does not in any way enable risky/experimental code not fit for production. This knob is for helping diagnose performance issues. G1 does have its fair share of experimental options, but all/most of these were from the initial import where G1 as a whole had been experimental (unstable) for some time. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381706769 From tschatzl at openjdk.org Fri Nov 3 13:59:05 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 3 Nov 2023 13:59:05 GMT Subject: RFR: 8319379: G1: gc/logging/TestUnifiedLoggingSwitchStress.java crashes after JDK-8318894 In-Reply-To: References: Message-ID: On Fri, 3 Nov 2023 13:32:02 GMT, Albert Mingkun Yang wrote: > Can reproduce the failure using `TEST=gc/logging/TestUnifiedLoggingSwitchStress.java JTREG="REPEAT_COUNT=10"`, but not any more after the fix. > > (The MIN2 was removed as part of JDK-8318894; added back now.) Lgtm. Thanks for the quick turnaround! ------------- Marked as reviewed by tschatzl (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16495#pullrequestreview-1712745543 From ayang at openjdk.org Fri Nov 3 14:17:07 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 3 Nov 2023 14:17:07 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v10] In-Reply-To: References: Message-ID: <1-i3-5OmZbuCNUlpfv31Kr3eiBXEd4Si8F5gsbPHuBQ=.1d97dcac-4662-4482-842c-ce86315ba61a@github.com> On Fri, 3 Nov 2023 13:46:51 GMT, Thomas Schatzl wrote: >> The JEP covers the idea very well, so I'm only covering some implementation details here: >> >> * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. >> >> * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: >> >> * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. >> >> * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). >> >> * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. >> >> * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) >> >> The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. >> >> I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. >> >> * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in a... > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > typos Marked as reviewed by ayang (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16342#pullrequestreview-1712794411 From tschatzl at openjdk.org Fri Nov 3 14:17:11 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 3 Nov 2023 14:17:11 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v9] In-Reply-To: References: Message-ID: On Fri, 3 Nov 2023 12:44:10 GMT, Albert Mingkun Yang wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> ayang review - renamings + documentation > > src/hotspot/share/gc/g1/g1YoungCollector.cpp line 87: > >> 85: GCCause::to_string(_pause_cause), >> 86: _collector->evacuation_pinned() ? " (Pinned)" : "", >> 87: _collector->evacuation_alloc_failed() ? " (Allocation Failure)" : ""); > >> GC(6) Pause Young (Normal) (Pinned) (Evacuation Failure) > > I wonder if the last two can be merged into one `()`, sth like `(Pinned / ...)`, because they are on the same abstraction level. Parsing the separate components is easier :) Not sure if these tags in any way ever indicated some level of abstraction. I do not have a strong opinion here. The combinations (Pinned) (Allocation Failure) (Pinned + Allocation Failure) // or the other way around, or some other symbol for "+" or no symbol at all? are fine with me (and I thought about doing something more elaborate here), but my concern has been that any complicated string makes it less unique (e.g. `(Allocation Failure)` vs. "Allocation Failure") and adds code both to implement and parse the result. Much more disrupting is likely that there is no "Evacuation Failure" string any more. But log messages are not part of the external interface, and we should not want to change them just because. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381716129 From ayang at openjdk.org Fri Nov 3 14:17:13 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 3 Nov 2023 14:17:13 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v9] In-Reply-To: References: Message-ID: On Fri, 3 Nov 2023 13:53:35 GMT, Thomas Schatzl wrote: >> src/hotspot/share/gc/g1/g1YoungCollector.cpp line 87: >> >>> 85: GCCause::to_string(_pause_cause), >>> 86: _collector->evacuation_pinned() ? " (Pinned)" : "", >>> 87: _collector->evacuation_alloc_failed() ? " (Allocation Failure)" : ""); >> >>> GC(6) Pause Young (Normal) (Pinned) (Evacuation Failure) >> >> I wonder if the last two can be merged into one `()`, sth like `(Pinned / ...)`, because they are on the same abstraction level. > > Parsing the separate components is easier :) Not sure if these tags in any way ever indicated some level of abstraction. > > I do not have a strong opinion here. The combinations > > (Pinned) > (Allocation Failure) > (Pinned + Allocation Failure) // or the other way around, or some other symbol for "+" or no symbol at all? > > are fine with me (and I thought about doing something more elaborate here), but my concern has been that any complicated string makes it less unique (e.g. `(Allocation Failure)` vs. "Allocation Failure") and adds code both to implement and parse the result. > > Much more disrupting is likely that there is no "Evacuation Failure" string any more. But log messages are not part of the external interface, and we should not want to change them just because. The example looks good to me. >> src/hotspot/share/gc/g1/g1_globals.hpp line 327: >> >>> 325: range(1, 256) \ >>> 326: \ >>> 327: product(uint, G1NumCollectionsKeepPinned, 8, DIAGNOSTIC, \ >> >> Any particular reason this is not `EXPERIMENTAL`? > > Changing this does not in any way enable risky/experimental code not fit for production. This knob is for helping diagnose performance issues. > > G1 does have its fair share of experimental options, but all/most of these were from the initial import where G1 as a whole had been experimental (unstable) for some time. This flag conceptually related (or similar) to `G1RetainRegionLiveThresholdPercent`, which is an exp, so I thought they should be the same category. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381748512 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381747902 From iwalulya at openjdk.org Fri Nov 3 21:25:13 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Fri, 3 Nov 2023 21:25:13 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v10] In-Reply-To: References: Message-ID: On Fri, 3 Nov 2023 13:46:51 GMT, Thomas Schatzl wrote: >> The JEP covers the idea very well, so I'm only covering some implementation details here: >> >> * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. >> >> * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: >> >> * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. >> >> * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). >> >> * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. >> >> * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) >> >> The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. >> >> I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. >> >> * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in a... > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > typos LGTM! Nits: src/hotspot/share/gc/g1/g1FullCollector.cpp line 465: > 463: continue; > 464: } else if (is_compaction_target(region_index)) { > 465: assert(!hr->has_pinned_objects(), "pinned objects should not be compaction targets"); Suggestion: assert(!hr->has_pinned_objects(), "pinned regions should not be compaction targets"); src/hotspot/share/gc/g1/g1YoungCollector.cpp line 430: > 428: _claimer(_g1h->workers()->active_workers()), > 429: _humongous_total(0), > 430: _humongous_candidates(0) { } Suggestion: _humongous_candidates(0) { } ------------- Marked as reviewed by iwalulya (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16342#pullrequestreview-1707910606 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1382164253 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1382135577 From iwalulya at openjdk.org Fri Nov 3 21:25:16 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Fri, 3 Nov 2023 21:25:16 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v5] In-Reply-To: References: Message-ID: On Tue, 31 Oct 2023 19:14:13 GMT, Thomas Schatzl wrote: >> The JEP covers the idea very well, so I'm only covering some implementation details here: >> >> * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. >> >> * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: >> >> * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. >> >> * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). >> >> * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. >> >> * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) >> >> The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. >> >> I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. >> >> * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in a... > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > Fix compilation src/hotspot/share/gc/g1/g1CollectionSet.cpp line 355: > 353: move_pinned_marking_to_retained(&pinned_marking_regions); > 354: // Drop pinned retained regions to make progress with retained regions. Regions > 355: // in that list have must have been pinned for at least Suggestion: // in that list must have been pinned for at least ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1378608614 From kbarrett at openjdk.org Sat Nov 4 11:31:20 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Sat, 4 Nov 2023 11:31:20 GMT Subject: RFR: 8319439: Move BufferNode from PtrQueue files to new files Message-ID: Please review this simple (trivial?) change that moves the BufferNode class from share/gc/shared/ptrQueue.[ch]pp to new files share/gc/shared/bufferNode.[ch]pp. The code (as opposed to the surrounding boiler plate and #include's) was moved unchanged. This led to adding #include's of the new header file to a bunch of other files (include what you use), in some cases eliminating the need to #include ptrQueue.hpp. Also tidied up the #include's for ptrQueue.[ch]pp to take advantage of the split. Also, the gtest gc/shared/test_ptrQueueBufferAllocator.cpp is renamed to gc/shared/test_bufferNodeAllocator.cpp, with some adjustments to includes and test naming. Testing: mach5 tier1 ------------- Commit messages: - fixup gtest after rename - move test - move BufferNode to new files Changes: https://git.openjdk.org/jdk/pull/16506/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16506&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319439 Stats: 383 lines in 17 files changed: 223 ins; 152 del; 8 mod Patch: https://git.openjdk.org/jdk/pull/16506.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16506/head:pull/16506 PR: https://git.openjdk.org/jdk/pull/16506 From dholmes at openjdk.org Mon Nov 6 06:00:07 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 6 Nov 2023 06:00:07 GMT Subject: RFR: 8319376: Parallel: Forwarded objects during heap inspection In-Reply-To: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Fri, 3 Nov 2023 12:42:46 GMT, Roman Kennke wrote: > See JBS issue for details. > > Testing: > - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC > - [x] tier1 -XX:+UseParallelGC > - [ ] tier2 -XX:+UseParallelGC > - [ ] hotspot_gc @rkennke could you update the title to "ParallelGC: Forwarded objects found during heap inspection" please. Thanks. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1794136928 From sjohanss at openjdk.org Mon Nov 6 10:50:13 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Mon, 6 Nov 2023 10:50:13 GMT Subject: RFR: 8319379: G1: gc/logging/TestUnifiedLoggingSwitchStress.java crashes after JDK-8318894 In-Reply-To: References: Message-ID: <4VjRX9Ie7Et0tsalTZxYEQt2oFX9L8TKs1wIEv3vQbA=.002574ec-adc5-4ba2-8c5a-4f07a72714df@github.com> On Fri, 3 Nov 2023 13:32:02 GMT, Albert Mingkun Yang wrote: > Can reproduce the failure using `TEST=gc/logging/TestUnifiedLoggingSwitchStress.java JTREG="REPEAT_COUNT=10"`, but not any more after the fix. > > (The MIN2 was removed as part of JDK-8318894; added back now.) Marked as reviewed by sjohanss (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16495#pullrequestreview-1714791256 From ayang at openjdk.org Mon Nov 6 11:09:22 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Mon, 6 Nov 2023 11:09:22 GMT Subject: RFR: 8319379: G1: gc/logging/TestUnifiedLoggingSwitchStress.java crashes after JDK-8318894 In-Reply-To: References: Message-ID: On Fri, 3 Nov 2023 13:32:02 GMT, Albert Mingkun Yang wrote: > Can reproduce the failure using `TEST=gc/logging/TestUnifiedLoggingSwitchStress.java JTREG="REPEAT_COUNT=10"`, but not any more after the fix. > > (The MIN2 was removed as part of JDK-8318894; added back now.) Thanks for the review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16495#issuecomment-1794573564 From ayang at openjdk.org Mon Nov 6 11:09:22 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Mon, 6 Nov 2023 11:09:22 GMT Subject: Integrated: 8319379: G1: gc/logging/TestUnifiedLoggingSwitchStress.java crashes after JDK-8318894 In-Reply-To: References: Message-ID: On Fri, 3 Nov 2023 13:32:02 GMT, Albert Mingkun Yang wrote: > Can reproduce the failure using `TEST=gc/logging/TestUnifiedLoggingSwitchStress.java JTREG="REPEAT_COUNT=10"`, but not any more after the fix. > > (The MIN2 was removed as part of JDK-8318894; added back now.) This pull request has now been integrated. Changeset: 8fb94fd4 Author: Albert Mingkun Yang URL: https://git.openjdk.org/jdk/commit/8fb94fd4fe46bc12885c7cc0c7ebbbc10fba47e5 Stats: 3 lines in 1 file changed: 3 ins; 0 del; 0 mod 8319379: G1: gc/logging/TestUnifiedLoggingSwitchStress.java crashes after JDK-8318894 Reviewed-by: tschatzl, sjohanss ------------- PR: https://git.openjdk.org/jdk/pull/16495 From kbarrett at openjdk.org Mon Nov 6 11:48:14 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Mon, 6 Nov 2023 11:48:14 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC In-Reply-To: References: Message-ID: On Fri, 3 Nov 2023 12:14:11 GMT, Albert Mingkun Yang wrote: > Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. > > It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. Just getting started on review, and noticed a few minor things. src/hotspot/share/gc/serial/cardTableRS.cpp line 354: > 352: auto is_word_aligned = [] (const void* const p) { > 353: return (((uintptr_t)p) % sizeof(Word)) == 0; > 354: }; We have `is_aligned` in `utilities/align.hpp`. Please use that. src/hotspot/share/gc/serial/cardTableRS.cpp line 359: > 357: constexpr Word clean_word = (Word)-1; > 358: > 359: CardValue* i = start_card; I don't much like naming a pointer variable "i". "i" is a really commonly used index variable name. src/hotspot/share/gc/serial/cardTableRS.cpp line 379: > 377: if (is_dirty(i + j)) { > 378: return i + j; > 379: } Instead of duplicating the byte search, this could just break out of the word search. ------------- Changes requested by kbarrett (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16492#pullrequestreview-1714894247 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1383163336 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1383165088 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1383168895 From ayang at openjdk.org Mon Nov 6 12:01:32 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Mon, 6 Nov 2023 12:01:32 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v2] In-Reply-To: References: Message-ID: > Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. > > It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: - review - Merge branch 'master' into s1-young-gc - s1-young-gc ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16492/files - new: https://git.openjdk.org/jdk/pull/16492/files/f1089de3..9dd97239 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=00-01 Stats: 1918 lines in 59 files changed: 894 ins; 279 del; 745 mod Patch: https://git.openjdk.org/jdk/pull/16492.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16492/head:pull/16492 PR: https://git.openjdk.org/jdk/pull/16492 From sjohanss at openjdk.org Mon Nov 6 12:14:21 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Mon, 6 Nov 2023 12:14:21 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v35] In-Reply-To: References: <6tngC-Jwyx8e25LGT8dAwKbaPb9qb_w5ONctnFieH3o=.61b013b2-beb9-4053-8c06-86a700208d77@github.com> Message-ID: <3iJqYOiXeO6bwtUmjzlO3tyFR9Uc28YAJ8aQbKbqKJM=.fda95001-86f3-4f09-8a74-e15be1987c4f@github.com> On Thu, 2 Nov 2023 01:20:59 GMT, Jonathan Joo wrote: > Could you elaborate a bit on what you were thinking of here? If we are assuming something like a thread that updates all other threads, I think this implementation could get a bit complicated. > > There are two main issues that we can see with a generic thread approach: > > 1. We would have to figure out how often to pull metrics from the various gc threads from the central thread, and possibly determine this frequency separately for every thread. Instead with our current implementation, we can manually trigger publishes based on when the GC thread is done doing work. > > 2. We would still need to tag each thread we want to track somewhere, and keep track of a mapping from thread to its counter name, etc. which doesn't seem to simplify things too much. (I imagine we will still need to touch numerous files to "tag" each thread with whether we want to track it or not?) I agree, I was not thinking about having a separate thread, more about trying to group this information in a way that it would be easier, for example, to provide periodic JFR events for the CPU times collected. Having something like a `CollectorCPUTimeCounters` (we already have `CollectorCounters`). Such a class could keep the different counters making it easier to get an overview of which CPU time counters are present. But this would also require a mapping between thread and counter (but it might be as simple as having an enum). I played around a bit instead of trying to explain what I mean and this is not very polished, but I was thinking something like this: https://github.com/openjdk/jdk/compare/pr/15082...kstefanj:jdk:pull/15082-idea What do you think? This way we don't add things to `CollectedHeap` as well, which is usually good unless really needed. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1794686146 From sjohanss at openjdk.org Mon Nov 6 12:20:21 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Mon, 6 Nov 2023 12:20:21 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v37] In-Reply-To: References: Message-ID: On Wed, 1 Nov 2023 23:56:25 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: > > - revert gitignore change > - Attempt to fix broken test > The existing sun.management:type=HotspotThreading MBean approach (discussed [here](https://mail.openjdk.org/pipermail/core-libs-dev/2023-September/111397.html)) could be another general way to track CPU. However, the discussion concludes that it is an internal API, and discourages users from using it. > @kstefanj , it is a pity that the `sun.management:type=HotspotThreading` MBean is not exported any more. If we move that or a similar functionality to a new MBean under `com.sun.management` (as proposed in the [cited discussion](https://mail.openjdk.org/pipermail/core-libs-dev/2023-September/111397.html)) then we might reuse these new hsperf counters in the same way this is already done by some other MBeans which already use hsperf counters as their information source. I think logging or JFR functionality could also easily be implemented on top of the new hsperf counters. I haven't looked at the details around this, but extracting some useful information from the internal bean and making it public sound reasonable to me. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1794693970 From ayang at openjdk.org Mon Nov 6 13:39:14 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Mon, 6 Nov 2023 13:39:14 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection In-Reply-To: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Fri, 3 Nov 2023 12:42:46 GMT, Roman Kennke wrote: > See JBS issue for details. > > Testing: > - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC > - [x] tier1 -XX:+UseParallelGC > - [ ] tier2 -XX:+UseParallelGC > - [ ] hotspot_gc > It is not totally clear how we can get a forwarded object in eden in the first place. This can happen when promotion-failure occurs. Serial and G1 have special code to fix self-forwarded (promotion-fail) objs. Maybe Parallel should do sth similar. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1794843503 From tschatzl at openjdk.org Mon Nov 6 14:51:49 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Mon, 6 Nov 2023 14:51:49 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v11] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 15 additional commits since the last revision: - Merge tag 'jdk-22+21' into 8318706-implementation-of-region-pinning-in-g1 Added tag jdk-22+21 for changeset d96f38b8 - iwalulya review - typos - ayang review - renamings + documentation - Add documentation about why and how we handle pinned regions in the young/old generation. - Renamings to (almost) consistently use the following nomenclature for evacuation failure and types of it: * evacuation failure is the general concept. It includes * pinned regions * allocation failure One region can both be pinned and experience an allocation failure. G1 GC messages use tags "(Pinned)" and "(Allocation Failure)" now instead of "(Evacuation Failure)" Did not rename the G1EvacFailureInjector since this adds a lot of noise. - NULL -> nullptr - Fix compilation - Improve TestPinnedOldObjectsEvacuation test - Move tests into gc.g1.pinnedobjs package - ... and 5 more: https://git.openjdk.org/jdk/compare/24b37ee3...251f4d38 ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/f9735539..251f4d38 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=10 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=09-10 Stats: 4795 lines in 166 files changed: 2660 ins; 1554 del; 581 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From tschatzl at openjdk.org Mon Nov 6 16:06:23 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Mon, 6 Nov 2023 16:06:23 GMT Subject: RFR: 8319541: G1: Rename g1EvacFailure.?pp Message-ID: Hi, please review this change that properly names the `g1EvacFailure.?pp` files according to the single class in the file. Testing: local compilation Thanks, Thomas ------------- Commit messages: - 8319541-remove-g1evacfailure-files Changes: https://git.openjdk.org/jdk/pull/16522/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16522&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319541 Stats: 124 lines in 4 files changed: 61 ins; 61 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16522.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16522/head:pull/16522 PR: https://git.openjdk.org/jdk/pull/16522 From ayang at openjdk.org Mon Nov 6 16:17:08 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Mon, 6 Nov 2023 16:17:08 GMT Subject: RFR: 8319541: G1: Rename g1EvacFailure.?pp In-Reply-To: References: Message-ID: <7ZagmptvHQIqnLDXYR8QYIVwZZI41ME0fho3E1Qaycw=.8973dd47-8d96-47e2-b66d-c19f0b2ce8d4@github.com> On Mon, 6 Nov 2023 15:59:50 GMT, Thomas Schatzl wrote: > Hi, > > please review this change that properly names the `g1EvacFailure.?pp` files according to the single class in the file. > > Testing: local compilation > > Thanks, > Thomas Seems that not many depend on this file/class, so can it be moved to the caller's cpp file? ------------- PR Review: https://git.openjdk.org/jdk/pull/16522#pullrequestreview-1715612704 From rkennke at openjdk.org Mon Nov 6 16:23:11 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Mon, 6 Nov 2023 16:23:11 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection In-Reply-To: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Fri, 3 Nov 2023 12:42:46 GMT, Roman Kennke wrote: > See JBS issue for details. > > Testing: > - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC > - [x] tier1 -XX:+UseParallelGC > - [ ] tier2 -XX:+UseParallelGC > - [ ] hotspot_gc > * JDK-8319376 > > It is not totally clear how we can get a forwarded object in eden in the first place. > > This can happen when promotion-failure occurs. Serial and G1 have special code to fix self-forwarded (promotion-fail) objs. Maybe Parallel should do sth similar. Maybe. But that is not (only) what is happening here. I added a check for self-forwarded into my proposed change and it asserts like this: # Internal Error (/home/rkennke/src/openjdk/jdk/src/hotspot/share/memory/heapInspection.cpp:217), pid=7955, tid=8208 # assert(obj->forwardee() == obj) failed: expect self-forwarded: obj: 0x00000000fac62258, fwd: 0x00000000fd6bac78 ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1795334683 From rkennke at openjdk.org Mon Nov 6 16:27:26 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Mon, 6 Nov 2023 16:27:26 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v2] In-Reply-To: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: <3PbfIJdubuqzX5XYYHQARHUB0HW1PSP3tRH9DMsSz_s=.72565146-c053-42d8-8d9f-dc41b76cfb54@github.com> > See JBS issue for details. > > Testing: > - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC > - [x] tier1 -XX:+UseParallelGC > - [ ] tier2 -XX:+UseParallelGC > - [ ] hotspot_gc Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: Handle self-forwarded objects correctly ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16494/files - new: https://git.openjdk.org/jdk/pull/16494/files/4db0df9f..08336840 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16494.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16494/head:pull/16494 PR: https://git.openjdk.org/jdk/pull/16494 From tschatzl at openjdk.org Mon Nov 6 17:20:39 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Mon, 6 Nov 2023 17:20:39 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v12] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 16 additional commits since the last revision: - Merge tag 'jdk-22+22' into 8318706-implementation-of-region-pinning-in-g1 Added tag jdk-22+22 for changeset d354141a - Merge tag 'jdk-22+21' into 8318706-implementation-of-region-pinning-in-g1 Added tag jdk-22+21 for changeset d96f38b8 - iwalulya review - typos - ayang review - renamings + documentation - Add documentation about why and how we handle pinned regions in the young/old generation. - Renamings to (almost) consistently use the following nomenclature for evacuation failure and types of it: * evacuation failure is the general concept. It includes * pinned regions * allocation failure One region can both be pinned and experience an allocation failure. G1 GC messages use tags "(Pinned)" and "(Allocation Failure)" now instead of "(Evacuation Failure)" Did not rename the G1EvacFailureInjector since this adds a lot of noise. - NULL -> nullptr - Fix compilation - Improve TestPinnedOldObjectsEvacuation test - ... and 6 more: https://git.openjdk.org/jdk/compare/3af918f1...2ad39680 ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/251f4d38..2ad39680 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=11 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=10-11 Stats: 27458 lines in 1170 files changed: 14959 ins; 4156 del; 8343 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From ayang at openjdk.org Mon Nov 6 17:42:09 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Mon, 6 Nov 2023 17:42:09 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v2] In-Reply-To: <3PbfIJdubuqzX5XYYHQARHUB0HW1PSP3tRH9DMsSz_s=.72565146-c053-42d8-8d9f-dc41b76cfb54@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> <3PbfIJdubuqzX5XYYHQARHUB0HW1PSP3tRH9DMsSz_s=.72565146-c053-42d8-8d9f-dc41b76cfb54@github.com> Message-ID: On Mon, 6 Nov 2023 16:27:26 GMT, Roman Kennke wrote: >> See JBS issue for details. >> >> Testing: >> - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC >> - [x] tier1 -XX:+UseParallelGC >> - [ ] tier2 -XX:+UseParallelGC >> - [ ] hotspot_gc > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Handle self-forwarded objects correctly I don't get the assertion any more after adding the following to `PSScavenge::clean_up_failed_promotion`: ParallelScavengeHeap* heap = ParallelScavengeHeap::heap(); struct ResetForwardedMarkWord : ObjectClosure { void do_object(oop obj) override { if (obj->is_forwarded()) { obj->init_mark(); } } } cl; heap->young_gen()->eden_space()->object_iterate(&cl); heap->young_gen()->from_space()->object_iterate(&cl); (Essentially copied from Serial.) ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1795627388 From rkennke at openjdk.org Mon Nov 6 18:15:00 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Mon, 6 Nov 2023 18:15:00 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v3] In-Reply-To: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: > See JBS issue for details. > > Testing: > - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC > - [x] tier1 -XX:+UseParallelGC > - [ ] tier2 -XX:+UseParallelGC > - [ ] hotspot_gc Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: Restore forwardings after failed promotions ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16494/files - new: https://git.openjdk.org/jdk/pull/16494/files/08336840..e65910d3 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=01-02 Stats: 16 lines in 2 files changed: 12 ins; 2 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16494.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16494/head:pull/16494 PR: https://git.openjdk.org/jdk/pull/16494 From rkennke at openjdk.org Mon Nov 6 19:48:29 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Mon, 6 Nov 2023 19:48:29 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v3] In-Reply-To: References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Mon, 6 Nov 2023 18:15:00 GMT, Roman Kennke wrote: >> See JBS issue for details. >> >> Testing: >> - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC >> - [x] tier1 -XX:+UseParallelGC >> - [ ] tier2 -XX:+UseParallelGC >> - [ ] hotspot_gc > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Restore forwardings after failed promotions > I don't get the assertion any more after adding the following to `PSScavenge::clean_up_failed_promotion`: > > ```c++ > ParallelScavengeHeap* heap = ParallelScavengeHeap::heap(); > > struct ResetForwardedMarkWord : ObjectClosure { > void do_object(oop obj) override { > if (obj->is_forwarded()) { > obj->init_mark(); > } > } > } cl; > > heap->young_gen()->eden_space()->object_iterate(&cl); > heap->young_gen()->from_space()->object_iterate(&cl); > ``` > > (Essentially copied from Serial.) Nice, I changed this PR accordingly. I'm curious (and puzzled), though. How would this situation normally have been repaired without this explicit header-resetting code? Would the failed-promotion slide directly into full-GC which repairs it? OTOH, I don't even see code in paralle mark-compact which would explicitely repair forwardings. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1796198151 From jjoo at openjdk.org Tue Nov 7 00:52:07 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Tue, 7 Nov 2023 00:52:07 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v38] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: Attempt to fix duplicate name error in test ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/9fb36a9e..ac780c5e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=37 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=36-37 Stats: 8 lines in 1 file changed: 0 ins; 5 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From manc at openjdk.org Tue Nov 7 01:08:33 2023 From: manc at openjdk.org (Man Cao) Date: Tue, 7 Nov 2023 01:08:33 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v35] In-Reply-To: <3iJqYOiXeO6bwtUmjzlO3tyFR9Uc28YAJ8aQbKbqKJM=.fda95001-86f3-4f09-8a74-e15be1987c4f@github.com> References: <6tngC-Jwyx8e25LGT8dAwKbaPb9qb_w5ONctnFieH3o=.61b013b2-beb9-4053-8c06-86a700208d77@github.com> <3iJqYOiXeO6bwtUmjzlO3tyFR9Uc28YAJ8aQbKbqKJM=.fda95001-86f3-4f09-8a74-e15be1987c4f@github.com> Message-ID: On Mon, 6 Nov 2023 12:11:47 GMT, Stefan Johansson wrote: > I played around a bit instead of trying to explain what I mean and this is not very polished, but I was thinking something like this: https://github.com/openjdk/jdk/compare/pr/15082...kstefanj:jdk:pull/15082-idea > > What do you think? This way we don't add things to CollectedHeap as well, which is usually good unless really needed. I think it looks great. It is mainly refactoring that consolidates the declarations/definitions of the hsperf counters in to a single file. Would it be better to name that class `CPUTimeCounters`, so we could move `sun.threads.cpu_time.vm` and `sun.threads.cpu_time.conc_dedup`, and future JIT thread CPU counters to that class? Then we could also change the constructor of `ThreadTotalCPUTimeClosure` to `ThreadTotalCPUTimeClosure(CPUTimeCounters* counters, CPUTimeGroups::Name name)`, then it could set `_update_gc_counters` based on `name`. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1797109487 From tschatzl at openjdk.org Tue Nov 7 08:31:54 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 7 Nov 2023 08:31:54 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v13] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: Fix tests after merge ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/2ad39680..d9ccccff Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=12 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=11-12 Stats: 36 lines in 3 files changed: 0 ins; 0 del; 36 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From iwalulya at openjdk.org Tue Nov 7 09:21:32 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Tue, 7 Nov 2023 09:21:32 GMT Subject: RFR: 8319306: Serial: Remove TenuredSpace::verify In-Reply-To: References: Message-ID: On Thu, 2 Nov 2023 10:11:04 GMT, Albert Mingkun Yang wrote: > I didn't find the BOT verification logic much useful while working on JDK-8318647, so I propose removing it to simplify the code. Marked as reviewed by iwalulya (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16471#pullrequestreview-1717186888 From sjohanss at openjdk.org Tue Nov 7 09:57:40 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Tue, 7 Nov 2023 09:57:40 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v35] In-Reply-To: References: <6tngC-Jwyx8e25LGT8dAwKbaPb9qb_w5ONctnFieH3o=.61b013b2-beb9-4053-8c06-86a700208d77@github.com> <3iJqYOiXeO6bwtUmjzlO3tyFR9Uc28YAJ8aQbKbqKJM=.fda95001-86f3-4f09-8a74-e15be1987c4f@github.com> Message-ID: <1CHY1oIjteVS6FU__jUkfO6nTijg9j3tRbv_QJl3QLI=.18e3b2ba-3ab2-4ee8-ab5f-1dd30323f671@github.com> On Tue, 7 Nov 2023 01:06:12 GMT, Man Cao wrote: > I think it looks great. It is mainly refactoring that consolidates the declarations/definitions of the hsperf counters in to a single file. Would it be better to name that class `CPUTimeCounters`, so we could move `sun.threads.cpu_time.vm` and `sun.threads.cpu_time.conc_dedup`, and future JIT thread CPU counters to that class? > Yes, mainly refactoring and I was thinking along the same lines, but since this patch was just for GC and we had `CollectorCounters` already I went with this. I think calling the class `CPUTimeCounters` would be good and place it outside GC makes sense if we plan to include even more CPU time counters. Another name that we could improve is `CPUTimeGroups` and maybe also the enum name `Name`, they are ok, but we might come up with something better. > Then we could also change the constructor of `ThreadTotalCPUTimeClosure` to `ThreadTotalCPUTimeClosure(CPUTimeCounters* counters, CPUTimeGroups::Name name)`, then it could set `_update_gc_counters` based on `name`. I was looking at this too, but had to restructure the code more to avoid circular deps. If we create the general `CPUTImeCounters` we could move this closure to that file and then things would fit better I believe. So I like your proposals. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1798170871 From ayang at openjdk.org Tue Nov 7 10:04:31 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 7 Nov 2023 10:04:31 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v3] In-Reply-To: References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: <1KlNGWoYqYFKUBBqosFwRiAST-R-fjH5aiYXv7BCImE=.c8b568ad-21b2-42c5-bae1-4413707a878f@github.com> On Mon, 6 Nov 2023 19:46:07 GMT, Roman Kennke wrote: > Would the failed-promotion slide directly into full-GC which repairs it? No, promotion-fail objects are specially handled, as documented in `PSPromotionManager::oop_promotion_failed`. Other successfully forwarded objects are essentially dead, so during a full garbage collection, they are skipped. If a young garbage collection is successful, the eden/from space is cleared, which is why object iteration doesn't expose the "forwarded" markword. The patch above is just intended to demonstrate a way of suppressing the assertion, but it's likely not the best or correct way to proceed. Perhaps object-iteration logic in ParallelGC should skip those with the "forwarded" markword (because they are dead objs)? (This makes sense to me semantically, but I'm not sure about the potential consequences, given that object iteration is possibly used in many places.) ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1798184871 From tschatzl at openjdk.org Tue Nov 7 10:23:42 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 7 Nov 2023 10:23:42 GMT Subject: RFR: 8319541: G1: Inline G1RemoveSelfForwardsTask into RestoreRetainedRegionsTask [v2] In-Reply-To: References: Message-ID: <2BkmtiASN46Oger9XNaWFaBCtYLik2i-KdzoxnSuYvk=.03bec17c-3c03-46d6-a2a8-d36ecab78a86@github.com> > Hi, > > please review this change that inlines the `G1RemoveSelfForwardsTask` into the `RestoreRetainedRegionsTask` because the former is only used by the latter, and the latter only forwards to the former anyway. > > Testing: local compilation > > Thanks, > Thomas Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: Inline G1RemoveSelfForwardsTask into evacuate task ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16522/files - new: https://git.openjdk.org/jdk/pull/16522/files/25f165ad..01fc1f3b Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16522&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16522&range=00-01 Stats: 473 lines in 4 files changed: 188 ins; 280 del; 5 mod Patch: https://git.openjdk.org/jdk/pull/16522.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16522/head:pull/16522 PR: https://git.openjdk.org/jdk/pull/16522 From ayang at openjdk.org Tue Nov 7 10:32:30 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 7 Nov 2023 10:32:30 GMT Subject: RFR: 8319541: G1: Inline G1RemoveSelfForwardsTask into RestoreRetainedRegionsTask [v2] In-Reply-To: <2BkmtiASN46Oger9XNaWFaBCtYLik2i-KdzoxnSuYvk=.03bec17c-3c03-46d6-a2a8-d36ecab78a86@github.com> References: <2BkmtiASN46Oger9XNaWFaBCtYLik2i-KdzoxnSuYvk=.03bec17c-3c03-46d6-a2a8-d36ecab78a86@github.com> Message-ID: On Tue, 7 Nov 2023 10:23:42 GMT, Thomas Schatzl wrote: >> Hi, >> >> please review this change that inlines the `G1RemoveSelfForwardsTask` into the `RestoreRetainedRegionsTask` because the former is only used by the latter, and the latter only forwards to the former anyway. >> >> Testing: local compilation >> >> Thanks, >> Thomas > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > Inline G1RemoveSelfForwardsTask into evacuate task Marked as reviewed by ayang (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16522#pullrequestreview-1717351859 From tschatzl at openjdk.org Tue Nov 7 10:52:35 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 7 Nov 2023 10:52:35 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v3] In-Reply-To: References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: <8-NYDad3UQZM_JyiOqxRmfLVcD4L0_W0tX6LrpKmuZk=.abc09ef7-b6bc-4720-a8ab-46c193b68330@github.com> On Mon, 6 Nov 2023 19:46:07 GMT, Roman Kennke wrote: > I'm curious (and puzzled), though. How would this situation normally have been repaired without this explicit header-resetting code? Would the failed-promotion slide directly into full-GC which repairs it? OTOH, I don't even see code in paralle mark-compact which would explicitely repair forwardings. After full gc the from/to/survivor spaces should be reset to have their tops point to bottom. Maybe this is not done for all of its space, e.g. only for eden and to-space. At least I could not find anything related at a quick glance. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1798264144 From tschatzl at openjdk.org Tue Nov 7 11:05:39 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 7 Nov 2023 11:05:39 GMT Subject: RFR: 8319620: Parallel: Remove unused PSPromotionManager::*_is_full getters and setters Message-ID: <1RTWnwXGclK3wJwbIZR8W7y1-NHha7XSGqcihvMI5oI=.0e5458c3-60b9-4665-92dd-9941663a38f0@github.com> Hello, please review this trivial removal of some unused member functions in `PSPromotionManager`. Testing: local compilation Thanks, Thomas ------------- Commit messages: - 8319620-remove-is-full-getters Changes: https://git.openjdk.org/jdk/pull/16539/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16539&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319620 Stats: 5 lines in 1 file changed: 0 ins; 5 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16539.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16539/head:pull/16539 PR: https://git.openjdk.org/jdk/pull/16539 From ayang at openjdk.org Tue Nov 7 11:10:32 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 7 Nov 2023 11:10:32 GMT Subject: RFR: 8319620: Parallel: Remove unused PSPromotionManager::*_is_full getters and setters In-Reply-To: <1RTWnwXGclK3wJwbIZR8W7y1-NHha7XSGqcihvMI5oI=.0e5458c3-60b9-4665-92dd-9941663a38f0@github.com> References: <1RTWnwXGclK3wJwbIZR8W7y1-NHha7XSGqcihvMI5oI=.0e5458c3-60b9-4665-92dd-9941663a38f0@github.com> Message-ID: On Tue, 7 Nov 2023 10:59:24 GMT, Thomas Schatzl wrote: > Hello, > > please review this trivial removal of some unused member functions in `PSPromotionManager`. > > Testing: local compilation > > Thanks, > Thomas Trivial. ------------- Marked as reviewed by ayang (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16539#pullrequestreview-1717426986 From tschatzl at openjdk.org Tue Nov 7 11:21:27 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 7 Nov 2023 11:21:27 GMT Subject: RFR: 8319439: Move BufferNode from PtrQueue files to new files In-Reply-To: References: Message-ID: On Sat, 4 Nov 2023 11:25:05 GMT, Kim Barrett wrote: > Please review this simple (trivial?) change that moves the BufferNode class > from share/gc/shared/ptrQueue.[ch]pp to new files > share/gc/shared/bufferNode.[ch]pp. The code (as opposed to the surrounding > boiler plate and #include's) was moved unchanged. > > This led to adding #include's of the new header file to a bunch of other files > (include what you use), in some cases eliminating the need to #include > ptrQueue.hpp. Also tidied up the #include's for ptrQueue.[ch]pp to take > advantage of the split. > > Also, the gtest gc/shared/test_ptrQueueBufferAllocator.cpp is renamed to > gc/shared/test_bufferNodeAllocator.cpp, with some adjustments to includes and > test naming. > > Testing: > mach5 tier1 lgtm ------------- Marked as reviewed by tschatzl (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16506#pullrequestreview-1717445517 From iwalulya at openjdk.org Tue Nov 7 11:24:31 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Tue, 7 Nov 2023 11:24:31 GMT Subject: RFR: 8319541: G1: Inline G1RemoveSelfForwardsTask into RestoreRetainedRegionsTask [v2] In-Reply-To: <2BkmtiASN46Oger9XNaWFaBCtYLik2i-KdzoxnSuYvk=.03bec17c-3c03-46d6-a2a8-d36ecab78a86@github.com> References: <2BkmtiASN46Oger9XNaWFaBCtYLik2i-KdzoxnSuYvk=.03bec17c-3c03-46d6-a2a8-d36ecab78a86@github.com> Message-ID: On Tue, 7 Nov 2023 10:23:42 GMT, Thomas Schatzl wrote: >> Hi, >> >> please review this change that inlines the `G1RemoveSelfForwardsTask` into the `RestoreRetainedRegionsTask` because the former is only used by the latter, and the latter only forwards to the former anyway. >> >> Testing: local compilation >> >> Thanks, >> Thomas > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > Inline G1RemoveSelfForwardsTask into evacuate task Marked as reviewed by iwalulya (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16522#pullrequestreview-1717449340 From rkennke at openjdk.org Tue Nov 7 12:40:32 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Tue, 7 Nov 2023 12:40:32 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v3] In-Reply-To: <1KlNGWoYqYFKUBBqosFwRiAST-R-fjH5aiYXv7BCImE=.c8b568ad-21b2-42c5-bae1-4413707a878f@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> <1KlNGWoYqYFKUBBqosFwRiAST-R-fjH5aiYXv7BCImE=.c8b568ad-21b2-42c5-bae1-4413707a878f@github.com> Message-ID: On Tue, 7 Nov 2023 10:02:04 GMT, Albert Mingkun Yang wrote: > > Would the failed-promotion slide directly into full-GC which repairs it? > > No, promotion-fail objects are specially handled, as documented in `PSPromotionManager::oop_promotion_failed`. Other successfully forwarded objects are essentially dead, so during a full garbage collection, they are skipped. Ok. Just to clarify: After failed promotion, all references to original copies of promoted objects are updated, and therefore the original copies are dead (and can safely be skipped)? Or is has no reference been updated yet, and therefore the new copies are dead and can be skipped? > If a young garbage collection is successful, the eden/from space is cleared, which is why object iteration doesn't expose the "forwarded" markword. Yes, that makes sense. > The patch above is just intended to demonstrate a way of suppressing the assertion, but it's likely not the best or correct way to proceed. I thought so. > Perhaps object-iteration logic in ParallelGC should skip those with the "forwarded" markword (because they are dead objs)? (This makes sense to me semantically, but I'm not sure about the potential consequences, given that object iteration is possibly used in many places.) If the first statement above is true, then this seems reasonable. I will give it a try. Roman ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1798419996 From ayang at openjdk.org Tue Nov 7 12:47:33 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 7 Nov 2023 12:47:33 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v3] In-Reply-To: References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> <1KlNGWoYqYFKUBBqosFwRiAST-R-fjH5aiYXv7BCImE=.c8b568ad-21b2-42c5-bae1-4413707a878f@github.com> Message-ID: <6nUhJZh2N3GblEkFDLXQ9o2QSEreD8DR8YbQQlJSooY=.2941018d-27ba-4639-99aa-7e6fa13136b3@github.com> On Tue, 7 Nov 2023 12:38:14 GMT, Roman Kennke wrote: > all references to original copies of promoted objects are updated, and therefore the original copies are dead (and can safely be skipped) Yes; after scavenging, all pointers are correct (same value for promotion-fail and new value otherwise). ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1798430442 From tschatzl at openjdk.org Tue Nov 7 13:50:43 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 7 Nov 2023 13:50:43 GMT Subject: RFR: 8319620: Parallel: Remove unused PSPromotionManager::*_is_full getters and setters In-Reply-To: References: <1RTWnwXGclK3wJwbIZR8W7y1-NHha7XSGqcihvMI5oI=.0e5458c3-60b9-4665-92dd-9941663a38f0@github.com> Message-ID: <_QPoYM7Y2fQQt-L5pwVr_ks88K7lChY84gwc1e2vN_c=.1d336e44-2b4b-4bbb-9a1e-2b54ca95c16e@github.com> On Tue, 7 Nov 2023 11:08:15 GMT, Albert Mingkun Yang wrote: >> Hello, >> >> please review this trivial removal of some unused member functions in `PSPromotionManager`. >> >> Testing: local compilation >> >> Thanks, >> Thomas > > Trivial. Thanks @albertnetymk for your review ------------- PR Comment: https://git.openjdk.org/jdk/pull/16539#issuecomment-1798553864 From tschatzl at openjdk.org Tue Nov 7 13:50:45 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 7 Nov 2023 13:50:45 GMT Subject: Integrated: 8319620: Parallel: Remove unused PSPromotionManager::*_is_full getters and setters In-Reply-To: <1RTWnwXGclK3wJwbIZR8W7y1-NHha7XSGqcihvMI5oI=.0e5458c3-60b9-4665-92dd-9941663a38f0@github.com> References: <1RTWnwXGclK3wJwbIZR8W7y1-NHha7XSGqcihvMI5oI=.0e5458c3-60b9-4665-92dd-9941663a38f0@github.com> Message-ID: On Tue, 7 Nov 2023 10:59:24 GMT, Thomas Schatzl wrote: > Hello, > > please review this trivial removal of some unused member functions in `PSPromotionManager`. > > Testing: local compilation > > Thanks, > Thomas This pull request has now been integrated. Changeset: 85e4cde3 Author: Thomas Schatzl URL: https://git.openjdk.org/jdk/commit/85e4cde3f815122730ad69aa16e4006139fe7a79 Stats: 5 lines in 1 file changed: 0 ins; 5 del; 0 mod 8319620: Parallel: Remove unused PSPromotionManager::*_is_full getters and setters Reviewed-by: ayang ------------- PR: https://git.openjdk.org/jdk/pull/16539 From tschatzl at openjdk.org Tue Nov 7 13:51:42 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 7 Nov 2023 13:51:42 GMT Subject: RFR: 8319541: G1: Inline G1RemoveSelfForwardsTask into RestoreRetainedRegionsTask [v2] In-Reply-To: References: <2BkmtiASN46Oger9XNaWFaBCtYLik2i-KdzoxnSuYvk=.03bec17c-3c03-46d6-a2a8-d36ecab78a86@github.com> Message-ID: On Tue, 7 Nov 2023 10:29:44 GMT, Albert Mingkun Yang wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> Inline G1RemoveSelfForwardsTask into evacuate task > > Marked as reviewed by ayang (Reviewer). Thanks @albertnetymk @walulyai for your reviews ------------- PR Comment: https://git.openjdk.org/jdk/pull/16522#issuecomment-1798552640 From tschatzl at openjdk.org Tue Nov 7 13:51:44 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 7 Nov 2023 13:51:44 GMT Subject: Integrated: 8319541: G1: Inline G1RemoveSelfForwardsTask into RestoreRetainedRegionsTask In-Reply-To: References: Message-ID: On Mon, 6 Nov 2023 15:59:50 GMT, Thomas Schatzl wrote: > Hi, > > please review this change that inlines the `G1RemoveSelfForwardsTask` into the `RestoreRetainedRegionsTask` because the former is only used by the latter, and the latter only forwards to the former anyway. > > Testing: local compilation > > Thanks, > Thomas This pull request has now been integrated. Changeset: c7600979 Author: Thomas Schatzl URL: https://git.openjdk.org/jdk/commit/c760097943d0fc893433d813502e2ff02bb4c82d Stats: 473 lines in 4 files changed: 188 ins; 280 del; 5 mod 8319541: G1: Inline G1RemoveSelfForwardsTask into RestoreRetainedRegionsTask Reviewed-by: ayang, iwalulya ------------- PR: https://git.openjdk.org/jdk/pull/16522 From tschatzl at openjdk.org Tue Nov 7 13:52:29 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 7 Nov 2023 13:52:29 GMT Subject: RFR: 8319204: G1: Change G1CMTask::_termination_time_ms to wallclock time In-Reply-To: <-XZQmjAaASQW0XC_6PfYBXhmVKXTsTO1BIw2FKk_YpM=.9935a72d-9b76-4c03-ab5a-0c63b8a1e255@github.com> References: <-XZQmjAaASQW0XC_6PfYBXhmVKXTsTO1BIw2FKk_YpM=.9935a72d-9b76-4c03-ab5a-0c63b8a1e255@github.com> Message-ID: On Wed, 1 Nov 2023 09:27:41 GMT, Albert Mingkun Yang wrote: > Simple switching from thread-cpu to wall-clock time for termination duration metric. lgtm. ------------- Marked as reviewed by tschatzl (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16446#pullrequestreview-1717731348 From tschatzl at openjdk.org Tue Nov 7 14:11:20 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 7 Nov 2023 14:11:20 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v14] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: "GCLocker Initiated GC" is not a valid GC cause for G1 any more ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/d9ccccff..c272a736 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=13 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=12-13 Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From rkennke at openjdk.org Tue Nov 7 14:31:49 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Tue, 7 Nov 2023 14:31:49 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v4] In-Reply-To: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: > See JBS issue for details. > > Testing: > - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC > - [x] tier1 -XX:+UseParallelGC > - [ ] tier2 -XX:+UseParallelGC > - [ ] hotspot_gc Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: Don't expose forwarded objects during object iteration ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16494/files - new: https://git.openjdk.org/jdk/pull/16494/files/e65910d3..78ffecb1 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=02-03 Stats: 16 lines in 2 files changed: 3 ins; 12 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16494.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16494/head:pull/16494 PR: https://git.openjdk.org/jdk/pull/16494 From ayang at openjdk.org Tue Nov 7 14:33:41 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 7 Nov 2023 14:33:41 GMT Subject: RFR: 8319204: G1: Change G1CMTask::_termination_time_ms to wallclock time In-Reply-To: <-XZQmjAaASQW0XC_6PfYBXhmVKXTsTO1BIw2FKk_YpM=.9935a72d-9b76-4c03-ab5a-0c63b8a1e255@github.com> References: <-XZQmjAaASQW0XC_6PfYBXhmVKXTsTO1BIw2FKk_YpM=.9935a72d-9b76-4c03-ab5a-0c63b8a1e255@github.com> Message-ID: On Wed, 1 Nov 2023 09:27:41 GMT, Albert Mingkun Yang wrote: > Simple switching from thread-cpu to wall-clock time for termination duration metric. Thanks for the review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16446#issuecomment-1798652164 From ayang at openjdk.org Tue Nov 7 14:33:42 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 7 Nov 2023 14:33:42 GMT Subject: Integrated: 8319204: G1: Change G1CMTask::_termination_time_ms to wallclock time In-Reply-To: <-XZQmjAaASQW0XC_6PfYBXhmVKXTsTO1BIw2FKk_YpM=.9935a72d-9b76-4c03-ab5a-0c63b8a1e255@github.com> References: <-XZQmjAaASQW0XC_6PfYBXhmVKXTsTO1BIw2FKk_YpM=.9935a72d-9b76-4c03-ab5a-0c63b8a1e255@github.com> Message-ID: On Wed, 1 Nov 2023 09:27:41 GMT, Albert Mingkun Yang wrote: > Simple switching from thread-cpu to wall-clock time for termination duration metric. This pull request has now been integrated. Changeset: bf9a93de Author: Albert Mingkun Yang URL: https://git.openjdk.org/jdk/commit/bf9a93de1fc2fceaad3fbfb68fc261b2da8c0ae6 Stats: 8 lines in 2 files changed: 0 ins; 6 del; 2 mod 8319204: G1: Change G1CMTask::_termination_time_ms to wallclock time Reviewed-by: iwalulya, tschatzl ------------- PR: https://git.openjdk.org/jdk/pull/16446 From ayang at openjdk.org Tue Nov 7 14:40:33 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 7 Nov 2023 14:40:33 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v4] In-Reply-To: References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Tue, 7 Nov 2023 14:31:49 GMT, Roman Kennke wrote: >> See JBS issue for details. >> >> Testing: >> - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC >> - [x] tier1 -XX:+UseParallelGC >> - [ ] tier2 -XX:+UseParallelGC >> - [ ] hotspot_gc > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Don't expose forwarded objects during object iteration src/hotspot/share/gc/parallel/mutableSpace.cpp line 239: > 237: while (p < top()) { > 238: oop obj = cast_to_oop(p); > 239: if (!obj->is_forwarded()) { Maybe some comment along these lines "When promotion-failure occurs during Young GC, eden/from space is not cleared, so we can encounter objs with "forwarded" markword. They are essentially dead, so skipping them." src/hotspot/share/gc/parallel/mutableSpace.cpp line 242: > 240: cl->do_object(cast_to_oop(p)); > 241: } > 242: p += cast_to_oop(p)->size(); One can use `obj` instead of `cast_to_oop`. src/hotspot/share/memory/heapInspection.cpp line 216: > 214: // of running out of space required to create a new entry. > 215: bool KlassInfoTable::record_instance(const oop obj) { > 216: assert(!obj->is_forwarded(), "must not be forwarded"); It'd be good if someone more familiar with this method/class can comment whether this assert makes sense in this context. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16494#discussion_r1385018851 PR Review Comment: https://git.openjdk.org/jdk/pull/16494#discussion_r1385014921 PR Review Comment: https://git.openjdk.org/jdk/pull/16494#discussion_r1385020892 From rkennke at openjdk.org Tue Nov 7 14:59:48 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Tue, 7 Nov 2023 14:59:48 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v5] In-Reply-To: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: > See JBS issue for details. > > Testing: > - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC > - [x] tier1 -XX:+UseParallelGC > - [ ] tier2 -XX:+UseParallelGC > - [ ] hotspot_gc Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: Review comments; expose self-forwarded objects ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16494/files - new: https://git.openjdk.org/jdk/pull/16494/files/78ffecb1..988b96e6 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=03-04 Stats: 7 lines in 2 files changed: 3 ins; 2 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16494.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16494/head:pull/16494 PR: https://git.openjdk.org/jdk/pull/16494 From ayang at openjdk.org Tue Nov 7 15:13:34 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 7 Nov 2023 15:13:34 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v5] In-Reply-To: References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Tue, 7 Nov 2023 14:59:48 GMT, Roman Kennke wrote: >> See JBS issue for details. >> >> Testing: >> - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC >> - [x] tier1 -XX:+UseParallelGC >> - [ ] tier2 -XX:+UseParallelGC >> - [ ] hotspot_gc > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Review comments; expose self-forwarded objects src/hotspot/share/gc/parallel/mutableSpace.cpp line 242: > 240: // so we can encounter objects with "forwarded" markword. > 241: // They are essentially dead, so skipping them > 242: if (!obj->is_forwarded() || obj->forwardee() == obj) { Self-forwarded ones should already be handled by `PSPromotionManager::restore_preserved_marks`, so the second condition will always be false. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16494#discussion_r1385076675 From tschatzl at openjdk.org Tue Nov 7 18:11:52 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 7 Nov 2023 18:11:52 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v15] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 19 commits: - Merge branch 'master' into 8318706-implementation-of-region-pinning-in-g1 - "GCLocker Initiated GC" is not a valid GC cause for G1 any more - Fix tests after merge - Merge tag 'jdk-22+22' into 8318706-implementation-of-region-pinning-in-g1 Added tag jdk-22+22 for changeset d354141a - Merge tag 'jdk-22+21' into 8318706-implementation-of-region-pinning-in-g1 Added tag jdk-22+21 for changeset d96f38b8 - iwalulya review - typos - ayang review - renamings + documentation - Add documentation about why and how we handle pinned regions in the young/old generation. - Renamings to (almost) consistently use the following nomenclature for evacuation failure and types of it: * evacuation failure is the general concept. It includes * pinned regions * allocation failure One region can both be pinned and experience an allocation failure. G1 GC messages use tags "(Pinned)" and "(Allocation Failure)" now instead of "(Evacuation Failure)" Did not rename the G1EvacFailureInjector since this adds a lot of noise. - ... and 9 more: https://git.openjdk.org/jdk/compare/45e68ae2...83eff9fe ------------- Changes: https://git.openjdk.org/jdk/pull/16342/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=14 Stats: 1820 lines in 59 files changed: 1147 ins; 430 del; 243 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From rkennke at openjdk.org Tue Nov 7 18:12:45 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Tue, 7 Nov 2023 18:12:45 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v6] In-Reply-To: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: > See JBS issue for details. > > Testing: > - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC > - [x] tier1 -XX:+UseParallelGC > - [ ] tier2 -XX:+UseParallelGC > - [ ] hotspot_gc Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: Do *not* handle self-forwarded objects ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16494/files - new: https://git.openjdk.org/jdk/pull/16494/files/988b96e6..6ba7c713 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=04-05 Stats: 6 lines in 1 file changed: 5 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16494.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16494/head:pull/16494 PR: https://git.openjdk.org/jdk/pull/16494 From ayang at openjdk.org Tue Nov 7 18:17:31 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 7 Nov 2023 18:17:31 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v6] In-Reply-To: References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Tue, 7 Nov 2023 18:12:45 GMT, Roman Kennke wrote: >> See JBS issue for details. >> >> Testing: >> - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC >> - [x] tier1 -XX:+UseParallelGC >> - [ ] tier2 -XX:+UseParallelGC >> - [ ] hotspot_gc > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Do *not* handle self-forwarded objects Marked as reviewed by ayang (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16494#pullrequestreview-1718395065 From ayang at openjdk.org Wed Nov 8 09:41:57 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 8 Nov 2023 09:41:57 GMT Subject: RFR: 8319456: jdk/jfr/event/gc/collection/TestGCCauseWith[Serial|Parallel].java : GC cause 'GCLocker Initiated GC' not in the valid causes In-Reply-To: <3vf4CilrHIDJRN6KbtezqVAz3YBMkyEiy8syWXesfvI=.99460479-ef0e-4d30-b918-2c2542ff2a2b@github.com> References: <3vf4CilrHIDJRN6KbtezqVAz3YBMkyEiy8syWXesfvI=.99460479-ef0e-4d30-b918-2c2542ff2a2b@github.com> Message-ID: On Tue, 7 Nov 2023 14:02:37 GMT, Thomas Schatzl wrote: > Hi all, > > please review these fixes to the `jdk/jfr/event/gc/collection/TestGCCauseWith[Serial|Parallel].java` tests that fail if the GC cause has been "GCLocker Initiated GC". > > This is a valid gc cause, just extremely rare (interestingly the corresponding G1 tests added it). > > Thanks, > Thomas Marked as reviewed by ayang (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16542#pullrequestreview-1719819307 From iwalulya at openjdk.org Wed Nov 8 09:50:57 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Wed, 8 Nov 2023 09:50:57 GMT Subject: RFR: 8319456: jdk/jfr/event/gc/collection/TestGCCauseWith[Serial|Parallel].java : GC cause 'GCLocker Initiated GC' not in the valid causes In-Reply-To: <3vf4CilrHIDJRN6KbtezqVAz3YBMkyEiy8syWXesfvI=.99460479-ef0e-4d30-b918-2c2542ff2a2b@github.com> References: <3vf4CilrHIDJRN6KbtezqVAz3YBMkyEiy8syWXesfvI=.99460479-ef0e-4d30-b918-2c2542ff2a2b@github.com> Message-ID: On Tue, 7 Nov 2023 14:02:37 GMT, Thomas Schatzl wrote: > Hi all, > > please review these fixes to the `jdk/jfr/event/gc/collection/TestGCCauseWith[Serial|Parallel].java` tests that fail if the GC cause has been "GCLocker Initiated GC". > > This is a valid gc cause, just extremely rare (interestingly the corresponding G1 tests added it). > > Thanks, > Thomas Marked as reviewed by iwalulya (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16542#pullrequestreview-1719836277 From ayang at openjdk.org Wed Nov 8 10:59:34 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 8 Nov 2023 10:59:34 GMT Subject: RFR: 8319703: Serial: Remove generationSpec Message-ID: Simple cleanup of removing an unnecessary indirection. The real change is only 11 LOC, using the global variables directly for young/old gen. ------------- Commit messages: - s1-remove-gen-spec Changes: https://git.openjdk.org/jdk/pull/16554/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16554&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319703 Stats: 265 lines in 12 files changed: 1 ins; 254 del; 10 mod Patch: https://git.openjdk.org/jdk/pull/16554.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16554/head:pull/16554 PR: https://git.openjdk.org/jdk/pull/16554 From tschatzl at openjdk.org Wed Nov 8 11:44:59 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Wed, 8 Nov 2023 11:44:59 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v6] In-Reply-To: References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Tue, 7 Nov 2023 18:12:45 GMT, Roman Kennke wrote: >> See JBS issue for details. >> >> Testing: >> - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC >> - [x] tier1 -XX:+UseParallelGC >> - [x] tier2 -XX:+UseParallelGC >> - [x] hotspot_gc > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Do *not* handle self-forwarded objects Marked as reviewed by tschatzl (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16494#pullrequestreview-1720129165 From tschatzl at openjdk.org Wed Nov 8 11:46:55 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Wed, 8 Nov 2023 11:46:55 GMT Subject: RFR: 8319306: Serial: Remove TenuredSpace::verify In-Reply-To: References: Message-ID: On Thu, 2 Nov 2023 10:11:04 GMT, Albert Mingkun Yang wrote: > I didn't find the BOT verification logic much useful while working on JDK-8318647, so I propose removing it to simplify the code. Marked as reviewed by tschatzl (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16471#pullrequestreview-1720131725 From tschatzl at openjdk.org Wed Nov 8 11:47:56 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Wed, 8 Nov 2023 11:47:56 GMT Subject: RFR: 8319158: Parallel: Make TestObjectTenuringFlags use createTestJavaProcessBuilder In-Reply-To: References: Message-ID: On Tue, 31 Oct 2023 09:55:17 GMT, Leo Korinth wrote: > Tested with: > `make run-test TEST=jtreg:test/hotspot/jtreg/gc/arguments/TestObjectTenuringFlags.java JTREG='VERBOSE=all;JAVA_OPTIONS='` -> pass 1 > `make run-test TEST=jtreg:test/hotspot/jtreg/gc/arguments/TestObjectTenuringFlags.java JTREG='VERBOSE=all;JAVA_OPTIONS=-XX:+UseSerialGC'` -> pass 0 > `make run-test TEST=jtreg:test/hotspot/jtreg/gc/arguments/TestObjectTenuringFlags.java JTREG='VERBOSE=all;JAVA_OPTIONS=-XX:+NeverTenure'` -> pass 0 > > I will later (before integrating) run this test together with more tests through high tier testing to see that we do not fail when rotating flags. Marked as reviewed by tschatzl (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16432#pullrequestreview-1720133520 From tschatzl at openjdk.org Wed Nov 8 11:50:59 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Wed, 8 Nov 2023 11:50:59 GMT Subject: RFR: 8319161: GC: Make TestParallelGCThreads use createTestJavaProcessBuilder In-Reply-To: References: Message-ID: On Tue, 31 Oct 2023 16:26:19 GMT, Leo Korinth wrote: > `make run-test TEST=jtreg:test/hotspot/jtreg/gc/arguments/TestParallelGCThreads.java JTREG='VERBOSE=all;JAVA_OPTIONS='` -> pass 1 > `make run-test TEST=jtreg:test/hotspot/jtreg/gc/arguments/TestParallelGCThreads.java JTREG='VERBOSE=all;JAVA_OPTIONS=-XX:ParallelGCThreads=42' ` --> total 0 > `make run-test TEST=jtreg:test/hotspot/jtreg/gc/arguments/TestParallelGCThreads.java JTREG='VERBOSE=all;JAVA_OPTIONS=-XX:+UseG1GC' ` --> total 0 > `make run-test TEST=jtreg:test/hotspot/jtreg/gc/arguments/TestParallelGCThreads.java JTREG='VERBOSE=all;JAVA_OPTIONS=-XX:+UseParallelGC' ` -> total 0 > `make run-test TEST=jtreg:test/hotspot/jtreg/gc/arguments/TestParallelGCThreads.java JTREG='VERBOSE=all;JAVA_OPTIONS=-XX:+UseSerialGC' ` -> total 0 > `make run-test TEST=jtreg:test/hotspot/jtreg/gc/arguments/TestParallelGCThreads.java JTREG='VERBOSE=all;JAVA_OPTIONS=-XX:+UseZGC' `-> total 0 > `make run-test TEST=jtreg:test/hotspot/jtreg/gc/arguments/TestParallelGCThreads.java JTREG='VERBOSE=all;JAVA_OPTIONS=-XX:+UseShenandoahGC' `-> total 0 > `make run-test TEST=jtreg:test/hotspot/jtreg/gc/arguments/TestParallelGCThreads.java JTREG='VERBOSE=all;JAVA_OPTIONS=-XX:+UnlockExperimentalVMOptions -XX:+UseEpsilonGC' ` -> total 0 > > I will later (before integrating) run this test together with more tests through high tier testing to see that we do not fail when rotating flags. Marked as reviewed by tschatzl (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16434#pullrequestreview-1720138072 From ayang at openjdk.org Wed Nov 8 11:58:06 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 8 Nov 2023 11:58:06 GMT Subject: RFR: 8319306: Serial: Remove TenuredSpace::verify In-Reply-To: References: Message-ID: On Thu, 2 Nov 2023 10:11:04 GMT, Albert Mingkun Yang wrote: > I didn't find the BOT verification logic much useful while working on JDK-8318647, so I propose removing it to simplify the code. Thanks for the review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16471#issuecomment-1801739267 From ayang at openjdk.org Wed Nov 8 11:58:07 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 8 Nov 2023 11:58:07 GMT Subject: Integrated: 8319306: Serial: Remove TenuredSpace::verify In-Reply-To: References: Message-ID: On Thu, 2 Nov 2023 10:11:04 GMT, Albert Mingkun Yang wrote: > I didn't find the BOT verification logic much useful while working on JDK-8318647, so I propose removing it to simplify the code. This pull request has now been integrated. Changeset: cd9719bc Author: Albert Mingkun Yang URL: https://git.openjdk.org/jdk/commit/cd9719bc1d5266faa6c3a0009fb5881a2016e7e1 Stats: 48 lines in 2 files changed: 0 ins; 47 del; 1 mod 8319306: Serial: Remove TenuredSpace::verify Reviewed-by: iwalulya, tschatzl ------------- PR: https://git.openjdk.org/jdk/pull/16471 From tschatzl at openjdk.org Wed Nov 8 13:51:03 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Wed, 8 Nov 2023 13:51:03 GMT Subject: RFR: 8317755: G1: Periodic GC interval should test for the last whole heap GC [v2] In-Reply-To: References: Message-ID: On Wed, 18 Oct 2023 14:27:26 GMT, Aleksey Shipilev wrote: >> See the description in the bug. Fortunately, we already track the last whole-heap GC. The new regression test verifies the behavior. >> >> Additional testing: >> - [x] Linux x86_64 fastdebug `tier1 tier2 tier3` > > Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: > > - Merge branch 'master' into JDK-8317755-g1-periodic-whole > - Optional flag > - Merge branch 'master' into JDK-8317755-g1-periodic-whole > - Keep the cast > - Fix Sorry for the late reply, I have been busy with getting the 423 JEP out the door :) > _Mailing list message from [Kirk Pepperdine](mailto:kirk at kodewerk.com) on [hotspot-gc-dev](mailto:hotspot-gc-dev at mail.openjdk.org):_ > > Don?t we have already have a GCInterval with a default value of Long.MAX_VALUE? There is no such flag. >When I hear the word idle I immediately start thinking, CPU idle. In this case however, I quickly shifted to memory idle which I think translates nicely into how idle are the allocators. Thus basing heap sizing ergonomics on allocation rates seems like a reasonable metric until you consider the edge cases. The most significant ?edge case? IME is when GC overheads start exceeding 20%. In those cases GC will throttle allocations and that in turn cases ergonomics to reduce sizes instead of increasing them (to reduce GC overhead). My conclusion from this is that ergonomics should consider both allocation rates and GC overhead when deciding on how to resize heap at the end of a collection. Allocation rate directly impacts GC overhead by causing more GCs. > Fortunately there are a steady stream of GC events that create a convenient point in time to make an ergonomic adjustment. Not having allocations and as a result not having the collector run implies one has to manufacture a convenient point in time to make an ergonomic sizing decision. See [JDK-8238687](https://bugs.openjdk.org/browse/JDK-8238687). That CR and this proposed feature complement each other, GC should do something when there is no steady stream of gc events. > [...] > As for returning memory, two issues, there appears to be no definition for ?unused memory?. Secondly, what I can say after looking at 1000s of GC logs is that the amount of floating garbage that G1 leaves behind even after several concurrent cycles is not insignificant. G1 currently respects `Min/MaxHeapFreeRatio` which is tunable. They may be fairly lenient. >I also write a G1 heap fragmentation viewer and what it revealed is that heap remains highly fragmented and scattered after each GC cycle. All this suggests that heap will need to be compacted with a full collection in order to return a significantly large enough block of memory to make the entire effort worthwhile. There is no harm with fragmentation on a region level; G1 will give back memory on a region level anyway if able to do so (see `Min/MaxHeapFreeRatio` default values). There is now a humongous-object compacting full gc stage if necessary, so keeping region-level fragmentation in check is less of an issue now. >Again, if the application is idle, then no harm no foul. However, for those applications that are memory-idle but not CPU-idle this might not be a great course of action. Hence the flag described in the JEP. The reason why the defaults disable the CPU-idle check is that there is no good default. > > In my mind, any trigger for a speculative collection would need to take into consideration, allocation rates, GC overhead, and mutator busyness (for cases when GC and allocator activity is low to 0). > It does (if told to do so appropriately). >From @shipilev : >> This is corroborated by experience from me working with end users actually doing that (i.e. force a whole heap analysis externally). > >Yes, and that is the beauty of periodic GCs that are driven by GC itself, when it knows it should not start the periodic GCs if there was recent activity. That part if captured in JEP, I think. What this PR does, is extends the usefulness of periodic GCs to the cases where you need a whole heap analysis to make the best call about what to do next. In G1 case, maybe discovering that we need to start running mixed collections to unclutter the heap. I think there is still a misconception with the current "periodic gcs" here: it is first and foremost thought to be applicable for catching the idle case with all its options (also consider cpu idle, do full or concurrent gc). This is the reason for the suggestion to phase out the current option `G1PeriodicGCInterval` to use a different name, and add this functionality (regular whole heap analysis to clean out cruft) as a new option (something like `WholeHeapAnalysisInterval`, or something more aligned with other collectors) I could easily imagine the use case where one would want to act fairly quickly (and allowing a disruptive full gc) on the detected idle case ("shortly" after detecting idle), and be more relaxed about general cleanup (long period, being least disruptive using a concurrent mark). Thomas ------------- PR Comment: https://git.openjdk.org/jdk/pull/16107#issuecomment-1801926929 From tschatzl at openjdk.org Wed Nov 8 13:54:11 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Wed, 8 Nov 2023 13:54:11 GMT Subject: RFR: 8319456: jdk/jfr/event/gc/collection/TestGCCauseWith[Serial|Parallel].java : GC cause 'GCLocker Initiated GC' not in the valid causes In-Reply-To: References: <3vf4CilrHIDJRN6KbtezqVAz3YBMkyEiy8syWXesfvI=.99460479-ef0e-4d30-b918-2c2542ff2a2b@github.com> Message-ID: On Wed, 8 Nov 2023 09:39:41 GMT, Albert Mingkun Yang wrote: >> Hi all, >> >> please review these fixes to the `jdk/jfr/event/gc/collection/TestGCCauseWith[Serial|Parallel].java` tests that fail if the GC cause has been "GCLocker Initiated GC". >> >> This is a valid gc cause, just extremely rare (interestingly the corresponding G1 tests added it). >> >> Thanks, >> Thomas > > Marked as reviewed by ayang (Reviewer). Thanks @albertnetymk @walulyai for your reviews ------------- PR Comment: https://git.openjdk.org/jdk/pull/16542#issuecomment-1801928063 From tschatzl at openjdk.org Wed Nov 8 13:54:13 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Wed, 8 Nov 2023 13:54:13 GMT Subject: Integrated: 8319456: jdk/jfr/event/gc/collection/TestGCCauseWith[Serial|Parallel].java : GC cause 'GCLocker Initiated GC' not in the valid causes In-Reply-To: <3vf4CilrHIDJRN6KbtezqVAz3YBMkyEiy8syWXesfvI=.99460479-ef0e-4d30-b918-2c2542ff2a2b@github.com> References: <3vf4CilrHIDJRN6KbtezqVAz3YBMkyEiy8syWXesfvI=.99460479-ef0e-4d30-b918-2c2542ff2a2b@github.com> Message-ID: On Tue, 7 Nov 2023 14:02:37 GMT, Thomas Schatzl wrote: > Hi all, > > please review these fixes to the `jdk/jfr/event/gc/collection/TestGCCauseWith[Serial|Parallel].java` tests that fail if the GC cause has been "GCLocker Initiated GC". > > This is a valid gc cause, just extremely rare (interestingly the corresponding G1 tests added it). > > Thanks, > Thomas This pull request has now been integrated. Changeset: 7c7f8ea3 Author: Thomas Schatzl URL: https://git.openjdk.org/jdk/commit/7c7f8ea30da7fe552bcd4f2b593fa9aad27dcdb4 Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod 8319456: jdk/jfr/event/gc/collection/TestGCCauseWith[Serial|Parallel].java : GC cause 'GCLocker Initiated GC' not in the valid causes Reviewed-by: ayang, iwalulya ------------- PR: https://git.openjdk.org/jdk/pull/16542 From ayang at openjdk.org Wed Nov 8 13:56:21 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 8 Nov 2023 13:56:21 GMT Subject: RFR: 8319713: Parallel: Remove PSAdaptiveSizePolicy::should_full_GC Message-ID: Simple removing too-conservative full-gc heuristic. ------------- Commit messages: - pgc-conservative-full-gc Changes: https://git.openjdk.org/jdk/pull/16560/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16560&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319713 Stats: 24 lines in 3 files changed: 0 ins; 23 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16560.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16560/head:pull/16560 PR: https://git.openjdk.org/jdk/pull/16560 From sjohanss at openjdk.org Wed Nov 8 14:49:00 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Wed, 8 Nov 2023 14:49:00 GMT Subject: RFR: 8318706: Implement JEP 423: Region Pinning for G1 [v9] In-Reply-To: References: Message-ID: <6_MNWrkq_LnvB4EqrBFNo6e6kVHLgffcRbkxhRPzTpg=.1a05d8af-b181-4e80-9922-6aea9a517564@github.com> On Fri, 3 Nov 2023 14:14:36 GMT, Albert Mingkun Yang wrote: >> Parsing the separate components is easier :) Not sure if these tags in any way ever indicated some level of abstraction. >> >> I do not have a strong opinion here. The combinations >> >> (Pinned) >> (Allocation Failure) >> (Pinned + Allocation Failure) // or the other way around, or some other symbol for "+" or no symbol at all? >> >> are fine with me (and I thought about doing something more elaborate here), but my concern has been that any complicated string makes it less unique (e.g. `(Allocation Failure)` vs. "Allocation Failure") and adds code both to implement and parse the result. >> >> Much more disrupting is likely that there is no "Evacuation Failure" string any more. But log messages are not part of the external interface, and we should not want to change them just because. > > The example looks good to me. Have the final output looking something like this was agreed on during internal discussion: GC(6) Pause Young (Normal) (Evacuation Failure: Pinned) 1M->1M(22M) 36.16ms GC(6) Pause Young (Normal) (Evacuation Failure: Allocation) 1M->1M(22M) 36.16ms GC(6) Pause Young (Normal) (Evacuation Failure: Allocation / Pinned) 1M->1M(22M) 36.16ms ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1386736860 From rkennke at openjdk.org Wed Nov 8 15:21:06 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Wed, 8 Nov 2023 15:21:06 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v6] In-Reply-To: References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Tue, 7 Nov 2023 18:12:45 GMT, Roman Kennke wrote: >> See JBS issue for details. >> >> Testing: >> - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC >> - [x] tier1 -XX:+UseParallelGC >> - [x] tier2 -XX:+UseParallelGC >> - [x] hotspot_gc > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Do *not* handle self-forwarded objects Thanks! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1802098998 From rkennke at openjdk.org Wed Nov 8 15:21:08 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Wed, 8 Nov 2023 15:21:08 GMT Subject: Integrated: 8319376: ParallelGC: Forwarded objects found during heap inspection In-Reply-To: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Fri, 3 Nov 2023 12:42:46 GMT, Roman Kennke wrote: > See JBS issue for details. > > Testing: > - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC > - [x] tier1 -XX:+UseParallelGC > - [x] tier2 -XX:+UseParallelGC > - [x] hotspot_gc This pull request has now been integrated. Changeset: 59e9981e Author: Roman Kennke URL: https://git.openjdk.org/jdk/commit/59e9981ec21258b8aa5f93cb1fb9b0ccf9f846af Stats: 12 lines in 1 file changed: 11 ins; 0 del; 1 mod 8319376: ParallelGC: Forwarded objects found during heap inspection Co-authored-by: Albert Mingkun Yang Reviewed-by: ayang, tschatzl ------------- PR: https://git.openjdk.org/jdk/pull/16494 From shade at openjdk.org Wed Nov 8 16:01:06 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 8 Nov 2023 16:01:06 GMT Subject: RFR: 8319725: G1: Subtracting virtual time from wall time after JDK-8319204 Message-ID: Noticed this when looking at the [JDK-8319204](https://bugs.openjdk.org/browse/JDK-8319204) changeset: we end subtracting `elapsedVTime()` vs `elapsedTime()`, which looks incorrect, as it does math on two unrelated quantities. ------------- Commit messages: - Fix Changes: https://git.openjdk.org/jdk/pull/16564/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16564&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319725 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16564.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16564/head:pull/16564 PR: https://git.openjdk.org/jdk/pull/16564 From ayang at openjdk.org Wed Nov 8 16:36:00 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 8 Nov 2023 16:36:00 GMT Subject: RFR: 8319725: G1: Subtracting virtual time from wall time after JDK-8319204 In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 15:52:50 GMT, Aleksey Shipilev wrote: > Noticed this when looking at the [JDK-8319204](https://bugs.openjdk.org/browse/JDK-8319204) changeset: we end subtracting `elapsedVTime()` vs `elapsedTime()`, which looks incorrect, as it does math on two unrelated quantities. Thank you for catching this. ------------- Marked as reviewed by ayang (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16564#pullrequestreview-1720794998 From iwalulya at openjdk.org Wed Nov 8 16:36:01 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Wed, 8 Nov 2023 16:36:01 GMT Subject: RFR: 8319725: G1: Subtracting virtual time from wall time after JDK-8319204 In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 15:52:50 GMT, Aleksey Shipilev wrote: > Noticed this when looking at the [JDK-8319204](https://bugs.openjdk.org/browse/JDK-8319204) changeset: we end subtracting `elapsedVTime()` vs `elapsedTime()`, which looks incorrect, as it does math on two unrelated quantities. Marked as reviewed by iwalulya (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16564#pullrequestreview-1720796275 From rkennke at openjdk.org Wed Nov 8 16:36:13 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Wed, 8 Nov 2023 16:36:13 GMT Subject: RFR: 8319726: Parallel GC: Re-use object in object-iterator Message-ID: As a minor follow-up/clean-up to [JDK-8319376](https://bugs.openjdk.org/browse/JDK-8319376), we can use the local obj field that we casted a few lines up, instead of casting from p again. ------------- Commit messages: - 8319726: Parallel GC: Re-use object in object-iterator Changes: https://git.openjdk.org/jdk/pull/16565/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16565&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319726 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16565.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16565/head:pull/16565 PR: https://git.openjdk.org/jdk/pull/16565 From ayang at openjdk.org Wed Nov 8 16:43:58 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 8 Nov 2023 16:43:58 GMT Subject: RFR: 8319726: Parallel GC: Re-use object in object-iterator In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 16:29:57 GMT, Roman Kennke wrote: > As a minor follow-up/clean-up to [JDK-8319376](https://bugs.openjdk.org/browse/JDK-8319376), we can use the local obj field that we casted a few lines up, instead of casting from p again. Marked as reviewed by ayang (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16565#pullrequestreview-1720812242 From shade at openjdk.org Wed Nov 8 16:47:57 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 8 Nov 2023 16:47:57 GMT Subject: RFR: 8319725: G1: Subtracting virtual time from wall time after JDK-8319204 In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 15:52:50 GMT, Aleksey Shipilev wrote: > Noticed this when looking at the [JDK-8319204](https://bugs.openjdk.org/browse/JDK-8319204) changeset: we end subtracting `elapsedVTime()` vs `elapsedTime()`, which looks incorrect, as it does math on two unrelated quantities. Thank you for quick reviews! Trivial, or? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16564#issuecomment-1802271556 From ayang at openjdk.org Wed Nov 8 16:52:01 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 8 Nov 2023 16:52:01 GMT Subject: RFR: 8319725: G1: Subtracting virtual time from wall time after JDK-8319204 In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 15:52:50 GMT, Aleksey Shipilev wrote: > Noticed this when looking at the [JDK-8319204](https://bugs.openjdk.org/browse/JDK-8319204) changeset: we end subtracting `elapsedVTime()` vs `elapsedTime()`, which looks incorrect, as it does math on two unrelated quantities. I think it's trivial. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16564#issuecomment-1802278397 From cjplummer at openjdk.org Wed Nov 8 17:33:57 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 8 Nov 2023 17:33:57 GMT Subject: RFR: 8319703: Serial: Remove generationSpec In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 10:51:19 GMT, Albert Mingkun Yang wrote: > Simple cleanup of removing an unnecessary indirection. The real change is only 11 LOC, using the global variables directly for young/old gen. SA changes look good. ------------- Marked as reviewed by cjplummer (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16554#pullrequestreview-1720915415 From phh at openjdk.org Wed Nov 8 19:14:57 2023 From: phh at openjdk.org (Paul Hohensee) Date: Wed, 8 Nov 2023 19:14:57 GMT Subject: RFR: 8319726: Parallel GC: Re-use object in object-iterator In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 16:29:57 GMT, Roman Kennke wrote: > As a minor follow-up/clean-up to [JDK-8319376](https://bugs.openjdk.org/browse/JDK-8319376), we can use the local obj field that we casted a few lines up, instead of casting from p again. Marked as reviewed by phh (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16565#pullrequestreview-1721090596 From jjoo at openjdk.org Thu Nov 9 00:42:33 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Thu, 9 Nov 2023 00:42:33 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v39] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: Refactor changes to counters, successful build ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/ac780c5e..22ccb909 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=38 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=37-38 Stats: 149 lines in 16 files changed: 26 ins; 112 del; 11 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Thu Nov 9 05:27:40 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Thu, 9 Nov 2023 05:27:40 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v40] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: Add missing cpuTimeCounters files ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/22ccb909..41771db6 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=39 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=38-39 Stats: 220 lines in 2 files changed: 220 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From shade at openjdk.org Thu Nov 9 07:39:07 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Thu, 9 Nov 2023 07:39:07 GMT Subject: RFR: 8319725: G1: Subtracting virtual time from wall time after JDK-8319204 In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 15:52:50 GMT, Aleksey Shipilev wrote: > Noticed this when looking at the [JDK-8319204](https://bugs.openjdk.org/browse/JDK-8319204) changeset: we end subtracting `elapsedVTime()` vs `elapsedTime()`, which looks incorrect, as it does math on two unrelated quantities. Thanks! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16564#issuecomment-1803298266 From shade at openjdk.org Thu Nov 9 07:39:07 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Thu, 9 Nov 2023 07:39:07 GMT Subject: Integrated: 8319725: G1: Subtracting virtual time from wall time after JDK-8319204 In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 15:52:50 GMT, Aleksey Shipilev wrote: > Noticed this when looking at the [JDK-8319204](https://bugs.openjdk.org/browse/JDK-8319204) changeset: we end subtracting `elapsedVTime()` vs `elapsedTime()`, which looks incorrect, as it does math on two unrelated quantities. This pull request has now been integrated. Changeset: b0fc8082 Author: Aleksey Shipilev URL: https://git.openjdk.org/jdk/commit/b0fc8082f02ee5ed5e17474221aae4dd9285e78f Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod 8319725: G1: Subtracting virtual time from wall time after JDK-8319204 Reviewed-by: ayang, iwalulya ------------- PR: https://git.openjdk.org/jdk/pull/16564 From duke at openjdk.org Thu Nov 9 08:00:19 2023 From: duke at openjdk.org (duke) Date: Thu, 9 Nov 2023 08:00:19 GMT Subject: Withdrawn: 8310160: Make GC APIs for handling archive heap objects agnostic of GC policy In-Reply-To: References: Message-ID: On Fri, 16 Jun 2023 14:26:09 GMT, Ashutosh Mehra wrote: > This PR adds GC APIs to be implemented by all collectors for proper handling of archive heap space. Currently only G1 is updated to use these APIs which just involves renaming the existing G1 APIs. > In addition to that filemap.cpp is updated to replace calls to `G1CollectedHeap::heap()` with `Universe::heap()` to avoid G1 specific code as much as possible. > > At many places in filemap.cpp heap range is requested from GC. All collectors except ZGC have contiguous heap and set `CollectedHeap::_reserved` to the heap range, so it can be easily exposed to the CDS code. This is done in this patch through `CollectedHeap::reserved` API. But for ZGC the heap can be discontiguous which makes it tricky to expose the heap range. > Another point to note is that most of the usage for heap range is for logging purpose, but there is one place where it is used for setting the `mapping_offset` in `FileMapInfo::write_region()` based on the heap start. So purely based on the functional requirement, we only need the heap start address, not the range. > > To keep things simple and considering ZGC does not currently support archive heap, i refrained from tackling the issue of discontiguous heap range in this PR. This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/jdk/pull/14520 From tschatzl at openjdk.org Thu Nov 9 10:44:29 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 9 Nov 2023 10:44:29 GMT Subject: RFR: 8318706: Implement JEP 423: Region Pinning for G1 [v16] In-Reply-To: References: Message-ID: <1kvCiT9zD5ZqoLH_HFRtPsh8M78WaFwE6R5ODemjUMs=.2e5b019c-4a3b-4184-8171-49ff5a84c841@github.com> > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: Modify evacuation failure log message as suggested by sjohanss: Use "Evacuation Failure" with a cause description (either "Allocation" or "Pinned") ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/83eff9fe..6395696a Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=15 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=14-15 Stats: 16 lines in 3 files changed: 11 ins; 1 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From tschatzl at openjdk.org Thu Nov 9 10:44:30 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 9 Nov 2023 10:44:30 GMT Subject: RFR: 8318706: Implement JEP 423: Region Pinning for G1 [v9] In-Reply-To: <6_MNWrkq_LnvB4EqrBFNo6e6kVHLgffcRbkxhRPzTpg=.1a05d8af-b181-4e80-9922-6aea9a517564@github.com> References: <6_MNWrkq_LnvB4EqrBFNo6e6kVHLgffcRbkxhRPzTpg=.1a05d8af-b181-4e80-9922-6aea9a517564@github.com> Message-ID: On Wed, 8 Nov 2023 14:46:16 GMT, Stefan Johansson wrote: >> The example looks good to me. > > Have the final output looking something like this was agreed on during internal discussion: > GC(6) Pause Young (Normal) (Evacuation Failure: Pinned) 1M->1M(22M) 36.16ms > GC(6) Pause Young (Normal) (Evacuation Failure: Allocation) 1M->1M(22M) 36.16ms > GC(6) Pause Young (Normal) (Evacuation Failure: Allocation / Pinned) 1M->1M(22M) 36.16ms Done. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1387817157 From sjohanss at openjdk.org Thu Nov 9 10:48:08 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Thu, 9 Nov 2023 10:48:08 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v40] In-Reply-To: References: Message-ID: <4EVNbC2fI1AQGHMkRMfI6SDJrw98KEX0xuRJR1s361o=.d1ff2f1a-ffc9-4454-9755-e6e9e14d9110@github.com> On Thu, 9 Nov 2023 05:27:40 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Add missing cpuTimeCounters files A few more comments. src/hotspot/share/gc/g1/g1ConcurrentRefineThread.cpp line 82: > 80: _vtime_accum = 0.0; > 81: } > 82: maybe_update_threads_cpu_time(); I think the lines above here: if (os::supports_vtime()) { _vtime_accum = (os::elapsedVTime() - _vtime_start); } else { _vtime_accum = 0.0; } Should be extracted out into the new method and instead of calling it `maybe_update_threads_cpu_time()` just call `track_usage()` or `track_cpu_time()`. The the implementation in the primary thread can then call this and do the extra tracking. src/hotspot/share/gc/g1/g1ConcurrentRefineThread.cpp line 189: > 187: void G1PrimaryConcurrentRefineThread::maybe_update_threads_cpu_time() { > 188: if (UsePerfData && os::is_thread_cpu_time_supported()) { > 189: cr()->update_concurrent_refine_threads_cpu_time(); I think we should pull the tracking closure in here and that way leave the concurrent refine class untouched. Suggestion: // The primary thread is responsible for updating the CPU time for all workers. CPUTimeCounters* counters = G1CollectedHeap::heap()->cpu_time_counters(); ThreadTotalCPUTimeClosure tttc(counters, CPUTimeGroups::gc_conc_refine); cr()->threads_do(&tttc); This is more or less a copy from `G1ConcurrentRefineThreadControl::update_threads_cpu_time()` which if we go with this solution can be removed. The above needs some new includes though. I change the comment a because I could not fully understand it, the primary thread is the one always checking and starting more threads so it is not stopped first. Also not sure when a terminated thread could be read. Even the stopped threads are still present so should be fine. If I'm missing something feel free to add back the comment. src/hotspot/share/gc/g1/g1ServiceThread.cpp line 138: > 136: ThreadTotalCPUTimeClosure tttc(counters, CPUTimeGroups::gc_service); > 137: tttc.do_thread(task->_service_thread); > 138: } Please extract this to a function, similar to the other cases something like `track_cpu_time()`. ------------- Changes requested by sjohanss (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/15082#pullrequestreview-1722194318 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1387787912 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1387803508 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1387805665 From sjohanss at openjdk.org Thu Nov 9 11:03:11 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Thu, 9 Nov 2023 11:03:11 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v40] In-Reply-To: References: Message-ID: <9yVx0YyUxW40p2wV4ee2VB5m6WLOL9CxVcEqUE8W81Q=.a68ded01-b31f-4b09-91dc-04e893ff15eb@github.com> On Thu, 9 Nov 2023 05:27:40 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Add missing cpuTimeCounters files src/hotspot/share/gc/g1/g1ConcurrentMark.cpp line 2093: > 2091: tttc.do_thread(cm_thread()); > 2092: threads_do(&tttc); > 2093: } Any particular reason for having this in `G1ConcurrentMark` instead of keeping it where it is called in `G1ConcurrentMarkThread`. The implementation would be more or less the same apart from the two last lines: tttc.do_thread(this); cm()->threads_do(&tttc); ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1387839879 From sjohanss at openjdk.org Thu Nov 9 11:13:06 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Thu, 9 Nov 2023 11:13:06 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v40] In-Reply-To: References: Message-ID: On Thu, 9 Nov 2023 05:27:40 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Add missing cpuTimeCounters files One more thing I noticed is that the parallel worker time from the Remark pause is not updated after the remark pause. So I guess we should add a call to update that. It will still be account the next time we do a normal GC, but if we want the counter to be up to date as much as possible we should update it after the remark pause. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1803629858 From shade at openjdk.org Thu Nov 9 12:53:59 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Thu, 9 Nov 2023 12:53:59 GMT Subject: RFR: 8317755: G1: Periodic GC interval should test for the last whole heap GC [v2] In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 13:48:39 GMT, Thomas Schatzl wrote: > I think there is still a misconception with the current "periodic gcs" here: it is first and foremost thought to be applicable for catching the idle case with all its options (also consider cpu idle, do full or concurrent gc). Yes, "idle case". This PR adds another option that lets users to further fine-tune what "idle case" is. > This is the reason for the suggestion to phase out the current option `G1PeriodicGCInterval` to use a different name, IMO, the cost/benefit for renaming flags is too high to be worthwhile here. It would add to further confusion for users, as we change more and more flags across JDK releases. I would argue that proposed scheme fits the existing amendments to the periodic GC configurations: 1. `-XX:G1PeriodicGCInterval=X`: "do periodic GCs with this frequency, unless we had other GCs recently" 2. `-XX:G1PeriodicGCSystemLoadThreshold=X`: "...except when the CPU load is this high" 3. `-XX:+G1PeriodicGCCheckWholeHeap`: "...except when other GCs did not inspect the whole heap" > and add this functionality (regular whole heap analysis to clean out cruft) as a new option (something like WholeHeapAnalysisInterval, or something more aligned with other collectors) All right, so Shenandoah has `ShenandoahGuaranteedGCInterval` to control periodic GCs. Generational Shenandoah has `ShenandoahGuaranteedOldGCInterval` and `ShenandoahGuaranteedYoungGCInterval`. Would you like to have `G1GuaranteedOldGCInterval` and `G1GuaranteedYoungGCInterval` (which is nearly what `G1PeriodicGCInterval` currently is)? It looks to me as creating the parallel hierarchy of periodic GC control options. If we phase out `G1Periodic*` after this, it would be effectively a rename of `G1PeriodicGCInterval` -> `G1GuaranteedYoungGCInterval`, `G1PeriodicGCSystemLoadThreshold` -> `G1GuaranteedGCSystemLoadThreshold` right? Would it be better to just introduce `G1PeriodicOldGCInterval` to fit the existing option set? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16107#issuecomment-1803772775 From rkennke at openjdk.org Thu Nov 9 13:43:14 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Thu, 9 Nov 2023 13:43:14 GMT Subject: Integrated: 8319726: Parallel GC: Re-use object in object-iterator In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 16:29:57 GMT, Roman Kennke wrote: > As a minor follow-up/clean-up to [JDK-8319376](https://bugs.openjdk.org/browse/JDK-8319376), we can use the local obj field that we casted a few lines up, instead of casting from p again. This pull request has now been integrated. Changeset: f57b78c9 Author: Roman Kennke URL: https://git.openjdk.org/jdk/commit/f57b78c9c3b3526a2cd5c436d16de1522ac040e4 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod 8319726: Parallel GC: Re-use object in object-iterator Reviewed-by: ayang, phh ------------- PR: https://git.openjdk.org/jdk/pull/16565 From rkennke at openjdk.org Thu Nov 9 20:05:39 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Thu, 9 Nov 2023 20:05:39 GMT Subject: RFR: 8305896: Alternative full GC forwarding [v55] In-Reply-To: References: Message-ID: > Currently, the full-GC modes of Serial, Shenandoah and G1 GCs are forwarding objects by over-writing the object header with the new object location. Unfortunately, for compact object headers ([JDK-8294992](https://bugs.openjdk.org/browse/JDK-8294992)) this would not work, because the crucial class information is also stored in the header, and we could no longer iterate over objects until the headers would be restored. Also, the preserved-headers tables would grow quite large. > > I propose to use an alternative algorithm for full-GC (sliding-GC) forwarding that uses a special encoding so that the forwarding information fits into the lowest 32 bits of the header. > > It exploits the insight that, with sliding GCs, objects from one region will only ever be forwarded to one of two possible target regions. For this to work, we need to divide the heap into equal-sized regions. This is already the case for Shenandoah and G1, and can easily be overlaid for Serial GC, by assuming either the whole heap as a single region (if it fits) or by using SpaceAlignment-sized virtual regions. > > We also build and maintain a table that has N elements, where N is the number of regions. Each entry is two addresses, which are the start-address of the possible target regions for each source region. > > With this, forwarding information would be encoded like this: > - Bits 0 and 1: same as before, we put in '11' to indicate that the object is forwarded. > - Bit 2: Used for 'fallback'-forwarding (see below) > - Bit 3: Selects the target region 0 or 1. Look up the base address in the table (see above) > - Bits 4..31 The number of heap words from the target base address > > This works well for all sliding GCs in Serial, G1 and Shenandoah. The exception is in G1, there is a special mode called 'serial compaction' which acts as a last-last-ditch effort to squeeze more space out of the heap by re-forwarding the tails of the compaction chains. Unfortunately, this breaks the assumption of the sliding-forwarding-table. When that happens, we initialize a fallback table, which is a simple open hash-table, and set the Bit 2 in the forwarding to indicate that we shall look up the forwardee in the fallback-table. > > All the table accesses can be done unsynchronized because: > - Serial GC is single-threaded anyway > - In G1 and Shenandoah, GC worker threads divide up the work such that each worker does disjoint sets of regions. > - G1 serial compaction is single-threaded > > The change introduces a new (experimental) flag -... Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 122 commits: - Merge remote-tracking branch 'upstream/master' into JDK-8305896 - Fix build - Merge remote-tracking branch 'upstream/master' into JDK-8305896 - Merge remote-tracking branch 'upstream/master' into JDK-8305896 - Merge branch 'master' into JDK-8305896 - Various cleanups - Some @shipilev comments from downstream review - Further templatize Serial GC's adjust_pointers() - Merge branch 'master' into JDK-8305896 - Specialize full-GC loops to get UseAltGCForwarding flag check out of hot paths - ... and 112 more: https://git.openjdk.org/jdk/compare/38745eca...bdc8c823 ------------- Changes: https://git.openjdk.org/jdk/pull/13582/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=13582&range=54 Stats: 1203 lines in 39 files changed: 1079 ins; 10 del; 114 mod Patch: https://git.openjdk.org/jdk/pull/13582.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13582/head:pull/13582 PR: https://git.openjdk.org/jdk/pull/13582 From rkennke at openjdk.org Thu Nov 9 20:11:33 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Thu, 9 Nov 2023 20:11:33 GMT Subject: RFR: 8305898: Alternative self-forwarding mechanism [v22] In-Reply-To: References: Message-ID: > Currently, the Serial, Parallel and G1 GCs store a pointer to self into object headers, when compaction fails, to indicate that the object has been looked at, but failed compaction into to-space. This is problematic for compact object headers ([JDK-8294992](https://bugs.openjdk.org/browse/JDK-8294992)) because it would (temporarily) over-write the crucial class information, which we need for heap parsing. I would like to propose an alternative: use the bit #3 (previously biased-locking bit) to indicate that an object is 'self-forwarded'. That preserves the crucial class information in the upper bits of the header until the full header gets restored. Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 34 commits: - Merge branch 'JDK-8305896' into JDK-8305898 - Merge branch 'JDK-8305896' into JDK-8305898 - Merge branch 'JDK-8305896' into JDK-8305898 - Merge branch 'JDK-8305896' into JDK-8305898 - Merge branch 'JDK-8305896' into JDK-8305898 - Merge branch 'JDK-8305896' into JDK-8305898 - Update comment about mark-word layout - Merge branch 'JDK-8305896' into JDK-8305898 - Fix tests on 32bit builds - Merge branch 'JDK-8305896' into JDK-8305898 - ... and 24 more: https://git.openjdk.org/jdk/compare/bdc8c823...eee8ab57 ------------- Changes: https://git.openjdk.org/jdk/pull/13779/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=13779&range=21 Stats: 101 lines in 8 files changed: 85 ins; 2 del; 14 mod Patch: https://git.openjdk.org/jdk/pull/13779.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13779/head:pull/13779 PR: https://git.openjdk.org/jdk/pull/13779 From tschatzl at openjdk.org Fri Nov 10 09:09:00 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 10 Nov 2023 09:09:00 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v2] In-Reply-To: References: Message-ID: On Mon, 6 Nov 2023 12:01:32 GMT, Albert Mingkun Yang wrote: >> Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. >> >> It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. > > Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: > > - review > - Merge branch 'master' into s1-young-gc > - s1-young-gc src/hotspot/share/gc/serial/cardTableRS.cpp line 352: > 350: using Word = uintptr_t; > 351: > 352: auto is_word_aligned = [] (const void* const p) { Just my opinion, feel free to ignore: don't use `auto` if unavoidable, in this case use the specific type `bool`. Apart from that I am not sure the gain of defining a single-use-single-line expression is worth the effort. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389108116 From tschatzl at openjdk.org Fri Nov 10 10:19:10 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 10 Nov 2023 10:19:10 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v2] In-Reply-To: References: Message-ID: On Mon, 6 Nov 2023 12:01:32 GMT, Albert Mingkun Yang wrote: >> Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. >> >> It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. > > Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: > > - review > - Merge branch 'master' into s1-young-gc > - s1-young-gc Changes requested by tschatzl (Reviewer). src/hotspot/share/gc/serial/cardTableRS.cpp line 353: > 351: > 352: auto is_word_aligned = [] (const void* const p) { > 353: return is_aligned(p, sizeof(Word)); The include for `is_aligned` seems to be missing. src/hotspot/share/gc/serial/cardTableRS.cpp line 372: > 370: > 371: // Word comparison > 372: while (i_card + sizeof(Word) <= end_card) { Not sure if it is worth mentioning, but this should probably increment by `sizeof(Word) / sizeof(CardValue)`. src/hotspot/share/gc/serial/cardTableRS.cpp line 375: > 373: Word* i_word = reinterpret_cast(i_card); > 374: if (*i_word != clean_word) { > 375: // Found sth in this word; fall back to byte-comparison Suggestion: // Found a clean card in this word; fall back to per-CardValue comparison. src/hotspot/share/gc/serial/cardTableRS.cpp line 378: > 376: break; > 377: } > 378: i_card += sizeof(Word); `sizeof(Word) / sizeof(CardValue)` ? Or assert that `sizeof(CardValue) == 1` somewhere. src/hotspot/share/gc/serial/cardTableRS.cpp line 381: > 379: } > 380: > 381: // Byte comparison // Per-CardValue comparison. src/hotspot/share/gc/serial/cardTableRS.cpp line 402: > 400: } > 401: > 402: // a potential candidate Suggestion: // A potential candidate. src/hotspot/share/gc/serial/cardTableRS.cpp line 410: > 408: } > 409: > 410: // final obj in dirty-chunk crosses card-boundary Suggestion: // Last object in dirty chunk crosses card-boundary. I think "last" is the more appropriate term compared to "final" as to me "final" indicates that this is the overall very last object. src/hotspot/share/gc/serial/cardTableRS.cpp line 413: > 411: oop obj = cast_to_oop(obj_start_addr); > 412: if (obj->is_objArray()) { > 413: // precised-marked Suggestion: // ObjArrays are always precisely-marked so we are not allowed to jump to the end of the current object. src/hotspot/share/gc/serial/cardTableRS.cpp line 417: > 415: } > 416: > 417: // final card occupied by this obj Suggestion: // Final card occupied by this obj. We can use this to return and continue our search because this object is imprecisely marked and is always looked at for references as a whole anyway. src/hotspot/share/gc/serial/cardTableRS.cpp line 423: > 421: } > 422: > 423: // continue the search... Suggestion: // Continue the search at the end of the current object... src/hotspot/share/gc/serial/cardTableRS.cpp line 458: > 456: HeapWord* start_addr; > 457: HeapWord* end_addr; > 458: } cached_obj {nullptr, mr.start()}; Suggestion: } cached_obj { nullptr, mr.start() }; src/hotspot/share/gc/serial/cardTableRS.cpp line 477: > 475: > 476: CardValue* const clear_limit_card = is_card_aligned(mr.end()) ? end_card - 1 > 477: : end_card - 2; Please add a comment similar to the one in the `PSStripeShadowCardTable` constructor here, otherwise this code will be too hard to understand in six months. src/hotspot/share/gc/serial/cardTableRS.cpp line 483: > 481: > 482: if (dirty_l == end_card) { > 483: // no dirty cards Suggestion: // No dirty cards to iterate. src/hotspot/share/gc/serial/cardTableRS.cpp line 509: > 507: > 508: if (is_obj_array) { > 509: // precise-marked Suggestion: // ObjArrays are always precise-marked. src/hotspot/share/gc/serial/cardTableRS.hpp line 59: > 57: CardValue* const end_card); > 58: template > 59: CardTable::CardValue* find_first_clean_card(CardValue* const start_card, Already mentioned that this method should probably be renamed or at least documented more exactly what it does with all its optimizations. ------------- PR Review: https://git.openjdk.org/jdk/pull/16492#pullrequestreview-1724331448 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389115818 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389127249 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389125502 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389127985 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389126089 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389112450 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389112730 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389134383 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389113011 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389113241 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389136146 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389142118 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389169318 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389172947 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389179817 From tschatzl at openjdk.org Fri Nov 10 10:19:12 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 10 Nov 2023 10:19:12 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v2] In-Reply-To: References: Message-ID: On Fri, 10 Nov 2023 09:08:48 GMT, Thomas Schatzl wrote: >> Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: >> >> - review >> - Merge branch 'master' into s1-young-gc >> - s1-young-gc > > src/hotspot/share/gc/serial/cardTableRS.cpp line 417: > >> 415: } >> 416: >> 417: // final card occupied by this obj > > Suggestion: > > // Final card occupied by this obj. We can use this to return and continue our search because this object is imprecisely marked and is always looked at for references as a whole anyway. I think this method should be renamed, its name is misleading. Or it should have additional documentation. It does not find the first clean card, but the first clean card after the recently processed object. Also the `objArray` exception should be documented, see my suggestions, but it is maybe better to collect this information at the top. I.e. in this case Heap | ObjectVeryLong | Cards | x | | | x | | | x = dirty card the method does not find the first clean card (the second card), but the last card covering the object (assuming it's not aligned). This is not a problem because the caller cleans up the extra dirty marks, but still it is surprising. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389165921 From ayang at openjdk.org Fri Nov 10 10:57:25 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 10 Nov 2023 10:57:25 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v3] In-Reply-To: References: Message-ID: <1yviiZHm8F0574W-Gcl9TT6pav486kyYw3q-gyljlxY=.bc2dc42d-16be-4ce9-8c84-051af9e7bac8@github.com> > Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. > > It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. Albert Mingkun Yang has updated the pull request incrementally with four additional commits since the last revision: - Update src/hotspot/share/gc/serial/cardTableRS.cpp Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> - Update src/hotspot/share/gc/serial/cardTableRS.cpp Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> - Update src/hotspot/share/gc/serial/cardTableRS.cpp Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> - Update src/hotspot/share/gc/serial/cardTableRS.cpp Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16492/files - new: https://git.openjdk.org/jdk/pull/16492/files/9dd97239..063e67ff Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=01-02 Stats: 4 lines in 1 file changed: 0 ins; 0 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/16492.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16492/head:pull/16492 PR: https://git.openjdk.org/jdk/pull/16492 From ayang at openjdk.org Fri Nov 10 10:57:28 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 10 Nov 2023 10:57:28 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v2] In-Reply-To: References: Message-ID: <_7gpweNEi-vJO9JsdoSUBdiKOAcpIgYP5AxTSoTiG5E=.98c933b1-ffc0-47ba-9eaf-7c8c8670f2e9@github.com> On Fri, 10 Nov 2023 09:08:30 GMT, Thomas Schatzl wrote: >> Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: >> >> - review >> - Merge branch 'master' into s1-young-gc >> - s1-young-gc > > src/hotspot/share/gc/serial/cardTableRS.cpp line 410: > >> 408: } >> 409: >> 410: // final obj in dirty-chunk crosses card-boundary > > Suggestion: > > // Last object in dirty chunk crosses card-boundary. > > > I think "last" is the more appropriate term compared to "final" as to me "final" indicates that this is the overall very last object. "last" can mean "previous" so I tried to avoid it. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389246270 From sjohanss at openjdk.org Fri Nov 10 11:02:06 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Fri, 10 Nov 2023 11:02:06 GMT Subject: RFR: 8318706: Implement JEP 423: Region Pinning for G1 [v16] In-Reply-To: <1kvCiT9zD5ZqoLH_HFRtPsh8M78WaFwE6R5ODemjUMs=.2e5b019c-4a3b-4184-8171-49ff5a84c841@github.com> References: <1kvCiT9zD5ZqoLH_HFRtPsh8M78WaFwE6R5ODemjUMs=.2e5b019c-4a3b-4184-8171-49ff5a84c841@github.com> Message-ID: On Thu, 9 Nov 2023 10:44:29 GMT, Thomas Schatzl wrote: >> The JEP covers the idea very well, so I'm only covering some implementation details here: >> >> * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. >> >> * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: >> >> * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. >> >> * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). >> >> * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. >> >> * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) >> >> The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. >> >> I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. >> >> * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in a... > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > Modify evacuation failure log message as suggested by sjohanss: Use "Evacuation Failure" with a cause description (either "Allocation" or "Pinned") Looks good. Just a few small things. src/hotspot/share/gc/g1/g1ParScanThreadState.cpp line 466: > 464: if (region_attr.is_pinned() && klass->is_typeArray_klass()) { > 465: return handle_evacuation_failure_par(old, old_mark, word_sz, true /* cause_pinned */); > 466: } I think it would make sense to add a comment here that this is an optimization, that we know that only type array can be pinned so we only care if the region is pinned or not for type arrays. src/hotspot/share/gc/g1/g1YoungGCPostEvacuateTasks.cpp line 925: > 923: _evac_failure_regions->num_regions_alloc_failed(), > 924: G1GCPhaseTimes::RestoreEvacFailureRegionsAllocFailedNum); > 925: The counts used here are the totals, but they are added to thread work item, so total count will be wrong. Discussed this a bit with Thomas and we should probably count those things in this task instead and could that way make `G1EvacFailureRegions` store a bit less information. src/hotspot/share/gc/shared/collectedHeap.hpp line 169: > 167: static inline size_t filler_array_hdr_size(); > 168: public: > 169: static size_t filler_array_min_size(); Is this needed or some leftover from earlier iterations? ------------- Marked as reviewed by sjohanss (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16342#pullrequestreview-1722530941 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1389227404 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1389211990 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1387998596 From ayang at openjdk.org Fri Nov 10 11:03:10 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 10 Nov 2023 11:03:10 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v4] In-Reply-To: References: Message-ID: > Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. > > It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. Albert Mingkun Yang has updated the pull request incrementally with two additional commits since the last revision: - Update src/hotspot/share/gc/serial/cardTableRS.cpp Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> - Update src/hotspot/share/gc/serial/cardTableRS.cpp Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16492/files - new: https://git.openjdk.org/jdk/pull/16492/files/063e67ff..24cc1dec Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=02-03 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16492.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16492/head:pull/16492 PR: https://git.openjdk.org/jdk/pull/16492 From tschatzl at openjdk.org Fri Nov 10 11:25:06 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 10 Nov 2023 11:25:06 GMT Subject: RFR: 8318706: Implement JEP 423: Region Pinning for G1 [v16] In-Reply-To: References: <1kvCiT9zD5ZqoLH_HFRtPsh8M78WaFwE6R5ODemjUMs=.2e5b019c-4a3b-4184-8171-49ff5a84c841@github.com> Message-ID: On Thu, 9 Nov 2023 13:26:06 GMT, Stefan Johansson wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> Modify evacuation failure log message as suggested by sjohanss: Use "Evacuation Failure" with a cause description (either "Allocation" or "Pinned") > > src/hotspot/share/gc/shared/collectedHeap.hpp line 169: > >> 167: static inline size_t filler_array_hdr_size(); >> 168: public: >> 169: static size_t filler_array_min_size(); > > Is this needed or some leftover from earlier iterations? Removed. I started doing an implementation that properly formats smaller filler arrays into the dead areas taking potential `typeArray`s into account, but ultimately opted for just allowing oversized filler arrays. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1389274174 From ayang at openjdk.org Fri Nov 10 11:29:17 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 10 Nov 2023 11:29:17 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v5] In-Reply-To: References: Message-ID: > Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. > > It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: Update src/hotspot/share/gc/serial/cardTableRS.cpp Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16492/files - new: https://git.openjdk.org/jdk/pull/16492/files/24cc1dec..5ae74167 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=03-04 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16492.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16492/head:pull/16492 PR: https://git.openjdk.org/jdk/pull/16492 From ayang at openjdk.org Fri Nov 10 12:07:17 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 10 Nov 2023 12:07:17 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v2] In-Reply-To: References: Message-ID: On Fri, 10 Nov 2023 09:21:35 GMT, Thomas Schatzl wrote: >> Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: >> >> - review >> - Merge branch 'master' into s1-young-gc >> - s1-young-gc > > src/hotspot/share/gc/serial/cardTableRS.cpp line 378: > >> 376: break; >> 377: } >> 378: i_card += sizeof(Word); > > `sizeof(Word) / sizeof(CardValue)` ? Or assert that `sizeof(CardValue) == 1` somewhere. I don't think the impl assumes each card is one-byte. In Word-comparison, the increment is word-size regardless of the card-size. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389311758 From ayang at openjdk.org Fri Nov 10 12:07:19 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 10 Nov 2023 12:07:19 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v2] In-Reply-To: References: Message-ID: On Fri, 10 Nov 2023 09:46:57 GMT, Thomas Schatzl wrote: >> src/hotspot/share/gc/serial/cardTableRS.cpp line 417: >> >>> 415: } >>> 416: >>> 417: // final card occupied by this obj >> >> Suggestion: >> >> // Final card occupied by this obj. We can use this to return and continue our search because this object is imprecisely marked and is always looked at for references as a whole anyway. > > I think this method should be renamed, its name is misleading. Or it should have additional documentation. It does not find the first clean card, but the first clean card after the recently processed object. > > Also the `objArray` exception should be documented, see my suggestions, but it is maybe better to collect this information at the top. > > I.e. in this case > > Heap | ObjectVeryLong | > Cards | x | | | x | | | > > x = dirty card > > > the method does not find the first clean card (the second card), but the last card covering the object (assuming it's not aligned). > This is not a problem because the caller cleans up the extra dirty marks, but still it is surprising. Added doc. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1389311062 From ayang at openjdk.org Fri Nov 10 12:07:14 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 10 Nov 2023 12:07:14 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v6] In-Reply-To: References: Message-ID: > Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. > > It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: review ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16492/files - new: https://git.openjdk.org/jdk/pull/16492/files/5ae74167..de18a885 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=04-05 Stats: 19 lines in 1 file changed: 10 ins; 4 del; 5 mod Patch: https://git.openjdk.org/jdk/pull/16492.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16492/head:pull/16492 PR: https://git.openjdk.org/jdk/pull/16492 From tschatzl at openjdk.org Fri Nov 10 12:14:19 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 10 Nov 2023 12:14:19 GMT Subject: RFR: 8318706: Implement JEP 423: Region Pinning for G1 [v17] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: stefanj review - fix counting of pinned/allocation failed regions in log - some cleanup of evacuation failure code, removing unnecessary members - comments ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/6395696a..d6df3b06 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=16 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=15-16 Stats: 54 lines in 6 files changed: 16 ins; 28 del; 10 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From ayang at openjdk.org Fri Nov 10 12:53:16 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 10 Nov 2023 12:53:16 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v7] In-Reply-To: References: Message-ID: > Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. > > It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 13 additional commits since the last revision: - review - Merge branch 'master' into s1-young-gc - review - Update src/hotspot/share/gc/serial/cardTableRS.cpp Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> - Update src/hotspot/share/gc/serial/cardTableRS.cpp Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> - Update src/hotspot/share/gc/serial/cardTableRS.cpp Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> - Update src/hotspot/share/gc/serial/cardTableRS.cpp Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> - Update src/hotspot/share/gc/serial/cardTableRS.cpp Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> - Update src/hotspot/share/gc/serial/cardTableRS.cpp Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> - Update src/hotspot/share/gc/serial/cardTableRS.cpp Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> - ... and 3 more: https://git.openjdk.org/jdk/compare/9a3ec1ff...41439f4a ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16492/files - new: https://git.openjdk.org/jdk/pull/16492/files/de18a885..41439f4a Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=05-06 Stats: 16589 lines in 250 files changed: 4894 ins; 8175 del; 3520 mod Patch: https://git.openjdk.org/jdk/pull/16492.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16492/head:pull/16492 PR: https://git.openjdk.org/jdk/pull/16492 From ayang at openjdk.org Fri Nov 10 13:33:12 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 10 Nov 2023 13:33:12 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v8] In-Reply-To: References: Message-ID: > Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. > > It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: next-obj-card ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16492/files - new: https://git.openjdk.org/jdk/pull/16492/files/41439f4a..e8b8e0ba Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=07 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=06-07 Stats: 6 lines in 1 file changed: 0 ins; 0 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/16492.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16492/head:pull/16492 PR: https://git.openjdk.org/jdk/pull/16492 From jjoo at openjdk.org Sat Nov 11 00:23:28 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Sat, 11 Nov 2023 00:23:28 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v41] In-Reply-To: References: Message-ID: <3kqtWcZA2lY3fPjUgyo5aO_-4SicTOPzF6AnKGyRCBA=.53d5b175-7e0f-4f6b-91a7-e4cfea62cb7a@github.com> > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: - Refactor ConcurrentRefine logic - Make CPUTimeCounters a singleton class ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/41771db6..533af850 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=40 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=39-40 Stats: 151 lines in 17 files changed: 62 ins; 62 del; 27 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Sat Nov 11 00:23:29 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Sat, 11 Nov 2023 00:23:29 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v40] In-Reply-To: <4EVNbC2fI1AQGHMkRMfI6SDJrw98KEX0xuRJR1s361o=.d1ff2f1a-ffc9-4454-9755-e6e9e14d9110@github.com> References: <4EVNbC2fI1AQGHMkRMfI6SDJrw98KEX0xuRJR1s361o=.d1ff2f1a-ffc9-4454-9755-e6e9e14d9110@github.com> Message-ID: <1NfEWhHC6ZHOtFiDpwcpZwuxDzbf-sfY7IenNn3nv9M=.d189f48e-8772-4ec2-bb17-bd912a28c5ab@github.com> On Thu, 9 Nov 2023 10:29:29 GMT, Stefan Johansson wrote: >> Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: >> >> Add missing cpuTimeCounters files > > src/hotspot/share/gc/g1/g1ConcurrentRefineThread.cpp line 189: > >> 187: void G1PrimaryConcurrentRefineThread::maybe_update_threads_cpu_time() { >> 188: if (UsePerfData && os::is_thread_cpu_time_supported()) { >> 189: cr()->update_concurrent_refine_threads_cpu_time(); > > I think we should pull the tracking closure in here and that way leave the concurrent refine class untouched. > > Suggestion: > > // The primary thread is responsible for updating the CPU time for all workers. > CPUTimeCounters* counters = G1CollectedHeap::heap()->cpu_time_counters(); > ThreadTotalCPUTimeClosure tttc(counters, CPUTimeGroups::gc_conc_refine); > cr()->threads_do(&tttc); > > > This is more or less a copy from `G1ConcurrentRefineThreadControl::update_threads_cpu_time()` which if we go with this solution can be removed. The above needs some new includes though. > > I change the comment a because I could not fully understand it, the primary thread is the one always checking and starting more threads so it is not stopped first. Also not sure when a terminated thread could be read. Even the stopped threads are still present so should be fine. If I'm missing something feel free to add back the comment. Thank you for the review! Do you think you could take a look at the newest update and see if that aligns with what you were thinking? I wanted to make sure I understood your comments correctly. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1390054625 From jjoo at openjdk.org Sat Nov 11 00:30:07 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Sat, 11 Nov 2023 00:30:07 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v41] In-Reply-To: <3kqtWcZA2lY3fPjUgyo5aO_-4SicTOPzF6AnKGyRCBA=.53d5b175-7e0f-4f6b-91a7-e4cfea62cb7a@github.com> References: <3kqtWcZA2lY3fPjUgyo5aO_-4SicTOPzF6AnKGyRCBA=.53d5b175-7e0f-4f6b-91a7-e4cfea62cb7a@github.com> Message-ID: On Sat, 11 Nov 2023 00:23:28 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: > > - Refactor ConcurrentRefine logic > - Make CPUTimeCounters a singleton class Will address the Remark pause update in my next commit! ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1806593319 From duke at openjdk.org Sat Nov 11 10:25:15 2023 From: duke at openjdk.org (Lei Zaakjyu) Date: Sat, 11 Nov 2023 10:25:15 GMT Subject: RFR: JDK-8234502 : Merge GenCollectedHeap and SerialHeap [v2] In-Reply-To: References: Message-ID: > JDK-8234502 : Merge GenCollectedHeap and SerialHeap Lei Zaakjyu has updated the pull request incrementally with one additional commit since the last revision: include 'serialVMOperations.hpp' ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16623/files - new: https://git.openjdk.org/jdk/pull/16623/files/08f1f0a8..4e74cc25 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16623&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16623&range=00-01 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16623.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16623/head:pull/16623 PR: https://git.openjdk.org/jdk/pull/16623 From duke at openjdk.org Sat Nov 11 11:49:33 2023 From: duke at openjdk.org (Lei Zaakjyu) Date: Sat, 11 Nov 2023 11:49:33 GMT Subject: RFR: JDK-8234502 : Merge GenCollectedHeap and SerialHeap [v3] In-Reply-To: References: Message-ID: > JDK-8234502 : Merge GenCollectedHeap and SerialHeap Lei Zaakjyu has updated the pull request incrementally with one additional commit since the last revision: Fix 'young_gen' function in 'genCollectedHeap.cpp' ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16623/files - new: https://git.openjdk.org/jdk/pull/16623/files/4e74cc25..883dd7b1 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16623&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16623&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16623.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16623/head:pull/16623 PR: https://git.openjdk.org/jdk/pull/16623 From tschatzl at openjdk.org Mon Nov 13 09:44:05 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Mon, 13 Nov 2023 09:44:05 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v8] In-Reply-To: References: Message-ID: On Fri, 10 Nov 2023 13:33:12 GMT, Albert Mingkun Yang wrote: >> Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. >> >> It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. > > Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: > > next-obj-card lgtm ------------- Marked as reviewed by tschatzl (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16492#pullrequestreview-1726892212 From tschatzl at openjdk.org Mon Nov 13 09:50:57 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Mon, 13 Nov 2023 09:50:57 GMT Subject: RFR: 8319713: Parallel: Remove PSAdaptiveSizePolicy::should_full_GC In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 13:44:56 GMT, Albert Mingkun Yang wrote: > Simple removing too-conservative full-gc heuristic. While I generally agree that this looks like the preemptive gcs for g1, do you have some numbers for these being a problem for Parallel gc as well? The CR explicitly mentions that Parallel is too conservative with the estimation, do you have an idea how far off Parallel GC is here as well? Promotion failures should not be that expensive - it's parallelized and there is nothing to do but restoring the preserved marks (compared to g1) but still I have no idea if/how expensive they are compared to this preemptive full gcs. ------------- PR Review: https://git.openjdk.org/jdk/pull/16560#pullrequestreview-1726906059 From tschatzl at openjdk.org Mon Nov 13 10:32:00 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Mon, 13 Nov 2023 10:32:00 GMT Subject: RFR: 8317755: G1: Periodic GC interval should test for the last whole heap GC [v2] In-Reply-To: References: Message-ID: On Thu, 9 Nov 2023 12:50:58 GMT, Aleksey Shipilev wrote: > > I think there is still a misconception with the current "periodic gcs" here: it is first and foremost thought to be applicable for catching the idle case with all its options (also consider cpu idle, do full or concurrent gc). > > Yes, "idle case". This PR adds another option that lets users to further fine-tune what "idle case" is. > > > This is the reason for the suggestion to phase out the current option `G1PeriodicGCInterval` to use a different name, > > IMO, the cost/benefit for renaming flags is too high to be worthwhile here. It would add to further confusion for users, as we change more and more flags across JDK releases. I would argue that proposed scheme fits the existing amendments to the periodic GC configurations: > > 1. `-XX:G1PeriodicGCInterval=X`: "do periodic GCs with this frequency, unless we had other GCs recently" > > 2. `-XX:G1PeriodicGCSystemLoadThreshold=X`: "...except when the CPU load is this high" > > 3. `-XX:+G1PeriodicGCCheckWholeHeap`: "...except when other GCs did not inspect the whole heap" Doing periodic whole heap inspections to free resources should not be tied to some notion of "idle" (ideally gc would schedule them at that time, but that is another matter). E.g. the problem mentioned in https://mail.openjdk.org/pipermail/hotspot-gc-dev/2020-March/028857.html is independent of idle cleanup. Same issue applies to finalizers/symbol table/etc (also written up in https://bugs.openjdk.org/browse/JDK-8213198). As far as I understand your proposal, in combination with `-XX:G1PeriodicGCSystemLoadThreshold` g1 may then never trigger whole heap collections (and the expected cleanups). An application may be non-idle for days without the need for concurrent mark, accumulating cruft in these data structures that can only be cleaned out with a concurrent mark. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16107#issuecomment-1807888303 From tschatzl at openjdk.org Mon Nov 13 10:38:01 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Mon, 13 Nov 2023 10:38:01 GMT Subject: RFR: 8317755: G1: Periodic GC interval should test for the last whole heap GC [v2] In-Reply-To: References: Message-ID: On Wed, 18 Oct 2023 14:27:26 GMT, Aleksey Shipilev wrote: >> See the description in the bug. Fortunately, we already track the last whole-heap GC. The new regression test verifies the behavior. >> >> Additional testing: >> - [x] Linux x86_64 fastdebug `tier1 tier2 tier3` > > Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: > > - Merge branch 'master' into JDK-8317755-g1-periodic-whole > - Optional flag > - Merge branch 'master' into JDK-8317755-g1-periodic-whole > - Keep the cast > - Fix Now one can argue that it is up to the user of `G1PeriodicGCSystemLoadThreshold` to make sure there are enough idle periods... let me discuss this internally too; but my impression about the original JEP was that the idle case was the significant one where one would always use `G1PeriodicGCSystemLoadThreshold`. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16107#issuecomment-1807894609 From tschatzl at openjdk.org Mon Nov 13 11:23:02 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Mon, 13 Nov 2023 11:23:02 GMT Subject: RFR: 8317755: G1: Periodic GC interval should test for the last whole heap GC [v2] In-Reply-To: References: Message-ID: On Wed, 18 Oct 2023 14:27:26 GMT, Aleksey Shipilev wrote: >> See the description in the bug. Fortunately, we already track the last whole-heap GC. The new regression test verifies the behavior. >> >> Additional testing: >> - [x] Linux x86_64 fastdebug `tier1 tier2 tier3` > > Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: > > - Merge branch 'master' into JDK-8317755-g1-periodic-whole > - Optional flag > - Merge branch 'master' into JDK-8317755-g1-periodic-whole > - Keep the cast > - Fix There is also the option to trigger full gcs on "idle"; would be fairly bad if they were triggered regularly with the `G1PeriodicGCCheckWholeHeap` option, and completely overkill for the https://bugs.openjdk.org/browse/JDK-8213198 case. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16107#issuecomment-1807967924 From ayang at openjdk.org Mon Nov 13 12:37:59 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Mon, 13 Nov 2023 12:37:59 GMT Subject: RFR: 8319713: Parallel: Remove PSAdaptiveSizePolicy::should_full_GC In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 13:44:56 GMT, Albert Mingkun Yang wrote: > Simple removing too-conservative full-gc heuristic. The check in `PSScavenge::should_attempt_scavenge` is constrained by used-in-young to offer a less-conservative heuristic, i.e. #promoted-bytes can't be larger than used-in-young. In terms of avoiding promotion-failure during young-gc, both heuristics are equally effective. Therefore, the end result is that a full-gc starts unnecessarily. (Also attached a small bm to illustrate the problem.) ------------- PR Comment: https://git.openjdk.org/jdk/pull/16560#issuecomment-1808084970 From ayang at openjdk.org Mon Nov 13 12:44:58 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Mon, 13 Nov 2023 12:44:58 GMT Subject: RFR: JDK-8234502 : Merge GenCollectedHeap and SerialHeap [v3] In-Reply-To: References: Message-ID: On Sat, 11 Nov 2023 11:49:33 GMT, Lei Zaakjyu wrote: >> JDK-8234502 : Merge GenCollectedHeap and SerialHeap > > Lei Zaakjyu has updated the pull request incrementally with one additional commit since the last revision: > > Fix 'young_gen' function in 'genCollectedHeap.cpp' Maybe there is some misinterpretation about the ticket description, but I'd expect `class GenCollectedHeap` to be removed completely with this ticket. Could you enable GHA (https://wiki.openjdk.org/display/SKARA/Testing) to catch potential issues for all platforms? It should show sth like "19 successful checks", e.g. in https://github.com/openjdk/jdk/pull/16560. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16623#issuecomment-1808094234 From tschatzl at openjdk.org Tue Nov 14 07:45:48 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 14 Nov 2023 07:45:48 GMT Subject: RFR: 8319955: Improve dependencies removal during class unloading Message-ID: Hi all, please review this performance improvement to dependency removal during class unloading: instead of moving the dependencies item by item via a `cmpxchg`, batch this operation. This makes that part of class unloading ~7x faster in my measurements (class unloading stress test, loading/unloading 6k classes). Testing: tier1-7 Thanks, Thomas ------------- Commit messages: - fix compilation - Remove timing code - Cleanup - initial version, with logging code Changes: https://git.openjdk.org/jdk/pull/16639/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16639&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319955 Stats: 42 lines in 2 files changed: 38 ins; 2 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16639.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16639/head:pull/16639 PR: https://git.openjdk.org/jdk/pull/16639 From dholmes at openjdk.org Tue Nov 14 08:02:27 2023 From: dholmes at openjdk.org (David Holmes) Date: Tue, 14 Nov 2023 08:02:27 GMT Subject: RFR: 8319955: Improve dependencies removal during class unloading In-Reply-To: References: Message-ID: On Mon, 13 Nov 2023 18:52:47 GMT, Thomas Schatzl wrote: > Hi all, > > please review this performance improvement to dependency removal during class unloading: instead of moving the dependencies item by item via a `cmpxchg`, batch this operation. > > This makes that part of class unloading ~7x faster in my measurements (class unloading stress test, loading/unloading 6k classes). > > Testing: tier1-7 > > Thanks, > Thomas So this batches the operation by moving the batch to the purge list, but it doesn't actually release anything. Who/what will process the purge list and when? The implications of this change in terms of concurrent activities is far from clear. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16639#issuecomment-1809705815 From tschatzl at openjdk.org Tue Nov 14 10:27:26 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 14 Nov 2023 10:27:26 GMT Subject: RFR: 8319955: Improve dependencies removal during class unloading In-Reply-To: References: Message-ID: On Tue, 14 Nov 2023 07:59:57 GMT, David Holmes wrote: > So this batches the operation by moving the batch to the purge list, but it doesn't actually release anything. Who/what will process the purge list and when? The implications of this change in terms of concurrent activities is far from clear. There is no observable difference (apart from being faster) here - the move of the dependencies from the (single) Klass to the purge list is and has been under the `ClassLoaderData_lock` anyway, locking out "everything" already. The current basic structure is like: (indentation means call level) lock ClassLoaderDataGraph_lock CLGD::do_unloading() CLD::unload foreach (Klass in CLD) { remove_all_dependents() } unlock ClassLoaderDataGraph_lock The only thing that change is the performance of `remove_all_dependents`, from void DependencyContext::remove_all_dependents() { for (all dependents x) { loadload barrier cmpxchg x into purge list } } to void DependencyContext::remove_all_dependents() { find last pointer of dependents list of the dead(!) Klass, i.e. verified unreachable by the mutator cmpxchg first/last into purge list } Literally no change in locking or the scope of the `ClassLoaderDataGraph_lock`. Everything else is already locked out (for a really long time in that stress test). The only change is the amount of atomic operations. Actually, the time to hold the `CLDG_lock` is *much shorter* now as indicated by the numbers in the CR. The purge list is processed as before at some point afterwards. No change here. Unfortunately the code still needs to traverse the linked list (and there are really many linked lists here and in nmethod management :( ) which are not very amenable to performance improvements (e.g. parallelization). It is a non-goal for this CR to change data structures. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16639#issuecomment-1809932543 From tschatzl at openjdk.org Tue Nov 14 10:32:28 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 14 Nov 2023 10:32:28 GMT Subject: RFR: 8319955: Improve dependencies removal during class unloading In-Reply-To: References: Message-ID: On Mon, 13 Nov 2023 18:52:47 GMT, Thomas Schatzl wrote: > Hi all, > > please review this performance improvement to dependency removal during class unloading: instead of moving the dependencies item by item via a `cmpxchg`, batch this operation. > > This makes that part of class unloading ~7x faster in my measurements (class unloading stress test, loading/unloading 6k classes). > > Testing: tier1-7 > > Thanks, > Thomas Sorry forgot to answer this: > Who/what will process the purge list and when? There is a call to `ClassLoaderDataGraph::purge` which does that, executed later at the garbage collector's convenience. This change does not change the amount of elements (apart from the order of the elements) too, so no difference here. (During class unloading `DependencyContext::delete_on_release()` is and has always been false, and the existing code always moves entries to the purge list - there is an appropriate assert in the new code). ------------- PR Comment: https://git.openjdk.org/jdk/pull/16639#issuecomment-1809940071 From tschatzl at openjdk.org Tue Nov 14 10:40:28 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 14 Nov 2023 10:40:28 GMT Subject: RFR: 8319955: Improve dependencies removal during class unloading In-Reply-To: References: Message-ID: On Mon, 13 Nov 2023 18:52:47 GMT, Thomas Schatzl wrote: > Hi all, > > please review this performance improvement to dependency removal during class unloading: instead of moving the dependencies item by item via a `cmpxchg`, batch this operation. > > This makes that part of class unloading ~7x faster in my measurements (class unloading stress test, loading/unloading 6k classes). > > Testing: tier1-7 > > Thanks, > Thomas Concurrent additions to the purge list (during (concurrent) class unloading) should be fine due to the `cmpxchg` used, i.e. callers of `DependencyContext::release()` while the first part of class unloading (i.e. CLD unlinking) is in progress (i.e. `!delete_on_release()`). This is as before afaict. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16639#issuecomment-1809952061 From dholmes at openjdk.org Tue Nov 14 12:01:31 2023 From: dholmes at openjdk.org (David Holmes) Date: Tue, 14 Nov 2023 12:01:31 GMT Subject: RFR: 8319955: Improve dependencies removal during class unloading In-Reply-To: References: Message-ID: On Mon, 13 Nov 2023 18:52:47 GMT, Thomas Schatzl wrote: > Hi all, > > please review this performance improvement to dependency removal during class unloading: instead of moving the dependencies item by item via a `cmpxchg`, batch this operation. > > This makes that part of class unloading ~7x faster in my measurements (class unloading stress test, loading/unloading 6k classes). > > Testing: tier1-7 > > Thanks, > Thomas Changes look good. Thanks ------------- Marked as reviewed by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16639#pullrequestreview-1729575808 From dholmes at openjdk.org Tue Nov 14 12:01:33 2023 From: dholmes at openjdk.org (David Holmes) Date: Tue, 14 Nov 2023 12:01:33 GMT Subject: RFR: 8319955: Improve dependencies removal during class unloading In-Reply-To: References: Message-ID: On Tue, 14 Nov 2023 10:29:46 GMT, Thomas Schatzl wrote: > (During class unloading DependencyContext::delete_on_release() is and has always been false, and the existing code always moves entries to the purge list - there is an appropriate assert in the new code). Thanks I can see that now. So all this does is move a chunk of elements to the purge-list instead of doing it one at a time. Makes me wonder why this wasn't noticed sooner. :) ------------- PR Comment: https://git.openjdk.org/jdk/pull/16639#issuecomment-1810069754 From eosterlund at openjdk.org Tue Nov 14 13:28:30 2023 From: eosterlund at openjdk.org (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Tue, 14 Nov 2023 13:28:30 GMT Subject: RFR: 8319955: Improve dependencies removal during class unloading In-Reply-To: References: Message-ID: On Mon, 13 Nov 2023 18:52:47 GMT, Thomas Schatzl wrote: > Hi all, > > please review this performance improvement to dependency removal during class unloading: instead of moving the dependencies item by item via a `cmpxchg`, batch this operation. > > This makes that part of class unloading ~7x faster in my measurements (class unloading stress test, loading/unloading 6k classes). > > Testing: tier1-7 > > Thanks, > Thomas Looks good. ------------- Marked as reviewed by eosterlund (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16639#pullrequestreview-1729726128 From duke at openjdk.org Tue Nov 14 15:26:46 2023 From: duke at openjdk.org (Lei Zaakjyu) Date: Tue, 14 Nov 2023 15:26:46 GMT Subject: RFR: JDK-8234502 : Merge GenCollectedHeap and SerialHeap [v4] In-Reply-To: References: Message-ID: > JDK-8234502 : Merge GenCollectedHeap and SerialHeap Lei Zaakjyu has updated the pull request incrementally with one additional commit since the last revision: Completely removed 'GenCollectedHeap' ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16623/files - new: https://git.openjdk.org/jdk/pull/16623/files/883dd7b1..8f164430 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16623&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16623&range=02-03 Stats: 74 lines in 3 files changed: 4 ins; 69 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16623.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16623/head:pull/16623 PR: https://git.openjdk.org/jdk/pull/16623 From duke at openjdk.org Tue Nov 14 15:53:11 2023 From: duke at openjdk.org (Lei Zaakjyu) Date: Tue, 14 Nov 2023 15:53:11 GMT Subject: RFR: JDK-8234502 : Merge GenCollectedHeap and SerialHeap [v5] In-Reply-To: References: Message-ID: <_YhRVRgg-b-GnysoG7zHG1lVODxbofxAi4VipF-6jO0=.47e30bef-31e8-4bbe-9c4b-a3cf464d7a58@github.com> > JDK-8234502 : Merge GenCollectedHeap and SerialHeap Lei Zaakjyu has updated the pull request incrementally with one additional commit since the last revision: add some headers ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16623/files - new: https://git.openjdk.org/jdk/pull/16623/files/8f164430..bf022a87 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16623&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16623&range=03-04 Stats: 2 lines in 2 files changed: 2 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16623.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16623/head:pull/16623 PR: https://git.openjdk.org/jdk/pull/16623 From iwalulya at openjdk.org Tue Nov 14 19:57:28 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Tue, 14 Nov 2023 19:57:28 GMT Subject: RFR: 8319439: Move BufferNode from PtrQueue files to new files In-Reply-To: References: Message-ID: On Sat, 4 Nov 2023 11:25:05 GMT, Kim Barrett wrote: > Please review this simple (trivial?) change that moves the BufferNode class > from share/gc/shared/ptrQueue.[ch]pp to new files > share/gc/shared/bufferNode.[ch]pp. The code (as opposed to the surrounding > boiler plate and #include's) was moved unchanged. > > This led to adding #include's of the new header file to a bunch of other files > (include what you use), in some cases eliminating the need to #include > ptrQueue.hpp. Also tidied up the #include's for ptrQueue.[ch]pp to take > advantage of the split. > > Also, the gtest gc/shared/test_ptrQueueBufferAllocator.cpp is renamed to > gc/shared/test_bufferNodeAllocator.cpp, with some adjustments to includes and > test naming. > > Testing: > mach5 tier1 Marked as reviewed by iwalulya (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16506#pullrequestreview-1730648862 From jjoo at openjdk.org Tue Nov 14 21:33:37 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Tue, 14 Nov 2023 21:33:37 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v42] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: Update parallel workers time after Remark ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/533af850..189d1852 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=41 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=40-41 Stats: 10 lines in 4 files changed: 5 ins; 2 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Tue Nov 14 21:33:38 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Tue, 14 Nov 2023 21:33:38 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v41] In-Reply-To: <3kqtWcZA2lY3fPjUgyo5aO_-4SicTOPzF6AnKGyRCBA=.53d5b175-7e0f-4f6b-91a7-e4cfea62cb7a@github.com> References: <3kqtWcZA2lY3fPjUgyo5aO_-4SicTOPzF6AnKGyRCBA=.53d5b175-7e0f-4f6b-91a7-e4cfea62cb7a@github.com> Message-ID: On Sat, 11 Nov 2023 00:23:28 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: > > - Refactor ConcurrentRefine logic > - Make CPUTimeCounters a singleton class I believe all comments have been addressed and this PR is once again RFR! ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1811353654 From kbarrett at openjdk.org Wed Nov 15 01:04:17 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Wed, 15 Nov 2023 01:04:17 GMT Subject: RFR: 8319439: Move BufferNode from PtrQueue files to new files [v2] In-Reply-To: References: Message-ID: <5K288rm4MtykYCyHrEPdanQsctdzzPeFeTA8EvHUAUI=.c8bfc38a-f926-408f-a158-084b07864170@github.com> > Please review this simple (trivial?) change that moves the BufferNode class > from share/gc/shared/ptrQueue.[ch]pp to new files > share/gc/shared/bufferNode.[ch]pp. The code (as opposed to the surrounding > boiler plate and #include's) was moved unchanged. > > This led to adding #include's of the new header file to a bunch of other files > (include what you use), in some cases eliminating the need to #include > ptrQueue.hpp. Also tidied up the #include's for ptrQueue.[ch]pp to take > advantage of the split. > > Also, the gtest gc/shared/test_ptrQueueBufferAllocator.cpp is renamed to > gc/shared/test_bufferNodeAllocator.cpp, with some adjustments to includes and > test naming. > > Testing: > mach5 tier1 Kim Barrett has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits: - Merge branch 'master' into buffer-node - fixup gtest after rename - move test - move BufferNode to new files ------------- Changes: https://git.openjdk.org/jdk/pull/16506/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16506&range=01 Stats: 383 lines in 17 files changed: 223 ins; 152 del; 8 mod Patch: https://git.openjdk.org/jdk/pull/16506.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16506/head:pull/16506 PR: https://git.openjdk.org/jdk/pull/16506 From kbarrett at openjdk.org Wed Nov 15 01:11:49 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Wed, 15 Nov 2023 01:11:49 GMT Subject: RFR: 8319439: Move BufferNode from PtrQueue files to new files [v2] In-Reply-To: References: Message-ID: On Tue, 7 Nov 2023 11:19:03 GMT, Thomas Schatzl wrote: >> Kim Barrett has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits: >> >> - Merge branch 'master' into buffer-node >> - fixup gtest after rename >> - move test >> - move BufferNode to new files > > lgtm Thanks @tschatzl and @walulyai for reviews. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16506#issuecomment-1811638624 From kbarrett at openjdk.org Wed Nov 15 01:11:52 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Wed, 15 Nov 2023 01:11:52 GMT Subject: Integrated: 8319439: Move BufferNode from PtrQueue files to new files In-Reply-To: References: Message-ID: <5I5wDXCF5lDItoDQ594oZX8VrEEk1WDX-LvvkhJQphk=.d55c81a9-841b-49b7-8b05-51b8f18945f5@github.com> On Sat, 4 Nov 2023 11:25:05 GMT, Kim Barrett wrote: > Please review this simple (trivial?) change that moves the BufferNode class > from share/gc/shared/ptrQueue.[ch]pp to new files > share/gc/shared/bufferNode.[ch]pp. The code (as opposed to the surrounding > boiler plate and #include's) was moved unchanged. > > This led to adding #include's of the new header file to a bunch of other files > (include what you use), in some cases eliminating the need to #include > ptrQueue.hpp. Also tidied up the #include's for ptrQueue.[ch]pp to take > advantage of the split. > > Also, the gtest gc/shared/test_ptrQueueBufferAllocator.cpp is renamed to > gc/shared/test_bufferNodeAllocator.cpp, with some adjustments to includes and > test naming. > > Testing: > mach5 tier1 This pull request has now been integrated. Changeset: 1e76ba0c Author: Kim Barrett URL: https://git.openjdk.org/jdk/commit/1e76ba0cd0de38da6fc2b8147627496ee2f2837b Stats: 383 lines in 17 files changed: 223 ins; 152 del; 8 mod 8319439: Move BufferNode from PtrQueue files to new files Reviewed-by: tschatzl, iwalulya ------------- PR: https://git.openjdk.org/jdk/pull/16506 From tschatzl at openjdk.org Wed Nov 15 08:36:42 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Wed, 15 Nov 2023 08:36:42 GMT Subject: Integrated: 8319955: Improve dependencies removal during class unloading In-Reply-To: References: Message-ID: On Mon, 13 Nov 2023 18:52:47 GMT, Thomas Schatzl wrote: > Hi all, > > please review this performance improvement to dependency removal during class unloading: instead of moving the dependencies item by item via a `cmpxchg`, batch this operation. > > This makes that part of class unloading ~7x faster in my measurements (class unloading stress test, loading/unloading 6k classes). > > Testing: tier1-7 > > Thanks, > Thomas This pull request has now been integrated. Changeset: fbe19378 Author: Thomas Schatzl URL: https://git.openjdk.org/jdk/commit/fbe19378c38decaff221f6570d1b78bb801a5e5d Stats: 42 lines in 2 files changed: 38 ins; 2 del; 2 mod 8319955: Improve dependencies removal during class unloading Reviewed-by: dholmes, eosterlund ------------- PR: https://git.openjdk.org/jdk/pull/16639 From tschatzl at openjdk.org Wed Nov 15 08:36:41 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Wed, 15 Nov 2023 08:36:41 GMT Subject: RFR: 8319955: Improve dependencies removal during class unloading In-Reply-To: References: Message-ID: On Tue, 14 Nov 2023 11:58:10 GMT, David Holmes wrote: >> Sorry forgot to answer this: >> >>> Who/what will process the purge list and when? >> >> There is a call to `ClassLoaderDataGraph::purge` which does that, executed later at the garbage collector's convenience. This change does not change the amount of elements (apart from the order of the elements) too, so no difference here. >> >> (During class unloading `DependencyContext::delete_on_release()` is and has always been false, and the existing code always moves entries to the purge list - there is an appropriate assert in the new code). > >> (During class unloading DependencyContext::delete_on_release() is and has always been false, and the existing code always moves entries to the purge list - there is an appropriate assert in the new code). > > Thanks I can see that now. So all this does is move a chunk of elements to the purge-list instead of doing it one at a time. Makes me wonder why this wasn't noticed sooner. :) Thanks @dholmes-ora @fisk for your reviews ------------- PR Comment: https://git.openjdk.org/jdk/pull/16639#issuecomment-1812010732 From sjohanss at openjdk.org Wed Nov 15 09:39:46 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Wed, 15 Nov 2023 09:39:46 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v42] In-Reply-To: References: Message-ID: On Tue, 14 Nov 2023 21:33:37 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Update parallel workers time after Remark Thanks for addressing my comments. I have a few more things: - I think all changes to `test_g1ServiceThread.cpp` can be reverted. Should not be needed now - Please fix all whitespace issues - Should we move the VMThread and StringDedup counters into `CPUTimeCounters` as well? Any problem with this? ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1812105608 From duke at openjdk.org Wed Nov 15 14:32:49 2023 From: duke at openjdk.org (Lei Zaakjyu) Date: Wed, 15 Nov 2023 14:32:49 GMT Subject: RFR: JDK-8234502 : Merge GenCollectedHeap and SerialHeap [v6] In-Reply-To: References: Message-ID: > JDK-8234502 : Merge GenCollectedHeap and SerialHeap Lei Zaakjyu has updated the pull request incrementally with one additional commit since the last revision: fix include statements ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16623/files - new: https://git.openjdk.org/jdk/pull/16623/files/bf022a87..1202a7bd Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16623&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16623&range=04-05 Stats: 4 lines in 4 files changed: 2 ins; 2 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16623.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16623/head:pull/16623 PR: https://git.openjdk.org/jdk/pull/16623 From duke at openjdk.org Wed Nov 15 20:23:29 2023 From: duke at openjdk.org (Lei Zaakjyu) Date: Wed, 15 Nov 2023 20:23:29 GMT Subject: RFR: JDK-8234502 : Merge GenCollectedHeap and SerialHeap [v6] In-Reply-To: References: Message-ID: On Wed, 15 Nov 2023 14:32:49 GMT, Lei Zaakjyu wrote: >> JDK-8234502 : Merge GenCollectedHeap and SerialHeap > > Lei Zaakjyu has updated the pull request incrementally with one additional commit since the last revision: > > fix include statements done! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16623#issuecomment-1813204231 From jjoo at openjdk.org Wed Nov 15 23:43:57 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Wed, 15 Nov 2023 23:43:57 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v43] In-Reply-To: References: Message-ID: <0PG8RL0P8m4dbQe-gmltDKvcQYPJvUvUshV-anYvFFM=.41402e8d-c895-43b9-9567-036723afcabc@github.com> > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: - Move vm and conc_dedup counters to cpuTimeCounters class - Revert test changes and fix whitespace issues ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/189d1852..4db8f09f Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=42 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=41-42 Stats: 30 lines in 9 files changed: 11 ins; 5 del; 14 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Wed Nov 15 23:50:02 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Wed, 15 Nov 2023 23:50:02 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v44] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: Fix whitespace ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/4db8f09f..ce7dbfcf Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=43 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=42-43 Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Thu Nov 16 00:17:42 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Thu, 16 Nov 2023 00:17:42 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v42] In-Reply-To: References: Message-ID: On Wed, 15 Nov 2023 09:36:47 GMT, Stefan Johansson wrote: >> Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: >> >> Update parallel workers time after Remark > > Thanks for addressing my comments. I have a few more things: > > - I think all changes to `test_g1ServiceThread.cpp` can be reverted. Should not be needed now > - Please fix all whitespace issues > - Should we move the VMThread and StringDedup counters into `CPUTimeCounters` as well? Any problem with this? @kstefanj Good points - done! ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1813500906 From manc at openjdk.org Thu Nov 16 00:27:44 2023 From: manc at openjdk.org (Man Cao) Date: Thu, 16 Nov 2023 00:27:44 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v44] In-Reply-To: References: Message-ID: On Wed, 15 Nov 2023 23:50:02 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Fix whitespace Overall looks good, a few details could be improved. src/hotspot/share/gc/g1/g1CollectedHeap.hpp line 59: > 57: #include "memory/iterator.hpp" > 58: #include "memory/memRegion.hpp" > 59: #include "runtime/cpuTimeCounters.hpp" Probably move this include to the .cpp file? src/hotspot/share/gc/g1/g1ServiceThread.cpp line 161: > 159: if (UsePerfData && os::is_thread_cpu_time_supported()) { > 160: ThreadTotalCPUTimeClosure tttc(CPUTimeCounters::get_instance(), CPUTimeGroups::gc_service); > 161: tttc.do_thread(task->_service_thread); It could just use `do_thread(this)`, then it can remove the `task` parameter. src/hotspot/share/gc/shared/stringdedup/stringDedupProcessor.cpp line 72: > 70: _processor = new Processor(); > 71: if (UsePerfData && os::is_thread_cpu_time_supported()) { > 72: EXCEPTION_MARK; This whole `if` block could be updated to `CPUTimeCounters::get_instance()->create_counter(CPUTimeGroups::conc_dedup)`. We can also remove the `_concurrent_dedup_thread_cpu_time` field and the `ThreadTotalCPUTimeClosure(PerfCounter* counter)` constructor. In `StringDedup::Processor::run()`, it can call if (UsePerfData && os::is_thread_cpu_time_supported()) { ThreadTotalCPUTimeClosure tttc(CPUTimeCounters::get_instance(), CPUTimeGroups::conc_dedup); tttc.do_thread(thread); } Similar, this can be applied to vmThread. src/hotspot/share/runtime/init.cpp line 124: > 122: codeCache_init(); > 123: VM_Version_init(); // depends on codeCache_init for emitting code > 124: // Initialize CPUTimeCounters object, which must be done before creation of the heap. Would it be possible to move this inside `universe_init()` in universe.cpp, somewhere above `Universe::initialize_heap()`? There's a similar `MetaspaceCounters::initialize_performance_counters()` in `universe_init()`. ------------- Changes requested by manc (Committer). PR Review: https://git.openjdk.org/jdk/pull/15082#pullrequestreview-1731128372 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1395001799 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1395007511 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1395009733 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1395000735 From manc at openjdk.org Thu Nov 16 00:27:54 2023 From: manc at openjdk.org (Man Cao) Date: Thu, 16 Nov 2023 00:27:54 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v42] In-Reply-To: References: Message-ID: On Tue, 14 Nov 2023 21:33:37 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Update parallel workers time after Remark src/hotspot/share/gc/g1/g1CollectedHeap.cpp line 1519: > 1517: > 1518: CPUTimeCounters* instance = CPUTimeCounters::get_instance(); > 1519: assert(instance != nullptr, "no instance found"); It's probably better to move this `assert` inside the `CPUTimeCounters::get_instance()` body. src/hotspot/share/runtime/cpuTimeCounters.cpp line 43: > 41: case gc_service: > 42: return "gc_service"; > 43: case COUNT: "default" seems more appropriate than COUNT. This seems better? default: ShouldNotReachHere(); src/hotspot/share/runtime/cpuTimeCounters.cpp line 65: > 63: } > 64: > 65: CPUTimeCounters* CPUTimeCounters::_instance = nullptr; No need for extra whitespaces. Single space should be fine. src/hotspot/share/runtime/cpuTimeCounters.hpp line 2: > 1: /* > 2: * Copyright (c) 2002, 2019, Oracle and/or its affiliates. All rights reserved. Year should be "2023". src/hotspot/share/runtime/cpuTimeCounters.hpp line 32: > 30: #include "runtime/perfData.hpp" > 31: #include "runtime/perfDataTypes.hpp" > 32: Include "memory/iterator.hpp" and remove this include from perfData.hpp? src/hotspot/share/runtime/cpuTimeCounters.hpp line 35: > 33: class CPUTimeGroups : public AllStatic { > 34: public: > 35: enum CPUTimeType { I think new code should prefer `enum class` over plain `enum`. https://stackoverflow.com/q/18335861 src/hotspot/share/runtime/cpuTimeCounters.hpp line 36: > 34: public: > 35: enum CPUTimeType { > 36: total, Probably `gc_total` instead of `total`, since we will include non-GC counters in this class. Also for naming style, I find it better to be `GcTotal` or `GC_TOTAL` for public enum constants. But HotSpot has mixed styles for enums, so changing the names is optional. src/hotspot/share/runtime/cpuTimeCounters.hpp line 48: > 46: }; > 47: > 48: class CPUTimeCounters: public CHeapObj { Perhaps `mtServiceability` as hsperf counters are part of serviceability, and we will include non-GC hsperf counters. src/hotspot/share/runtime/cpuTimeCounters.hpp line 50: > 48: class CPUTimeCounters: public CHeapObj { > 49: private: > 50: // We want CPUTimeCounters to be a singleton instance accessed by the vm thread. Suggest remove "accessed by the vm thread". It is already access by non-VM threads like G1 concurrent mark thread and concurrent refine thread. src/hotspot/share/runtime/cpuTimeCounters.hpp line 61: > 59: // since the last time we called `publish_total_cpu_time()`. > 60: // It is incremented using Atomic::add() to prevent race conditions, and > 61: // is added to the `total` CPUTimeGroup at the end of GC. Ditto, better to have "gc" substring in these names: _total_cpu_time_diff inc_total_cpu_time() publish_total_cpu_time() src/hotspot/share/runtime/cpuTimeCounters.hpp line 80: > 78: void operator=(const CPUTimeCounters& copy) = delete; > 79: > 80: ~CPUTimeCounters(); No need to declare destructor since it is not overwritten. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1393555723 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394989485 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394992509 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394981679 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394997677 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394972807 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394976545 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394981147 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394969526 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394977769 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394983409 From ayang at openjdk.org Thu Nov 16 15:17:35 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Thu, 16 Nov 2023 15:17:35 GMT Subject: RFR: JDK-8234502 : Merge GenCollectedHeap and SerialHeap [v6] In-Reply-To: References: Message-ID: <9bW9Gq13fwD5HG4rS6IITvQGHBE8s0U5QrKVJa80r90=.722bda1b-2ec0-499e-9f20-cbd6bbc474aa@github.com> On Wed, 15 Nov 2023 14:32:49 GMT, Lei Zaakjyu wrote: >> JDK-8234502 : Merge GenCollectedHeap and SerialHeap > > Lei Zaakjyu has updated the pull request incrementally with one additional commit since the last revision: > > fix include statements src/hotspot/share/gc/serial/serialHeap.cpp line 94: > 92: > 93: SerialHeap::SerialHeap() : > 94: #if 0 Isn't this effectively dead code? src/hotspot/share/gc/serial/serialVMOperations.hpp line 66: > 64: > 65: > 66: #endif // SHARE_GC_SERIAL_SERIALVMOPERATIONS_HPP I think it misses a line-break for EOF, which is why this special icon shows here. (The same issue exists in some other files updated by this PR as well.) src/hotspot/share/gc/shenandoah/shenandoahMonitoringSupport.hpp line 29: > 27: > 28: #include "memory/allocation.hpp" > 29: #include "gc/shared/collectorCounters.hpp" If this is required, could it be made in its own PR? It's odd to see this PR changes files exclusively owned by other collectors. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16623#discussion_r1395875545 PR Review Comment: https://git.openjdk.org/jdk/pull/16623#discussion_r1395878314 PR Review Comment: https://git.openjdk.org/jdk/pull/16623#discussion_r1395869900 From duke at openjdk.org Thu Nov 16 16:49:54 2023 From: duke at openjdk.org (Lei Zaakjyu) Date: Thu, 16 Nov 2023 16:49:54 GMT Subject: RFR: JDK-8234502 : Merge GenCollectedHeap and SerialHeap [v7] In-Reply-To: References: Message-ID: > JDK-8234502 : Merge GenCollectedHeap and SerialHeap Lei Zaakjyu has updated the pull request incrementally with three additional commits since the last revision: - replace a necessary include statement - clean up - add line-breaks ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16623/files - new: https://git.openjdk.org/jdk/pull/16623/files/1202a7bd..0563686a Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16623&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16623&range=05-06 Stats: 10 lines in 5 files changed: 1 ins; 6 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/16623.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16623/head:pull/16623 PR: https://git.openjdk.org/jdk/pull/16623 From denghui.ddh at alibaba-inc.com Thu Nov 16 17:31:12 2023 From: denghui.ddh at alibaba-inc.com (Denghui Dong) Date: Fri, 17 Nov 2023 01:31:12 +0800 Subject: =?UTF-8?B?TGltaXQgdGhlIG51bWJlciBvZiBoZWFwIGR1bXBzIGdlbmVyYXRlZCBieSBIZWFwRHVtcEJl?= =?UTF-8?B?Zm9yZUZ1bGxHQy9BZnRlckZ1bGxHQw==?= Message-ID: <169addc8-8083-492c-b200-06f1cf512bf1.denghui.ddh@alibaba-inc.com> Hi team, HeapDumpBeforeFullGC/AfterFullGC are the two flags that are commonly used by developers when analyzing FGC problems. The current implementation of these two flags is triggering a heap dump when every FGC happens. Although these two flags can be turned off through jinfo, this behavior may still bring trouble(make the disk full) when FGC continuously occurs Unklie HeapDumpBeforeFullGC/AfterFullGC, HeapDumpOnOutOfMemoryError will only generate one dump. IMHO, we should improve the implementation.(e.g. Introducing a counter or interval to limit the number). What do you think? I'm sorry in advance if this issue has ever been discussed. Thanks, Denghui -------------- next part -------------- An HTML attachment was scrubbed... URL: From kbarrett at openjdk.org Thu Nov 16 19:19:44 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Thu, 16 Nov 2023 19:19:44 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v8] In-Reply-To: References: Message-ID: On Fri, 10 Nov 2023 13:33:12 GMT, Albert Mingkun Yang wrote: >> Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. >> >> It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. > > Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: > > next-obj-card Changes requested by kbarrett (Reviewer). src/hotspot/share/gc/serial/cardTableRS.cpp line 354: > 352: > 353: static_assert(clean_card_val() == (CardValue)-1, "inv"); > 354: constexpr Word clean_word = (Word)-1; CardTable provides `clean_card_row_val()` for this sort of thing. src/hotspot/share/gc/serial/cardTableRS.cpp line 370: > 368: // Word comparison > 369: while (i_card + sizeof(Word) <= end_card) { > 370: Word* i_word = reinterpret_cast(i_card); Maybe CardTable should provide a helper for accessing a "row" of cards? src/hotspot/share/gc/serial/cardTableRS.cpp line 372: > 370: Word* i_word = reinterpret_cast(i_card); > 371: if (*i_word != clean_word) { > 372: // Found a clean card in this word; fall back to per-CardValue comparison. s/clean/non-clean/ or s/clean/dirty/. src/hotspot/share/gc/serial/cardTableRS.cpp line 490: > 488: if (dirty_l == end_card) { > 489: // No dirty cards to iterate. > 490: break; Maybe s/break/return/ ? src/hotspot/share/gc/serial/cardTableRS.hpp line 56: > 54: static void clear_cards(CardValue* start, CardValue* end); > 55: > 56: static CardTable::CardValue* find_first_dirty_card(CardValue* const start_card, CardTable qualification for return type isn't needed here, or a couple lines later. ------------- PR Review: https://git.openjdk.org/jdk/pull/16492#pullrequestreview-1735202108 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1396202686 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1396204908 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1396198763 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1396209000 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1396189076 From ayang at openjdk.org Thu Nov 16 19:41:07 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Thu, 16 Nov 2023 19:41:07 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v9] In-Reply-To: References: Message-ID: > Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. > > It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 16 additional commits since the last revision: - Merge branch 'master' into s1-young-gc - review - next-obj-card - review - Merge branch 'master' into s1-young-gc - review - Update src/hotspot/share/gc/serial/cardTableRS.cpp Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> - Update src/hotspot/share/gc/serial/cardTableRS.cpp Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> - Update src/hotspot/share/gc/serial/cardTableRS.cpp Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> - Update src/hotspot/share/gc/serial/cardTableRS.cpp Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> - ... and 6 more: https://git.openjdk.org/jdk/compare/c0fe8d1f...8762a64a ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16492/files - new: https://git.openjdk.org/jdk/pull/16492/files/e8b8e0ba..8762a64a Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=08 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=07-08 Stats: 608510 lines in 887 files changed: 81377 ins; 469008 del; 58125 mod Patch: https://git.openjdk.org/jdk/pull/16492.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16492/head:pull/16492 PR: https://git.openjdk.org/jdk/pull/16492 From ayang at openjdk.org Thu Nov 16 19:41:08 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Thu, 16 Nov 2023 19:41:08 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v8] In-Reply-To: References: Message-ID: On Thu, 16 Nov 2023 19:05:03 GMT, Kim Barrett wrote: >> Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: >> >> next-obj-card > > src/hotspot/share/gc/serial/cardTableRS.cpp line 370: > >> 368: // Word comparison >> 369: while (i_card + sizeof(Word) <= end_card) { >> 370: Word* i_word = reinterpret_cast(i_card); > > Maybe CardTable should provide a helper for accessing a "row" of cards? g1RemSet.cpp also has sth like this. I will handle them together in another PR. > src/hotspot/share/gc/serial/cardTableRS.hpp line 56: > >> 54: static void clear_cards(CardValue* start, CardValue* end); >> 55: >> 56: static CardTable::CardValue* find_first_dirty_card(CardValue* const start_card, > > CardTable qualification for return type isn't needed here, or a couple lines later. Got compilation error without it. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1396241610 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1396241828 From tschatzl at openjdk.org Fri Nov 17 09:59:30 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 17 Nov 2023 09:59:30 GMT Subject: RFR: 8319713: Parallel: Remove PSAdaptiveSizePolicy::should_full_GC In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 13:44:56 GMT, Albert Mingkun Yang wrote: > Simple removing too-conservative full-gc heuristic. Marked as reviewed by tschatzl (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16560#pullrequestreview-1736513465 From tschatzl at openjdk.org Fri Nov 17 09:59:32 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 17 Nov 2023 09:59:32 GMT Subject: RFR: 8319713: Parallel: Remove PSAdaptiveSizePolicy::should_full_GC In-Reply-To: References: Message-ID: On Mon, 13 Nov 2023 12:35:40 GMT, Albert Mingkun Yang wrote: > The check in `PSScavenge::should_attempt_scavenge` is constrained by used-in-young to offer a less-conservative heuristic, i.e. #promoted-bytes can't be larger than used-in-young. Okay, got it. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16560#issuecomment-1816066096 From kbarrett at openjdk.org Fri Nov 17 18:28:36 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Fri, 17 Nov 2023 18:28:36 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v8] In-Reply-To: References: Message-ID: On Thu, 16 Nov 2023 19:37:44 GMT, Albert Mingkun Yang wrote: >> src/hotspot/share/gc/serial/cardTableRS.hpp line 56: >> >>> 54: static void clear_cards(CardValue* start, CardValue* end); >>> 55: >>> 56: static CardTable::CardValue* find_first_dirty_card(CardValue* const start_card, >> >> CardTable qualification for return type isn't needed here, or a couple lines later. > > Got compilation error without it. That is weird. Which compiler(s)? Unqualified CardValue should be fine here, and is used in other places in the same signature. It would be needed in the (out of class) definition, where the scoping is different, but shouldn't be needed here (within the class definition). (That scoping issue could be removed by using trailing return type, which has different scope too, but that would be strange style in HotSpot.) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1397718399 From ayang at openjdk.org Fri Nov 17 19:43:33 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 17 Nov 2023 19:43:33 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v8] In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 18:25:54 GMT, Kim Barrett wrote: >> Got compilation error without it. > > That is weird. Which compiler(s)? Unqualified CardValue should be fine here, and is used in other places in the same signature. It would be needed in the (out of class) definition, where the scoping is different, but shouldn't be needed here (within the class definition). (That scoping issue could be removed by using trailing return type, which has different scope too, but that would be strange style in HotSpot.) >From clang 14: cardTableRS.cpp:349:1: error: unknown type name 'CardValue' CardValue* CardTableRS::find_first_dirty_card(CardValue* const start_card, >From gcc 13: cardTableRS.cpp:349:1: error: 'CardValue' does not name a type; did you mean 'JavaValue'? 349 | CardValue* CardTableRS::find_first_dirty_card(CardValue* const start_card, Can you build successfully without the prefix? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1397790766 From kbarrett at openjdk.org Fri Nov 17 21:36:33 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Fri, 17 Nov 2023 21:36:33 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v8] In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 19:40:26 GMT, Albert Mingkun Yang wrote: >> That is weird. Which compiler(s)? Unqualified CardValue should be fine here, and is used in other places in the same signature. It would be needed in the (out of class) definition, where the scoping is different, but shouldn't be needed here (within the class definition). (That scoping issue could be removed by using trailing return type, which has different scope too, but that would be strange style in HotSpot.) > > From clang 14: > > > cardTableRS.cpp:349:1: error: unknown type name 'CardValue' > CardValue* CardTableRS::find_first_dirty_card(CardValue* const start_card, > > > From gcc 13: > > > cardTableRS.cpp:349:1: error: 'CardValue' does not name a type; did you mean 'JavaValue'? > 349 | CardValue* CardTableRS::find_first_dirty_card(CardValue* const start_card, > > > Can you build successfully without the prefix? My original comment was about the .hpp file. My followup comment noted that the qualification is needed in the .cpp file, because of the different scoping (in the class definition body vs outside it). The error you are mentioning is in the .cpp file. A leading return type is not in the scope of the function. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1397908653 From ayang at openjdk.org Fri Nov 17 22:23:32 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 17 Nov 2023 22:23:32 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v8] In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 21:33:43 GMT, Kim Barrett wrote: >> From clang 14: >> >> >> cardTableRS.cpp:349:1: error: unknown type name 'CardValue' >> CardValue* CardTableRS::find_first_dirty_card(CardValue* const start_card, >> >> >> From gcc 13: >> >> >> cardTableRS.cpp:349:1: error: 'CardValue' does not name a type; did you mean 'JavaValue'? >> 349 | CardValue* CardTableRS::find_first_dirty_card(CardValue* const start_card, >> >> >> Can you build successfully without the prefix? > > My original comment was about the .hpp file. My followup comment noted that the qualification is needed in > the .cpp file, because of the different scoping (in the class definition body vs outside it). The error you are > mentioning is in the .cpp file. A leading return type is not in the scope of the function. I didn't understand your request. Compiling works fine if I remove the prefix in header only. However, I think it's more consistent when the declaration in header and definition have the identical signature. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1397948336 From kbarrett at openjdk.org Sat Nov 18 05:13:30 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Sat, 18 Nov 2023 05:13:30 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v8] In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 22:20:30 GMT, Albert Mingkun Yang wrote: >> My original comment was about the .hpp file. My followup comment noted that the qualification is needed in >> the .cpp file, because of the different scoping (in the class definition body vs outside it). The error you are >> mentioning is in the .cpp file. A leading return type is not in the scope of the function. > > I didn't understand your request. Compiling works fine if I remove the prefix in header only. However, I think it's more consistent when the declaration in header and definition have the identical signature. There are a number of ways in which a declaration and a separate definition may or even must differ. I think this is a false consistency, and confusing because the qualifier is unnecessary and eye catching. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1398086819 From tschatzl at openjdk.org Mon Nov 20 11:31:50 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Mon, 20 Nov 2023 11:31:50 GMT Subject: RFR: 8320331: G1 Full GC Heap verification relies on metadata not reset before verification Message-ID: Hi all, please review this fix to full gc to properly set `parsable_bottom` for the heap regions marked through so that "during" verification (`-XX:+VerifyDuringGC`) does not access metadata already unlinked but not yet completely purged. The test case reproduces crashes fairly well without the patch; the new test case is run with debug VMs only purely to save testing time. Testing: gha, test case Thanks, Thomas ------------- Commit messages: - Only run test case on debug VMs, sufficient - 8320331 g1 full gc "during" verification accesses half-unloaded metadata Changes: https://git.openjdk.org/jdk/pull/16733/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16733&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8320331 Stats: 19 lines in 5 files changed: 16 ins; 0 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/16733.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16733/head:pull/16733 PR: https://git.openjdk.org/jdk/pull/16733 From tschatzl at openjdk.org Mon Nov 20 13:02:33 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Mon, 20 Nov 2023 13:02:33 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v8] In-Reply-To: References: Message-ID: On Sat, 18 Nov 2023 05:10:34 GMT, Kim Barrett wrote: >> I didn't understand your request. Compiling works fine if I remove the prefix in header only. However, I think it's more consistent when the declaration in header and definition have the identical signature. > > There are a number of ways in which a declaration and a separate definition may or even must differ. > I think this is a false consistency, and confusing because the qualifier is unnecessary and eye catching. +1 ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1399167241 From ayang at openjdk.org Mon Nov 20 15:19:58 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Mon, 20 Nov 2023 15:19:58 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v8] In-Reply-To: References: Message-ID: <7wErujxUEaztW9HSUcEBuMhsLN8cPkzNhSGkpRtvVP0=.d70a03e7-d619-49c6-9f4b-08aa96189126@github.com> On Mon, 20 Nov 2023 12:59:31 GMT, Thomas Schatzl wrote: >> There are a number of ways in which a declaration and a separate definition may or even must differ. >> I think this is a false consistency, and confusing because the qualifier is unnecessary and eye catching. > > +1 Changed. I still believe the declaration and definition should be trivially duplicated, because they are semantically the same thing. Subjective, I guess. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1399351655 From ayang at openjdk.org Mon Nov 20 15:19:58 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Mon, 20 Nov 2023 15:19:58 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v10] In-Reply-To: References: Message-ID: <4Rcj_9iG3FgCQN0sxu1TCRfGoLS5KXOnda3fQDl6v80=.7fc5f93f-c1c1-4cad-bbd0-70cc061d41e7@github.com> > Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. > > It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: review ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16492/files - new: https://git.openjdk.org/jdk/pull/16492/files/8762a64a..c5c7e181 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=09 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=08-09 Stats: 6 lines in 1 file changed: 0 ins; 0 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/16492.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16492/head:pull/16492 PR: https://git.openjdk.org/jdk/pull/16492 From kbarrett at openjdk.org Mon Nov 20 19:38:13 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Mon, 20 Nov 2023 19:38:13 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v10] In-Reply-To: <4Rcj_9iG3FgCQN0sxu1TCRfGoLS5KXOnda3fQDl6v80=.7fc5f93f-c1c1-4cad-bbd0-70cc061d41e7@github.com> References: <4Rcj_9iG3FgCQN0sxu1TCRfGoLS5KXOnda3fQDl6v80=.7fc5f93f-c1c1-4cad-bbd0-70cc061d41e7@github.com> Message-ID: On Mon, 20 Nov 2023 15:19:58 GMT, Albert Mingkun Yang wrote: >> Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. >> >> It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. > > Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: > > review A few more minor nits. src/hotspot/share/gc/serial/cardTableRS.cpp line 351: > 349: CardTable::CardValue* CardTableRS::find_first_dirty_card(CardValue* const start_card, > 350: CardValue* const end_card) { > 351: using Word = uintptr_t; I think it would be better if Word was consistent with clean_card_row_val, to eliminate casts now and in the future when you address my earlier comment about needing a clean-row accessor. So I suggest `using Word = decltype(clean_card_row_val());`. Maybe as part of the clean-row accessor also add a CardRow type alias. Note that I think that type should be unsigned (as currently here, e.g. uintptr_t) rather than signed (as currently provided by clean_card_row_val). src/hotspot/share/gc/serial/cardTableRS.cpp line 353: > 351: using Word = uintptr_t; > 352: > 353: static_assert(clean_card_val() == (CardValue)-1, "inv"); Now that clean_card_row_val() is being used, why do we need this static_assert? src/hotspot/share/gc/serial/cardTableRS.cpp line 354: > 352: > 353: static_assert(clean_card_val() == (CardValue)-1, "inv"); > 354: constexpr Word clean_word = (Word)CardTable::clean_card_row_val(); Why is clean_card_row_val qualified here? src/hotspot/share/gc/serial/cardTableRS.cpp line 356: > 354: constexpr Word clean_word = (Word)CardTable::clean_card_row_val(); > 355: > 356: CardValue* i_card = start_card; What does the "i_" prefix here mean? Something like "cur_card" might be a better name. src/hotspot/share/gc/serial/cardTableRS.cpp line 371: > 369: while (i_card + sizeof(Word) <= end_card) { > 370: Word* i_word = reinterpret_cast(i_card); > 371: if (*i_word != clean_word) { Why not just use clean_card_row_val() here directly? Rather than introducing the clean_word variable and only using it once, here. ------------- Changes requested by kbarrett (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16492#pullrequestreview-1740535865 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1399651020 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1399642167 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1399651910 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1399634692 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1399646352 From ayang at openjdk.org Mon Nov 20 20:35:49 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Mon, 20 Nov 2023 20:35:49 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v11] In-Reply-To: References: Message-ID: > Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. > > It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 19 additional commits since the last revision: - Merge branch 'master' into s1-young-gc - review - review - Merge branch 'master' into s1-young-gc - review - next-obj-card - review - Merge branch 'master' into s1-young-gc - review - Update src/hotspot/share/gc/serial/cardTableRS.cpp Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> - ... and 9 more: https://git.openjdk.org/jdk/compare/5906bea1...aeae83cd ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16492/files - new: https://git.openjdk.org/jdk/pull/16492/files/c5c7e181..aeae83cd Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=10 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=09-10 Stats: 7520 lines in 191 files changed: 3523 ins; 1342 del; 2655 mod Patch: https://git.openjdk.org/jdk/pull/16492.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16492/head:pull/16492 PR: https://git.openjdk.org/jdk/pull/16492 From ayang at openjdk.org Mon Nov 20 20:35:50 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Mon, 20 Nov 2023 20:35:50 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v10] In-Reply-To: References: <4Rcj_9iG3FgCQN0sxu1TCRfGoLS5KXOnda3fQDl6v80=.7fc5f93f-c1c1-4cad-bbd0-70cc061d41e7@github.com> Message-ID: On Mon, 20 Nov 2023 19:27:50 GMT, Kim Barrett wrote: >> Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: >> >> review > > src/hotspot/share/gc/serial/cardTableRS.cpp line 351: > >> 349: CardTable::CardValue* CardTableRS::find_first_dirty_card(CardValue* const start_card, >> 350: CardValue* const end_card) { >> 351: using Word = uintptr_t; > > I think it would be better if Word was consistent with clean_card_row_val, to eliminate casts now and in > the future when you address my earlier comment about needing a clean-row accessor. So I suggest > `using Word = decltype(clean_card_row_val());`. > Maybe as part of the clean-row accessor also add a CardRow type alias. > Note that I think that type should be unsigned (as currently here, e.g. uintptr_t) rather than signed > (as currently provided by clean_card_row_val). The similar card-scanning code in g1remset also defines `Word`. I will extract them together. This PR targets Serial-specific issue, so I wanna minimize touching the shared code, e.g. introducing a new `CardWord` type and operations around it. > src/hotspot/share/gc/serial/cardTableRS.cpp line 356: > >> 354: constexpr Word clean_word = (Word)CardTable::clean_card_row_val(); >> 355: >> 356: CardValue* i_card = start_card; > > What does the "i_" prefix here mean? Something like "cur_card" might be a better name. It's for "iterator". ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1399717744 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1399717186 From jjoo at openjdk.org Tue Nov 21 02:19:47 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Tue, 21 Nov 2023 02:19:47 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v45] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: Address comments and refactor TTTC class for simplification ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/ce7dbfcf..17a8eaf3 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=44 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=43-44 Stats: 97 lines in 16 files changed: 11 ins; 28 del; 58 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Tue Nov 21 02:19:48 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Tue, 21 Nov 2023 02:19:48 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v44] In-Reply-To: References: Message-ID: On Wed, 15 Nov 2023 23:50:02 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Fix whitespace Addressed comments, but am running into an assertion failure here when building: https://github.com/openjdk/jdk/pull/15082/files#diff-d1c5f7a125171a3828bdb3f4488327e03ceb96ff615eacfc9df406aca408cbe6R72 Will investigate tomorrow. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1820104586 From kbarrett at openjdk.org Tue Nov 21 07:56:14 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Tue, 21 Nov 2023 07:56:14 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v11] In-Reply-To: References: Message-ID: On Mon, 20 Nov 2023 20:35:49 GMT, Albert Mingkun Yang wrote: >> Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. >> >> It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. > > Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 19 additional commits since the last revision: > > - Merge branch 'master' into s1-young-gc > - review > - review > - Merge branch 'master' into s1-young-gc > - review > - next-obj-card > - review > - Merge branch 'master' into s1-young-gc > - review > - Update src/hotspot/share/gc/serial/cardTableRS.cpp > > Co-authored-by: Thomas Schatzl <59967451+tschatzl at users.noreply.github.com> > - ... and 9 more: https://git.openjdk.org/jdk/compare/e77dc44a...aeae83cd Changes requested by kbarrett (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16492#pullrequestreview-1741306094 From kbarrett at openjdk.org Tue Nov 21 07:56:16 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Tue, 21 Nov 2023 07:56:16 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v10] In-Reply-To: References: <4Rcj_9iG3FgCQN0sxu1TCRfGoLS5KXOnda3fQDl6v80=.7fc5f93f-c1c1-4cad-bbd0-70cc061d41e7@github.com> Message-ID: On Mon, 20 Nov 2023 20:30:02 GMT, Albert Mingkun Yang wrote: >> src/hotspot/share/gc/serial/cardTableRS.cpp line 356: >> >>> 354: constexpr Word clean_word = (Word)CardTable::clean_card_row_val(); >>> 355: >>> 356: CardValue* i_card = start_card; >> >> What does the "i_" prefix here mean? Something like "cur_card" might be a better name. > > It's for "iterator". I don't think I would have ever guessed that, which suggests it might not be an ideal naming choice. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1400148691 From denghui.ddh at alibaba-inc.com Tue Nov 21 09:53:22 2023 From: denghui.ddh at alibaba-inc.com (Denghui Dong) Date: Tue, 21 Nov 2023 17:53:22 +0800 Subject: =?UTF-8?B?UmU6IExpbWl0IHRoZSBudW1iZXIgb2YgaGVhcCBkdW1wcyBnZW5lcmF0ZWQgYnkgSGVhcER1?= =?UTF-8?B?bXBCZWZvcmVGdWxsR0MvQWZ0ZXJGdWxsR0M=?= In-Reply-To: <169addc8-8083-492c-b200-06f1cf512bf1.denghui.ddh@alibaba-inc.com> References: <169addc8-8083-492c-b200-06f1cf512bf1.denghui.ddh@alibaba-inc.com> Message-ID: Any Thoughts? ------------------------------------------------------------------ From:???(??) Send Time:2023?11?17?(???) 01:31 To:hotspot-gc-dev ; hotspot-runtime-dev Subject:Limit the number of heap dumps generated by HeapDumpBeforeFullGC/AfterFullGC Hi team, HeapDumpBeforeFullGC/AfterFullGC are the two flags that are commonly used by developers when analyzing FGC problems. The current implementation of these two flags is triggering a heap dump when every FGC happens. Although these two flags can be turned off through jinfo, this behavior may still bring trouble(make the disk full) when FGC continuously occurs Unklie HeapDumpBeforeFullGC/AfterFullGC, HeapDumpOnOutOfMemoryError will only generate one dump. IMHO, we should improve the implementation.(e.g. Introducing a counter or interval to limit the number). What do you think? I'm sorry in advance if this issue has ever been discussed. Thanks, Denghui -------------- next part -------------- An HTML attachment was scrubbed... URL: From ayang at openjdk.org Tue Nov 21 14:20:11 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 21 Nov 2023 14:20:11 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v10] In-Reply-To: References: <4Rcj_9iG3FgCQN0sxu1TCRfGoLS5KXOnda3fQDl6v80=.7fc5f93f-c1c1-4cad-bbd0-70cc061d41e7@github.com> Message-ID: On Tue, 21 Nov 2023 07:50:35 GMT, Kim Barrett wrote: >> It's for "iterator". > > I don't think I would have ever guessed that, which suggests it might not be an ideal naming choice. My concern regarding `cur_card` is that there is no contrasting `prev_` or `next_` in this context. The sole message I want to convey here is that we are iterating through cards. Using the conventional loop variable `i` plus `card` seems like the most natural approach, in my opinion. It is concise, so it doesn't introduce much overhead despite its frequent occurrence in the context. Therefore, I prefer the current name. By the way, the same style and convention are also employed in other methods in this PR. Additionally, similar naming conventions can be found in the already merged code of Parallel (`PSCardTable`) and G1 (`G1ScanHRForRegionClosure`). ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1400670786 From ayang at openjdk.org Tue Nov 21 14:22:09 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 21 Nov 2023 14:22:09 GMT Subject: RFR: JDK-8234502 : Merge GenCollectedHeap and SerialHeap [v7] In-Reply-To: References: Message-ID: On Thu, 16 Nov 2023 16:49:54 GMT, Lei Zaakjyu wrote: >> JDK-8234502 : Merge GenCollectedHeap and SerialHeap > > Lei Zaakjyu has updated the pull request incrementally with three additional commits since the last revision: > > - replace a necessary include statement > - clean up > - add line-breaks The general shape looks good. I wanna took a closer look after #16492 is merged. (I believe that can reduce the diff size of this PR slightly.) ------------- PR Comment: https://git.openjdk.org/jdk/pull/16623#issuecomment-1821014195 From tschatzl at openjdk.org Tue Nov 21 16:09:16 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 21 Nov 2023 16:09:16 GMT Subject: RFR: 8320525: G1: G1UpdateRemSetTrackingBeforeRebuild::distribute_marked_bytes accesses partially unloaded klass Message-ID: Hi all, please review this fix that removes the access to a partially unloaded (i.e. unlinked only) `Klass` used for debug code in `G1UpdateRemSetTrackingBeforeRebuild::distribute_marked_bytes`. This starts to fail if metadata purging happens before the call to this methods (as https://bugs.openjdk.org/browse/JDK-8317809 suggests). The test gc/g1/humongousObjects/TestHumongousClassLoader.java starts to crash on linux-x86 with 100% reproduction because it more aggressively uncommits memory when purging metaspace. The fix fixes the asserts to only access the klass when it should not be unloaded yet. Testing: failing test case not failing any more, gha Thanks, Thomas ------------- Commit messages: - 8320525 initial implementation Changes: https://git.openjdk.org/jdk/pull/16766/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16766&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8320525 Stats: 9 lines in 1 file changed: 1 ins; 4 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/16766.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16766/head:pull/16766 PR: https://git.openjdk.org/jdk/pull/16766 From tschatzl at openjdk.org Tue Nov 21 16:09:29 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 21 Nov 2023 16:09:29 GMT Subject: RFR: 8317809: Insertion of free code blobs into code cache can be very slow during class unloading Message-ID: <_dcFF70_w7IXSjb6w-HuHCBkPyS3a6NlzejtqdfdYnM=.74e0f9df-eca3-49b0-be68-2d5824c16003@github.com> Insert code blobs in a sorted fashion to exploit the finger-optimization when adding, making this procedure O(n) instead of O(n^2) Introduces a globally available ClassUnloadingContext that contains common methods pertaining to class and code unloading. GCs may use it to efficiently manage unlinked class loader datas and nmethods to allow use of common methods (unlink/merge). The steps typically are registering a new to be unlinked CLD/nmethod, and then purge its memory later. STW collectors perform this work in one big chunk taking the CodeCache_lock, for the entire duration, while concurrent collectors lock/unlock for every insertion to allow for concurrent users for the lock to progress. Some care has been taken to stay consistent with an "unloading = unlinking + purge" scheme; however particularly the existing CLD handling API (still) mixes unlinking and purging in its CLD::unload() call. To simplify this change that is mostly geared towards separating nmethod unlinking from purging, to make code blob freeing O(n) instead of O(n^2). Upcoming changes will * separate nmethod unregistering from nmethod purging to allow doing that in bulk (for the STW collectors); that can significantly reduce code purging time for the STW collectors. * better name the second stage of unlinking (called "cleaning" throughout, e.g. the work done in `G1CollectedHeap::complete_cleaning`) * untangle CLD unlinking and what's called "cleaning" now to allow moving more stuff into the second unlinking stage for better parallelism * G1: move some significant tasks from the remark pause to concurrent (unregistering nmethods, freeing code blobs and cld/metaspace purging) * Maybe move Serial/Parallel GC metaspace purging closer to other unlinking/purging code to keep things local and allow easier logging. Please also first looking into the (small) PR this depends on. The crash on linux-x86 is fixed by PR#16766 which I split out for quicker reviews. Testing: tier1-7 Thanks, Thomas ------------- Depends on: https://git.openjdk.org/jdk/pull/16733 Commit messages: - 8317809 Insert code blobs in a sorted fashion to exploit the finger-optimization when adding, making this procedure O(n) instead of O(n^2) Changes: https://git.openjdk.org/jdk/pull/16759/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16759&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8317809 Stats: 495 lines in 28 files changed: 368 ins; 83 del; 44 mod Patch: https://git.openjdk.org/jdk/pull/16759.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16759/head:pull/16759 PR: https://git.openjdk.org/jdk/pull/16759 From tschatzl at openjdk.org Tue Nov 21 16:21:21 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 21 Nov 2023 16:21:21 GMT Subject: RFR: 8317007: Add bulk removal of dead nmethods during class unloading Message-ID: Hi all, please review this change that bulk-removes dead nmethods for the STW collectors instead of unregistering nmethod by nmethod. This significantly speeds up the class unloading phase. For G1, this is almost 100% the code that has been removed from the review for JDK-8315503. For Serial and Parallel GC, the code is new. This change does not try to improve the situation for concurrent collectors - this would at first glance require extending the scope of the `CodeCache_lock` which I did not want to do. See the CR for more details. Also, no parallelism for Parallel GC: the existing data structure for code root remembered set is a linked list. There is in almost all cases no point in trying to parallelize what is basically a traversal of the linked list (with each element not having a lot of work to do). I filed JDK-8320067. There should already be a significant speedup with this change, as each `unregister_nmethod` call previously traversed the entire linked list to find that element (i.e. complexity reduction of O(n^2) -> O(n)). One more comment: * So it would be possible to merge the `ScavengableNMethod::prune_nmethods_not_into_young` which removes elements from the code root sets that do no longer need to be remembered (due to object movement) with removing dead/unlinked nmethods. However, this would mean to put that class unloading code to the end of gc as during phase 1 the new locations which are relevant for pruning uninteresting code root remembered set entries did not move yet. I wanted to keep determining the unlinking nmethods and class/code unloading them together in the code, but I could be convinced to move it. Testing: tier1-4 Depends on #16759, please review first. Thanks, Thomas ------------- Depends on: https://git.openjdk.org/jdk/pull/16759 Commit messages: - 8317007 Changes: https://git.openjdk.org/jdk/pull/16767/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16767&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8317007 Stats: 123 lines in 20 files changed: 108 ins; 4 del; 11 mod Patch: https://git.openjdk.org/jdk/pull/16767.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16767/head:pull/16767 PR: https://git.openjdk.org/jdk/pull/16767 From jjoo at openjdk.org Tue Nov 21 21:42:39 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Tue, 21 Nov 2023 21:42:39 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v46] In-Reply-To: References: Message-ID: <4S_iyhdkwxpMar7tdNxHobR6vaRcqKcikQQrrcNBwX0=.29697f18-b7cc-44a9-8900-90f7a3a1e780@github.com> > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: - Update memory tracking type for CPUTimeCounters - Fix assertion logic ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/17a8eaf3..4ca30f32 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=45 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=44-45 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Tue Nov 21 21:46:18 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Tue, 21 Nov 2023 21:46:18 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v46] In-Reply-To: <4S_iyhdkwxpMar7tdNxHobR6vaRcqKcikQQrrcNBwX0=.29697f18-b7cc-44a9-8900-90f7a3a1e780@github.com> References: <4S_iyhdkwxpMar7tdNxHobR6vaRcqKcikQQrrcNBwX0=.29697f18-b7cc-44a9-8900-90f7a3a1e780@github.com> Message-ID: On Tue, 21 Nov 2023 21:42:39 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: > > - Update memory tracking type for CPUTimeCounters > - Fix assertion logic All comments have been addressed and assertion failure fixed - this PR should once again be RFR. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1821740369 From manc at openjdk.org Wed Nov 22 02:22:24 2023 From: manc at openjdk.org (Man Cao) Date: Wed, 22 Nov 2023 02:22:24 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v46] In-Reply-To: <4S_iyhdkwxpMar7tdNxHobR6vaRcqKcikQQrrcNBwX0=.29697f18-b7cc-44a9-8900-90f7a3a1e780@github.com> References: <4S_iyhdkwxpMar7tdNxHobR6vaRcqKcikQQrrcNBwX0=.29697f18-b7cc-44a9-8900-90f7a3a1e780@github.com> Message-ID: On Tue, 21 Nov 2023 21:42:39 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: > > - Update memory tracking type for CPUTimeCounters > - Fix assertion logic Looks pretty clean. Only a few minor cleanups remain. src/hotspot/share/gc/g1/g1ConcurrentMark.cpp line 71: > 69: #include "oops/oop.inline.hpp" > 70: #include "runtime/atomic.hpp" > 71: #include "runtime/cpuTimeCounters.hpp" This include could be removed. src/hotspot/share/gc/g1/g1ConcurrentRefine.cpp line 38: > 36: #include "memory/allocation.inline.hpp" > 37: #include "memory/iterator.hpp" > 38: #include "runtime/cpuTimeCounters.hpp" This could be removed too. src/hotspot/share/gc/g1/g1ServiceThread.cpp line 26: > 24: > 25: #include "precompiled.hpp" > 26: #include "gc/g1/g1CollectedHeap.hpp" Could this include be removed? src/hotspot/share/gc/g1/g1ServiceThread.hpp line 30: > 28: #include "gc/shared/concurrentGCThread.hpp" > 29: #include "runtime/mutex.hpp" > 30: #include "runtime/perfData.hpp" This could be removed. src/hotspot/share/gc/shared/collectedHeap.cpp line 52: > 50: #include "oops/instanceMirrorKlass.hpp" > 51: #include "oops/oop.inline.hpp" > 52: #include "runtime/atomic.hpp" This could be removed. src/hotspot/share/gc/shared/stringdedup/stringDedupProcessor.cpp line 28: > 26: #include "classfile/javaClasses.inline.hpp" > 27: #include "classfile/stringTable.hpp" > 28: #include "gc/shared/collectedHeap.hpp" This include as well as the include for perfData.hpp (line 45) could be removed. src/hotspot/share/gc/shared/stringdedup/stringDedupProcessor.cpp line 70: > 68: void StringDedup::Processor::initialize() { > 69: _processor = new Processor(); > 70: if (UsePerfData && os::is_thread_cpu_time_supported()) { The if and EXCEPTION_MARK could be removed, because `create_counter()` does that internally. src/hotspot/share/gc/shared/stringdedup/stringDedupProcessor.hpp line 30: > 28: #include "gc/shared/stringdedup/stringDedup.hpp" > 29: #include "memory/allocation.hpp" > 30: #include "runtime/perfData.hpp" This could be removed. src/hotspot/share/runtime/cpuTimeCounters.hpp line 97: > 95: class ThreadTotalCPUTimeClosure: public ThreadClosure { > 96: private: > 97: jlong _gc_total; _total is a more appropriate name for this field. src/hotspot/share/runtime/vmThread.cpp line 140: > 138: PerfDataManager::create_counter(SUN_THREADS, "vmOperationTime", > 139: PerfData::U_Ticks, CHECK); > 140: if (os::is_thread_cpu_time_supported()) { The if could be remove, as `create_counter()` checks it internally. ------------- Marked as reviewed by manc (Committer). PR Review: https://git.openjdk.org/jdk/pull/15082#pullrequestreview-1743425918 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401413936 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401415033 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401416191 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401416622 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401416969 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401417321 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401418586 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401420582 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401410366 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401419999 From kbarrett at openjdk.org Wed Nov 22 09:25:12 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Wed, 22 Nov 2023 09:25:12 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v10] In-Reply-To: References: <4Rcj_9iG3FgCQN0sxu1TCRfGoLS5KXOnda3fQDl6v80=.7fc5f93f-c1c1-4cad-bbd0-70cc061d41e7@github.com> Message-ID: On Tue, 21 Nov 2023 14:17:30 GMT, Albert Mingkun Yang wrote: >> I don't think I would have ever guessed that, which suggests it might not be an ideal naming choice. > > My concern regarding `cur_card` is that there is no contrasting `prev_` or `next_` in this context. The sole message I want to convey here is that we are iterating through cards. Using the conventional loop variable `i` plus `card` seems like the most natural approach, in my opinion. It is concise, so it doesn't introduce much overhead despite its frequent occurrence in the context. Therefore, I prefer the current name. By the way, the same style and convention are also employed in other methods in this PR. Additionally, similar naming conventions can be found in the already merged code of Parallel (`PSCardTable`) and G1 (`G1ScanHRForRegionClosure`). I think "cur" is a frequently used naming convention for iteration, irrespective of there being related "prev" or "next". I don't like the similar naming in those other places you mention either. Like I said, "i" usually indicates an integral/index variable, and using it otherwise is (at least to me) confusing. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1401740722 From iwalulya at openjdk.org Wed Nov 22 10:19:05 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Wed, 22 Nov 2023 10:19:05 GMT Subject: RFR: 8320331: G1 Full GC Heap verification relies on metadata not reset before verification In-Reply-To: References: Message-ID: On Mon, 20 Nov 2023 11:24:12 GMT, Thomas Schatzl wrote: > Hi all, > > please review this fix to full gc to properly set `parsable_bottom` for the heap regions marked through so that "during" verification (`-XX:+VerifyDuringGC`) does not access metadata already unlinked but not yet completely purged. > > The test case reproduces crashes fairly well without the patch; the new test case is run with debug VMs only purely to save testing time. > > Testing: gha, test case > > Thanks, > Thomas LGTM! Minor: - update the comment associated with `_parsable_bottom` - Add a comment to `prepare_for_full_gc` to hint on why this is required. ------------- Marked as reviewed by iwalulya (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16733#pullrequestreview-1744030751 From tschatzl at openjdk.org Wed Nov 22 11:02:19 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Wed, 22 Nov 2023 11:02:19 GMT Subject: RFR: 8320331: G1 Full GC Heap verification relies on metadata not reset before verification [v2] In-Reply-To: References: Message-ID: > Hi all, > > please review this fix to full gc to properly set `parsable_bottom` for the heap regions marked through so that "during" verification (`-XX:+VerifyDuringGC`) does not access metadata already unlinked but not yet completely purged. > > The test case reproduces crashes fairly well without the patch; the new test case is run with debug VMs only purely to save testing time. > > Testing: gha, test case > > Thanks, > Thomas Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: iwaluya review, comments ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16733/files - new: https://git.openjdk.org/jdk/pull/16733/files/15812e5c..cc23c2d8 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16733&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16733&range=00-01 Stats: 14 lines in 2 files changed: 9 ins; 0 del; 5 mod Patch: https://git.openjdk.org/jdk/pull/16733.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16733/head:pull/16733 PR: https://git.openjdk.org/jdk/pull/16733 From ayang at openjdk.org Wed Nov 22 11:14:10 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 22 Nov 2023 11:14:10 GMT Subject: RFR: 8320331: G1 Full GC Heap verification relies on metadata not reset before verification [v2] In-Reply-To: References: Message-ID: On Wed, 22 Nov 2023 11:02:19 GMT, Thomas Schatzl wrote: >> Hi all, >> >> please review this fix to full gc to properly set `parsable_bottom` for the heap regions marked through so that "during" verification (`-XX:+VerifyDuringGC`) does not access metadata already unlinked but not yet completely purged. >> >> The test case reproduces crashes fairly well without the patch; the new test case is run with debug VMs only purely to save testing time. >> >> Testing: gha, test case >> >> Thanks, >> Thomas > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > iwaluya review, comments Marked as reviewed by ayang (Reviewer). src/hotspot/share/gc/g1/heapRegion.inline.hpp line 170: > 168: } > 169: > 170: inline void HeapRegion::prepare_for_full_gc() { I don't believe this API should be part of this class. It appears entirely caller-specific, and it's unclear why the implementation is just a field-update without knowledge of the caller context. While the comment helps mitigate the situation, making it acceptable for a bugfix, I think it would be more appropriate to relocate methods related to parsable-bottom outside of this class. ------------- PR Review: https://git.openjdk.org/jdk/pull/16733#pullrequestreview-1744134179 PR Review Comment: https://git.openjdk.org/jdk/pull/16733#discussion_r1401886735 From tschatzl at openjdk.org Wed Nov 22 11:42:05 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Wed, 22 Nov 2023 11:42:05 GMT Subject: RFR: 8320331: G1 Full GC Heap verification relies on metadata not reset before verification [v2] In-Reply-To: References: Message-ID: On Wed, 22 Nov 2023 11:10:54 GMT, Albert Mingkun Yang wrote: >I don't believe this API should be part of this class. It appears entirely caller-specific, All of these `_for_full_gc`/`_after_full_gc` methods (and `note_*_marking/scrubbing` methods for concurrent mark) are caller specific, for exactly one caller, for a particular situation the caller encounters. Is there something particularly problematic with this exact method compared to others? > and it's unclear why the implementation is just a field-update without knowledge of the caller context. While the comment helps mitigate the situation, making it acceptable for a bugfix, I think it would be more appropriate to relocate methods related to parsable-bottom outside of this class. So basically `_parsable_bottom` (and `_top_at_mark_start` since it is very similar to PB) fields should be somewhere else then? I think there is a CR for moving TAMS somewhere else, there is nothing for PB. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16733#discussion_r1401918339 From ayang at openjdk.org Wed Nov 22 11:51:04 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 22 Nov 2023 11:51:04 GMT Subject: RFR: 8320331: G1 Full GC Heap verification relies on metadata not reset before verification [v2] In-Reply-To: References: Message-ID: On Wed, 22 Nov 2023 11:39:25 GMT, Thomas Schatzl wrote: > Is there something particularly problematic with this exact method compared to others? No; all of them require caller-info to understand the impl. > fields should be somewhere else then? Yes, because they are constructed/maintained by sth external to heap-region, IMO. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16733#discussion_r1401928354 From tschatzl at openjdk.org Wed Nov 22 16:09:27 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Wed, 22 Nov 2023 16:09:27 GMT Subject: RFR: 8317809: Insertion of free code blobs into code cache can be very slow during class unloading [v2] In-Reply-To: <_dcFF70_w7IXSjb6w-HuHCBkPyS3a6NlzejtqdfdYnM=.74e0f9df-eca3-49b0-be68-2d5824c16003@github.com> References: <_dcFF70_w7IXSjb6w-HuHCBkPyS3a6NlzejtqdfdYnM=.74e0f9df-eca3-49b0-be68-2d5824c16003@github.com> Message-ID: > Insert code blobs in a sorted fashion to exploit the finger-optimization when adding, making this procedure O(n) instead of O(n^2) > > Introduces a globally available ClassUnloadingContext that contains common methods pertaining to class and code unloading. GCs may use it to efficiently manage unlinked class loader datas and nmethods to allow use of common methods (unlink/merge). > > The steps typically are registering a new to be unlinked CLD/nmethod, and then purge its memory later. STW collectors perform this work in one big chunk taking the CodeCache_lock, for the entire duration, while concurrent collectors lock/unlock for every insertion to allow for concurrent users for the lock to progress. > > Some care has been taken to stay consistent with an "unloading = unlinking + purge" scheme; however particularly the existing CLD handling API (still) mixes unlinking and purging in its CLD::unload() call. To simplify this change that is mostly geared towards separating nmethod unlinking from purging, to make code blob freeing O(n) instead of O(n^2). > > Upcoming changes will > * separate nmethod unregistering from nmethod purging to allow doing that in bulk (for the STW collectors); that can significantly reduce code purging time for the STW collectors. > * better name the second stage of unlinking (called "cleaning" throughout, e.g. the work done in `G1CollectedHeap::complete_cleaning`) > * untangle CLD unlinking and what's called "cleaning" now to allow moving more stuff into the second unlinking stage for better parallelism > * G1: move some significant tasks from the remark pause to concurrent (unregistering nmethods, freeing code blobs and cld/metaspace purging) > * Maybe move Serial/Parallel GC metaspace purging closer to other unlinking/purging code to keep things local and allow easier logging. > > Please also first looking into the (small) PR this depends on. > > The crash on linux-x86 is fixed by PR#16766 which I split out for quicker reviews. > > Testing: tier1-7 > > Thanks, > Thomas Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: iwalulya review, naming ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16759/files - new: https://git.openjdk.org/jdk/pull/16759/files/d63ff4a4..448232df Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16759&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16759&range=00-01 Stats: 14 lines in 2 files changed: 0 ins; 0 del; 14 mod Patch: https://git.openjdk.org/jdk/pull/16759.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16759/head:pull/16759 PR: https://git.openjdk.org/jdk/pull/16759 From tschatzl at openjdk.org Wed Nov 22 16:55:08 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Wed, 22 Nov 2023 16:55:08 GMT Subject: RFR: 8317809: Insertion of free code blobs into code cache can be very slow during class unloading [v2] In-Reply-To: References: <_dcFF70_w7IXSjb6w-HuHCBkPyS3a6NlzejtqdfdYnM=.74e0f9df-eca3-49b0-be68-2d5824c16003@github.com> Message-ID: On Wed, 22 Nov 2023 16:09:27 GMT, Thomas Schatzl wrote: >> Insert code blobs in a sorted fashion to exploit the finger-optimization when adding, making this procedure O(n) instead of O(n^2) >> >> Introduces a globally available ClassUnloadingContext that contains common methods pertaining to class and code unloading. GCs may use it to efficiently manage unlinked class loader datas and nmethods to allow use of common methods (unlink/merge). >> >> The steps typically are registering a new to be unlinked CLD/nmethod, and then purge its memory later. STW collectors perform this work in one big chunk taking the CodeCache_lock, for the entire duration, while concurrent collectors lock/unlock for every insertion to allow for concurrent users for the lock to progress. >> >> Some care has been taken to stay consistent with an "unloading = unlinking + purge" scheme; however particularly the existing CLD handling API (still) mixes unlinking and purging in its CLD::unload() call. To simplify this change that is mostly geared towards separating nmethod unlinking from purging, to make code blob freeing O(n) instead of O(n^2). >> >> Upcoming changes will >> * separate nmethod unregistering from nmethod purging to allow doing that in bulk (for the STW collectors); that can significantly reduce code purging time for the STW collectors. >> * better name the second stage of unlinking (called "cleaning" throughout, e.g. the work done in `G1CollectedHeap::complete_cleaning`) >> * untangle CLD unlinking and what's called "cleaning" now to allow moving more stuff into the second unlinking stage for better parallelism >> * G1: move some significant tasks from the remark pause to concurrent (unregistering nmethods, freeing code blobs and cld/metaspace purging) >> * Maybe move Serial/Parallel GC metaspace purging closer to other unlinking/purging code to keep things local and allow easier logging. >> >> These are the reason for the class hierarchy for `ClassUnloadingContext`: the goal is to ultimately have about this phasing (for G1): >> 1. collect all dead CLDs, using the `register_unloading_class_loader_data` method *only* >> 2. parallelize the stuff in `ClassLoaderData::unload()` in one way or another, adding them to the `complete_cleaning` (parallel) phase. >> 3. `purge_nmethods`, `free_code_blobs` and the `remove_unlinked_nmethods_from_code_root_set` (from JDK-8317007) will be concurrent. >> >> Particularly the split of `SystemDictionary::do_unloading` into "only" traversing the CLDs to find the dead ones and then in parallel process them in 2. a... > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > iwalulya review, naming I added some explanation of why there is a class hierarchy for `ClassUnloadingContext` in the description (and some further background). ------------- PR Comment: https://git.openjdk.org/jdk/pull/16759#issuecomment-1823139932 From tschatzl at openjdk.org Wed Nov 22 17:20:18 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Wed, 22 Nov 2023 17:20:18 GMT Subject: RFR: 8320331: G1 Full GC Heap verification relies on metadata not reset before verification [v2] In-Reply-To: References: Message-ID: On Wed, 22 Nov 2023 11:11:26 GMT, Albert Mingkun Yang wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> iwaluya review, comments > > Marked as reviewed by ayang (Reviewer). Thanks @albertnetymk @walulyai for your reviews ------------- PR Comment: https://git.openjdk.org/jdk/pull/16733#issuecomment-1823180379 From tschatzl at openjdk.org Wed Nov 22 17:20:20 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Wed, 22 Nov 2023 17:20:20 GMT Subject: Integrated: 8320331: G1 Full GC Heap verification relies on metadata not reset before verification In-Reply-To: References: Message-ID: <9dlOLbANiYO6vJnZsqx7K2U3mfQBvcMRrC5LfpruAIk=.c1b0f46c-6d8c-4238-86ac-aaf1beb7665a@github.com> On Mon, 20 Nov 2023 11:24:12 GMT, Thomas Schatzl wrote: > Hi all, > > please review this fix to full gc to properly set `parsable_bottom` for the heap regions marked through so that "during" verification (`-XX:+VerifyDuringGC`) does not access metadata already unlinked but not yet completely purged. > > The test case reproduces crashes fairly well without the patch; the new test case is run with debug VMs only purely to save testing time. > > Testing: gha, test case > > Thanks, > Thomas This pull request has now been integrated. Changeset: 1629a905 Author: Thomas Schatzl URL: https://git.openjdk.org/jdk/commit/1629a9059bd2e0f07559a384be4276c7dc13eff2 Stats: 33 lines in 5 files changed: 25 ins; 0 del; 8 mod 8320331: G1 Full GC Heap verification relies on metadata not reset before verification Reviewed-by: iwalulya, ayang ------------- PR: https://git.openjdk.org/jdk/pull/16733 From jjoo at openjdk.org Wed Nov 22 23:08:36 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Wed, 22 Nov 2023 23:08:36 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v47] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: Cleanup and address comments ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/4ca30f32..fcc7e471 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=46 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=45-46 Stats: 19 lines in 10 files changed: 0 ins; 13 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Wed Nov 22 23:08:38 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Wed, 22 Nov 2023 23:08:38 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v46] In-Reply-To: <4S_iyhdkwxpMar7tdNxHobR6vaRcqKcikQQrrcNBwX0=.29697f18-b7cc-44a9-8900-90f7a3a1e780@github.com> References: <4S_iyhdkwxpMar7tdNxHobR6vaRcqKcikQQrrcNBwX0=.29697f18-b7cc-44a9-8900-90f7a3a1e780@github.com> Message-ID: On Tue, 21 Nov 2023 21:42:39 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: > > - Update memory tracking type for CPUTimeCounters > - Fix assertion logic Addressed cleanups, PR should be RFR! ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1823609497 From amitkumar at openjdk.org Thu Nov 23 05:10:08 2023 From: amitkumar at openjdk.org (Amit Kumar) Date: Thu, 23 Nov 2023 05:10:08 GMT Subject: RFR: 8317809: Insertion of free code blobs into code cache can be very slow during class unloading [v2] In-Reply-To: References: <_dcFF70_w7IXSjb6w-HuHCBkPyS3a6NlzejtqdfdYnM=.74e0f9df-eca3-49b0-be68-2d5824c16003@github.com> Message-ID: On Wed, 22 Nov 2023 16:09:27 GMT, Thomas Schatzl wrote: >> Insert code blobs in a sorted fashion to exploit the finger-optimization when adding, making this procedure O(n) instead of O(n^2) >> >> Introduces a globally available ClassUnloadingContext that contains common methods pertaining to class and code unloading. GCs may use it to efficiently manage unlinked class loader datas and nmethods to allow use of common methods (unlink/merge). >> >> The steps typically are registering a new to be unlinked CLD/nmethod, and then purge its memory later. STW collectors perform this work in one big chunk taking the CodeCache_lock, for the entire duration, while concurrent collectors lock/unlock for every insertion to allow for concurrent users for the lock to progress. >> >> Some care has been taken to stay consistent with an "unloading = unlinking + purge" scheme; however particularly the existing CLD handling API (still) mixes unlinking and purging in its CLD::unload() call. To simplify this change that is mostly geared towards separating nmethod unlinking from purging, to make code blob freeing O(n) instead of O(n^2). >> >> Upcoming changes will >> * separate nmethod unregistering from nmethod purging to allow doing that in bulk (for the STW collectors); that can significantly reduce code purging time for the STW collectors. >> * better name the second stage of unlinking (called "cleaning" throughout, e.g. the work done in `G1CollectedHeap::complete_cleaning`) >> * untangle CLD unlinking and what's called "cleaning" now to allow moving more stuff into the second unlinking stage for better parallelism >> * G1: move some significant tasks from the remark pause to concurrent (unregistering nmethods, freeing code blobs and cld/metaspace purging) >> * Maybe move Serial/Parallel GC metaspace purging closer to other unlinking/purging code to keep things local and allow easier logging. >> >> These are the reason for the class hierarchy for `ClassUnloadingContext`: the goal is to ultimately have about this phasing (for G1): >> 1. collect all dead CLDs, using the `register_unloading_class_loader_data` method *only* >> 2. parallelize the stuff in `ClassLoaderData::unload()` in one way or another, adding them to the `complete_cleaning` (parallel) phase. >> 3. `purge_nmethods`, `free_code_blobs` and the `remove_unlinked_nmethods_from_code_root_set` (from JDK-8317007) will be concurrent. >> >> Particularly the split of `SystemDictionary::do_unloading` into "only" traversing the CLDs to find the dead ones and then in parallel process them in 2. a... > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > iwalulya review, naming src/hotspot/share/classfile/classLoaderData.cpp line 602: > 600: > 601: // Clean up class dependencies and tell serviceability tools > 602: // these classes are unloading. This must be called Suggestion: // these classes are unloading. This must be called ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16759#discussion_r1402920337 From sjohanss at openjdk.org Thu Nov 23 08:51:20 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Thu, 23 Nov 2023 08:51:20 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v47] In-Reply-To: References: Message-ID: On Wed, 22 Nov 2023 23:08:36 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Cleanup and address comments I think this looks good now. I've started some additional testing to make sure we don't run into anything unexpected with the newly added test on any platforms. Will approve once the testing is green. src/hotspot/share/runtime/cpuTimeCounters.hpp line 2: > 1: /* > 2: * Copyright (c) 2002, 2023, Oracle and/or its affiliates. All rights reserved. This is probably on me, but this 2002 is wrong: Suggestion: * Copyright (c) 2023, Oracle and/or its affiliates. All rights reserved. ------------- PR Review: https://git.openjdk.org/jdk/pull/15082#pullrequestreview-1745951725 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1403070166 From ayang at openjdk.org Thu Nov 23 13:52:28 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Thu, 23 Nov 2023 13:52:28 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v47] In-Reply-To: References: Message-ID: On Wed, 22 Nov 2023 23:08:36 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Cleanup and address comments src/hotspot/share/gc/g1/g1CollectedHeap.cpp line 2433: > 2431: } > 2432: WorkerThreads* worker_threads = workers(); > 2433: if (worker_threads != nullptr) { When will this be null? src/hotspot/share/runtime/cpuTimeCounters.cpp line 61: > 59: return true; > 60: case CPUTimeType::gc_service: > 61: return true; I think it would look cleaner if these are grouped into a single return. src/hotspot/share/runtime/cpuTimeCounters.cpp line 96: > 94: if (UsePerfData) { > 95: EXCEPTION_MARK; > 96: if (os::is_thread_cpu_time_supported()) { Why is this check inside the scope of `EXCEPTION_MARK`? (I'd expect sth like ` if (use-perf-data && is-cpu-time-supported)` at the top.) src/hotspot/share/runtime/cpuTimeCounters.cpp line 119: > 117: if (CPUTimeGroups::is_gc_counter(_name)) { > 118: instance->inc_gc_total_cpu_time(net_cpu_time); > 119: } I feel much of this is on the wrong abstraction level; `CPUTimeCounters::update_counter(_name, _total);` should be sufficient here. (The caller handles diff calculation and inc gc-counter if needed.) src/hotspot/share/runtime/cpuTimeCounters.cpp line 126: > 124: // pthread_getcpuclockid() and clock_gettime() must return 0. Thus caller > 125: // must ensure the thread exists and has not terminated. > 126: assert(os::is_thread_cpu_time_supported(), "os must support cpu time"); Could this assert be moved to the constructor? src/hotspot/share/runtime/cpuTimeCounters.hpp line 74: > 72: assert(_instance != nullptr, "no instance found"); > 73: return _instance; > 74: } Seems that this is needed solely for accessing the following instance methods. I wonder if it's possible to expose only static methods in the public API. src/hotspot/share/runtime/cpuTimeCounters.hpp line 83: > 81: // Prevent copy of singleton object. > 82: CPUTimeCounters(const CPUTimeCounters& copy) = delete; > 83: void operator=(const CPUTimeCounters& copy) = delete; I think `NONCOPYABLE` can be used here. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1403333939 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1403220339 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1403219626 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1403411402 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1403333335 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1403413301 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1403225116 From ayang at openjdk.org Thu Nov 23 14:22:08 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Thu, 23 Nov 2023 14:22:08 GMT Subject: RFR: 8320525: G1: G1UpdateRemSetTrackingBeforeRebuild::distribute_marked_bytes accesses partially unloaded klass In-Reply-To: References: Message-ID: On Tue, 21 Nov 2023 16:01:39 GMT, Thomas Schatzl wrote: > Hi all, > > please review this fix that removes the access to a partially unloaded (i.e. unlinked only) `Klass` used for debug code in `G1UpdateRemSetTrackingBeforeRebuild::distribute_marked_bytes`. > > This starts to fail if metadata purging happens before the call to this methods (as https://bugs.openjdk.org/browse/JDK-8317809 suggests). The test gc/g1/humongousObjects/TestHumongousClassLoader.java starts to crash on linux-x86 with 100% reproduction because it more aggressively uncommits memory when purging metaspace. > > The fix fixes the asserts to only access the klass when it should not be unloaded yet. > > Testing: failing test case not failing any more, gha > > Thanks, > Thomas src/hotspot/share/gc/g1/g1ConcurrentMark.cpp line 1137: > 1135: // regions; also, we should not access their header any more them as their > 1136: // klass may have been unloaded. > 1137: assert(marked_bytes == 0 || cast_to_oop(hr->bottom())->size() * HeapWordSize == marked_bytes, Is it possible to separate these two cases into two methods, for instance? Taking a step back, why do we even need to call `note_end_of_marking` on these effectively empty regions? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16766#discussion_r1403448011 From tschatzl at openjdk.org Fri Nov 24 08:38:09 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 24 Nov 2023 08:38:09 GMT Subject: RFR: 8320525: G1: G1UpdateRemSetTrackingBeforeRebuild::distribute_marked_bytes accesses partially unloaded klass In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 14:19:47 GMT, Albert Mingkun Yang wrote: >> Hi all, >> >> please review this fix that removes the access to a partially unloaded (i.e. unlinked only) `Klass` used for debug code in `G1UpdateRemSetTrackingBeforeRebuild::distribute_marked_bytes`. >> >> This starts to fail if metadata purging happens before the call to this methods (as https://bugs.openjdk.org/browse/JDK-8317809 suggests). The test gc/g1/humongousObjects/TestHumongousClassLoader.java starts to crash on linux-x86 with 100% reproduction because it more aggressively uncommits memory when purging metaspace. >> >> The fix fixes the asserts to only access the klass when it should not be unloaded yet. >> >> Testing: failing test case not failing any more, gha >> >> Thanks, >> Thomas > > src/hotspot/share/gc/g1/g1ConcurrentMark.cpp line 1137: > >> 1135: // regions; also, we should not access their header any more them as their >> 1136: // klass may have been unloaded. >> 1137: assert(marked_bytes == 0 || cast_to_oop(hr->bottom())->size() * HeapWordSize == marked_bytes, > > Is it possible to separate these two cases into two methods, for instance? Taking a step back, why do we even need to call `note_end_of_marking` on these effectively empty regions? We need to call `HeapRegion::note_end_of_marking` on them, even if they are empty for completeness. It's not completely necessary because reclamation will probably reset them correctly, but it's easier to reason if they (empty and nonempty regions) are handled the same to me. I.e. so _all_ regions have `note_start/end_of_marking` called. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16766#discussion_r1404075292 From tschatzl at openjdk.org Fri Nov 24 08:43:07 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 24 Nov 2023 08:43:07 GMT Subject: RFR: 8320525: G1: G1UpdateRemSetTrackingBeforeRebuild::distribute_marked_bytes accesses partially unloaded klass In-Reply-To: References: Message-ID: On Fri, 24 Nov 2023 08:34:51 GMT, Thomas Schatzl wrote: >> src/hotspot/share/gc/g1/g1ConcurrentMark.cpp line 1137: >> >>> 1135: // regions; also, we should not access their header any more them as their >>> 1136: // klass may have been unloaded. >>> 1137: assert(marked_bytes == 0 || cast_to_oop(hr->bottom())->size() * HeapWordSize == marked_bytes, >> >> Is it possible to separate these two cases into two methods, for instance? Taking a step back, why do we even need to call `note_end_of_marking` on these effectively empty regions? > > We need to call `HeapRegion::note_end_of_marking` on them, even if they are empty for completeness. It's not completely necessary because reclamation will probably reset them correctly, but it's easier to reason if they (empty and nonempty regions) are handled the same to me. > > I.e. so _all_ regions have `note_start/end_of_marking` called. I do not think it is easier to understand to separate these two cases (empty/non-empty) regions here: both distributing bytes (it's fine to distribute 0 bytes) and being consistent with calling the start/end notifications (and `note_end` works as expected on empty regions too) for all regions is easier to follow to me compared to having an unnecessary exception. Because then the question is: why have that exception? Even if they do not do anything "meaningful" other than resetting some internal state that is later overwritten for these specially handled empty regions anyway. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16766#discussion_r1404079467 From tschatzl at openjdk.org Fri Nov 24 08:45:06 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 24 Nov 2023 08:45:06 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v10] In-Reply-To: References: <4Rcj_9iG3FgCQN0sxu1TCRfGoLS5KXOnda3fQDl6v80=.7fc5f93f-c1c1-4cad-bbd0-70cc061d41e7@github.com> Message-ID: On Tue, 21 Nov 2023 14:17:30 GMT, Albert Mingkun Yang wrote: >> I don't think I would have ever guessed that, which suggests it might not be an ideal naming choice. > > My concern regarding `cur_card` is that there is no contrasting `prev_` or `next_` in this context. The sole message I want to convey here is that we are iterating through cards. Using the conventional loop variable `i` plus `card` seems like the most natural approach, in my opinion. It is concise, so it doesn't introduce much overhead despite its frequent occurrence in the context. Therefore, I prefer the current name. By the way, the same style and convention are also employed in other methods in this PR. Additionally, similar naming conventions can be found in the already merged code of Parallel (`PSCardTable`) and G1 (`G1ScanHRForRegionClosure`). I concur with Kim and would like to ask to rename these variables to use the `cur_` prefix as is much more common. I do not see a connection of `cur` with linked list prev/next pointers; in the previous recent PRs that introduced this naming I (in hindsight wrongly) did not speak up because although I did not like it, nobody else objected either. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1401654457 From tschatzl at openjdk.org Fri Nov 24 08:45:07 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 24 Nov 2023 08:45:07 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v10] In-Reply-To: References: <4Rcj_9iG3FgCQN0sxu1TCRfGoLS5KXOnda3fQDl6v80=.7fc5f93f-c1c1-4cad-bbd0-70cc061d41e7@github.com> Message-ID: On Wed, 22 Nov 2023 09:22:24 GMT, Kim Barrett wrote: >> I concur with Kim and would like to ask to rename these variables to use the `cur_` prefix as is much more common. I do not see a connection of `cur` with linked list prev/next pointers; in the previous recent PRs that introduced this naming I (in hindsight wrongly) did not speak up because although I did not like it, nobody else objected either. > > I think "cur" is a frequently used naming convention for iteration, irrespective of there being related > "prev" or "next". I don't like the similar naming in those other places you mention either. Like I said, > "i" usually indicates an integral/index variable, and using it otherwise is (at least to me) confusing. Given that there is now a strong objection to this naming style, and I did not really like the `i_` prefix in the other CRs either but did not speak up, I suggest we just use the existing common `cur_` prefix here. From my point of view there is no relation of `cur_` to `prev`/`next` either. The other uses can be fixed up separately. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1404055869 From tschatzl at openjdk.org Fri Nov 24 09:18:25 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 24 Nov 2023 09:18:25 GMT Subject: RFR: 8317809: Insertion of free code blobs into code cache can be very slow during class unloading [v3] In-Reply-To: <_dcFF70_w7IXSjb6w-HuHCBkPyS3a6NlzejtqdfdYnM=.74e0f9df-eca3-49b0-be68-2d5824c16003@github.com> References: <_dcFF70_w7IXSjb6w-HuHCBkPyS3a6NlzejtqdfdYnM=.74e0f9df-eca3-49b0-be68-2d5824c16003@github.com> Message-ID: > Insert code blobs in a sorted fashion to exploit the finger-optimization when adding, making this procedure O(n) instead of O(n^2) > > Introduces a globally available ClassUnloadingContext that contains common methods pertaining to class and code unloading. GCs may use it to efficiently manage unlinked class loader datas and nmethods to allow use of common methods (unlink/merge). > > The steps typically are registering a new to be unlinked CLD/nmethod, and then purge its memory later. STW collectors perform this work in one big chunk taking the CodeCache_lock, for the entire duration, while concurrent collectors lock/unlock for every insertion to allow for concurrent users for the lock to progress. > > Some care has been taken to stay consistent with an "unloading = unlinking + purge" scheme; however particularly the existing CLD handling API (still) mixes unlinking and purging in its CLD::unload() call. To simplify this change that is mostly geared towards separating nmethod unlinking from purging, to make code blob freeing O(n) instead of O(n^2). > > Upcoming changes will > * separate nmethod unregistering from nmethod purging to allow doing that in bulk (for the STW collectors); that can significantly reduce code purging time for the STW collectors. > * better name the second stage of unlinking (called "cleaning" throughout, e.g. the work done in `G1CollectedHeap::complete_cleaning`) > * untangle CLD unlinking and what's called "cleaning" now to allow moving more stuff into the second unlinking stage for better parallelism > * G1: move some significant tasks from the remark pause to concurrent (unregistering nmethods, freeing code blobs and cld/metaspace purging) > * Maybe move Serial/Parallel GC metaspace purging closer to other unlinking/purging code to keep things local and allow easier logging. > > These are the reason for the class hierarchy for `ClassUnloadingContext`: the goal is to ultimately have about this phasing (for G1): > 1. collect all dead CLDs, using the `register_unloading_class_loader_data` method *only* > 2. parallelize the stuff in `ClassLoaderData::unload()` in one way or another, adding them to the `complete_cleaning` (parallel) phase. > 3. `purge_nmethods`, `free_code_blobs` and the `remove_unlinked_nmethods_from_code_root_set` (from JDK-8317007) will be concurrent. > > Particularly the split of `SystemDictionary::do_unloading` into "only" traversing the CLDs to find the dead ones and then in parallel process them in 2. above warrants a separate `ClassUnloadingCo... Thomas Schatzl has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: - Merge branch 'master' into mergeme - iwalulya review, naming - 8317809 Insert code blobs in a sorted fashion to exploit the finger-optimization when adding, making this procedure O(n) instead of O(n^2) Introduce a globally available ClassUnloadingContext that contains common methods pertaining to class and code unloading. GCs may use it to efficiently manage unlinked class loader datas and nmethods to allow use of common methods (unlink/merge). The steps typically are registering a new to be unlinked CLD/nmethod, and then purge its memory later. STW collectors perform this work in one big chunk taking the CodeCache_lock, for the entire duration, while concurrent collectors lock/unlock for every insertion to allow for concurrent users for the lock to progress. Some care has been taken to stay consistent with an "unloading = unlinking + purge" scheme; however particularly the existing CLD handling API (still) mixes unlinking and purging in its CLD::unload() call. To simplify this change that is mostly geared towards separating nmethod unlinking from purging, to make code blob freeing O(n) instead of O(n^2). Upcoming changes will * separate nmethod unregistering from nmethod purging to allow doing that in bulk (for the STW collectors); that can significantly reduce code purging time for the STW collectors. * better name the second stage of unlinking (called "cleaning" throughout, e.g. the work done in `G1CollectedHeap::complete_cleaning`) * untangle CLD unlinking and what's called "cleaning" now to allow moving more stuff into the second unlinking stage for better parallelism * G1: move some signifcant tasks from the remark pause to concurrent (unregistering nmethods, freeing code blobs and cld/metaspace purging) * Maybe move Serial/Parallel GC metaspace purging closer to other unlinking/purging code to keep things local and allow easier logging. - Only run test case on debug VMs, sufficient - 8320331 g1 full gc "during" verification accesses half-unloaded metadata ------------- Changes: https://git.openjdk.org/jdk/pull/16759/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16759&range=02 Stats: 495 lines in 28 files changed: 368 ins; 83 del; 44 mod Patch: https://git.openjdk.org/jdk/pull/16759.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16759/head:pull/16759 PR: https://git.openjdk.org/jdk/pull/16759 From tschatzl at openjdk.org Fri Nov 24 09:18:28 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 24 Nov 2023 09:18:28 GMT Subject: RFR: 8317809: Insertion of free code blobs into code cache can be very slow during class unloading [v2] In-Reply-To: References: <_dcFF70_w7IXSjb6w-HuHCBkPyS3a6NlzejtqdfdYnM=.74e0f9df-eca3-49b0-be68-2d5824c16003@github.com> Message-ID: On Thu, 23 Nov 2023 05:06:53 GMT, Amit Kumar wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> iwalulya review, naming > > src/hotspot/share/classfile/classLoaderData.cpp line 602: > >> 600: >> 601: // Clean up class dependencies and tell serviceability tools >> 602: // these classes are unloading. This must be called > > Suggestion: > > // these classes are unloading. This must be called Hotspot code style allows comments with both two spaces and one space between sentences in a paragraph. I did not want to change the style there. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16759#discussion_r1404116626 From duke at openjdk.org Fri Nov 24 10:18:31 2023 From: duke at openjdk.org (Lei Zaakjyu) Date: Fri, 24 Nov 2023 10:18:31 GMT Subject: RFR: JDK-8234502 : Merge GenCollectedHeap and SerialHeap [v8] In-Reply-To: References: Message-ID: > JDK-8234502 : Merge GenCollectedHeap and SerialHeap Lei Zaakjyu has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 12 additional commits since the last revision: - Merge branch 'openjdk:master' into serialgc - replace a necessary include statement - clean up - add line-breaks - fix include statements - add some headers - Completely removed 'GenCollectedHeap' - Fix 'young_gen' function in 'genCollectedHeap.cpp' - include 'serialVMOperations.hpp' - fix trialing whitespace - ... and 2 more: https://git.openjdk.org/jdk/compare/8f060df7...12c680a3 ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16623/files - new: https://git.openjdk.org/jdk/pull/16623/files/0563686a..12c680a3 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16623&range=07 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16623&range=06-07 Stats: 627411 lines in 1531 files changed: 93131 ins; 472348 del; 61932 mod Patch: https://git.openjdk.org/jdk/pull/16623.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16623/head:pull/16623 PR: https://git.openjdk.org/jdk/pull/16623 From ayang at openjdk.org Fri Nov 24 12:49:05 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 24 Nov 2023 12:49:05 GMT Subject: RFR: 8320525: G1: G1UpdateRemSetTrackingBeforeRebuild::distribute_marked_bytes accesses partially unloaded klass In-Reply-To: References: Message-ID: On Fri, 24 Nov 2023 08:39:30 GMT, Thomas Schatzl wrote: >> We need to call `HeapRegion::note_end_of_marking` on them, even if they are empty for completeness. It's not completely necessary because reclamation will probably reset them correctly, but it's easier to reason if they (empty and nonempty regions) are handled the same to me. >> >> I.e. so _all_ regions have `note_start/end_of_marking` called. > > I do not think it is easier to understand to separate these two cases (empty/non-empty) regions here: both distributing bytes (it's fine to distribute 0 bytes) and being consistent with calling the start/end notifications (and `note_end` works as expected on empty regions too) for all regions is easier to follow to me compared to having an unnecessary exception. > Because then the question is: why have that exception? > > Even if they do not do anything "meaningful" other than resetting some internal state that is later overwritten for these specially handled empty regions anyway. > so all regions have note_start/end_of_marking called. Currently, the pair is invoked on all regions and the filtering (skipping young-region for example) is done inside these methods. However, the logic why a particular kind of region can (should) be skipped really belongs to the caller. The region itself doesn't know how to react to marking-start/end. (This is kind of tied to the ticket of moving marking-related fields outside region.) > why have that exception? Because live-region and effective-region are diff, and mixing them causes confusion. I think the existence of the new comment "we should not access their header any more them..." demonstrates that it's not super obvious why the current code (before this PR) is problematic. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16766#discussion_r1404318347 From tschatzl at openjdk.org Fri Nov 24 14:36:05 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 24 Nov 2023 14:36:05 GMT Subject: RFR: 8320525: G1: G1UpdateRemSetTrackingBeforeRebuild::distribute_marked_bytes accesses partially unloaded klass In-Reply-To: References: Message-ID: On Fri, 24 Nov 2023 12:46:32 GMT, Albert Mingkun Yang wrote: > > so all regions have note_start/end_of_marking called. > > Currently, the pair is invoked on all regions and the filtering (skipping young-region for example) is done inside these methods. However, the logic why a particular kind of region can (should) be skipped really belongs to the caller. The region itself doesn't know how to react to marking-start/end. (This is kind of tied to the ticket of moving marking-related fields outside region.) We already agreed that these `note*` methods are basically part of the caller, placed in the wrong location because the members it accesses are in the wrong location. I do not think messing with this here in this CR half-heartedly is a good idea. As soon as the work to move TAMS and PB starts, this is going to change anyway and is imho a more appropriate time to reconsider this (and will probably naturally fix itself). The problematic code is assertion code, which quite often accesses internals one normally would not. The regular code is independent of whether the region's klass is live or not after all. Deleting this assert would fix the issue at hand as well. Another option would be to just not do class unloading this early; there is no particular reason to do it right after marking completed. > > > why have that exception? > > Because live-region and effective-region are diff, and mixing them causes confusion. What is an "effective-region" in this context? I do not understand this sentence. >I think the existence of the new comment "we should not access their header any more them..." demonstrates that it's not super obvious why the current code (before this PR) is problematic. To me this change indicates that the sanity check code (this problematic statement is part of sanity check code - regular code does not use it) is doing things it should not. The original author (me I guess) correctly considered that we already unloaded classes super-early for some reason, wanted to have some extra check there, but then botched the refactoring (factoring out the `obj_size_in_words` calculation for the two uses). That particular new comment is only to make it abundantly clear to not factor out any kind of `obj_size` calculation any more (I removed the second use in this change). Maybe not adding the comment would have been better, as the `marked_bytes == 0` predicate already indicates just that (and the second use of `obj_size_in_words` is gone) Looking at the code again, another source for confusion is maybe wrong comment placement. I will improve these. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16766#discussion_r1404415875 From tschatzl at openjdk.org Fri Nov 24 14:43:18 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 24 Nov 2023 14:43:18 GMT Subject: RFR: 8320525: G1: G1UpdateRemSetTrackingBeforeRebuild::distribute_marked_bytes accesses partially unloaded klass [v2] In-Reply-To: References: Message-ID: > Hi all, > > please review this fix that removes the access to a partially unloaded (i.e. unlinked only) `Klass` used for debug code in `G1UpdateRemSetTrackingBeforeRebuild::distribute_marked_bytes`. > > This starts to fail if metadata purging happens before the call to this methods (as https://bugs.openjdk.org/browse/JDK-8317809 suggests). The test gc/g1/humongousObjects/TestHumongousClassLoader.java starts to crash on linux-x86 with 100% reproduction because it more aggressively uncommits memory when purging metaspace. > > The fix fixes the asserts to only access the klass when it should not be unloaded yet. > > Testing: failing test case not failing any more, gha > > Thanks, > Thomas Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: ayang review, improve documentation for assert. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16766/files - new: https://git.openjdk.org/jdk/pull/16766/files/eff72aa6..c86922f1 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16766&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16766&range=00-01 Stats: 5 lines in 1 file changed: 0 ins; 4 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16766.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16766/head:pull/16766 PR: https://git.openjdk.org/jdk/pull/16766 From ayang at openjdk.org Fri Nov 24 15:05:08 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 24 Nov 2023 15:05:08 GMT Subject: RFR: 8320525: G1: G1UpdateRemSetTrackingBeforeRebuild::distribute_marked_bytes accesses partially unloaded klass [v2] In-Reply-To: References: Message-ID: On Fri, 24 Nov 2023 14:43:18 GMT, Thomas Schatzl wrote: >> Hi all, >> >> please review this fix that removes the access to a partially unloaded (i.e. unlinked only) `Klass` used for debug code in `G1UpdateRemSetTrackingBeforeRebuild::distribute_marked_bytes`. >> >> This starts to fail if metadata purging happens before the call to this methods (as https://bugs.openjdk.org/browse/JDK-8317809 suggests). The test gc/g1/humongousObjects/TestHumongousClassLoader.java starts to crash on linux-x86 with 100% reproduction because it more aggressively uncommits memory when purging metaspace. >> >> The fix fixes the asserts to only access the klass when it should not be unloaded yet. >> >> Testing: failing test case not failing any more, gha >> >> Thanks, >> Thomas > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > ayang review, improve documentation for assert. > this is going to change anyway and is imho a more appropriate time to reconsider this (and will probably naturally fix itself). OK. > What is an "effective-region" in this context? I meant effectively-empty region. ------------- Marked as reviewed by ayang (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16766#pullrequestreview-1748096393 From sjohanss at openjdk.org Mon Nov 27 12:08:21 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Mon, 27 Nov 2023 12:08:21 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v47] In-Reply-To: References: Message-ID: On Wed, 22 Nov 2023 23:08:36 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Cleanup and address comments My additional testing found a failure when running: `sun/tools/jstat/jstatSnap1.sh` When running `jstat -snap 0` (which is what the test does) with this change the new counters are displayed even though they belong to the unsupported namespace "sun.". From what I can tell, the reason for this is that when creating a new subsystem namespace we need to create three of them, one for each top-level namespace. Otherwise the code trying to figure out if this is supported or not is tripped over. I've suggested two changes to do this and with those the test in question no longer fails. It would be good if someone better familiar with jstat/perf-counter code verifies that this is how it should be solved. src/hotspot/share/runtime/perfData.cpp line 76: > 74: "com.sun.threads", > 75: "sun.threads", > 76: "sun.threads.cpu_time", // Subsystem for Sun Threads CPU times Suggestion: "java.threads.cpu_time", //Thread CPU time name spaces "com.sun.threads.cpu_time", "sun.threads.cpu_time", src/hotspot/share/runtime/perfData.hpp line 64: > 62: COM_THREADS, > 63: SUN_THREADS, > 64: SUN_THREADS_CPUTIME, // Subsystem for Sun Threads CPU times Suggestion: JAVA_THREADS_CPUTIME, // Thread CPU time name spaces COM_THREADS_CPUTIME, SUN_THREADS_CPUTIME, ------------- Changes requested by sjohanss (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/15082#pullrequestreview-1750253595 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1406050140 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1406047123 From tschatzl at openjdk.org Mon Nov 27 13:57:07 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Mon, 27 Nov 2023 13:57:07 GMT Subject: RFR: 8319703: Serial: Remove generationSpec In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 10:51:19 GMT, Albert Mingkun Yang wrote: > Simple cleanup of removing an unnecessary indirection. The real change is only 11 LOC, using the global variables directly for young/old gen. Marked as reviewed by tschatzl (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16554#pullrequestreview-1750492324 From iwalulya at openjdk.org Mon Nov 27 14:59:13 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Mon, 27 Nov 2023 14:59:13 GMT Subject: RFR: 8317809: Insertion of free code blobs into code cache can be very slow during class unloading [v3] In-Reply-To: References: <_dcFF70_w7IXSjb6w-HuHCBkPyS3a6NlzejtqdfdYnM=.74e0f9df-eca3-49b0-be68-2d5824c16003@github.com> Message-ID: On Fri, 24 Nov 2023 09:18:25 GMT, Thomas Schatzl wrote: >> Insert code blobs in a sorted fashion to exploit the finger-optimization when adding, making this procedure O(n) instead of O(n^2) >> >> Introduces a globally available ClassUnloadingContext that contains common methods pertaining to class and code unloading. GCs may use it to efficiently manage unlinked class loader datas and nmethods to allow use of common methods (unlink/merge). >> >> The steps typically are registering a new to be unlinked CLD/nmethod, and then purge its memory later. STW collectors perform this work in one big chunk taking the CodeCache_lock, for the entire duration, while concurrent collectors lock/unlock for every insertion to allow for concurrent users for the lock to progress. >> >> Some care has been taken to stay consistent with an "unloading = unlinking + purge" scheme; however particularly the existing CLD handling API (still) mixes unlinking and purging in its CLD::unload() call. To simplify this change that is mostly geared towards separating nmethod unlinking from purging, to make code blob freeing O(n) instead of O(n^2). >> >> Upcoming changes will >> * separate nmethod unregistering from nmethod purging to allow doing that in bulk (for the STW collectors); that can significantly reduce code purging time for the STW collectors. >> * better name the second stage of unlinking (called "cleaning" throughout, e.g. the work done in `G1CollectedHeap::complete_cleaning`) >> * untangle CLD unlinking and what's called "cleaning" now to allow moving more stuff into the second unlinking stage for better parallelism >> * G1: move some significant tasks from the remark pause to concurrent (unregistering nmethods, freeing code blobs and cld/metaspace purging) >> * Maybe move Serial/Parallel GC metaspace purging closer to other unlinking/purging code to keep things local and allow easier logging. >> >> These are the reason for the class hierarchy for `ClassUnloadingContext`: the goal is to ultimately have about this phasing (for G1): >> 1. collect all dead CLDs, using the `register_unloading_class_loader_data` method *only* >> 2. parallelize the stuff in `ClassLoaderData::unload()` in one way or another, adding them to the `complete_cleaning` (parallel) phase. >> 3. `purge_nmethods`, `free_code_blobs` and the `remove_unlinked_nmethods_from_code_root_set` (from JDK-8317007) will be concurrent. >> >> Particularly the split of `SystemDictionary::do_unloading` into "only" traversing the CLDs to find the dead ones and then in parallel process them in 2. a... > > Thomas Schatzl has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: > > - Merge branch 'master' into mergeme > - iwalulya review, naming > - 8317809 Insert code blobs in a sorted fashion to exploit the finger-optimization when adding, making this procedure O(n) instead of O(n^2) > > Introduce a globally available ClassUnloadingContext that contains common methods pertaining to class and code unloading. > GCs may use it to efficiently manage unlinked class loader datas and nmethods to allow use of common methods (unlink/merge). > > The steps typically are registering a new to be unlinked CLD/nmethod, and then purge its memory later. STW collectors perform > this work in one big chunk taking the CodeCache_lock, for the entire duration, while concurrent collectors lock/unlock for every > insertion to allow for concurrent users for the lock to progress. > > Some care has been taken to stay consistent with an "unloading = unlinking + purge" scheme; however particularly the existing > CLD handling API (still) mixes unlinking and purging in its CLD::unload() call. To simplify this change that is mostly geared > towards separating nmethod unlinking from purging, to make code blob freeing O(n) instead of O(n^2). > > Upcoming changes will > * separate nmethod unregistering from nmethod purging to allow doing that in bulk (for the STW collectors); that can significantly > reduce code purging time for the STW collectors. > * better name the second stage of unlinking (called "cleaning" throughout, e.g. the work done in `G1CollectedHeap::complete_cleaning`) > * untangle CLD unlinking and what's called "cleaning" now to allow moving more stuff into the second unlinking stage for better > parallelism > * G1: move some signifcant tasks from the remark pause to concurrent (unregistering nmethods, freeing code blobs and cld/metaspace purging) > * Maybe move Serial/Parallel GC metaspace purging closer to other unlinking/purging code to keep things local and allow easier logging. > - Only run test case on debug VMs, sufficient > - 8320331 g1 full gc "during" verification accesses half-unloaded metadata Changes requested by iwalulya (Reviewer). src/hotspot/share/gc/shared/classUnloadingContext.cpp line 138: > 136: NMethodSet* nmethod_set = nullptr; > 137: > 138: bool is_parallel = _num_nmethod_unlink_workers != 1; Suggestion: bool is_parallel = _num_nmethod_unlink_workers > 1; Seems more intuitive. src/hotspot/share/gc/shared/classUnloadingContext.hpp line 66: > 64: ClassLoaderData* volatile _cld_head; > 65: > 66: uint _num_nmethod_unlink_workers; probably better to set `_num_nmethod_unlink_workers;` as const, the destructor depends on this being const through the lifetime of the object. src/hotspot/share/gc/shared/classUnloadingContext.hpp line 95: > 93: }; > 94: > 95: #endif // SHARE_GC_SHARED_CLASSUNLOADINGCONTEXT_HPP trailing space warning* ------------- PR Review: https://git.openjdk.org/jdk/pull/16759#pullrequestreview-1750594239 PR Review Comment: https://git.openjdk.org/jdk/pull/16759#discussion_r1406283555 PR Review Comment: https://git.openjdk.org/jdk/pull/16759#discussion_r1406260568 PR Review Comment: https://git.openjdk.org/jdk/pull/16759#discussion_r1406268113 From ayang at openjdk.org Mon Nov 27 14:59:17 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Mon, 27 Nov 2023 14:59:17 GMT Subject: RFR: 8317809: Insertion of free code blobs into code cache can be very slow during class unloading [v3] In-Reply-To: References: <_dcFF70_w7IXSjb6w-HuHCBkPyS3a6NlzejtqdfdYnM=.74e0f9df-eca3-49b0-be68-2d5824c16003@github.com> Message-ID: On Fri, 24 Nov 2023 09:18:25 GMT, Thomas Schatzl wrote: >> Insert code blobs in a sorted fashion to exploit the finger-optimization when adding, making this procedure O(n) instead of O(n^2) >> >> Introduces a globally available ClassUnloadingContext that contains common methods pertaining to class and code unloading. GCs may use it to efficiently manage unlinked class loader datas and nmethods to allow use of common methods (unlink/merge). >> >> The steps typically are registering a new to be unlinked CLD/nmethod, and then purge its memory later. STW collectors perform this work in one big chunk taking the CodeCache_lock, for the entire duration, while concurrent collectors lock/unlock for every insertion to allow for concurrent users for the lock to progress. >> >> Some care has been taken to stay consistent with an "unloading = unlinking + purge" scheme; however particularly the existing CLD handling API (still) mixes unlinking and purging in its CLD::unload() call. To simplify this change that is mostly geared towards separating nmethod unlinking from purging, to make code blob freeing O(n) instead of O(n^2). >> >> Upcoming changes will >> * separate nmethod unregistering from nmethod purging to allow doing that in bulk (for the STW collectors); that can significantly reduce code purging time for the STW collectors. >> * better name the second stage of unlinking (called "cleaning" throughout, e.g. the work done in `G1CollectedHeap::complete_cleaning`) >> * untangle CLD unlinking and what's called "cleaning" now to allow moving more stuff into the second unlinking stage for better parallelism >> * G1: move some significant tasks from the remark pause to concurrent (unregistering nmethods, freeing code blobs and cld/metaspace purging) >> * Maybe move Serial/Parallel GC metaspace purging closer to other unlinking/purging code to keep things local and allow easier logging. >> >> These are the reason for the class hierarchy for `ClassUnloadingContext`: the goal is to ultimately have about this phasing (for G1): >> 1. collect all dead CLDs, using the `register_unloading_class_loader_data` method *only* >> 2. parallelize the stuff in `ClassLoaderData::unload()` in one way or another, adding them to the `complete_cleaning` (parallel) phase. >> 3. `purge_nmethods`, `free_code_blobs` and the `remove_unlinked_nmethods_from_code_root_set` (from JDK-8317007) will be concurrent. >> >> Particularly the split of `SystemDictionary::do_unloading` into "only" traversing the CLDs to find the dead ones and then in parallel process them in 2. a... > > Thomas Schatzl has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: > > - Merge branch 'master' into mergeme > - iwalulya review, naming > - 8317809 Insert code blobs in a sorted fashion to exploit the finger-optimization when adding, making this procedure O(n) instead of O(n^2) > > Introduce a globally available ClassUnloadingContext that contains common methods pertaining to class and code unloading. > GCs may use it to efficiently manage unlinked class loader datas and nmethods to allow use of common methods (unlink/merge). > > The steps typically are registering a new to be unlinked CLD/nmethod, and then purge its memory later. STW collectors perform > this work in one big chunk taking the CodeCache_lock, for the entire duration, while concurrent collectors lock/unlock for every > insertion to allow for concurrent users for the lock to progress. > > Some care has been taken to stay consistent with an "unloading = unlinking + purge" scheme; however particularly the existing > CLD handling API (still) mixes unlinking and purging in its CLD::unload() call. To simplify this change that is mostly geared > towards separating nmethod unlinking from purging, to make code blob freeing O(n) instead of O(n^2). > > Upcoming changes will > * separate nmethod unregistering from nmethod purging to allow doing that in bulk (for the STW collectors); that can significantly > reduce code purging time for the STW collectors. > * better name the second stage of unlinking (called "cleaning" throughout, e.g. the work done in `G1CollectedHeap::complete_cleaning`) > * untangle CLD unlinking and what's called "cleaning" now to allow moving more stuff into the second unlinking stage for better > parallelism > * G1: move some signifcant tasks from the remark pause to concurrent (unregistering nmethods, freeing code blobs and cld/metaspace purging) > * Maybe move Serial/Parallel GC metaspace purging closer to other unlinking/purging code to keep things local and allow easier logging. > - Only run test case on debug VMs, sufficient > - 8320331 g1 full gc "during" verification accesses half-unloaded metadata src/hotspot/share/gc/shared/classUnloadingContext.cpp line 120: > 118: for (uint i = 0; i < _num_nmethod_unlink_workers; ++i) { > 119: NMethodSet* set = _unlinked_nmethods[i]; > 120: for (int j = 0; j < set->length(); ++j) { I wonder if one can use range-based for loop here. src/hotspot/share/gc/shared/classUnloadingContext.cpp line 171: > 169: ConditionalMutexLocker ml_inner(CodeCache_lock, _lock_codeblob_free_separately, Mutex::_no_safepoint_check_flag); > 170: CodeCache::free(nmethod_set->at(i)); > 171: } I feel it would be clearer if the for-loop is duplicated to handle either case separately. src/hotspot/share/gc/shared/classUnloadingContext.hpp line 63: > 61: }; > 62: > 63: class DefaultClassUnloadingContext : public ClassUnloadingContext { I don't understand why they need to be two classes, even after reading "These are the reason for the class hierarchy for...". The reference to future/other PR(s) in the description doesn't really help -- it's unclear what is *necessary* for the current PR and what is preparation for future PR(s). src/hotspot/share/gc/shared/classUnloadingContext.hpp line 95: > 93: }; > 94: > 95: #endif // SHARE_GC_SHARED_CLASSUNLOADINGCONTEXT_HPP Seems missing a newline. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16759#discussion_r1406262742 PR Review Comment: https://git.openjdk.org/jdk/pull/16759#discussion_r1406264755 PR Review Comment: https://git.openjdk.org/jdk/pull/16759#discussion_r1406282829 PR Review Comment: https://git.openjdk.org/jdk/pull/16759#discussion_r1406283473 From ayang at openjdk.org Mon Nov 27 16:12:18 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Mon, 27 Nov 2023 16:12:18 GMT Subject: RFR: 8319703: Serial: Remove generationSpec In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 10:51:19 GMT, Albert Mingkun Yang wrote: > Simple cleanup of removing an unnecessary indirection. The real change is only 11 LOC, using the global variables directly for young/old gen. Thanks for the review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16554#issuecomment-1828140398 From ayang at openjdk.org Mon Nov 27 16:12:20 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Mon, 27 Nov 2023 16:12:20 GMT Subject: Integrated: 8319703: Serial: Remove generationSpec In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 10:51:19 GMT, Albert Mingkun Yang wrote: > Simple cleanup of removing an unnecessary indirection. The real change is only 11 LOC, using the global variables directly for young/old gen. This pull request has now been integrated. Changeset: f9e9131e Author: Albert Mingkun Yang URL: https://git.openjdk.org/jdk/commit/f9e9131e126802af95f27abdd667e310033fd1de Stats: 265 lines in 12 files changed: 1 ins; 254 del; 10 mod 8319703: Serial: Remove generationSpec Reviewed-by: cjplummer, tschatzl ------------- PR: https://git.openjdk.org/jdk/pull/16554 From ysr at openjdk.org Mon Nov 27 20:50:08 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Mon, 27 Nov 2023 20:50:08 GMT Subject: RFR: 8317755: G1: Periodic GC interval should test for the last whole heap GC [v2] In-Reply-To: References: Message-ID: On Wed, 18 Oct 2023 14:27:26 GMT, Aleksey Shipilev wrote: >> See the description in the bug. Fortunately, we already track the last whole-heap GC. The new regression test verifies the behavior. >> >> Additional testing: >> - [x] Linux x86_64 fastdebug `tier1 tier2 tier3` > > Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: > > - Merge branch 'master' into JDK-8317755-g1-periodic-whole > - Optional flag > - Merge branch 'master' into JDK-8317755-g1-periodic-whole > - Keep the cast > - Fix LGTM. Very useful when there is no direct feedback loop from resource pressure that we might otherwise have used for signaling the need for a whole heap GC, so is a good escape hatch to put in the hands of service owners. Of course, like any escape hatch, it could be misused, but I don't think that is a concern about ejection seats from fighter jets for example. We trust those who fly the jets not to eject for no reason. I would suggest formally dissociating this from the JEP itself. The fix ups to the JEP's and its implementation shortcomings can be addressed fully in the fullness of time. My USD 0.02. ------------- Marked as reviewed by ysr (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16107#pullrequestreview-1751328467 From jjoo at openjdk.org Tue Nov 28 02:22:45 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Tue, 28 Nov 2023 02:22:45 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v48] In-Reply-To: References: Message-ID: <_lEBVrWV8wrVbmhOiu3AAqPJo_xBs718ZtA9V-VSzGM=.253c0ec8-256e-4dee-b125-90be6338e4b8@github.com> > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: - Fix namespace issues (2) Co-authored-by: Stefan Johansson <54407259+kstefanj at users.noreply.github.com> - Fix namespace issues Co-authored-by: Stefan Johansson <54407259+kstefanj at users.noreply.github.com> ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/fcc7e471..abb90258 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=47 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=46-47 Stats: 6 lines in 2 files changed: 4 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From ayang at openjdk.org Tue Nov 28 09:24:12 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 28 Nov 2023 09:24:12 GMT Subject: RFR: JDK-8234502 : Merge GenCollectedHeap and SerialHeap [v8] In-Reply-To: References: Message-ID: On Fri, 24 Nov 2023 10:18:31 GMT, Lei Zaakjyu wrote: >> JDK-8234502 : Merge GenCollectedHeap and SerialHeap > > Lei Zaakjyu has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 12 additional commits since the last revision: > > - Merge branch 'openjdk:master' into serialgc > - replace a necessary include statement > - clean up > - add line-breaks > - fix include statements > - add some headers > - Completely removed 'GenCollectedHeap' > - Fix 'young_gen' function in 'genCollectedHeap.cpp' > - include 'serialVMOperations.hpp' > - fix trialing whitespace > - ... and 2 more: https://git.openjdk.org/jdk/compare/165c291d...12c680a3 I just noticed I referred to the wrong PR number in my previous msg... Could you resolve the conflict now that that PR is merged? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16623#issuecomment-1829410192 From ayang at openjdk.org Tue Nov 28 09:36:24 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 28 Nov 2023 09:36:24 GMT Subject: RFR: 8320864: Serial: Extract out Full GC related fields from ContiguousSpace Message-ID: <3KhtxAuc0WQcaDeLSyUERRMAccss4EpRiiY3NlmwX6U=.e8447faa-5af7-4279-aca3-bc53e3f1eb6f@github.com> Simple moving some fields from the generic class to Full GC specific files. Ideally, on deciding wheter `to`-space should be compacted, one should use `set_incremental_collection_failed` to check if promotion-failure occurred during young-gc, but in some places, this flag is set even in the absence of promotion-failure, e.g. `heap->set_incremental_collection_failed(); // Slight lie: we did not even attempt one`. Therefore, I used `next_compaction_space` for this purpose, in order to keep this patch more focused and the diff-size small. Test: tier1-5; perf neutral using the attached benchmark. ------------- Commit messages: - s1-full-gc Changes: https://git.openjdk.org/jdk/pull/16842/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16842&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8320864 Stats: 857 lines in 11 files changed: 269 ins; 494 del; 94 mod Patch: https://git.openjdk.org/jdk/pull/16842.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16842/head:pull/16842 PR: https://git.openjdk.org/jdk/pull/16842 From sjohanss at openjdk.org Tue Nov 28 10:09:08 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Tue, 28 Nov 2023 10:09:08 GMT Subject: RFR: 8319713: Parallel: Remove PSAdaptiveSizePolicy::should_full_GC In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 13:44:56 GMT, Albert Mingkun Yang wrote: > Simple removing too-conservative full-gc heuristic. Looks good. ------------- Marked as reviewed by sjohanss (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16560#pullrequestreview-1752506631 From ayang at openjdk.org Tue Nov 28 10:45:17 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 28 Nov 2023 10:45:17 GMT Subject: RFR: 8319713: Parallel: Remove PSAdaptiveSizePolicy::should_full_GC In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 13:44:56 GMT, Albert Mingkun Yang wrote: > Simple removing too-conservative full-gc heuristic. Thanks for the review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16560#issuecomment-1829550696 From ayang at openjdk.org Tue Nov 28 10:45:18 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 28 Nov 2023 10:45:18 GMT Subject: Integrated: 8319713: Parallel: Remove PSAdaptiveSizePolicy::should_full_GC In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 13:44:56 GMT, Albert Mingkun Yang wrote: > Simple removing too-conservative full-gc heuristic. This pull request has now been integrated. Changeset: 4bcda602 Author: Albert Mingkun Yang URL: https://git.openjdk.org/jdk/commit/4bcda602668835c35e2ac6ff6702d15cd249bc2a Stats: 24 lines in 3 files changed: 0 ins; 23 del; 1 mod 8319713: Parallel: Remove PSAdaptiveSizePolicy::should_full_GC Reviewed-by: tschatzl, sjohanss ------------- PR: https://git.openjdk.org/jdk/pull/16560 From stuefe at openjdk.org Tue Nov 28 11:23:28 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Tue, 28 Nov 2023 11:23:28 GMT Subject: RFR: JDK-8320859: gtest high malloc footprint caused by BufferNodeAllocator stress test Message-ID: `BufferNodeAllocatorTest.stress_free_list_allocator_vm` is too expensive. On my box, this test accumulates ~1.5 GB of malloc footprint and raises the libc memory retention from 50m to 800m. It is quite alone in its hunger. The rest of the tests together accumulate just ~50mb of libc retention. The buffer does a stress test of the BufferNodeAllocator. Four "mutator threads" race four "gc threads". Mutator threads allocate buffers, GC threads release them. No processing is done on the buffers; this seems to be purely a test of the allocator and its freelist mechanism. The memory footprint of this test depends on the number of retained free buffers in the allocator. The allocator bulk-releases free buffers triggered by a free-count threshold. On my box, the number of free items in the unmodified stock VM is 100k..200k. It is not clear whether the fact that so many free list items exist indicates a problem with the allocator itself. It looks like the free list should be drained if there are more than 10 items in this list. In any case, each buffer's capacity is 1024 * sizeof(void*)+header, so 8KB+x, and with 100..200k of those things it explains the NMT-reported malloc footprint of ~1..2GB. An easy fix is to reduce the size of this buffer; since they are not processed, their size should not matter. --------- The patch decreases the size per buffer node to cache line size. I chose cache line size out of the vague feeling that I don't want to cause false sharing and thereby degrade the test. Interestingly enough, reducing the buffer size greatly increases the number of free items since it makes allocation cheaper. This also happens in release builds, so this is not us zapping stuff; my unproven assumption is that this is the libc just being slower when allocating 8K vs 64-byte blocks. But with smaller buffers, we spend more time in freelist management and less time in the libc, which is good for a stress test. For more details, see JBS issue and its comments. This patch also increases the number of "processor" threads vs "mutator" threads. Since allocation seems to have a speed edge over deallocation, this reduces the number of free items somewhat. This reduces the malloc spike seen from this thread from 1.5-2GB to ~160MB (release) 3xxMB (debug). We could reduce it a lot more if we reduced the buffer size to its minimum (1 slot); that would risk false sharing and may degrade test performance. ------------- Commit messages: - more processor threads than mutators - increase number of processor threads - reduce test buffer node size Changes: https://git.openjdk.org/jdk/pull/16845/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16845&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8320859 Stats: 13 lines in 1 file changed: 5 ins; 0 del; 8 mod Patch: https://git.openjdk.org/jdk/pull/16845.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16845/head:pull/16845 PR: https://git.openjdk.org/jdk/pull/16845 From ayang at openjdk.org Tue Nov 28 11:35:10 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 28 Nov 2023 11:35:10 GMT Subject: RFR: JDK-8320859: gtest high malloc footprint caused by BufferNodeAllocator stress test In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 11:02:24 GMT, Thomas Stuefe wrote: > `BufferNodeAllocatorTest.stress_free_list_allocator_vm` is too expensive. On my box, this test accumulates ~1.5 GB of malloc footprint and raises the libc memory retention from 50m to 800m. > > It is quite alone in its hunger. The rest of the tests together accumulate just ~50mb of libc retention. > > The buffer does a stress test of the BufferNodeAllocator. Four "mutator threads" race four "gc threads". Mutator threads allocate buffers, GC threads release them. No processing is done on the buffers; this seems to be purely a test of the allocator and its freelist mechanism. The memory footprint of this test depends on the number of retained free buffers in the allocator. The allocator bulk-releases free buffers triggered by a free-count threshold. On my box, the number of free items in the unmodified stock VM is 100k..200k. > > It is not clear whether the fact that so many free list items exist indicates a problem with the allocator itself. It looks like the free list should be drained if there are more than 10 items in this list. > > In any case, each buffer's capacity is 1024 * sizeof(void*)+header, so 8KB+x, and with 100..200k of those things it explains the NMT-reported malloc footprint of ~1..2GB. An easy fix is to reduce the size of this buffer; since they are not processed, their size should not matter. > > --------- > > The patch decreases the size per buffer node to cache line size. I chose cache line size out of the vague feeling that I don't want to cause false sharing and thereby degrade the test. > > Interestingly enough, reducing the buffer size greatly increases the number of free items since it makes allocation cheaper. This also happens in release builds, so this is not us zapping stuff; my unproven assumption is that this is the libc just being slower when allocating 8K vs 64-byte blocks. But with smaller buffers, we spend more time in freelist management and less time in the libc, which is good for a stress test. > > This patch also increases the number of "processor" threads vs "mutator" threads. Since allocation seems to have a speed edge over deallocation, this reduces the number of free items somewhat. > > This reduces the malloc spike seen from this thread from 1.5-2GB to ~160MB (release) 3xxMB (debug). We could reduce it a lot more if we reduced the buffer size to its minimum (1 slot); that would risk false sharing and may degrade test performance. Marked as reviewed by ayang (Reviewer). test/hotspot/gtest/gc/shared/test_bufferNodeAllocator.cpp line 193: > 191: // free nodes (footprint) > 192: const uint nthreads_mut = 4; > 193: const uint nthreads_proc = 6; Naming them "num_processor_threads" and "num_allocator_threads" is more consistent with the corresponding types `ProcessorThread` and `AllocatorThread`, IMO. ------------- PR Review: https://git.openjdk.org/jdk/pull/16845#pullrequestreview-1752667075 PR Review Comment: https://git.openjdk.org/jdk/pull/16845#discussion_r1407620010 From stuefe at openjdk.org Tue Nov 28 12:23:07 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Tue, 28 Nov 2023 12:23:07 GMT Subject: RFR: JDK-8320859: gtest high malloc footprint caused by BufferNodeAllocator stress test In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 11:31:58 GMT, Albert Mingkun Yang wrote: >> `BufferNodeAllocatorTest.stress_free_list_allocator_vm` is too expensive. On my box, this test accumulates ~1.5 GB of malloc footprint and raises the libc memory retention from 50m to 800m. >> >> It is quite alone in its hunger. The rest of the tests together accumulate just ~50mb of libc retention. >> >> The buffer does a stress test of the BufferNodeAllocator. Four "mutator threads" race four "gc threads". Mutator threads allocate buffers, GC threads release them. No processing is done on the buffers; this seems to be purely a test of the allocator and its freelist mechanism. The memory footprint of this test depends on the number of retained free buffers in the allocator. The allocator bulk-releases free buffers triggered by a free-count threshold. On my box, the number of free items in the unmodified stock VM is 100k..200k. >> >> It is not clear whether the fact that so many free list items exist indicates a problem with the allocator itself. It looks like the free list should be drained if there are more than 10 items in this list. >> >> In any case, each buffer's capacity is 1024 * sizeof(void*)+header, so 8KB+x, and with 100..200k of those things it explains the NMT-reported malloc footprint of ~1..2GB. An easy fix is to reduce the size of this buffer; since they are not processed, their size should not matter. >> >> --------- >> >> The patch decreases the size per buffer node to cache line size. I chose cache line size out of the vague feeling that I don't want to cause false sharing and thereby degrade the test. >> >> Interestingly enough, reducing the buffer size greatly increases the number of free items since it makes allocation cheaper. This also happens in release builds, so this is not us zapping stuff; my unproven assumption is that this is the libc just being slower when allocating 8K vs 64-byte blocks. But with smaller buffers, we spend more time in freelist management and less time in the libc, which is good for a stress test. >> >> This patch also increases the number of "processor" threads vs "mutator" threads. Since allocation seems to have a speed edge over deallocation, this reduces the number of free items somewhat. >> >> This reduces the malloc spike seen from this thread from 1.5-2GB to ~160MB (release) 3xxMB (debug). We could reduce it a lot more if we reduced the buffer size to its minimum (1 slot); that would risk false sharing and may degrade test performance. > > test/hotspot/gtest/gc/shared/test_bufferNodeAllocator.cpp line 193: > >> 191: // free nodes (footprint) >> 192: const uint nthreads_mut = 4; >> 193: const uint nthreads_proc = 6; > > Naming them "num_processor_threads" and "num_allocator_threads" is more consistent with the corresponding types `ProcessorThread` and `AllocatorThread`, IMO. Okay ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16845#discussion_r1407675200 From kbarrett at openjdk.org Tue Nov 28 12:26:03 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Tue, 28 Nov 2023 12:26:03 GMT Subject: RFR: JDK-8320859: gtest high malloc footprint caused by BufferNodeAllocator stress test In-Reply-To: References: Message-ID: <7SibSGQ103DkKdyDzVuQLB2oeR1eZKZpsmQFiMysjRQ=.403e52b5-a125-455a-a888-4073ad3e4017@github.com> On Tue, 28 Nov 2023 11:02:24 GMT, Thomas Stuefe wrote: > `BufferNodeAllocatorTest.stress_free_list_allocator_vm` is too expensive. On my box, this test accumulates ~1.5 GB of malloc footprint and raises the libc memory retention from 50m to 800m. > > It is quite alone in its hunger. The rest of the tests together accumulate just ~50mb of libc retention. > > The buffer does a stress test of the BufferNodeAllocator. Four "mutator threads" race four "gc threads". Mutator threads allocate buffers, GC threads release them. No processing is done on the buffers; this seems to be purely a test of the allocator and its freelist mechanism. The memory footprint of this test depends on the number of retained free buffers in the allocator. The allocator bulk-releases free buffers triggered by a free-count threshold. On my box, the number of free items in the unmodified stock VM is 100k..200k. > > It is not clear whether the fact that so many free list items exist indicates a problem with the allocator itself. It looks like the free list should be drained if there are more than 10 items in this list. > > In any case, each buffer's capacity is 1024 * sizeof(void*)+header, so 8KB+x, and with 100..200k of those things it explains the NMT-reported malloc footprint of ~1..2GB. An easy fix is to reduce the size of this buffer; since they are not processed, their size should not matter. > > --------- > > The patch decreases the size per buffer node to cache line size. I chose cache line size out of the vague feeling that I don't want to cause false sharing and thereby degrade the test. > > Interestingly enough, reducing the buffer size greatly increases the number of free items since it makes allocation cheaper. This also happens in release builds, so this is not us zapping stuff; my unproven assumption is that this is the libc just being slower when allocating 8K vs 64-byte blocks. But with smaller buffers, we spend more time in freelist management and less time in the libc, which is good for a stress test. > > This patch also increases the number of "processor" threads vs "mutator" threads. Since allocation seems to have a speed edge over deallocation, this reduces the number of free items somewhat. > > This reduces the malloc spike seen from this thread from 1.5-2GB to ~160MB (release) 3xxMB (debug). We could reduce it a lot more if we reduced the buffer size to its minimum (1 slot); that would risk false sharing and may degrade test performance. Looks good. ------------- Marked as reviewed by kbarrett (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16845#pullrequestreview-1752759826 From stuefe at openjdk.org Tue Nov 28 12:54:23 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Tue, 28 Nov 2023 12:54:23 GMT Subject: RFR: JDK-8320859: gtest high malloc footprint caused by BufferNodeAllocator stress test [v2] In-Reply-To: References: Message-ID: > `BufferNodeAllocatorTest.stress_free_list_allocator_vm` is too expensive. On my box, this test accumulates ~1.5 GB of malloc footprint and raises the libc memory retention from 50m to 800m. > > It is quite alone in its hunger. The rest of the tests together accumulate just ~50mb of libc retention. > > The buffer does a stress test of the BufferNodeAllocator. Four "mutator threads" race four "gc threads". Mutator threads allocate buffers, GC threads release them. No processing is done on the buffers; this seems to be purely a test of the allocator and its freelist mechanism. The memory footprint of this test depends on the number of retained free buffers in the allocator. The allocator bulk-releases free buffers triggered by a free-count threshold. On my box, the number of free items in the unmodified stock VM is 100k..200k. > > It is not clear whether the fact that so many free list items exist indicates a problem with the allocator itself. It looks like the free list should be drained if there are more than 10 items in this list. > > In any case, each buffer's capacity is 1024 * sizeof(void*)+header, so 8KB+x, and with 100..200k of those things it explains the NMT-reported malloc footprint of ~1..2GB. An easy fix is to reduce the size of this buffer; since they are not processed, their size should not matter. > > --------- > > The patch decreases the size per buffer node to cache line size. I chose cache line size out of the vague feeling that I don't want to cause false sharing and thereby degrade the test. > > Interestingly enough, reducing the buffer size greatly increases the number of free items since it makes allocation cheaper. This also happens in release builds, so this is not us zapping stuff; my unproven assumption is that this is the libc just being slower when allocating 8K vs 64-byte blocks. But with smaller buffers, we spend more time in freelist management and less time in the libc, which is good for a stress test. > > This patch also increases the number of "processor" threads vs "mutator" threads. Since allocation seems to have a speed edge over deallocation, this reduces the number of free items somewhat. > > This reduces the malloc spike seen from this thread from 1.5-2GB to ~160MB (release) 3xxMB (debug). We could reduce it a lot more if we reduced the buffer size to its minimum (1 slot); that would risk false sharing and may degrade test performance. Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: Rename vars ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16845/files - new: https://git.openjdk.org/jdk/pull/16845/files/7a8a694f..255591ca Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16845&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16845&range=00-01 Stats: 9 lines in 1 file changed: 0 ins; 0 del; 9 mod Patch: https://git.openjdk.org/jdk/pull/16845.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16845/head:pull/16845 PR: https://git.openjdk.org/jdk/pull/16845 From stuefe at openjdk.org Tue Nov 28 12:54:24 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Tue, 28 Nov 2023 12:54:24 GMT Subject: RFR: JDK-8320859: gtest high malloc footprint caused by BufferNodeAllocator stress test [v2] In-Reply-To: <7SibSGQ103DkKdyDzVuQLB2oeR1eZKZpsmQFiMysjRQ=.403e52b5-a125-455a-a888-4073ad3e4017@github.com> References: <7SibSGQ103DkKdyDzVuQLB2oeR1eZKZpsmQFiMysjRQ=.403e52b5-a125-455a-a888-4073ad3e4017@github.com> Message-ID: On Tue, 28 Nov 2023 12:23:32 GMT, Kim Barrett wrote: >> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: >> >> Rename vars > > Looks good. Thanks @kimbarrett and @albertnetymk. I will wait the obligatory 24 hrs before pushing tomorrow. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16845#issuecomment-1829781564 From iwalulya at openjdk.org Tue Nov 28 14:56:00 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Tue, 28 Nov 2023 14:56:00 GMT Subject: RFR: 8320525: G1: G1UpdateRemSetTrackingBeforeRebuild::distribute_marked_bytes accesses partially unloaded klass [v2] In-Reply-To: References: Message-ID: On Fri, 24 Nov 2023 14:43:18 GMT, Thomas Schatzl wrote: >> Hi all, >> >> please review this fix that removes the access to a partially unloaded (i.e. unlinked only) `Klass` used for debug code in `G1UpdateRemSetTrackingBeforeRebuild::distribute_marked_bytes`. >> >> This starts to fail if metadata purging happens before the call to this methods (as https://bugs.openjdk.org/browse/JDK-8317809 suggests). The test gc/g1/humongousObjects/TestHumongousClassLoader.java starts to crash on linux-x86 with 100% reproduction because it more aggressively uncommits memory when purging metaspace. >> >> The fix fixes the asserts to only access the klass when it should not be unloaded yet. >> >> Testing: failing test case not failing any more, gha >> >> Thanks, >> Thomas > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > ayang review, improve documentation for assert. Marked as reviewed by iwalulya (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16766#pullrequestreview-1753103517 From shade at openjdk.org Tue Nov 28 15:46:17 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Tue, 28 Nov 2023 15:46:17 GMT Subject: RFR: 8320907: Shenandoah: Remove ShenandoahSelfFixing flag Message-ID: We added `ShenandoahSelfFixing` flag during LRB work to exercise LRB mid-paths better during stress tests. Today, we do not need the flag. Actually, if we ever call LRB for the side-effect of updating the location (e.g. somewhere in roots), then disabling this flag would introduce more bugs. Therefore, we need to remove the flag and make self-fixing unconditional. Additional testing: - [x] MacOS AArch64 server fastdebug, `hotspot_gc_shenandoah` ------------- Commit messages: - Fix Changes: https://git.openjdk.org/jdk/pull/16855/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16855&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8320907 Stats: 7 lines in 3 files changed: 0 ins; 5 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16855.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16855/head:pull/16855 PR: https://git.openjdk.org/jdk/pull/16855 From ayang at openjdk.org Tue Nov 28 15:57:53 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 28 Nov 2023 15:57:53 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v12] In-Reply-To: References: Message-ID: > Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. > > It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: review ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16492/files - new: https://git.openjdk.org/jdk/pull/16492/files/aeae83cd..a504b4bb Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=11 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=10-11 Stats: 23 lines in 1 file changed: 0 ins; 0 del; 23 mod Patch: https://git.openjdk.org/jdk/pull/16492.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16492/head:pull/16492 PR: https://git.openjdk.org/jdk/pull/16492 From wkemper at openjdk.org Tue Nov 28 16:31:04 2023 From: wkemper at openjdk.org (William Kemper) Date: Tue, 28 Nov 2023 16:31:04 GMT Subject: RFR: 8320907: Shenandoah: Remove ShenandoahSelfFixing flag In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 15:40:24 GMT, Aleksey Shipilev wrote: > We added `ShenandoahSelfFixing` flag during LRB work to exercise LRB mid-paths better during stress tests. Today, we do not need the flag. Actually, if we ever call LRB for the side-effect of updating the location (e.g. somewhere in roots), then disabling this flag would introduce more bugs. Therefore, we need to remove the flag and make self-fixing unconditional. > > Additional testing: > - [x] MacOS AArch64 server fastdebug, `hotspot_gc_shenandoah` Marked as reviewed by wkemper (Author). ------------- PR Review: https://git.openjdk.org/jdk/pull/16855#pullrequestreview-1753353777 From zgu at openjdk.org Tue Nov 28 16:50:09 2023 From: zgu at openjdk.org (Zhengyu Gu) Date: Tue, 28 Nov 2023 16:50:09 GMT Subject: RFR: 8320907: Shenandoah: Remove ShenandoahSelfFixing flag In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 15:40:24 GMT, Aleksey Shipilev wrote: > We added `ShenandoahSelfFixing` flag during LRB work to exercise LRB mid-paths better during stress tests. Today, we do not need the flag. Actually, if we ever call LRB for the side-effect of updating the location (e.g. somewhere in roots), then disabling this flag would introduce more bugs. Therefore, we need to remove the flag and make self-fixing unconditional. > > Additional testing: > - [x] MacOS AArch64 server fastdebug, `hotspot_gc_shenandoah` LGTM ------------- Marked as reviewed by zgu (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16855#pullrequestreview-1753394728 From kbarrett at openjdk.org Tue Nov 28 16:58:12 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Tue, 28 Nov 2023 16:58:12 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v12] In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 15:57:53 GMT, Albert Mingkun Yang wrote: >> Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. >> >> It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. > > Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: > > review Changes requested by kbarrett (Reviewer). src/hotspot/share/gc/serial/cardTableRS.hpp line 57: > 55: > 56: static CardValue* find_first_dirty_card(CardValue* const start_card, > 57: CardValue* const end_card); const qualifiers are noise here, having no effect. Similarly in find_first_clean_card. (And why isn't ct similarly qualified there?) The second const qualifiers in is_dirty/clean don't add much, and it's not typical usage, but at least they have an effect. The const qualifiers in the implementations of find_first_xxx are fine, its an implementation detail that those variables don't get modified. That implementation detail doesn't belong here though. src/hotspot/share/gc/serial/cardTableRS.hpp line 58: > 56: static CardValue* find_first_dirty_card(CardValue* const start_card, > 57: CardValue* const end_card); > 58: template Please add a blank line between these two function declarations. src/hotspot/share/gc/serial/cardTableRS.hpp line 62: > 60: CardValue* const end_card, > 61: CardTableRS* ct, > 62: Func&& object_start); Neither rvalue references nor universal references are permitted in HotSpot code (yet). Usual style is to take function arguments by value. The copy may not even matter (may be elided/optimized away), given this function is called once and potentially inlined. Maybe declare this function inline to encourage that? I'm waffling over rejecting this, since I think we ought to at least permit universal references, but that hasn't been proposed yet. ------------- PR Review: https://git.openjdk.org/jdk/pull/16492#pullrequestreview-1753267658 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1408000710 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1407999822 PR Review Comment: https://git.openjdk.org/jdk/pull/16492#discussion_r1408087620 From shade at openjdk.org Tue Nov 28 17:51:31 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Tue, 28 Nov 2023 17:51:31 GMT Subject: RFR: 8317755: G1: Periodic GC interval should test for the last whole heap GC [v3] In-Reply-To: References: Message-ID: > See the description in the bug. Fortunately, we already track the last whole-heap GC. The new regression test verifies the behavior. > > Additional testing: > - [x] Linux x86_64 fastdebug `tier1 tier2 tier3` Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision: - Merge branch 'master' into JDK-8317755-g1-periodic-whole - Merge branch 'master' into JDK-8317755-g1-periodic-whole - Optional flag - Merge branch 'master' into JDK-8317755-g1-periodic-whole - Keep the cast - Fix ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16107/files - new: https://git.openjdk.org/jdk/pull/16107/files/4025fd02..145e96b2 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16107&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16107&range=01-02 Stats: 714801 lines in 3525 files changed: 137625 ins; 497951 del; 79225 mod Patch: https://git.openjdk.org/jdk/pull/16107.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16107/head:pull/16107 PR: https://git.openjdk.org/jdk/pull/16107 From shade at openjdk.org Tue Nov 28 17:52:21 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Tue, 28 Nov 2023 17:52:21 GMT Subject: RFR: 8320877: Shenandoah: Remove ShenandoahUnloadClassesFrequency support Message-ID: Flag cleanup. `ShenandoahUnloadClassesFrequency` was added long time ago when Shenandoah did not support concurrent class unloading. We kept it in to simplify backports. At this point, most releases support concurrent class unloading, and there is no reason to keep the flag in. The default is "1" since [JDK-8228720](https://bugs.openjdk.org/browse/JDK-8228720). Additional testing: - [x] MacOS AArch64 server fastdebug, `hotspot_gc_shenandoah` ------------- Commit messages: - Revert incorrect cleanup - More cleanups - Fix Changes: https://git.openjdk.org/jdk/pull/16847/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16847&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8320877 Stats: 37 lines in 5 files changed: 0 ins; 34 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/16847.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16847/head:pull/16847 PR: https://git.openjdk.org/jdk/pull/16847 From wkemper at openjdk.org Tue Nov 28 17:52:21 2023 From: wkemper at openjdk.org (William Kemper) Date: Tue, 28 Nov 2023 17:52:21 GMT Subject: RFR: 8320877: Shenandoah: Remove ShenandoahUnloadClassesFrequency support In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 12:05:16 GMT, Aleksey Shipilev wrote: > Flag cleanup. `ShenandoahUnloadClassesFrequency` was added long time ago when Shenandoah did not support concurrent class unloading. We kept it in to simplify backports. At this point, most releases support concurrent class unloading, and there is no reason to keep the flag in. The default is "1" since [JDK-8228720](https://bugs.openjdk.org/browse/JDK-8228720). > > Additional testing: > - [x] MacOS AArch64 server fastdebug, `hotspot_gc_shenandoah` Marked as reviewed by wkemper (Author). ------------- PR Review: https://git.openjdk.org/jdk/pull/16847#pullrequestreview-1753348581 From ayang at openjdk.org Tue Nov 28 18:26:51 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 28 Nov 2023 18:26:51 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v13] In-Reply-To: References: Message-ID: > Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. > > It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 22 additional commits since the last revision: - review - Merge branch 'master' into s1-young-gc - review - Merge branch 'master' into s1-young-gc - review - review - Merge branch 'master' into s1-young-gc - review - next-obj-card - review - ... and 12 more: https://git.openjdk.org/jdk/compare/a8e5c20f...aba888cd ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16492/files - new: https://git.openjdk.org/jdk/pull/16492/files/a504b4bb..aba888cd Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=12 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16492&range=11-12 Stats: 16499 lines in 661 files changed: 11961 ins; 2619 del; 1919 mod Patch: https://git.openjdk.org/jdk/pull/16492.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16492/head:pull/16492 PR: https://git.openjdk.org/jdk/pull/16492 From kdnilsen at openjdk.org Tue Nov 28 18:30:05 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Tue, 28 Nov 2023 18:30:05 GMT Subject: RFR: 8320877: Shenandoah: Remove ShenandoahUnloadClassesFrequency support In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 12:05:16 GMT, Aleksey Shipilev wrote: > Flag cleanup. `ShenandoahUnloadClassesFrequency` was added long time ago when Shenandoah did not support concurrent class unloading. We kept it in to simplify backports. At this point, most releases support concurrent class unloading, and there is no reason to keep the flag in. The default is "1" since [JDK-8228720](https://bugs.openjdk.org/browse/JDK-8228720). > > Additional testing: > - [x] MacOS AArch64 server fastdebug, `hotspot_gc_shenandoah` Marked as reviewed by kdnilsen (no project role). ------------- PR Review: https://git.openjdk.org/jdk/pull/16847#pullrequestreview-1753605554 From kdnilsen at openjdk.org Tue Nov 28 18:31:13 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Tue, 28 Nov 2023 18:31:13 GMT Subject: RFR: 8320907: Shenandoah: Remove ShenandoahSelfFixing flag In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 15:40:24 GMT, Aleksey Shipilev wrote: > We added `ShenandoahSelfFixing` flag during LRB work to exercise LRB mid-paths better during stress tests. Today, we do not need the flag. Actually, if we ever call LRB for the side-effect of updating the location (e.g. somewhere in roots), then disabling this flag would introduce more bugs. Therefore, we need to remove the flag and make self-fixing unconditional. > > Additional testing: > - [x] MacOS AArch64 server fastdebug, `hotspot_gc_shenandoah` Marked as reviewed by kdnilsen (no project role). ------------- PR Review: https://git.openjdk.org/jdk/pull/16855#pullrequestreview-1753617205 From tschatzl at openjdk.org Tue Nov 28 20:58:17 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 28 Nov 2023 20:58:17 GMT Subject: RFR: 8320525: G1: G1UpdateRemSetTrackingBeforeRebuild::distribute_marked_bytes accesses partially unloaded klass [v2] In-Reply-To: References: Message-ID: On Fri, 24 Nov 2023 15:02:41 GMT, Albert Mingkun Yang wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> ayang review, improve documentation for assert. > >> this is going to change anyway and is imho a more appropriate time to reconsider this (and will probably naturally fix itself). > > OK. > >> What is an "effective-region" in this context? > > I meant effectively-empty region. Thanks @albertnetymk @walulyai for your reviews ------------- PR Comment: https://git.openjdk.org/jdk/pull/16766#issuecomment-1830718070 From tschatzl at openjdk.org Tue Nov 28 20:58:19 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 28 Nov 2023 20:58:19 GMT Subject: Integrated: 8320525: G1: G1UpdateRemSetTrackingBeforeRebuild::distribute_marked_bytes accesses partially unloaded klass In-Reply-To: References: Message-ID: <6zI7Jm4wHWbw7qVQskhmgeVMo8ZK5D_aEwAEtEJguyQ=.70e20c84-c159-47e2-ba73-c83eade394ec@github.com> On Tue, 21 Nov 2023 16:01:39 GMT, Thomas Schatzl wrote: > Hi all, > > please review this fix that removes the access to a partially unloaded (i.e. unlinked only) `Klass` used for debug code in `G1UpdateRemSetTrackingBeforeRebuild::distribute_marked_bytes`. > > This starts to fail if metadata purging happens before the call to this methods (as https://bugs.openjdk.org/browse/JDK-8317809 suggests). The test gc/g1/humongousObjects/TestHumongousClassLoader.java starts to crash on linux-x86 with 100% reproduction because it more aggressively uncommits memory when purging metaspace. > > The fix fixes the asserts to only access the klass when it should not be unloaded yet. > > Testing: failing test case not failing any more, gha > > Thanks, > Thomas This pull request has now been integrated. Changeset: 21d361e9 Author: Thomas Schatzl URL: https://git.openjdk.org/jdk/commit/21d361e944a599e82e5ed93aa81ad485b4601843 Stats: 10 lines in 1 file changed: 0 ins; 7 del; 3 mod 8320525: G1: G1UpdateRemSetTrackingBeforeRebuild::distribute_marked_bytes accesses partially unloaded klass Reviewed-by: ayang, iwalulya ------------- PR: https://git.openjdk.org/jdk/pull/16766 From kbarrett at openjdk.org Tue Nov 28 21:20:13 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Tue, 28 Nov 2023 21:20:13 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v13] In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 18:26:51 GMT, Albert Mingkun Yang wrote: >> Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. >> >> It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. > > Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 22 additional commits since the last revision: > > - review > - Merge branch 'master' into s1-young-gc > - review > - Merge branch 'master' into s1-young-gc > - review > - review > - Merge branch 'master' into s1-young-gc > - review > - next-obj-card > - review > - ... and 12 more: https://git.openjdk.org/jdk/compare/7e4045cd...aba888cd Looks good. ------------- Marked as reviewed by kbarrett (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16492#pullrequestreview-1753964997 From duke at openjdk.org Wed Nov 29 05:03:23 2023 From: duke at openjdk.org (duke) Date: Wed, 29 Nov 2023 05:03:23 GMT Subject: Withdrawn: 8308507: G1: GClocker induced GCs can starve threads requiring memory leading to OOME In-Reply-To: References: Message-ID: On Mon, 22 May 2023 11:44:24 GMT, Ivan Walulya wrote: > Please review this change which fixes the thread starvation problem during allocation for G1. > > The starvation problem is not limited to GCLocker, however, currently, it manifests as an OOME only when GCLocker is active. In other cases, the starvation only affects the "starved" thread as it may loop indefinitely. > > Starvation with an active GCLocker happens as below: > > 1. Thread A tries to allocate memory as normal, and tries to start a GC; the GCLocker is active and so the thread gets stalled waiting for the GC. > 2. GCLocker induced GC executes and frees some memory. > 3. Thread A does not get any of that memory, but other threads also waiting for memory. > 4. Goto 1 until the gclocker retry count has been reached. > > In this change, we take the general approach to solving starvation problems with announcement tables (request queues). On slow allocation, a thread that wishes to complete an Allocation GC and then attempt an allocation, announces its allocation request before proceeding to participate in a race to execute a GC safepoint. Whichever thread succeeds in executing the Allocation GC safepoint will be tasked with completing all allocation requests that were announced before the safepoint. This guarantees that all announced allocation requests are either satisfied during the safepoint, or failed in case there is not enough memory to complete all requests. This effectively deals with the starvation issue and reduces the number of allocation GCs triggered. > > Note: The change also adopts ZList from ZGC and makes it available under utilities as DoublyLinkedList with slight modifications. > > Testing: Tier 1-7 This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/jdk/pull/14077 From sjohanss at openjdk.org Wed Nov 29 08:36:25 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Wed, 29 Nov 2023 08:36:25 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v48] In-Reply-To: <_lEBVrWV8wrVbmhOiu3AAqPJo_xBs718ZtA9V-VSzGM=.253c0ec8-256e-4dee-b125-90be6338e4b8@github.com> References: <_lEBVrWV8wrVbmhOiu3AAqPJo_xBs718ZtA9V-VSzGM=.253c0ec8-256e-4dee-b125-90be6338e4b8@github.com> Message-ID: On Tue, 28 Nov 2023 02:22:45 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: > > - Fix namespace issues (2) > > Co-authored-by: Stefan Johansson <54407259+kstefanj at users.noreply.github.com> > - Fix namespace issues > > Co-authored-by: Stefan Johansson <54407259+kstefanj at users.noreply.github.com> >From a testing point of view I think this is looking good now, re-ran the failing test and some other jstat tests as well and they all pass. Please address the comments from Albert and we can hopefully finish this before RDP1. src/hotspot/share/runtime/cpuTimeCounters.cpp line 91: > 89: } while (old_value != fetched_value); > 90: get_counter(CPUTimeGroups::CPUTimeType::gc_total)->inc(fetched_value); > 91: } Why do we have to do this publish dance? Couldn't the closure that update the diff instead just update the counter? From what I can tell we never have multiple closures active at the same time so should be no race there? ------------- PR Review: https://git.openjdk.org/jdk/pull/15082#pullrequestreview-1754686415 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1408914724 From sjohanss at openjdk.org Wed Nov 29 08:36:28 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Wed, 29 Nov 2023 08:36:28 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v47] In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 13:46:54 GMT, Albert Mingkun Yang wrote: >> Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: >> >> Cleanup and address comments > > src/hotspot/share/runtime/cpuTimeCounters.cpp line 119: > >> 117: if (CPUTimeGroups::is_gc_counter(_name)) { >> 118: instance->inc_gc_total_cpu_time(net_cpu_time); >> 119: } > > I feel much of this is on the wrong abstraction level; `CPUTimeCounters::update_counter(_name, _total);` should be sufficient here. (The caller handles diff calculation and inc gc-counter if needed.) We could add a new closure just used by GC that 's a sub-class of `ThreadTotalCPUTimeClosure` and just adds this to the constructor: instance->inc_gc_total_cpu_time(net_cpu_time); That way we could get rid of `CPUTimeGroups::is_gc_counter()` as well since all those counters should use the "GC closure" or we can keep it and assert that no GC closure uses the wrong closure. What do you think about that Albert, would that address your concerns? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1408923331 From tschatzl at openjdk.org Wed Nov 29 10:06:27 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Wed, 29 Nov 2023 10:06:27 GMT Subject: RFR: 8318706: Implement JEP 423: Region Pinning for G1 [v10] In-Reply-To: <1-i3-5OmZbuCNUlpfv31Kr3eiBXEd4Si8F5gsbPHuBQ=.1d97dcac-4662-4482-842c-ce86315ba61a@github.com> References: <1-i3-5OmZbuCNUlpfv31Kr3eiBXEd4Si8F5gsbPHuBQ=.1d97dcac-4662-4482-842c-ce86315ba61a@github.com> Message-ID: On Fri, 3 Nov 2023 14:14:50 GMT, Albert Mingkun Yang wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> typos > > Marked as reviewed by ayang (Reviewer). Thanks @albertnetymk @kstefanj @walulyai for your reviews! Given that the JEP is now targeted, I will integrate. This has been a fairly long journey until today... :) ------------- PR Comment: https://git.openjdk.org/jdk/pull/16342#issuecomment-1831581034 From tschatzl at openjdk.org Wed Nov 29 10:06:30 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Wed, 29 Nov 2023 10:06:30 GMT Subject: Integrated: 8318706: Implement JEP 423: Region Pinning for G1 In-Reply-To: References: Message-ID: On Tue, 24 Oct 2023 09:56:57 GMT, Thomas Schatzl wrote: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... This pull request has now been integrated. Changeset: 38cfb220 Author: Thomas Schatzl URL: https://git.openjdk.org/jdk/commit/38cfb220ddadbb401cc15f313aadb8234f626210 Stats: 1823 lines in 59 files changed: 1150 ins; 435 del; 238 mod 8318706: Implement JEP 423: Region Pinning for G1 Reviewed-by: ayang, iwalulya, sjohanss ------------- PR: https://git.openjdk.org/jdk/pull/16342 From ayang at openjdk.org Wed Nov 29 10:31:22 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 29 Nov 2023 10:31:22 GMT Subject: RFR: 8319373: Serial: Refactor dirty cards scanning during Young GC [v13] In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 18:26:51 GMT, Albert Mingkun Yang wrote: >> Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. >> >> It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. > > Albert Mingkun Yang has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 22 additional commits since the last revision: > > - review > - Merge branch 'master' into s1-young-gc > - review > - Merge branch 'master' into s1-young-gc > - review > - review > - Merge branch 'master' into s1-young-gc > - review > - next-obj-card > - review > - ... and 12 more: https://git.openjdk.org/jdk/compare/9e50b0b4...aba888cd Thanks for the review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16492#issuecomment-1831618504 From ayang at openjdk.org Wed Nov 29 10:31:23 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 29 Nov 2023 10:31:23 GMT Subject: Integrated: 8319373: Serial: Refactor dirty cards scanning during Young GC In-Reply-To: References: Message-ID: On Fri, 3 Nov 2023 12:14:11 GMT, Albert Mingkun Yang wrote: > Reading the new code directly is probably easier. The structure is quite similar to its counterpart in Parallel. > > It's mostly perf-neutral, except when dirty cards are scarce. Using `card_scan.java` in [JDK-8310031](https://bugs.openjdk.org/browse/JDK-8310031), I observed ~40% reduction in young-gc pause time with `stride = 32 * 64`. This pull request has now been integrated. Changeset: 77d604a2 Author: Albert Mingkun Yang URL: https://git.openjdk.org/jdk/commit/77d604a29f486b62db53be3d56b921be46e56fd9 Stats: 470 lines in 4 files changed: 186 ins; 269 del; 15 mod 8319373: Serial: Refactor dirty cards scanning during Young GC Reviewed-by: kbarrett, tschatzl ------------- PR: https://git.openjdk.org/jdk/pull/16492 From shade at openjdk.org Wed Nov 29 13:04:15 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 29 Nov 2023 13:04:15 GMT Subject: RFR: 8312023: Parallel pretouch should shortcut when only 1 thread is needed Message-ID: This is a follow-up for code introduced in JDK 16 by [JDK-8252221](https://bugs.openjdk.org/browse/JDK-8252221). Unfortunately, we do pre-touch not only at startup, but also during the application lifetime when space boundary moves, see [JDK-8312021](https://bugs.openjdk.org/browse/JDK-8312021). From that code, we start the pre-touch threads unconditionally, even when we only need one thread anyway. This is clearly visible with `-Xlog:gc+heap=debug`. The actual pretouch takes single-digit microseconds, but the round-trip through the worker pool incurs latency of about 100us (2x 50us) in my setups. This is significant for very short GC pauses. Additional testing: - [ ] Ad-hoc benchmarks - [ ] Linux x86_64 server fastdebug, `tier{1,2,3}` with `-XX:+UseParallelGC -XX:+AlwaysPreTouch` ------------- Commit messages: - Work Changes: https://git.openjdk.org/jdk/pull/16882/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16882&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8312023 Stats: 14 lines in 1 file changed: 8 ins; 2 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/16882.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16882/head:pull/16882 PR: https://git.openjdk.org/jdk/pull/16882 From stuefe at openjdk.org Wed Nov 29 13:21:23 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 29 Nov 2023 13:21:23 GMT Subject: Integrated: JDK-8320859: gtest high malloc footprint caused by BufferNodeAllocator stress test In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 11:02:24 GMT, Thomas Stuefe wrote: > `BufferNodeAllocatorTest.stress_free_list_allocator_vm` is too expensive. On my box, this test accumulates ~1.5 GB of malloc footprint and raises the libc memory retention from 50m to 800m. > > It is quite alone in its hunger. The rest of the tests together accumulate just ~50mb of libc retention. > > The buffer does a stress test of the BufferNodeAllocator. Four "mutator threads" race four "gc threads". Mutator threads allocate buffers, GC threads release them. No processing is done on the buffers; this seems to be purely a test of the allocator and its freelist mechanism. The memory footprint of this test depends on the number of retained free buffers in the allocator. The allocator bulk-releases free buffers triggered by a free-count threshold. On my box, the number of free items in the unmodified stock VM is 100k..200k. > > It is not clear whether the fact that so many free list items exist indicates a problem with the allocator itself. It looks like the free list should be drained if there are more than 10 items in this list. > > In any case, each buffer's capacity is 1024 * sizeof(void*)+header, so 8KB+x, and with 100..200k of those things it explains the NMT-reported malloc footprint of ~1..2GB. An easy fix is to reduce the size of this buffer; since they are not processed, their size should not matter. > > --------- > > The patch decreases the size per buffer node to cache line size. I chose cache line size out of the vague feeling that I don't want to cause false sharing and thereby degrade the test. > > Interestingly enough, reducing the buffer size greatly increases the number of free items since it makes allocation cheaper. This also happens in release builds, so this is not us zapping stuff; my unproven assumption is that this is the libc just being slower when allocating 8K vs 64-byte blocks. But with smaller buffers, we spend more time in freelist management and less time in the libc, which is good for a stress test. > > This patch also increases the number of "processor" threads vs "mutator" threads. Since allocation seems to have a speed edge over deallocation, this reduces the number of free items somewhat. > > This reduces the malloc spike seen from this thread from 1.5-2GB to ~160MB (release) 3xxMB (debug). We could reduce it a lot more if we reduced the buffer size to its minimum (1 slot); that would risk false sharing and may degrade test performance. This pull request has now been integrated. Changeset: abf2e490 Author: Thomas Stuefe URL: https://git.openjdk.org/jdk/commit/abf2e4909733550c228bd66ccebe33a492269e7f Stats: 14 lines in 1 file changed: 5 ins; 0 del; 9 mod 8320859: gtest high malloc footprint caused by BufferNodeAllocator stress test Reviewed-by: ayang, kbarrett ------------- PR: https://git.openjdk.org/jdk/pull/16845 From ayang at openjdk.org Wed Nov 29 14:59:23 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 29 Nov 2023 14:59:23 GMT Subject: RFR: 8321013: Parallel: Refactor ObjectStartArray Message-ID: <2D-p1OeMo-tVqBfZ0e_e-PF8sZR38VpltNckqRvXK5k=.0765bbf4-62a0-4167-8277-52c9a939ff67@github.com> Port the block-offset-table algorithm from Serial/G1 to Parallel. There is ~20% reduction in young-gc pause, using card_scan.java from https://bugs.openjdk.org/browse/JDK-8310031 ------------- Commit messages: - pgc-bot Changes: https://git.openjdk.org/jdk/pull/16884/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16884&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8321013 Stats: 319 lines in 11 files changed: 67 ins; 167 del; 85 mod Patch: https://git.openjdk.org/jdk/pull/16884.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16884/head:pull/16884 PR: https://git.openjdk.org/jdk/pull/16884 From ayang at openjdk.org Wed Nov 29 15:05:42 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 29 Nov 2023 15:05:42 GMT Subject: RFR: 8321018: Parallel: Make some methods in ParCompactionManager private Message-ID: Trivial reducing visibility of some methods. ------------- Commit messages: - pgc-private-field Changes: https://git.openjdk.org/jdk/pull/16885/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16885&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8321018 Stats: 3 lines in 1 file changed: 1 ins; 1 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16885.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16885/head:pull/16885 PR: https://git.openjdk.org/jdk/pull/16885 From ayang at openjdk.org Wed Nov 29 15:27:52 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 29 Nov 2023 15:27:52 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v48] In-Reply-To: References: <_lEBVrWV8wrVbmhOiu3AAqPJo_xBs718ZtA9V-VSzGM=.253c0ec8-256e-4dee-b125-90be6338e4b8@github.com> Message-ID: On Wed, 29 Nov 2023 08:24:22 GMT, Stefan Johansson wrote: >> Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: >> >> - Fix namespace issues (2) >> >> Co-authored-by: Stefan Johansson <54407259+kstefanj at users.noreply.github.com> >> - Fix namespace issues >> >> Co-authored-by: Stefan Johansson <54407259+kstefanj at users.noreply.github.com> > > src/hotspot/share/runtime/cpuTimeCounters.cpp line 91: > >> 89: } while (old_value != fetched_value); >> 90: get_counter(CPUTimeGroups::CPUTimeType::gc_total)->inc(fetched_value); >> 91: } > > Why do we have to do this publish dance? Couldn't the closure that update the diff instead just update the counter? From what I can tell we never have multiple closures active at the same time so should be no race there? This two-step update does seem unnecessary, IMO. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1409452326 From ayang at openjdk.org Wed Nov 29 15:27:55 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 29 Nov 2023 15:27:55 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v47] In-Reply-To: References: Message-ID: On Wed, 29 Nov 2023 08:31:37 GMT, Stefan Johansson wrote: >> src/hotspot/share/runtime/cpuTimeCounters.cpp line 119: >> >>> 117: if (CPUTimeGroups::is_gc_counter(_name)) { >>> 118: instance->inc_gc_total_cpu_time(net_cpu_time); >>> 119: } >> >> I feel much of this is on the wrong abstraction level; `CPUTimeCounters::update_counter(_name, _total);` should be sufficient here. (The ~caller~ callee handles diff calculation and inc gc-counter if needed.) > > We could add a new closure just used by GC that 's a sub-class of `ThreadTotalCPUTimeClosure` and just adds this to the constructor: > > instance->inc_gc_total_cpu_time(net_cpu_time); > > > That way we could get rid of `CPUTimeGroups::is_gc_counter()` as well since all those counters should use the "GC closure" or we can keep it and assert that no GC closure uses the wrong closure. > > What do you think about that Albert, would that address your concerns? (I just realized that I made a typo in my previous msg; should be *callee* instead.) That is what I have in mind. void CPUTimeCounters::update_counter(name, total) { auto counter = get_counter(name); auto old_v = counter->get_value(); auto diff = total - old_v; counter->inc(diff); if (counter->is_gc_counter()) { counter->inc(diff); } } ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1409450168 From sjohanss at openjdk.org Wed Nov 29 16:02:18 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Wed, 29 Nov 2023 16:02:18 GMT Subject: RFR: 8320916: jdk/jfr/event/gc/stacktrace/TestParallelMarkSweepAllocationPendingStackTrace.java failed with "OutOfMemoryError: GC overhead limit exceeded" In-Reply-To: References: Message-ID: On Wed, 29 Nov 2023 00:35:47 GMT, Albert Mingkun Yang wrote: > Simple fix to reduce live set so that after the triggered full-gc, there is still some memory left. > > Test: ~2/10 failure before the fix and no failure observed for 100 iterations. Looks good. ------------- Marked as reviewed by sjohanss (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16870#pullrequestreview-1755634055 From shade at openjdk.org Wed Nov 29 17:03:23 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 29 Nov 2023 17:03:23 GMT Subject: RFR: 8320969: Shenandoah: Enforce stable number of GC workers Message-ID: <54bP7p1EQWWE8YXN94AlKVabtJUOFpw-wZ7fXOmxG_0=.7ad5f3d4-fe36-4ea3-8e38-0561020cc043@github.com> See bug for rationale. Additional testing: - [x] MacOS AArch64 server fastdebug, `hotspot_gc_shenandoah` - [x] Linux x86_64 server fastdebug, `tier{1,2,3}` with `-XX:+UseShenandoahGC` - [x] Linux AArch64 server fastdebug, `tier{1,2,3}` with `-XX:+UseShenandoahGC` ------------- Commit messages: - No dynamic GC workers Changes: https://git.openjdk.org/jdk/pull/16878/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16878&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8320969 Stats: 206 lines in 5 files changed: 10 ins; 175 del; 21 mod Patch: https://git.openjdk.org/jdk/pull/16878.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16878/head:pull/16878 PR: https://git.openjdk.org/jdk/pull/16878 From rkennke at openjdk.org Wed Nov 29 17:18:07 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Wed, 29 Nov 2023 17:18:07 GMT Subject: RFR: 8320877: Shenandoah: Remove ShenandoahUnloadClassesFrequency support In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 12:05:16 GMT, Aleksey Shipilev wrote: > Flag cleanup. `ShenandoahUnloadClassesFrequency` was added long time ago when Shenandoah did not support concurrent class unloading. We kept it in to simplify backports. At this point, most releases support concurrent class unloading, and there is no reason to keep the flag in. The default is "1" since [JDK-8228720](https://bugs.openjdk.org/browse/JDK-8228720). > > Additional testing: > - [x] MacOS AArch64 server fastdebug, `hotspot_gc_shenandoah` Makes sense and looks good, thank you! ------------- Marked as reviewed by rkennke (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16847#pullrequestreview-1755822674 From rkennke at openjdk.org Wed Nov 29 17:19:04 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Wed, 29 Nov 2023 17:19:04 GMT Subject: RFR: 8320907: Shenandoah: Remove ShenandoahSelfFixing flag In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 15:40:24 GMT, Aleksey Shipilev wrote: > We added `ShenandoahSelfFixing` flag during LRB work to exercise LRB mid-paths better during stress tests. Today, we do not need the flag. Actually, if we ever call LRB for the side-effect of updating the location (e.g. somewhere in roots), then disabling this flag would introduce more bugs. Therefore, we need to remove the flag and make self-fixing unconditional. > > Additional testing: > - [x] MacOS AArch64 server fastdebug, `hotspot_gc_shenandoah` Looks good, thank you! ------------- Marked as reviewed by rkennke (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16855#pullrequestreview-1755824941 From shade at openjdk.org Wed Nov 29 17:25:15 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 29 Nov 2023 17:25:15 GMT Subject: RFR: 8320907: Shenandoah: Remove ShenandoahSelfFixing flag In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 15:40:24 GMT, Aleksey Shipilev wrote: > We added `ShenandoahSelfFixing` flag during LRB work to exercise LRB mid-paths better during stress tests. Today, we do not need the flag. Actually, if we ever call LRB for the side-effect of updating the location (e.g. somewhere in roots), then disabling this flag would introduce more bugs. Therefore, we need to remove the flag and make self-fixing unconditional. > > Additional testing: > - [x] MacOS AArch64 server fastdebug, `hotspot_gc_shenandoah` Thank you all! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16855#issuecomment-1832378026 From shade at openjdk.org Wed Nov 29 17:25:16 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 29 Nov 2023 17:25:16 GMT Subject: Integrated: 8320907: Shenandoah: Remove ShenandoahSelfFixing flag In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 15:40:24 GMT, Aleksey Shipilev wrote: > We added `ShenandoahSelfFixing` flag during LRB work to exercise LRB mid-paths better during stress tests. Today, we do not need the flag. Actually, if we ever call LRB for the side-effect of updating the location (e.g. somewhere in roots), then disabling this flag would introduce more bugs. Therefore, we need to remove the flag and make self-fixing unconditional. > > Additional testing: > - [x] MacOS AArch64 server fastdebug, `hotspot_gc_shenandoah` This pull request has now been integrated. Changeset: 43c7f6a6 Author: Aleksey Shipilev URL: https://git.openjdk.org/jdk/commit/43c7f6a673c2fa0b4dbec232e92b621619a98246 Stats: 7 lines in 3 files changed: 0 ins; 5 del; 2 mod 8320907: Shenandoah: Remove ShenandoahSelfFixing flag Reviewed-by: wkemper, zgu, kdnilsen, rkennke ------------- PR: https://git.openjdk.org/jdk/pull/16855 From shade at openjdk.org Wed Nov 29 17:28:24 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 29 Nov 2023 17:28:24 GMT Subject: RFR: 8320877: Shenandoah: Remove ShenandoahUnloadClassesFrequency support In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 12:05:16 GMT, Aleksey Shipilev wrote: > Flag cleanup. `ShenandoahUnloadClassesFrequency` was added long time ago when Shenandoah did not support concurrent class unloading. We kept it in to simplify backports. At this point, most releases support concurrent class unloading, and there is no reason to keep the flag in. The default is "1" since [JDK-8228720](https://bugs.openjdk.org/browse/JDK-8228720). > > Additional testing: > - [x] MacOS AArch64 server fastdebug, `hotspot_gc_shenandoah` Thanks! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16847#issuecomment-1832380194 From shade at openjdk.org Wed Nov 29 17:28:25 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 29 Nov 2023 17:28:25 GMT Subject: Integrated: 8320877: Shenandoah: Remove ShenandoahUnloadClassesFrequency support In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 12:05:16 GMT, Aleksey Shipilev wrote: > Flag cleanup. `ShenandoahUnloadClassesFrequency` was added long time ago when Shenandoah did not support concurrent class unloading. We kept it in to simplify backports. At this point, most releases support concurrent class unloading, and there is no reason to keep the flag in. The default is "1" since [JDK-8228720](https://bugs.openjdk.org/browse/JDK-8228720). > > Additional testing: > - [x] MacOS AArch64 server fastdebug, `hotspot_gc_shenandoah` This pull request has now been integrated. Changeset: b65ccff3 Author: Aleksey Shipilev URL: https://git.openjdk.org/jdk/commit/b65ccff357e2e294b027f693ceb3d25410236a6b Stats: 37 lines in 5 files changed: 0 ins; 34 del; 3 mod 8320877: Shenandoah: Remove ShenandoahUnloadClassesFrequency support Reviewed-by: wkemper, kdnilsen, rkennke ------------- PR: https://git.openjdk.org/jdk/pull/16847 From ysr at openjdk.org Wed Nov 29 17:31:05 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Wed, 29 Nov 2023 17:31:05 GMT Subject: RFR: 8320969: Shenandoah: Enforce stable number of GC workers In-Reply-To: <54bP7p1EQWWE8YXN94AlKVabtJUOFpw-wZ7fXOmxG_0=.7ad5f3d4-fe36-4ea3-8e38-0561020cc043@github.com> References: <54bP7p1EQWWE8YXN94AlKVabtJUOFpw-wZ7fXOmxG_0=.7ad5f3d4-fe36-4ea3-8e38-0561020cc043@github.com> Message-ID: On Wed, 29 Nov 2023 11:01:32 GMT, Aleksey Shipilev wrote: > See bug for rationale. > > Additional testing: > - [x] MacOS AArch64 server fastdebug, `hotspot_gc_shenandoah` > - [x] Linux x86_64 server fastdebug, `tier{1,2,3}` with `-XX:+UseShenandoahGC` > - [x] Linux AArch64 server fastdebug, `tier{1,2,3}` with `-XX:+UseShenandoahGC` @shipilev can you share any performance numbers with this change? The infra to modify the number of threads per phase would seem to be useful for adaptively changing the number of threads, specializing the per-GC/collector/phase policy for doing so, instead of removing it entirely. The problem you point out with the existing implementation was that it deferred completely to the shared policy; couldn't we just specialize that for Shenandoah or Generational Shenandoah (and for now just fix the number of workers at the values you chose here, but allow this to be more intelligently adapted in the future)? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16878#issuecomment-1832388065 From shade at openjdk.org Wed Nov 29 20:16:31 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 29 Nov 2023 20:16:31 GMT Subject: RFR: 8312023: Parallel pretouch should shortcut when only 1 thread is needed [v2] In-Reply-To: References: Message-ID: > This is a follow-up for code introduced in JDK 16 by [JDK-8252221](https://bugs.openjdk.org/browse/JDK-8252221). > > Unfortunately, we do pre-touch not only at startup, but also during the application lifetime when space boundary moves, see [JDK-8312021](https://bugs.openjdk.org/browse/JDK-8312021). From that code, we start the pre-touch threads unconditionally, even when we only need one thread anyway. This is clearly visible with `-Xlog:gc+heap=debug`. The actual pretouch takes single-digit microseconds, but the round-trip through the worker pool incurs latency of about 100us (2x 50us) in my setups. This is significant for very short GC pauses. > > Additional testing: > - [x] Ad-hoc benchmarks > - [ ] Linux x86_64 server fastdebug, `tier{1,2,3}` with `-XX:+UseParallelGC -XX:+AlwaysPreTouch` Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: Restore calculation, hoping to fix Windows build ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16882/files - new: https://git.openjdk.org/jdk/pull/16882/files/fc0d4c33..bfe2cfe4 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16882&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16882&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16882.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16882/head:pull/16882 PR: https://git.openjdk.org/jdk/pull/16882 From bchristi at openjdk.org Wed Nov 29 23:42:06 2023 From: bchristi at openjdk.org (Brent Christian) Date: Wed, 29 Nov 2023 23:42:06 GMT Subject: RFR: 8314480: Memory ordering spec updates in java.lang.ref In-Reply-To: References: Message-ID: On Tue, 21 Nov 2023 09:12:49 GMT, Viktor Klang wrote: >> Classes in the `java.lang.ref` package would benefit from an update to bring the spec in line with how the VM already behaves. The changes would focus on _happens-before_ edges at some key points during reference processing. >> >> A couple key things we want to be able to say are: >> - `Reference.reachabilityFence(x)` _happens-before_ reference processing occurs for 'x'. >> - `Cleaner.register()` _happens-before_ the Cleaner thread runs the registered cleaning action. >> >> This will bring Cleaner in line (or close) with the memory visibility guarantees made for finalizers in [JLS 17.4.5](https://docs.oracle.com/javase/specs/jls/se18/html/jls-17.html#jls-17.4.5): >> _"There is a happens-before edge from the end of a constructor of an object to the start of a finalizer (?12.6) for that object."_ > > src/java.base/share/classes/java/lang/ref/Reference.java line 497: > >> 495: * or {@link ReferenceQueue#remove}. >> 496: * >> 497: *

This method is invoked only by Java code; when the garbage collector > > Perhaps something along the lines of: > > Suggestion: > > *

This method is only invoked explicitly; when the garbage collector I think the important part to convey is that the GC won't call this method. So how about _just_:

When the garbage collector enqueues references it does so directly, without invoking this method. ? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16644#discussion_r1409993103 From jjoo at openjdk.org Thu Nov 30 00:06:18 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Thu, 30 Nov 2023 00:06:18 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v47] In-Reply-To: References: Message-ID: <4CkUYwnpxAAaKyV86ybEHbzNJw-7yzaU4X2CJsmD4MQ=.4d5cc389-6009-4e01-9a79-5044fe011bfe@github.com> On Thu, 23 Nov 2023 12:43:00 GMT, Albert Mingkun Yang wrote: >> Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: >> >> Cleanup and address comments > > src/hotspot/share/gc/g1/g1CollectedHeap.cpp line 2433: > >> 2431: } >> 2432: WorkerThreads* worker_threads = workers(); >> 2433: if (worker_threads != nullptr) { > > When will this be null? I guess it shouldn't be, since it is set when we call initialize() and also never unset. I'll remove this conditional. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1410005989 From bchristi at openjdk.org Thu Nov 30 00:53:04 2023 From: bchristi at openjdk.org (Brent Christian) Date: Thu, 30 Nov 2023 00:53:04 GMT Subject: RFR: 8314480: Memory ordering spec updates in java.lang.ref In-Reply-To: References: Message-ID: On Tue, 21 Nov 2023 22:46:32 GMT, Roger Riggs wrote: >> src/java.base/share/classes/java/lang/ref/Reference.java line 489: >> >>> 487: *

If this reference was already enqueued (by the garbage collector, or a >>> 488: * previous call to {@code enqueue}), this method is not successful, >>> 489: * and returns false. >> >> If we're going to talk about successful and unsuccessful calls, we should define what success is first. I guess that would be something like: if this ref is registered with a queue and not already enqueued, it is enqueued successfully and the method returns true. Otherwise, not successful, and returns false. > > This could be worded as a post condition, after the call the ref is enqueued with the queue; the return is true iff the ref was not previously enqueued. > The enqueuing is not conditional (assuming the queue is non-null). I'll give that some thought. The enqueuing is not conditional, but the _happens-before_ is... ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16644#discussion_r1410031119 From jjoo at openjdk.org Thu Nov 30 01:48:47 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Thu, 30 Nov 2023 01:48:47 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v49] In-Reply-To: References: Message-ID: <5zh0f9THaoeVoSuLRa8Poxm9ex9jnLpBcyK5JoEIEHQ=.c00b21e1-69ab-424b-a708-be2962b3f4ed@github.com> > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: Change APIs to be all-static, address other comments ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/abb90258..91100e81 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=48 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=47-48 Stats: 80 lines in 6 files changed: 21 ins; 21 del; 38 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Thu Nov 30 01:48:47 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Thu, 30 Nov 2023 01:48:47 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v47] In-Reply-To: References: Message-ID: On Wed, 29 Nov 2023 15:23:55 GMT, Albert Mingkun Yang wrote: >> We could add a new closure just used by GC that 's a sub-class of `ThreadTotalCPUTimeClosure` and just adds this to the constructor: >> >> instance->inc_gc_total_cpu_time(net_cpu_time); >> >> >> That way we could get rid of `CPUTimeGroups::is_gc_counter()` as well since all those counters should use the "GC closure" or we can keep it and assert that no GC closure uses the wrong closure. >> >> What do you think about that Albert, would that address your concerns? > > (I just realized that I made a typo in my previous msg; should be *callee* instead.) That is what I have in mind. > > > void CPUTimeCounters::update_counter(name, total) { > auto counter = get_counter(name); > auto old_v = counter->get_value(); > auto diff = total - old_v; > counter->inc(diff); > if (counter->is_gc_counter()) { > counter->inc(diff); > } > } I'm not sure I understood correctly, could you let me know if this latest commit addresses your comment in the way you were intending? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1410059322 From jjoo at openjdk.org Thu Nov 30 01:52:46 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Thu, 30 Nov 2023 01:52:46 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v50] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: remove whitespace ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/91100e81..d11d6960 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=49 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=48-49 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Thu Nov 30 01:59:58 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Thu, 30 Nov 2023 01:59:58 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v51] In-Reply-To: References: Message-ID: <-NwH6KcjpOF6QwMu8IJvYFjEPrjgQ_CjSzS-kGUGBrI=.9623e3ce-7687-4ccc-86cc-415fdde20096@github.com> > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 56 commits: - Merge branch 'openjdk:master' into master - remove whitespace - Change APIs to be all-static, address other comments - Fix namespace issues (2) Co-authored-by: Stefan Johansson <54407259+kstefanj at users.noreply.github.com> - Fix namespace issues Co-authored-by: Stefan Johansson <54407259+kstefanj at users.noreply.github.com> - Cleanup and address comments - Update memory tracking type for CPUTimeCounters - Fix assertion logic - Address comments and refactor TTTC class for simplification - Fix whitespace - ... and 46 more: https://git.openjdk.org/jdk/compare/940f67c1...63adf78c ------------- Changes: https://git.openjdk.org/jdk/pull/15082/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=50 Stats: 437 lines in 19 files changed: 432 ins; 4 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Thu Nov 30 02:04:41 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Thu, 30 Nov 2023 02:04:41 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v52] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: fix whitespace again ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/63adf78c..e6726ab6 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=51 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=50-51 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Thu Nov 30 02:42:18 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Thu, 30 Nov 2023 02:42:18 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v48] In-Reply-To: References: <_lEBVrWV8wrVbmhOiu3AAqPJo_xBs718ZtA9V-VSzGM=.253c0ec8-256e-4dee-b125-90be6338e4b8@github.com> Message-ID: On Wed, 29 Nov 2023 15:24:52 GMT, Albert Mingkun Yang wrote: >> src/hotspot/share/runtime/cpuTimeCounters.cpp line 91: >> >>> 89: } while (old_value != fetched_value); >>> 90: get_counter(CPUTimeGroups::CPUTimeType::gc_total)->inc(fetched_value); >>> 91: } >> >> Why do we have to do this publish dance? Couldn't the closure that update the diff instead just update the counter? From what I can tell we never have multiple closures active at the same time so should be no race there? > > This two-step update does seem unnecessary, IMO. I agree that in the case that multiple closures are not active at the same time, we wouldn't have to implement it in this way. However, isn't it possible to have multiple closures active simultaneously, e.g. vm thread, concurrent mark thread, concurrent refine thread? I think to account for the races there, we can only update the `_gc_total_cpu_time_diff` member variable atomically during these closure destructions, and then publish the actual updated `gc_total` counter at manually specified times via `publish_gc_total_cpu_time()`. If we were to call `publish_gc_total_cpu_time` as part of the thread closure, I believe it would be difficult to prevent races when accessing the underlying counter from the various gc-related threads. Or maybe there is another strategy that I'm not seeing? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1410089070 From jjoo at openjdk.org Thu Nov 30 02:45:45 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Thu, 30 Nov 2023 02:45:45 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v53] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: Add missing include ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/e6726ab6..7e4cdcd3 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=52 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=51-52 Stats: 2 lines in 2 files changed: 1 ins; 1 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From ayang at openjdk.org Thu Nov 30 06:48:20 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Thu, 30 Nov 2023 06:48:20 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v48] In-Reply-To: References: <_lEBVrWV8wrVbmhOiu3AAqPJo_xBs718ZtA9V-VSzGM=.253c0ec8-256e-4dee-b125-90be6338e4b8@github.com> Message-ID: On Thu, 30 Nov 2023 02:39:39 GMT, Jonathan Joo wrote: >> This two-step update does seem unnecessary, IMO. > > I agree that in the case that multiple closures are not active at the same time, we wouldn't have to implement it in this way. However, isn't it possible to have multiple closures active simultaneously, e.g. vm thread, concurrent mark thread, concurrent refine thread? I think to account for the races there, we can only update the `_gc_total_cpu_time_diff` member variable atomically during these closure destructions, and then publish the actual updated `gc_total` counter at manually specified times via `publish_gc_total_cpu_time()`. If we were to call `publish_gc_total_cpu_time` as part of the thread closure, I believe it would be difficult to prevent races when accessing the underlying counter from the various gc-related threads. > > Or maybe there is another strategy that I'm not seeing? Both `publish_gc_total_cpu_time` and `~ThreadTotalCPUTimeClosure` are called by the vm-thread inside a safepoint, so there shouldn't be any other threads running simultaneously, I believe. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1410221646 From sjohanss at openjdk.org Thu Nov 30 09:33:34 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Thu, 30 Nov 2023 09:33:34 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v48] In-Reply-To: References: <_lEBVrWV8wrVbmhOiu3AAqPJo_xBs718ZtA9V-VSzGM=.253c0ec8-256e-4dee-b125-90be6338e4b8@github.com> Message-ID: On Thu, 30 Nov 2023 06:45:02 GMT, Albert Mingkun Yang wrote: >> I agree that in the case that multiple closures are not active at the same time, we wouldn't have to implement it in this way. However, isn't it possible to have multiple closures active simultaneously, e.g. vm thread, concurrent mark thread, concurrent refine thread? I think to account for the races there, we can only update the `_gc_total_cpu_time_diff` member variable atomically during these closure destructions, and then publish the actual updated `gc_total` counter at manually specified times via `publish_gc_total_cpu_time()`. If we were to call `publish_gc_total_cpu_time` as part of the thread closure, I believe it would be difficult to prevent races when accessing the underlying counter from the various gc-related threads. >> >> Or maybe there is another strategy that I'm not seeing? > > Both `publish_gc_total_cpu_time` and `~ThreadTotalCPUTimeClosure` are called by the vm-thread inside a safepoint, so there shouldn't be any other threads running simultaneously, I believe. Me and Albert just spoke and we do see the problem that two concurrent threads could be executing the closure at the same time. So if having a total counter we need to sync the updates. But when talking we started to question how useful it is to have the gc_total counter. It is just an aggregate of the other gc-counters, but it is out of sync between safepoints. So you will always get a more accurate value by looking at the individual gc-counters. We came to the conclusion that it would probably be easier to drop `gc_total` right now, rather than trying to keep it in sync for all updates to the individual counters. Because having them out of sync doesn't feel like a great option. Are we missing anything or do you agree? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1410394993 From sjohanss at openjdk.org Thu Nov 30 09:58:29 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Thu, 30 Nov 2023 09:58:29 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v53] In-Reply-To: References: Message-ID: <-GX8bATX2hz3YWgnJbhTNEYbi4t8HxfdhYqBP-ulyGg=.0080d7b0-8e43-4b81-b885-1d4a742048cc@github.com> On Thu, 30 Nov 2023 02:45:45 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Add missing include Few more comments after the latest changes. src/hotspot/share/gc/parallel/parallelScavengeHeap.cpp line 905: > 903: gc_threads_do(&tttc); > 904: > 905: CPUTimeCounters::publish_gc_total_cpu_time(); As I suggested in the other comment, maybe we should not keep the total counter, but if we do we need to make sure the destructor of the closure is run before the call to `publish_gc_total_cpu_time()`. Otherwise we will publish a not yet updated value. src/hotspot/share/runtime/cpuTimeCounters.hpp line 59: > 57: NONCOPYABLE(CPUTimeCounters); > 58: > 59: static CPUTimeCounters* _instance; I would prefer if we made the whole class static and got rid of the instance and just made the `_cpu_time_counters` array static. The only drawback I/we (discussed this with Albert as well) can see is that the memory for the array would not be accounted in NMT, but this array will always be very small so should not be a big problem. Do you see any other concerns? ------------- Changes requested by sjohanss (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/15082#pullrequestreview-1757031262 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1410415366 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1410410265 From sjohanss at openjdk.org Thu Nov 30 10:20:27 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Thu, 30 Nov 2023 10:20:27 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v53] In-Reply-To: References: Message-ID: On Thu, 30 Nov 2023 02:45:45 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Add missing include src/hotspot/share/runtime/cpuTimeCounters.cpp line 47: > 45: return "conc_dedup"; > 46: default: > 47: ShouldNotReachHere(); My IDE complained and I guess depending on warning level we might need a return here. Suggestion: ShouldNotReachHere(); return ""; ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1410454940 From tschatzl at openjdk.org Thu Nov 30 10:37:11 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 30 Nov 2023 10:37:11 GMT Subject: RFR: 8317809: Insertion of free code blobs into code cache can be very slow during class unloading [v3] In-Reply-To: References: <_dcFF70_w7IXSjb6w-HuHCBkPyS3a6NlzejtqdfdYnM=.74e0f9df-eca3-49b0-be68-2d5824c16003@github.com> Message-ID: On Mon, 27 Nov 2023 14:53:34 GMT, Albert Mingkun Yang wrote: >> Thomas Schatzl has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: >> >> - Merge branch 'master' into mergeme >> - iwalulya review, naming >> - 8317809 Insert code blobs in a sorted fashion to exploit the finger-optimization when adding, making this procedure O(n) instead of O(n^2) >> >> Introduce a globally available ClassUnloadingContext that contains common methods pertaining to class and code unloading. >> GCs may use it to efficiently manage unlinked class loader datas and nmethods to allow use of common methods (unlink/merge). >> >> The steps typically are registering a new to be unlinked CLD/nmethod, and then purge its memory later. STW collectors perform >> this work in one big chunk taking the CodeCache_lock, for the entire duration, while concurrent collectors lock/unlock for every >> insertion to allow for concurrent users for the lock to progress. >> >> Some care has been taken to stay consistent with an "unloading = unlinking + purge" scheme; however particularly the existing >> CLD handling API (still) mixes unlinking and purging in its CLD::unload() call. To simplify this change that is mostly geared >> towards separating nmethod unlinking from purging, to make code blob freeing O(n) instead of O(n^2). >> >> Upcoming changes will >> * separate nmethod unregistering from nmethod purging to allow doing that in bulk (for the STW collectors); that can significantly >> reduce code purging time for the STW collectors. >> * better name the second stage of unlinking (called "cleaning" throughout, e.g. the work done in `G1CollectedHeap::complete_cleaning`) >> * untangle CLD unlinking and what's called "cleaning" now to allow moving more stuff into the second unlinking stage for better >> parallelism >> * G1: move some signifcant tasks from the remark pause to concurrent (unregistering nmethods, freeing code blobs and cld/metaspace purging) >> * Maybe move Serial/Parallel GC metaspace purging closer to other unlinking/purging code to keep things local and allow easier logging. >> - Only run test case on debug VMs, sufficient >> - 8320331 g1 full gc "during" verification accesses half-unloaded metadata > > src/hotspot/share/gc/shared/classUnloadingContext.hpp line 63: > >> 61: }; >> 62: >> 63: class DefaultClassUnloadingContext : public ClassUnloadingContext { > > I don't understand why they need to be two classes, even after reading "These are the reason for the class hierarchy for...". The reference to future/other PR(s) in the description doesn't really help -- it's unclear what is *necessary* for the current PR and what is preparation for future PR(s). The base class is unnecessary for this change, but very nice to have for future changes. I'll just merge them for now, and separate them again later. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16759#discussion_r1410476983 From tschatzl at openjdk.org Thu Nov 30 11:38:28 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 30 Nov 2023 11:38:28 GMT Subject: RFR: 8317809: Insertion of free code blobs into code cache can be very slow during class unloading [v4] In-Reply-To: <_dcFF70_w7IXSjb6w-HuHCBkPyS3a6NlzejtqdfdYnM=.74e0f9df-eca3-49b0-be68-2d5824c16003@github.com> References: <_dcFF70_w7IXSjb6w-HuHCBkPyS3a6NlzejtqdfdYnM=.74e0f9df-eca3-49b0-be68-2d5824c16003@github.com> Message-ID: <2_ONmN3qxsdTIEJMbQhE82nBn10l_RnZm5-DZAmQn2I=.9ad49c74-3721-4299-8a9e-b8c1973eb494@github.com> > Insert code blobs in a sorted fashion to exploit the finger-optimization when adding, making this procedure O(n) instead of O(n^2) > > Introduces a globally available ClassUnloadingContext that contains common methods pertaining to class and code unloading. GCs may use it to efficiently manage unlinked class loader datas and nmethods to allow use of common methods (unlink/merge). > > The steps typically are registering a new to be unlinked CLD/nmethod, and then purge its memory later. STW collectors perform this work in one big chunk taking the CodeCache_lock, for the entire duration, while concurrent collectors lock/unlock for every insertion to allow for concurrent users for the lock to progress. > > Some care has been taken to stay consistent with an "unloading = unlinking + purge" scheme; however particularly the existing CLD handling API (still) mixes unlinking and purging in its CLD::unload() call. To simplify this change that is mostly geared towards separating nmethod unlinking from purging, to make code blob freeing O(n) instead of O(n^2). > > Upcoming changes will > * separate nmethod unregistering from nmethod purging to allow doing that in bulk (for the STW collectors); that can significantly reduce code purging time for the STW collectors. > * better name the second stage of unlinking (called "cleaning" throughout, e.g. the work done in `G1CollectedHeap::complete_cleaning`) > * untangle CLD unlinking and what's called "cleaning" now to allow moving more stuff into the second unlinking stage for better parallelism > * G1: move some significant tasks from the remark pause to concurrent (unregistering nmethods, freeing code blobs and cld/metaspace purging) > * Maybe move Serial/Parallel GC metaspace purging closer to other unlinking/purging code to keep things local and allow easier logging. > > These are the reason for the class hierarchy for `ClassUnloadingContext`: the goal is to ultimately have about this phasing (for G1): > 1. collect all dead CLDs, using the `register_unloading_class_loader_data` method *only* > 2. parallelize the stuff in `ClassLoaderData::unload()` in one way or another, adding them to the `complete_cleaning` (parallel) phase. > 3. `purge_nmethods`, `free_code_blobs` and the `remove_unlinked_nmethods_from_code_root_set` (from JDK-8317007) will be concurrent. > > Particularly the split of `SystemDictionary::do_unloading` into "only" traversing the CLDs to find the dead ones and then in parallel process them in 2. above warrants a separate `ClassUnloadingCo... Thomas Schatzl has updated the pull request incrementally with three additional commits since the last revision: - remove trailing whitespace - fix indentation after recent commit - Address ayang/iwalulya review comments, remove inheritance in ClassUnloadingContext for now as unnecessary for this change, use iterators, other review comments ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16759/files - new: https://git.openjdk.org/jdk/pull/16759/files/e8c8477a..801426a2 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16759&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16759&range=02-03 Stats: 103 lines in 9 files changed: 19 ins; 40 del; 44 mod Patch: https://git.openjdk.org/jdk/pull/16759.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16759/head:pull/16759 PR: https://git.openjdk.org/jdk/pull/16759 From sjohanss at openjdk.org Thu Nov 30 13:38:06 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Thu, 30 Nov 2023 13:38:06 GMT Subject: RFR: 8321018: Parallel: Make some methods in ParCompactionManager private In-Reply-To: References: Message-ID: On Wed, 29 Nov 2023 14:57:24 GMT, Albert Mingkun Yang wrote: > Trivial reducing visibility of some methods. Looks good. ------------- Marked as reviewed by sjohanss (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16885#pullrequestreview-1757484337 From duke at openjdk.org Thu Nov 30 15:05:08 2023 From: duke at openjdk.org (Lei Zaakjyu) Date: Thu, 30 Nov 2023 15:05:08 GMT Subject: RFR: JDK-8234502 : Merge GenCollectedHeap and SerialHeap [v8] In-Reply-To: References: Message-ID: <182QWwu6PO97q3Jf3RaiiNLUKN6f-Xz5VjPb8rd0O50=.5f4a12df-3c3e-4ba7-8ba4-10627d7bbcdd@github.com> On Tue, 28 Nov 2023 09:20:58 GMT, Albert Mingkun Yang wrote: > I just noticed I referred to the wrong PR number in my previous msg... > > Could you resolve the conflict now that that PR is merged? ok! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16623#issuecomment-1833952980 From tschatzl at openjdk.org Thu Nov 30 15:42:06 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 30 Nov 2023 15:42:06 GMT Subject: RFR: 8320916: jdk/jfr/event/gc/stacktrace/TestParallelMarkSweepAllocationPendingStackTrace.java failed with "OutOfMemoryError: GC overhead limit exceeded" In-Reply-To: References: Message-ID: On Wed, 29 Nov 2023 00:35:47 GMT, Albert Mingkun Yang wrote: > Simple fix to reduce live set so that after the triggered full-gc, there is still some memory left. > > Test: ~2/10 failure before the fix and no failure observed for 100 iterations. Marked as reviewed by tschatzl (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16870#pullrequestreview-1757771064 From tschatzl at openjdk.org Thu Nov 30 15:48:05 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 30 Nov 2023 15:48:05 GMT Subject: RFR: 8312023: Parallel pretouch should shortcut when only 1 thread is needed [v2] In-Reply-To: References: Message-ID: On Wed, 29 Nov 2023 20:16:31 GMT, Aleksey Shipilev wrote: >> This is a follow-up for code introduced in JDK 16 by [JDK-8252221](https://bugs.openjdk.org/browse/JDK-8252221). >> >> Unfortunately, we do pre-touch not only at startup, but also during the application lifetime when space boundary moves, see [JDK-8312021](https://bugs.openjdk.org/browse/JDK-8312021). From that code, we start the pre-touch threads unconditionally, even when we only need one thread anyway. This is clearly visible with `-Xlog:gc+heap=debug`. The actual pretouch takes single-digit microseconds, but the round-trip through the worker pool incurs latency of about 100us (2x 50us) in my setups. This is significant for very short GC pauses. >> >> Additional testing: >> - [x] Ad-hoc benchmarks >> - [x] Linux x86_64 server fastdebug, `tier{1,2,3}` with `-XX:+UseParallelGC -XX:+AlwaysPreTouch` > > Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: > > Restore calculation, hoping to fix Windows build Marked as reviewed by tschatzl (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16882#pullrequestreview-1757782295 From ayang at openjdk.org Thu Nov 30 15:51:25 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Thu, 30 Nov 2023 15:51:25 GMT Subject: RFR: 8320916: jdk/jfr/event/gc/stacktrace/TestParallelMarkSweepAllocationPendingStackTrace.java failed with "OutOfMemoryError: GC overhead limit exceeded" In-Reply-To: References: Message-ID: On Wed, 29 Nov 2023 00:35:47 GMT, Albert Mingkun Yang wrote: > Simple fix to reduce live set so that after the triggered full-gc, there is still some memory left. > > Test: ~2/10 failure before the fix and no failure observed for 100 iterations. Thanks for the review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16870#issuecomment-1834033077 From ayang at openjdk.org Thu Nov 30 15:51:25 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Thu, 30 Nov 2023 15:51:25 GMT Subject: Integrated: 8320916: jdk/jfr/event/gc/stacktrace/TestParallelMarkSweepAllocationPendingStackTrace.java failed with "OutOfMemoryError: GC overhead limit exceeded" In-Reply-To: References: Message-ID: On Wed, 29 Nov 2023 00:35:47 GMT, Albert Mingkun Yang wrote: > Simple fix to reduce live set so that after the triggered full-gc, there is still some memory left. > > Test: ~2/10 failure before the fix and no failure observed for 100 iterations. This pull request has now been integrated. Changeset: 69384745 Author: Albert Mingkun Yang URL: https://git.openjdk.org/jdk/commit/693847452f208446a34186f142fe2c56a49ceceb Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod 8320916: jdk/jfr/event/gc/stacktrace/TestParallelMarkSweepAllocationPendingStackTrace.java failed with "OutOfMemoryError: GC overhead limit exceeded" Reviewed-by: sjohanss, tschatzl ------------- PR: https://git.openjdk.org/jdk/pull/16870 From ayang at openjdk.org Thu Nov 30 16:00:06 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Thu, 30 Nov 2023 16:00:06 GMT Subject: RFR: 8312023: Parallel pretouch should shortcut when only 1 thread is needed [v2] In-Reply-To: References: Message-ID: On Wed, 29 Nov 2023 20:16:31 GMT, Aleksey Shipilev wrote: >> This is a follow-up for code introduced in JDK 16 by [JDK-8252221](https://bugs.openjdk.org/browse/JDK-8252221). >> >> Unfortunately, we do pre-touch not only at startup, but also during the application lifetime when space boundary moves, see [JDK-8312021](https://bugs.openjdk.org/browse/JDK-8312021). From that code, we start the pre-touch threads unconditionally, even when we only need one thread anyway. This is clearly visible with `-Xlog:gc+heap=debug`. The actual pretouch takes single-digit microseconds, but the round-trip through the worker pool incurs latency of about 100us (2x 50us) in my setups. This is significant for very short GC pauses. >> >> Additional testing: >> - [x] Ad-hoc benchmarks >> - [x] Linux x86_64 server fastdebug, `tier{1,2,3}` with `-XX:+UseParallelGC -XX:+AlwaysPreTouch` > > Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: > > Restore calculation, hoping to fix Windows build Marked as reviewed by ayang (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16882#pullrequestreview-1757809045 From shade at openjdk.org Thu Nov 30 16:16:41 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Thu, 30 Nov 2023 16:16:41 GMT Subject: RFR: 8321120: Shenandoah: Remove ShenandoahElasticTLAB flag Message-ID: We have added `ShenandoahElasticTLAB` a long time ago, to provide the escape hatch if elastic TLABs would misbehave. We have been running with elastic TLABs for years without problems. Taking care of elastic/non-elastic TLAB allocs adds to technical debt. Additional testing: - [x] hotspot_gc_shenandoah ------------- Commit messages: - Remove stale comment as well - Fix Changes: https://git.openjdk.org/jdk/pull/16907/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16907&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8321120 Stats: 92 lines in 5 files changed: 0 ins; 87 del; 5 mod Patch: https://git.openjdk.org/jdk/pull/16907.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16907/head:pull/16907 PR: https://git.openjdk.org/jdk/pull/16907 From tschatzl at openjdk.org Thu Nov 30 16:23:22 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 30 Nov 2023 16:23:22 GMT Subject: RFR: 8321018: Parallel: Make some methods in ParCompactionManager private In-Reply-To: References: Message-ID: <5taa59u-oIeQQk79aOloRndnMqMO6QqhotDyMTZR0u0=.d78ef939-0b4d-4d88-b439-0f3f317fdf42@github.com> On Wed, 29 Nov 2023 14:57:24 GMT, Albert Mingkun Yang wrote: > Trivial reducing visibility of some methods. Marked as reviewed by tschatzl (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16885#pullrequestreview-1757860534 From shade at openjdk.org Thu Nov 30 16:34:06 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Thu, 30 Nov 2023 16:34:06 GMT Subject: RFR: 8321122: Shenandoah: Remove ShenandoahLoopOptsAfterExpansion flag Message-ID: More flags cleanup. `ShenandoahLoopOptsAfterExpansion` was added long time ago to gate more aggressive barrier optimizations in C2. It was default since then. There seem to be no reason to keep it as the flag. Just checking, @rwestrel -- you don't need it anymore, right? Additional testing: - [x] `hotspot_gc_shenandoah` ------------- Commit messages: - Fix Changes: https://git.openjdk.org/jdk/pull/16908/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16908&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8321122 Stats: 18 lines in 2 files changed: 5 ins; 11 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16908.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16908/head:pull/16908 PR: https://git.openjdk.org/jdk/pull/16908 From ayang at openjdk.org Thu Nov 30 16:35:27 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Thu, 30 Nov 2023 16:35:27 GMT Subject: RFR: 8321018: Parallel: Make some methods in ParCompactionManager private In-Reply-To: References: Message-ID: On Wed, 29 Nov 2023 14:57:24 GMT, Albert Mingkun Yang wrote: > Trivial reducing visibility of some methods. Thanks for the review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16885#issuecomment-1834115963 From ayang at openjdk.org Thu Nov 30 16:35:27 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Thu, 30 Nov 2023 16:35:27 GMT Subject: Integrated: 8321018: Parallel: Make some methods in ParCompactionManager private In-Reply-To: References: Message-ID: On Wed, 29 Nov 2023 14:57:24 GMT, Albert Mingkun Yang wrote: > Trivial reducing visibility of some methods. This pull request has now been integrated. Changeset: b1cbf558 Author: Albert Mingkun Yang URL: https://git.openjdk.org/jdk/commit/b1cbf55859c26cfca89c556b0db0e551ad58e4a6 Stats: 3 lines in 1 file changed: 1 ins; 1 del; 1 mod 8321018: Parallel: Make some methods in ParCompactionManager private Reviewed-by: sjohanss, tschatzl ------------- PR: https://git.openjdk.org/jdk/pull/16885 From kdnilsen at openjdk.org Thu Nov 30 16:38:16 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Thu, 30 Nov 2023 16:38:16 GMT Subject: RFR: 8321120: Shenandoah: Remove ShenandoahElasticTLAB flag In-Reply-To: References: Message-ID: On Thu, 30 Nov 2023 15:50:52 GMT, Aleksey Shipilev wrote: > We have added `ShenandoahElasticTLAB` a long time ago, to provide the escape hatch if elastic TLABs would misbehave. We have been running with elastic TLABs for years without problems. Taking care of elastic/non-elastic TLAB allocs adds to technical debt. > > Additional testing: > - [x] hotspot_gc_shenandoah I agree to make elastic tlab behavior the only supported configuration. FWIW, GenShen code has not thoroughly tested with this flag disabled. ------------- Marked as reviewed by kdnilsen (no project role). PR Review: https://git.openjdk.org/jdk/pull/16907#pullrequestreview-1757903383 From kdnilsen at openjdk.org Thu Nov 30 16:44:05 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Thu, 30 Nov 2023 16:44:05 GMT Subject: RFR: 8321122: Shenandoah: Remove ShenandoahLoopOptsAfterExpansion flag In-Reply-To: References: Message-ID: On Thu, 30 Nov 2023 16:00:17 GMT, Aleksey Shipilev wrote: > More flags cleanup. `ShenandoahLoopOptsAfterExpansion` was added long time ago to gate more aggressive barrier optimizations in C2. It was default since then. There seem to be no reason to keep it as the flag. Just checking, @rwestrel -- you don't need it anymore, right? > > Additional testing: > - [x] `hotspot_gc_shenandoah` This looks good to me, though I don't have full understanding of all the relevant issues. ------------- Marked as reviewed by kdnilsen (no project role). PR Review: https://git.openjdk.org/jdk/pull/16908#pullrequestreview-1757916271 From kbarrett at openjdk.org Thu Nov 30 20:12:09 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Thu, 30 Nov 2023 20:12:09 GMT Subject: RFR: 8312023: Parallel pretouch should shortcut when only 1 thread is needed [v2] In-Reply-To: References: Message-ID: On Wed, 29 Nov 2023 20:16:31 GMT, Aleksey Shipilev wrote: >> This is a follow-up for code introduced in JDK 16 by [JDK-8252221](https://bugs.openjdk.org/browse/JDK-8252221). >> >> Unfortunately, we do pre-touch not only at startup, but also during the application lifetime when space boundary moves, see [JDK-8312021](https://bugs.openjdk.org/browse/JDK-8312021). From that code, we start the pre-touch threads unconditionally, even when we only need one thread anyway. This is clearly visible with `-Xlog:gc+heap=debug`. The actual pretouch takes single-digit microseconds, but the round-trip through the worker pool incurs latency of about 100us (2x 50us) in my setups. This is significant for very short GC pauses. >> >> Additional testing: >> - [x] Ad-hoc benchmarks >> - [x] Linux x86_64 server fastdebug, `tier{1,2,3}` with `-XX:+UseParallelGC -XX:+AlwaysPreTouch` > > Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: > > Restore calculation, hoping to fix Windows build I recall several discussions about avoiding use of available workers when the amount of work to be done is sufficiently small that the number of workers that would be requested is one. The alternative is to go ahead and use the workers, but have the worker implementation recognize the special case of only one requested thread and just run the task on the current thread. That avoids making all uses of workers specially handle this case. I thought there was an RFE for this, but can't find it. There may have been some concern that some task might care about the kind of thread it runs on? So I'm not convinced this change should be made. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16882#issuecomment-1834471328 From ysr at openjdk.org Thu Nov 30 20:14:10 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Thu, 30 Nov 2023 20:14:10 GMT Subject: RFR: 8321120: Shenandoah: Remove ShenandoahElasticTLAB flag In-Reply-To: References: Message-ID: On Thu, 30 Nov 2023 15:50:52 GMT, Aleksey Shipilev wrote: > We have added `ShenandoahElasticTLAB` a long time ago, to provide the escape hatch if elastic TLABs would misbehave. We have been running with elastic TLABs for years without problems. Taking care of elastic/non-elastic TLAB allocs adds to technical debt. > > Additional testing: > - [x] hotspot_gc_shenandoah Marked as reviewed by ysr (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16907#pullrequestreview-1758364128 From shade at openjdk.org Thu Nov 30 20:56:04 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Thu, 30 Nov 2023 20:56:04 GMT Subject: RFR: 8312023: Parallel pretouch should shortcut when only 1 thread is needed [v2] In-Reply-To: References: Message-ID: On Thu, 30 Nov 2023 20:09:06 GMT, Kim Barrett wrote: > There may have been some concern that some task might care about the kind of thread it runs on? Exactly, in generic case you don't know if caller thread is set up to do the task in the way the worker threads are. This pretouching code is one of the cases where we know running in caller is fine. Existing code even does it when no worker pool is available. > So I'm not convinced this change should be made. I believe we want to do it. Round-tripping to worker pool is asking for a latency hiccup. Even if things go well, we lose double-digit microseconds on this path. This is a common optimization we do. Last time [we did it in safepoints](https://bugs.openjdk.org/browse/JDK-8306738) with [great results](https://github.com/openjdk/jdk/pull/13616#issuecomment-1538206572). ------------- PR Comment: https://git.openjdk.org/jdk/pull/16882#issuecomment-1834544347 From jjoo at openjdk.org Thu Nov 30 21:21:48 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Thu, 30 Nov 2023 21:21:48 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v54] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: Return after ShouldNotReachHere Co-authored-by: Stefan Johansson <54407259+kstefanj at users.noreply.github.com> ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/7e4cdcd3..fcf00cfe Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=53 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=52-53 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Thu Nov 30 21:21:49 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Thu, 30 Nov 2023 21:21:49 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v48] In-Reply-To: References: <_lEBVrWV8wrVbmhOiu3AAqPJo_xBs718ZtA9V-VSzGM=.253c0ec8-256e-4dee-b125-90be6338e4b8@github.com> Message-ID: On Thu, 30 Nov 2023 09:30:14 GMT, Stefan Johansson wrote: >> Both `publish_gc_total_cpu_time` and `~ThreadTotalCPUTimeClosure` are called by the vm-thread inside a safepoint, so there shouldn't be any other threads running simultaneously, I believe. > > Me and Albert just spoke and we do see the problem that two concurrent threads could be executing the closure at the same time. So if having a total counter we need to sync the updates. But when talking we started to question how useful it is to have the gc_total counter. It is just an aggregate of the other gc-counters, but it is out of sync between safepoints. So you will always get a more accurate value by looking at the individual gc-counters. > > We came to the conclusion that it would probably be easier to drop `gc_total` right now, rather than trying to keep it in sync for all updates to the individual counters. Because having them out of sync doesn't feel like a great option. > > Are we missing anything or do you agree? @simonis was the original suggester of this counter, so I will defer to his expertise. I do agree that dropping the counter would simplify things, but it also might not hurt to just leave it in. I'm okay with either option! ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1411270546 From manc at openjdk.org Thu Nov 30 21:47:21 2023 From: manc at openjdk.org (Man Cao) Date: Thu, 30 Nov 2023 21:47:21 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v48] In-Reply-To: References: <_lEBVrWV8wrVbmhOiu3AAqPJo_xBs718ZtA9V-VSzGM=.253c0ec8-256e-4dee-b125-90be6338e4b8@github.com> Message-ID: On Thu, 30 Nov 2023 21:17:14 GMT, Jonathan Joo wrote: >> Me and Albert just spoke and we do see the problem that two concurrent threads could be executing the closure at the same time. So if having a total counter we need to sync the updates. But when talking we started to question how useful it is to have the gc_total counter. It is just an aggregate of the other gc-counters, but it is out of sync between safepoints. So you will always get a more accurate value by looking at the individual gc-counters. >> >> We came to the conclusion that it would probably be easier to drop `gc_total` right now, rather than trying to keep it in sync for all updates to the individual counters. Because having them out of sync doesn't feel like a great option. >> >> Are we missing anything or do you agree? > > @simonis was the original suggester of this counter, so I will defer to his expertise. I do agree that dropping the counter would simplify things, but it also might not hurt to just leave it in. I'm okay with either option! Right, see @simonis 's comments at https://github.com/openjdk/jdk/pull/15082#pullrequestreview-1613868256, https://github.com/openjdk/jdk/pull/15082#discussion_r1321703912. I initially had similar thought that `gc_total` isn't necessary and provides redundant data. Now I agree with @simonis that the `gc_total` is valuable to users. It saves users from extra work of aggregating different sets of counters for different garbage collectors, and potential mistakes of missing some counters. It is also future-proof that if GC implementation changes that add additional threads, users wouldn't need to change their code to add the counter for additional threads. I think the maintenance overhead is quite small for `gc_total` since it is mostly in this class. The benefit to users is worth it. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1411293819 From bchristi at openjdk.org Thu Nov 30 21:56:09 2023 From: bchristi at openjdk.org (Brent Christian) Date: Thu, 30 Nov 2023 21:56:09 GMT Subject: RFR: 8314480: Memory ordering spec updates in java.lang.ref In-Reply-To: References: Message-ID: On Tue, 21 Nov 2023 09:13:59 GMT, Viktor Klang wrote: >> src/java.base/share/classes/java/lang/ref/Reference.java line 505: >> >>> 503: * to return this reference even though the referent is still strongly >>> 504: * reachable. That is, before the referent has reached the expected >>> 505: * reachability level. >> >> The wording here is kind of awkward. We could try to wordsmith it, but there is also the possibility that there are valid use cases for using enqueue(), which makes this caution seem misplaced. An alternative would simply be to omit this paragraph. > > Might be better to leave out the last sencence? I'll take out the last sentence, and we can continue discussing whether to keep this note at all. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16644#discussion_r1411305306 From manc at openjdk.org Thu Nov 30 22:03:28 2023 From: manc at openjdk.org (Man Cao) Date: Thu, 30 Nov 2023 22:03:28 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v53] In-Reply-To: <-GX8bATX2hz3YWgnJbhTNEYbi4t8HxfdhYqBP-ulyGg=.0080d7b0-8e43-4b81-b885-1d4a742048cc@github.com> References: <-GX8bATX2hz3YWgnJbhTNEYbi4t8HxfdhYqBP-ulyGg=.0080d7b0-8e43-4b81-b885-1d4a742048cc@github.com> Message-ID: On Thu, 30 Nov 2023 09:41:55 GMT, Stefan Johansson wrote: >> Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: >> >> Add missing include > > src/hotspot/share/runtime/cpuTimeCounters.hpp line 59: > >> 57: NONCOPYABLE(CPUTimeCounters); >> 58: >> 59: static CPUTimeCounters* _instance; > > I would prefer if we made the whole class static and got rid of the instance and just made the `_cpu_time_counters` array static. The only drawback I/we (discussed this with Albert as well) can see is that the memory for the array would not be accounted in NMT, but this array will always be very small so should not be a big problem. > > Do you see any other concerns? I thought it is typically preferred to initialize a singleton object on the heap, rather than using several static variables. It is easier to control the initialization order and timing of an on-heap singleton object than statics. Moreover, for this class, `initialize()` could also check `if (UsePerfData)`, and only create the singleton object under `UsePerfData`. This could save some memory when `UsePerfData` is false. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1411317828 From ysr at openjdk.org Thu Nov 30 22:23:06 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Thu, 30 Nov 2023 22:23:06 GMT Subject: RFR: 8312023: Parallel pretouch should shortcut when only 1 thread is needed [v2] In-Reply-To: References: Message-ID: On Wed, 29 Nov 2023 20:16:31 GMT, Aleksey Shipilev wrote: >> This is a follow-up for code introduced in JDK 16 by [JDK-8252221](https://bugs.openjdk.org/browse/JDK-8252221). >> >> Unfortunately, we do pre-touch not only at startup, but also during the application lifetime when space boundary moves, see [JDK-8312021](https://bugs.openjdk.org/browse/JDK-8312021). From that code, we start the pre-touch threads unconditionally, even when we only need one thread anyway. This is clearly visible with `-Xlog:gc+heap=debug`. The actual pretouch takes single-digit microseconds, but the round-trip through the worker pool incurs latency of about 100us (2x 50us) in my setups. This is significant for very short GC pauses. >> >> Additional testing: >> - [x] Ad-hoc benchmarks >> - [x] Linux x86_64 server fastdebug, `tier{1,2,3}` with `-XX:+UseParallelGC -XX:+AlwaysPreTouch` > > Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: > > Restore calculation, hoping to fix Windows build Marked as reviewed by ysr (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16882#pullrequestreview-1758561282 From bchristi at openjdk.org Thu Nov 30 22:42:07 2023 From: bchristi at openjdk.org (Brent Christian) Date: Thu, 30 Nov 2023 22:42:07 GMT Subject: RFR: 8314480: Memory ordering spec updates in java.lang.ref In-Reply-To: References: Message-ID: <-EMuHypBSxQF40bnTvD13GKpGbx59CRml904UB6owkc=.c173d608-9e4e-4455-aab2-d0327733f236@github.com> On Tue, 21 Nov 2023 22:58:09 GMT, Roger Riggs wrote: >> Classes in the `java.lang.ref` package would benefit from an update to bring the spec in line with how the VM already behaves. The changes would focus on _happens-before_ edges at some key points during reference processing. >> >> A couple key things we want to be able to say are: >> - `Reference.reachabilityFence(x)` _happens-before_ reference processing occurs for 'x'. >> - `Cleaner.register()` _happens-before_ the Cleaner thread runs the registered cleaning action. >> >> This will bring Cleaner in line (or close) with the memory visibility guarantees made for finalizers in [JLS 17.4.5](https://docs.oracle.com/javase/specs/jls/se18/html/jls-17.html#jls-17.4.5): >> _"There is a happens-before edge from the end of a constructor of an object to the start of a finalizer (?12.6) for that object."_ > > src/java.base/share/classes/java/lang/ref/Reference.java line 552: > >> 550: * of this method. >> 551: * Invocation of this method does not itself initiate reference processing, >> 552: * garbage collection, or finalization. > > My understanding was that it is not the object instance that is being guarded, only that the reference holding the object is considered a strong root and is only used to delimit a range of bytecodes for which the reference is considered to be strong. > In particular, the invocation of the method itself has no semantics, only that a control flow could reach that statement and the reference was considered strong as long as the reference was in a scope that included the reachability fence. The existing docs refer to "invocation" of this method. I've continued with that, and in general have kept this at a bit higher level, in order to simplify understanding. I'm open to more detailed wording, if it would improve understandability, (or if what I have is not functionally accurate). ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16644#discussion_r1411372474 From bchristi at openjdk.org Thu Nov 30 23:09:09 2023 From: bchristi at openjdk.org (Brent Christian) Date: Thu, 30 Nov 2023 23:09:09 GMT Subject: RFR: 8314480: Memory ordering spec updates in java.lang.ref In-Reply-To: References: Message-ID: On Wed, 15 Nov 2023 22:45:46 GMT, Stuart Marks wrote: >> Classes in the `java.lang.ref` package would benefit from an update to bring the spec in line with how the VM already behaves. The changes would focus on _happens-before_ edges at some key points during reference processing. >> >> A couple key things we want to be able to say are: >> - `Reference.reachabilityFence(x)` _happens-before_ reference processing occurs for 'x'. >> - `Cleaner.register()` _happens-before_ the Cleaner thread runs the registered cleaning action. >> >> This will bring Cleaner in line (or close) with the memory visibility guarantees made for finalizers in [JLS 17.4.5](https://docs.oracle.com/javase/specs/jls/se18/html/jls-17.html#jls-17.4.5): >> _"There is a happens-before edge from the end of a constructor of an object to the start of a finalizer (?12.6) for that object."_ > > src/java.base/share/classes/java/lang/ref/ReferenceQueue.java line 187: > >> 185: * {@link Reference#enqueue()} instead of by the garbage collector, its >> 186: * former referent (which has since been cleared) could still be strongly >> 187: * reachable. > > See my comments on the corresponding Reference.enqueue() apiNote. I'm not sure we want to have this issue sprinkled around in the apiNotes for several methods, including others here on ReferenceQueue. > > If we do want to address this issue -- and it's not clear to me that we do -- then perhaps we should do it in a central location? We can cover this issue just in `Reference.enqueue()`. I'll take out these `@apiNote`s, and put in `@see Reference.enqueue()`. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16644#discussion_r1411395204