From stuefe at openjdk.org  Fri Aug  4 16:07:33 2023
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Fri, 4 Aug 2023 16:07:33 GMT
Subject: RFR: JDK-8310388: Shenandoah: Auxiliary bitmap is not madvised for
 THP [v2]
In-Reply-To: <v3gYJXBLGIv43FkBuha0068hhRYXzhdqV-_uawhsiEA=.abe308f3-4cb7-4496-b996-4b64081ffe3c@github.com>
References: <2KUO9EbhXSV-XrJFWWAY8KvDCDvkdQE7w8ew_MMzUEk=.3fc29a02-fa49-4b6a-8800-aab84525ba43@github.com>
 <v3gYJXBLGIv43FkBuha0068hhRYXzhdqV-_uawhsiEA=.abe308f3-4cb7-4496-b996-4b64081ffe3c@github.com>
Message-ID: <UKodZttfoHg3bi7k1OgUrCGM4JCeyizX_gsr5_To2N4=.4d2f49fe-6b1a-4677-9857-59a3e616c02e@github.com>

On Thu, 20 Jul 2023 16:39:47 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> See details in JBS isse.
>> 
>> Note that there is no actual functional difference. AUX bitmap did not use THPs before this patch and does not now either. The only difference is that before, the JVM *thought* the AUX bitmap uses THPs when in fact it did not. That caused confusion.
>> 
>> (All this illuminates how badly thought out the ReservedSpace API really is. We pass in page size to the constructor, but then need to commit manually; THPs actually use madvise on commit, not on reservation, so we need to pass page size in *again* to commit. Ideally, ReservedSpace should handle committing for us with the page size it saved from reservation.)
>> 
>> Note that this only takes care of preventing THP formation in "madvise" mode. In "always" mode, the kernel will do THP coalescation always. We could prevent it from doing so by advising against it via madvise, but that would require extension of the platform generic reservation APIs and is left for a different RFE. Ideally, nobody should use THP "always" mode.
>
> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Alekseys feedback

Friendly ping... still need a second review.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/14953#issuecomment-1665854118

From wkemper at openjdk.org  Fri Aug  4 16:25:27 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Aug 2023 16:25:27 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <e6xLb1L4u_nfLcl5YIj26lqikXOUKAsgYIUIfzMtyzg=.3766fc6a-cc22-4514-9dcf-db764d0519e7@github.com>

Merges tag jdk-22+9. Includes fixes to GHA infrastructure.

-------------

Commit messages:
 - Merge tag 'jdk-22+9' into merge-jdk-22-9
 - 8306582: Remove MetaspaceShared::exit_after_static_dump()
 - 8313368: (fc) FileChannel.size returns 0 on block special files
 - 8312078: [PPC] JcmdScale.java Failing on AIX
 - 8312617: SIGSEGV in ConnectionGraph::verify_ram_nodes
 - 8313322: RISC-V: implement MD5 intrinsic
 - 8313593: Generational ZGC: NMT assert when the heap fails to expand
 - 8313402: C1: Incorrect LoadIndexed value numbering
 - 8311989: Test java/lang/Thread/virtual/Reflection.java timed out
 - 8310311: Serial: move Generation::contribute_scratch to DefNewGeneration
 - ... and 101 more: https://git.openjdk.org/shenandoah/compare/6ca2eeaa...1219a6af

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah/pull/303/files
  Stats: 14244 lines in 454 files changed: 7487 ins; 3171 del; 3586 mod
  Patch: https://git.openjdk.org/shenandoah/pull/303.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/303/head:pull/303

PR: https://git.openjdk.org/shenandoah/pull/303

From wkemper at openjdk.org  Fri Aug  4 16:41:24 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Aug 2023 16:41:24 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <e6xLb1L4u_nfLcl5YIj26lqikXOUKAsgYIUIfzMtyzg=.3766fc6a-cc22-4514-9dcf-db764d0519e7@github.com>
References: <e6xLb1L4u_nfLcl5YIj26lqikXOUKAsgYIUIfzMtyzg=.3766fc6a-cc22-4514-9dcf-db764d0519e7@github.com>
Message-ID: <FZs553mKlpiNM6H69PNMtcigx_jblJdthij14-Vx5NU=.f6192bd6-aac0-4d39-b4ae-5672b8f5adff@github.com>

> Merges tag jdk-22+9. Includes fixes to GHA infrastructure.

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 305 commits:

 - Merge tag 'jdk-22+9' into merge-jdk-22-9
   
   Added tag jdk-22+9 for changeset cff25dd5
 - 8312610: GenShen: Old generation available is unintentionally restricted by mutator's available memory
   
   Reviewed-by: kdnilsen, ysr
 - Merge
 - Merge
 - 8311602: GenShen: Decouple generational mode heuristics
   
   Reviewed-by: kdnilsen, ysr
 - 8312321: GenShen: Remembered set scan may encounter garbage objects
   
   Reviewed-by: kdnilsen
 - 8312422: GenShen: In-place region promotion state may carry over when evacuation fails
   
   Reviewed-by: kdnilsen
 - 8312322: GenShen: Cancelled GCs may become stuck in self-cancellation loop
   
   Reviewed-by: kdnilsen
 - Merge
 - 8311599: GenShen: Missing card mark barrier when processing references
   
   Reviewed-by: ysr
 - ... and 295 more: https://git.openjdk.org/shenandoah/compare/cff25dd5...1219a6af

-------------

Changes: https://git.openjdk.org/shenandoah/pull/303/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=303&range=01
  Stats: 20548 lines in 208 files changed: 18689 ins; 903 del; 956 mod
  Patch: https://git.openjdk.org/shenandoah/pull/303.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/303/head:pull/303

PR: https://git.openjdk.org/shenandoah/pull/303

From wkemper at openjdk.org  Fri Aug  4 16:41:26 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Aug 2023 16:41:26 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <e6xLb1L4u_nfLcl5YIj26lqikXOUKAsgYIUIfzMtyzg=.3766fc6a-cc22-4514-9dcf-db764d0519e7@github.com>
References: <e6xLb1L4u_nfLcl5YIj26lqikXOUKAsgYIUIfzMtyzg=.3766fc6a-cc22-4514-9dcf-db764d0519e7@github.com>
Message-ID: <nDRmJOTng5UZNiIXMRzt518Lokn7gwLWjoKlp__r9k4=.7f77f512-f3a9-4913-8ecd-931f01620f6d@github.com>

On Fri, 4 Aug 2023 16:20:10 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-22+9. Includes fixes to GHA infrastructure.

This pull request has now been integrated.

Changeset: 3535f1a0
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/3535f1a05e55ff1de5a963b1907b675f60981dfe
Stats:     14244 lines in 454 files changed: 7487 ins; 3171 del; 3586 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/303

From wkemper at openjdk.org  Fri Aug  4 18:25:21 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 4 Aug 2023 18:25:21 GMT
Subject: RFR: 8313574: GenShen: Completing a global mark should also complete
 an old mark
Message-ID: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>

The mark context is shared, but each generation maintains a separate flag (`ShenandoahGeneration::_is_marking_complete`) to record the status of the mark. The remembered set scan will use the mark bitmap only if the old generation mark is complete. Otherwise, it walks the old regions and requires them to be parseable.

Presently, we make the heap parseable after global marking finishes, but this work can be deferred to the start of an old mark if we had global mark record completion of old marking.

-------------

Commit messages:
 - Merge branch upstream into global-completes-old-mark
 - Have global mark also complete young and old marks

Changes: https://git.openjdk.org/shenandoah/pull/304/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=304&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8313574
  Stats: 90 lines in 9 files changed: 18 ins; 59 del; 13 mod
  Patch: https://git.openjdk.org/shenandoah/pull/304.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/304/head:pull/304

PR: https://git.openjdk.org/shenandoah/pull/304

From kdnilsen at openjdk.org  Fri Aug  4 18:38:32 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 4 Aug 2023 18:38:32 GMT
Subject: RFR: JDK-8310388: Shenandoah: Auxiliary bitmap is not madvised for
 THP [v2]
In-Reply-To: <v3gYJXBLGIv43FkBuha0068hhRYXzhdqV-_uawhsiEA=.abe308f3-4cb7-4496-b996-4b64081ffe3c@github.com>
References: <2KUO9EbhXSV-XrJFWWAY8KvDCDvkdQE7w8ew_MMzUEk=.3fc29a02-fa49-4b6a-8800-aab84525ba43@github.com>
 <v3gYJXBLGIv43FkBuha0068hhRYXzhdqV-_uawhsiEA=.abe308f3-4cb7-4496-b996-4b64081ffe3c@github.com>
Message-ID: <w6eEJX2hzmBJH6338kmWUk_b8SO-JgcV9H2R-OAVoHs=.b8a50795-c1c5-4916-b0cb-68cb40684f74@github.com>

On Thu, 20 Jul 2023 16:39:47 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> See details in JBS isse.
>> 
>> Note that there is no actual functional difference. AUX bitmap did not use THPs before this patch and does not now either. The only difference is that before, the JVM *thought* the AUX bitmap uses THPs when in fact it did not. That caused confusion.
>> 
>> (All this illuminates how badly thought out the ReservedSpace API really is. We pass in page size to the constructor, but then need to commit manually; THPs actually use madvise on commit, not on reservation, so we need to pass page size in *again* to commit. Ideally, ReservedSpace should handle committing for us with the page size it saved from reservation.)
>> 
>> Note that this only takes care of preventing THP formation in "madvise" mode. In "always" mode, the kernel will do THP coalescation always. We could prevent it from doing so by advising against it via madvise, but that would require extension of the platform generic reservation APIs and is left for a different RFE. Ideally, nobody should use THP "always" mode.
>
> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Alekseys feedback

Looks good to me.

-------------

Marked as reviewed by kdnilsen (no project role).

PR Review: https://git.openjdk.org/jdk/pull/14953#pullrequestreview-1563376599

From stuefe at openjdk.org  Fri Aug  4 18:42:43 2023
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Fri, 4 Aug 2023 18:42:43 GMT
Subject: RFR: JDK-8310388: Shenandoah: Auxiliary bitmap is not madvised for
 THP [v2]
In-Reply-To: <v3gYJXBLGIv43FkBuha0068hhRYXzhdqV-_uawhsiEA=.abe308f3-4cb7-4496-b996-4b64081ffe3c@github.com>
References: <2KUO9EbhXSV-XrJFWWAY8KvDCDvkdQE7w8ew_MMzUEk=.3fc29a02-fa49-4b6a-8800-aab84525ba43@github.com>
 <v3gYJXBLGIv43FkBuha0068hhRYXzhdqV-_uawhsiEA=.abe308f3-4cb7-4496-b996-4b64081ffe3c@github.com>
Message-ID: <FSFlpeYAmOm6iUSE-JYWIklnGFBJWgcROri7eqsI058=.b0138e93-1a62-4b80-9353-20b31685cbb5@github.com>

On Thu, 20 Jul 2023 16:39:47 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> See details in JBS isse.
>> 
>> Note that there is no actual functional difference. AUX bitmap did not use THPs before this patch and does not now either. The only difference is that before, the JVM *thought* the AUX bitmap uses THPs when in fact it did not. That caused confusion.
>> 
>> (All this illuminates how badly thought out the ReservedSpace API really is. We pass in page size to the constructor, but then need to commit manually; THPs actually use madvise on commit, not on reservation, so we need to pass page size in *again* to commit. Ideally, ReservedSpace should handle committing for us with the page size it saved from reservation.)
>> 
>> Note that this only takes care of preventing THP formation in "madvise" mode. In "always" mode, the kernel will do THP coalescation always. We could prevent it from doing so by advising against it via madvise, but that would require extension of the platform generic reservation APIs and is left for a different RFE. Ideally, nobody should use THP "always" mode.
>
> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Alekseys feedback

Thanks Kelvin!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/14953#issuecomment-1666031278

From stuefe at openjdk.org  Fri Aug  4 18:42:44 2023
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Fri, 4 Aug 2023 18:42:44 GMT
Subject: Integrated: JDK-8310388: Shenandoah: Auxiliary bitmap is not madvised
 for THP
In-Reply-To: <2KUO9EbhXSV-XrJFWWAY8KvDCDvkdQE7w8ew_MMzUEk=.3fc29a02-fa49-4b6a-8800-aab84525ba43@github.com>
References: <2KUO9EbhXSV-XrJFWWAY8KvDCDvkdQE7w8ew_MMzUEk=.3fc29a02-fa49-4b6a-8800-aab84525ba43@github.com>
Message-ID: <m_G5F38iGF5lJKnBq0AhTvCW7yb7ekxno1l38mr6qxU=.27c4a5e5-c1b1-48e0-92bf-a43bb2320b8e@github.com>

On Thu, 20 Jul 2023 11:19:53 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> See details in JBS isse.
> 
> Note that there is no actual functional difference. AUX bitmap did not use THPs before this patch and does not now either. The only difference is that before, the JVM *thought* the AUX bitmap uses THPs when in fact it did not. That caused confusion.
> 
> (All this illuminates how badly thought out the ReservedSpace API really is. We pass in page size to the constructor, but then need to commit manually; THPs actually use madvise on commit, not on reservation, so we need to pass page size in *again* to commit. Ideally, ReservedSpace should handle committing for us with the page size it saved from reservation.)
> 
> Note that this only takes care of preventing THP formation in "madvise" mode. In "always" mode, the kernel will do THP coalescation always. We could prevent it from doing so by advising against it via madvise, but that would require extension of the platform generic reservation APIs and is left for a different RFE. Ideally, nobody should use THP "always" mode.

This pull request has now been integrated.

Changeset: 017e0c78
Author:    Thomas Stuefe <stuefe at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/017e0c7850e305877e3e0b1d4644b5605225e07c
Stats:     11 lines in 1 file changed: 9 ins; 0 del; 2 mod

8310388: Shenandoah: Auxiliary bitmap is not madvised for THP

Reviewed-by: shade, kdnilsen

-------------

PR: https://git.openjdk.org/jdk/pull/14953

From kdnilsen at openjdk.org  Fri Aug  4 19:24:10 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 4 Aug 2023 19:24:10 GMT
Subject: RFR: 8313574: GenShen: Completing a global mark should also
 complete an old mark
In-Reply-To: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>
References: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>
Message-ID: <wKbw1TM-wVK6bA1TwpWd_4oBhKog2OjMsKn-uwMSLic=.bad9a72e-0531-43a7-8d73-bbff1a51b163@github.com>

On Fri, 4 Aug 2023 18:17:29 GMT, William Kemper <wkemper at openjdk.org> wrote:

> The mark context is shared, but each generation maintains a separate flag (`ShenandoahGeneration::_is_marking_complete`) to record the status of the mark. The remembered set scan will use the mark bitmap only if the old generation mark is complete. Otherwise, it walks the old regions and requires them to be parseable.
> 
> Presently, we make the heap parseable after global marking finishes, but this work can be deferred to the start of an old mark if we had global mark record completion of old marking.

Looks good.  Thanks for prompt fix.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/304#pullrequestreview-1563449376

From ysr at openjdk.org  Fri Aug  4 19:45:03 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 4 Aug 2023 19:45:03 GMT
Subject: RFR: 8313574: GenShen: Completing a global mark should also
 complete an old mark
In-Reply-To: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>
References: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>
Message-ID: <AKR_JonhZ-OoDwF3IdEXg4ErCmUt1fTGSXWSfEBlSq0=.3cbc78e8-b074-41fe-a822-d3f11ee011eb@github.com>

On Fri, 4 Aug 2023 18:17:29 GMT, William Kemper <wkemper at openjdk.org> wrote:

> The mark context is shared, but each generation maintains a separate flag (`ShenandoahGeneration::_is_marking_complete`) to record the status of the mark. The remembered set scan will use the mark bitmap only if the old generation mark is complete. Otherwise, it walks the old regions and requires them to be parseable.
> 
> Presently, we make the heap parseable after global marking finishes, but this work can be deferred to the start of an old mark if we had global mark record completion of old marking.

LGTM.

Please add a stack retrace of the failure you'd typically see (and perhaps the failing test) that identified this issue into the associated JBS ticket.

Fix looks good.

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/304#pullrequestreview-1563469340

From ysr at openjdk.org  Fri Aug  4 19:45:03 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 4 Aug 2023 19:45:03 GMT
Subject: RFR: 8313574: GenShen: Completing a global mark should also
 complete an old mark
In-Reply-To: <AKR_JonhZ-OoDwF3IdEXg4ErCmUt1fTGSXWSfEBlSq0=.3cbc78e8-b074-41fe-a822-d3f11ee011eb@github.com>
References: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>
 <AKR_JonhZ-OoDwF3IdEXg4ErCmUt1fTGSXWSfEBlSq0=.3cbc78e8-b074-41fe-a822-d3f11ee011eb@github.com>
Message-ID: <3pmB8E9XwOkEGMA8oULv5hRo3CbwhihP2Ui2q8NcL8E=.0299aacf-b9c7-4247-abe5-e06e0c1ef9f4@github.com>

On Fri, 4 Aug 2023 19:40:08 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> LGTM.
> 
> Please add a stack retrace of the failure you'd typically see (and perhaps the failing test) that identified this issue into the associated JBS ticket.
> 
> Fix looks good.

Also I am guessing the JBS issue should be a "bug" rather than just called an enhancement (which I agree it is also :-). Or may be the bug is tracked in a different ticket and will be closed as a duplicate of this enhancement?

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/304#issuecomment-1666089123

From ysr at openjdk.org  Tue Aug  8 22:16:51 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 8 Aug 2023 22:16:51 GMT
Subject: RFR: 8311883: [Genshen] Adaptive tenuring threshold [v10]
In-Reply-To: <JtI01IJXIhWHSX4VI7paTeI2CnBmKQ5eqqJznX0RbGM=.d2535a64-074c-45bb-b3df-c77d2473f369@github.com>
References: <JtI01IJXIhWHSX4VI7paTeI2CnBmKQ5eqqJznX0RbGM=.d2535a64-074c-45bb-b3df-c77d2473f369@github.com>
Message-ID: <OMoOkF7lA1xfYBJUE96-O8vpoywhlSv0zjKiIIjOri0=.f65a4714-b421-4f50-b428-dd00a1a303dc@github.com>

> JDK-8311883 [GenShen] Adaptive tenuring
> 
> I am opening this previously draft PR for formal preliminary review. It has already benefited from review feedback from a code walkthrough of an earlier version of the code. Most of that feedback and the corrections thereof are to be found in the comments in this PR. I have addressed a large majority of those comments, and am working on the last one that I plan to address as part of this PR. For the ones that I don't plan to address in this PR, I will create follow up tickets. Those will be added in the responses for the remaining feedback comments recorded in this PR's conversation.
> 
> Preliminary testing w/SPECjbb didn't yield reliable performance data from which to infer any performance improvements stemming from enabling adaptive tenuring. I believe that was because of the way SPECjbb is run, which causes excessive degenerate and full gc's. I plan to collect SPECjbb numbers with a fixed lower max HBIR so as to be able to discern performance differences from this change, as well as Extremem workloads. Those will be added here once ready over the next few days.

Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 81 commits:

 - Merge branch 'master' into adaptive_tenuring
 - Reposition a mis-positioned assert as well as surrounding if-test order.
 - Fix global flags to make adaptive tenuring with census at mark the
   default, along with ignoring mortality rates of cohorts older than the
   selected tenuring age at previous epoch.
 - Flag to ignore mortality rate of cohorts older than the tenuring age of
   most recent preious cycle when computing new tenuring threshold.
 - Clear partial census data before a new marking cycle.
   Allow consideration of mortality rate of youngest cohort for tenuring
   decisions. (It was previously excluded because census data didn't
   include the youngest cohort allocated in the current epoch.)
   
   Still to do: avoid tenuring revisionism a la Kelvin's suggestion.
 - jcheck whitespace.
 - Merge branch 'master' into adaptive_tenuring
 - Separate out clearing and testing of local and global portions of census
   data, towards expunging partial/interrupted census data as a next step.
 - Merge branch 'master' into adaptive_tenuring
 - Changes from @earthling-amzn review.
 - ... and 71 more: https://git.openjdk.org/shenandoah/compare/3535f1a0...6bde9d23

-------------

Changes: https://git.openjdk.org/shenandoah/pull/289/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=289&range=09
  Stats: 994 lines in 30 files changed: 876 ins; 45 del; 73 mod
  Patch: https://git.openjdk.org/shenandoah/pull/289.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/289/head:pull/289

PR: https://git.openjdk.org/shenandoah/pull/289

From rkennke at openjdk.org  Fri Aug 11 18:50:00 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 11 Aug 2023 18:50:00 GMT
Subject: RFR: 8305896: Alternative full GC forwarding [v51]
In-Reply-To: <yFs73BPFgB89xCPY4k63X4VTz13N8ELJf1lGQlE6UpA=.85ee4f25-a4ee-41d0-b4c2-ed9ff2cb1bb1@github.com>
References: <yFs73BPFgB89xCPY4k63X4VTz13N8ELJf1lGQlE6UpA=.85ee4f25-a4ee-41d0-b4c2-ed9ff2cb1bb1@github.com>
Message-ID: <00gSQKVRc45BizoIAs77R6-ei2xC9h2sG6Fn5ZwV19Q=.4103b586-038c-41bb-b4ca-b3d6aacea083@github.com>

> Currently, the full-GC modes of Serial, Shenandoah and G1 GCs are forwarding objects by over-writing the object header with the new object location. Unfortunately, for compact object headers ([JDK-8294992](https://bugs.openjdk.org/browse/JDK-8294992)) this would not work, because the crucial class information is also stored in the header, and we could no longer iterate over objects until the headers would be restored. Also, the preserved-headers tables would grow quite large.
> 
> I propose to use an alternative algorithm for full-GC (sliding-GC) forwarding that uses a special encoding so that the forwarding information fits into the lowest 32 bits of the header.
> 
> It exploits the insight that, with sliding GCs, objects from one region will only ever be forwarded to one of two possible target regions. For this to work, we need to divide the heap into equal-sized regions. This is already the case for Shenandoah and G1, and can easily be overlaid for Serial GC, by assuming either the whole heap as a single region (if it fits) or by using SpaceAlignment-sized virtual regions.
> 
> We also build and maintain a table that has N elements, where N is the number of regions. Each entry is two addresses, which are the start-address of the possible target regions for each source region.
> 
> With this, forwarding information would be encoded like this:
>  - Bits 0 and 1: same as before, we put in '11' to indicate that the object is forwarded.
>  - Bit 2: Used for 'fallback'-forwarding (see below)
>  - Bit 3: Selects the target region 0 or 1. Look up the base address in the table (see above)
>  - Bits 4..31 The number of heap words from the target base address
> 
> This works well for all sliding GCs in Serial, G1 and Shenandoah. The exception is in G1, there is a special mode called 'serial compaction' which acts as a last-last-ditch effort to squeeze more space out of the heap by re-forwarding the tails of the compaction chains. Unfortunately, this breaks the assumption of the sliding-forwarding-table. When that happens, we initialize a fallback table, which is a simple open hash-table, and set the Bit 2 in the forwarding to indicate that we shall look up the forwardee in the fallback-table.
> 
> All the table accesses can be done unsynchronized because:
> - Serial GC is single-threaded anyway
> - In G1 and Shenandoah, GC worker threads divide up the work such that each worker does disjoint sets of regions.
> - G1 serial compaction is single-threaded
> 
> The change introduces a new (experimental) flag -...

Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 118 commits:

 - Merge branch 'master' into JDK-8305896
 - Various cleanups
 - Some @shipilev comments from downstream review
 - Further templatize Serial GC's adjust_pointers()
 - Merge branch 'master' into JDK-8305896
 - Specialize full-GC loops to get UseAltGCForwarding flag check out of hot paths
 - Remove G1-only assert for fallback forwarding, and comment with explanation
 - Merge branch 'master' into JDK-8305896
 - Replace homegrown FallbackTable with a ResourceHashtable based impl
 - Merge remote-tracking branch 'origin/JDK-8305896' into JDK-8305896
 - ... and 108 more: https://git.openjdk.org/jdk/compare/12326770...d2b41e25

-------------

Changes: https://git.openjdk.org/jdk/pull/13582/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=13582&range=50
  Stats: 1201 lines in 39 files changed: 1077 ins; 10 del; 114 mod
  Patch: https://git.openjdk.org/jdk/pull/13582.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/13582/head:pull/13582

PR: https://git.openjdk.org/jdk/pull/13582

From wkemper at openjdk.org  Fri Aug 11 18:54:28 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 11 Aug 2023 18:54:28 GMT
Subject: Integrated: Merge openjdk/jdk:master
Message-ID: <JtZTswOEbOQYX1jOyfoH2J9NZ-oZNZ_bnWpjdXI9DmM=.dc081377-da0e-4cd7-b680-93200596278f@github.com>

Merge tag jdk-22+10

-------------

Commit messages:
 - Merge tag 'jdk-22+10' into merge-jdk-22-10
 - Merge
 - 8312610: GenShen: Old generation available is unintentionally restricted by mutator's available memory
 - Merge
 - Merge
 - 8311602: GenShen: Decouple generational mode heuristics
 - 8312321: GenShen: Remembered set scan may encounter garbage objects
 - 8312422: GenShen: In-place region promotion state may carry over when evacuation fails
 - 8312322: GenShen: Cancelled GCs may become stuck in self-cancellation loop
 - Merge
 - ... and 296 more: https://git.openjdk.org/shenandoah/compare/242a2e63...54fced09

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah/pull/305/files
  Stats: 20548 lines in 208 files changed: 18689 ins; 903 del; 956 mod
  Patch: https://git.openjdk.org/shenandoah/pull/305.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/305/head:pull/305

PR: https://git.openjdk.org/shenandoah/pull/305

From wkemper at openjdk.org  Fri Aug 11 18:54:29 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 11 Aug 2023 18:54:29 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <JtZTswOEbOQYX1jOyfoH2J9NZ-oZNZ_bnWpjdXI9DmM=.dc081377-da0e-4cd7-b680-93200596278f@github.com>
References: <JtZTswOEbOQYX1jOyfoH2J9NZ-oZNZ_bnWpjdXI9DmM=.dc081377-da0e-4cd7-b680-93200596278f@github.com>
Message-ID: <4nHJZAFRe6LmMhmtdbohoHxlhkDeYprAd5DmJen-QsY=.7bde428b-e22e-48b2-8713-3e956cbeffd6@github.com>

On Fri, 11 Aug 2023 18:35:19 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merge tag jdk-22+10

This pull request has now been integrated.

Changeset: a4f6425e
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/a4f6425e80a16bdc81d2e5e2782adac158148fe6
Stats:     13151 lines in 635 files changed: 5121 ins; 4057 del; 3973 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/305

From wkemper at openjdk.org  Mon Aug 14 19:47:17 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 14 Aug 2023 19:47:17 GMT
Subject: RFR: 8313574: GenShen: Completing a global mark should also
 complete an old mark [v2]
In-Reply-To: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>
References: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>
Message-ID: <CHef0YpeYMXjVHC7QYk75nGhLUFIVprRAE-YQho9PNM=.d7387a48-15a8-4362-8184-8bacc2d4d94b@github.com>

> The mark context is shared, but each generation maintains a separate flag (`ShenandoahGeneration::_is_marking_complete`) to record the status of the mark. The remembered set scan will use the mark bitmap only if the old generation mark is complete. Otherwise, it walks the old regions and requires them to be parseable.
> 
> Presently, we make the heap parseable after global marking finishes, but this work can be deferred to the start of an old mark if we had global mark record completion of old marking.

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:

 - Merge branch 'shenandoah-master' into global-completes-old-mark
 - Fix assertions and update comments
 - Prepare for mixed collections after global cycle
 - Merge branch upstream into global-completes-old-mark
 - Have global mark also complete young and old marks
   
   This will have the remembered set scan use the mark bitmap following a global mark and eliminates the need to coalesce and fill during a global collection.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/304/files
  - new: https://git.openjdk.org/shenandoah/pull/304/files/be1bf322..2bf50f3f

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=304&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=304&range=00-01

  Stats: 13236 lines in 641 files changed: 5149 ins; 4068 del; 4019 mod
  Patch: https://git.openjdk.org/shenandoah/pull/304.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/304/head:pull/304

PR: https://git.openjdk.org/shenandoah/pull/304

From wkemper at openjdk.org  Mon Aug 14 19:47:17 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 14 Aug 2023 19:47:17 GMT
Subject: RFR: 8313574: GenShen: Completing a global mark should also
 complete an old mark
In-Reply-To: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>
References: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>
Message-ID: <P6bF1ca0ViNrGjyULGRmBh6psxOeF-8Ur1YPgVAmKFk=.e75ed8d6-2092-4f52-b17b-71a8430851ec@github.com>

On Fri, 4 Aug 2023 18:17:29 GMT, William Kemper <wkemper at openjdk.org> wrote:

> The mark context is shared, but each generation maintains a separate flag (`ShenandoahGeneration::_is_marking_complete`) to record the status of the mark. The remembered set scan will use the mark bitmap only if the old generation mark is complete. Otherwise, it walks the old regions and requires them to be parseable.
> 
> Presently, we make the heap parseable after global marking finishes, but this work can be deferred to the start of an old mark if we had global mark record completion of old marking.

Converted to draft because of failures in Dacapo suite.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/304#issuecomment-1666136456

From ysr at openjdk.org  Mon Aug 14 20:58:25 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 14 Aug 2023 20:58:25 GMT
Subject: RFR: 8311883: [Genshen] Adaptive tenuring threshold [v11]
In-Reply-To: <JtI01IJXIhWHSX4VI7paTeI2CnBmKQ5eqqJznX0RbGM=.d2535a64-074c-45bb-b3df-c77d2473f369@github.com>
References: <JtI01IJXIhWHSX4VI7paTeI2CnBmKQ5eqqJznX0RbGM=.d2535a64-074c-45bb-b3df-c77d2473f369@github.com>
Message-ID: <ZbF82OtcrsNnycozj7G0lfK2tLj6VgvFrWTZqiPMStM=.664618c7-5060-418b-bd92-b7dc48b3443d@github.com>

> JDK-8311883 [GenShen] Adaptive tenuring
> 
> I am opening this previously draft PR for formal preliminary review. It has already benefited from review feedback from a code walkthrough of an earlier version of the code. Most of that feedback and the corrections thereof are to be found in the comments in this PR. I have addressed a large majority of those comments, and am working on the last one that I plan to address as part of this PR. For the ones that I don't plan to address in this PR, I will create follow up tickets. Those will be added in the responses for the remaining feedback comments recorded in this PR's conversation.
> 
> Preliminary testing w/SPECjbb didn't yield reliable performance data from which to infer any performance improvements stemming from enabling adaptive tenuring. I believe that was because of the way SPECjbb is run, which causes excessive degenerate and full gc's. I plan to collect SPECjbb numbers with a fixed lower max HBIR so as to be able to discern performance differences from this change, as well as Extremem workloads. Those will be added here once ready over the next few days.

Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 82 commits:

 - Merge branch 'master' into adaptive_tenuring
 - Merge branch 'master' into adaptive_tenuring
 - Reposition a mis-positioned assert as well as surrounding if-test order.
 - Fix global flags to make adaptive tenuring with census at mark the
   default, along with ignoring mortality rates of cohorts older than the
   selected tenuring age at previous epoch.
 - Flag to ignore mortality rate of cohorts older than the tenuring age of
   most recent preious cycle when computing new tenuring threshold.
 - Clear partial census data before a new marking cycle.
   Allow consideration of mortality rate of youngest cohort for tenuring
   decisions. (It was previously excluded because census data didn't
   include the youngest cohort allocated in the current epoch.)
   
   Still to do: avoid tenuring revisionism a la Kelvin's suggestion.
 - jcheck whitespace.
 - Merge branch 'master' into adaptive_tenuring
 - Separate out clearing and testing of local and global portions of census
   data, towards expunging partial/interrupted census data as a next step.
 - Merge branch 'master' into adaptive_tenuring
 - ... and 72 more: https://git.openjdk.org/shenandoah/compare/a4f6425e...13fa9ac9

-------------

Changes: https://git.openjdk.org/shenandoah/pull/289/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=289&range=10
  Stats: 994 lines in 30 files changed: 876 ins; 45 del; 73 mod
  Patch: https://git.openjdk.org/shenandoah/pull/289.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/289/head:pull/289

PR: https://git.openjdk.org/shenandoah/pull/289

From ysr at openjdk.org  Mon Aug 14 23:01:42 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 14 Aug 2023 23:01:42 GMT
Subject: RFR: 8313574: GenShen: Completing a global mark should also
 complete an old mark [v2]
In-Reply-To: <CHef0YpeYMXjVHC7QYk75nGhLUFIVprRAE-YQho9PNM=.d7387a48-15a8-4362-8184-8bacc2d4d94b@github.com>
References: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>
 <CHef0YpeYMXjVHC7QYk75nGhLUFIVprRAE-YQho9PNM=.d7387a48-15a8-4362-8184-8bacc2d4d94b@github.com>
Message-ID: <QqIa6DIZ57-NHeLgq1i_K_QX_Bstug-JZ1b5pd5b3mw=.ced61261-5850-4563-9385-65d0f4367b83@github.com>

On Mon, 14 Aug 2023 19:47:17 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> The mark context is shared, but each generation maintains a separate flag (`ShenandoahGeneration::_is_marking_complete`) to record the status of the mark. The remembered set scan will use the mark bitmap only if the old generation mark is complete. Otherwise, it walks the old regions and requires them to be parseable.
>> 
>> Presently, we make the heap parseable after global marking finishes, but this work can be deferred to the start of an old mark if we had global mark record completion of old marking.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
> 
>  - Merge branch 'shenandoah-master' into global-completes-old-mark
>  - Fix assertions and update comments
>  - Prepare for mixed collections after global cycle
>  - Merge branch upstream into global-completes-old-mark
>  - Have global mark also complete young and old marks
>    
>    This will have the remembered set scan use the mark bitmap following a global mark and eliminates the need to coalesce and fill during a global collection.

Left a few comments, and going over all of the changes again to make sure I understand the changes completely again before I sign off on the review.

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.hpp line 149:

> 147:   unsigned int get_coalesce_and_fill_candidates(ShenandoahHeapRegion** buffer);
> 148:   bool has_coalesce_and_fill_candidates() const { return _last_old_region > 0; }
> 149:   size_t coalesce_and_fill_candidates() const { return _last_old_region; }

The return value from `coalesce_and_fill_candidates()` (which by the way could be confused for an imperative action verb) lready available in `last_old_region_index()` although might be a good idea to rename appropriately? Also fine to leave as is, but it sounds to me like one name might work for both uses.

src/hotspot/share/gc/shenandoah/shenandoahDegeneratedGC.cpp line 105:

> 103:       assert(!heap->is_old_gc_active(), "Old GC should not be active during global cycle");
> 104:     }
> 105: 

Is this not true?

You could rearrange the test into the assertion check though:


    assert(!_generation->is_global() || !heap->is_old_gc_active, "Old GC should not be active during global cycle");

(or its de-Morganization, if you prefer).

src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 417:

> 415: //           |   |        |     +-----------------+
> 416: //           |   |        |     |    WAITING FOR  |
> 417: //           |   +--------|---> |    EVACUATIONS  |

This transition has arrows at both ends (IDLE and WAITING FOR EVACUATIONS).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/304#pullrequestreview-1577748885
PR Review Comment: https://git.openjdk.org/shenandoah/pull/304#discussion_r1294028628
PR Review Comment: https://git.openjdk.org/shenandoah/pull/304#discussion_r1294031288
PR Review Comment: https://git.openjdk.org/shenandoah/pull/304#discussion_r1294041386

From ysr at openjdk.org  Mon Aug 14 23:28:43 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 14 Aug 2023 23:28:43 GMT
Subject: RFR: 8313574: GenShen: Completing a global mark should also
 complete an old mark [v2]
In-Reply-To: <CHef0YpeYMXjVHC7QYk75nGhLUFIVprRAE-YQho9PNM=.d7387a48-15a8-4362-8184-8bacc2d4d94b@github.com>
References: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>
 <CHef0YpeYMXjVHC7QYk75nGhLUFIVprRAE-YQho9PNM=.d7387a48-15a8-4362-8184-8bacc2d4d94b@github.com>
Message-ID: <5TSTsertEpQj0W-tHtduKQA4KqUGxy96z7Tk5rIfoGo=.abbe3e6c-e56b-4b73-a4cf-8ecc6b976a88@github.com>

On Mon, 14 Aug 2023 19:47:17 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> The mark context is shared, but each generation maintains a separate flag (`ShenandoahGeneration::_is_marking_complete`) to record the status of the mark. The remembered set scan will use the mark bitmap only if the old generation mark is complete. Otherwise, it walks the old regions and requires them to be parseable.
>> 
>> Presently, we make the heap parseable after global marking finishes, but this work can be deferred to the start of an old mark if we had global mark record completion of old marking.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
> 
>  - Merge branch 'shenandoah-master' into global-completes-old-mark
>  - Fix assertions and update comments
>  - Prepare for mixed collections after global cycle
>  - Merge branch upstream into global-completes-old-mark
>  - Have global mark also complete young and old marks
>    
>    This will have the remembered set scan use the mark bitmap following a global mark and eliminates the need to coalesce and fill during a global collection.

Changes LGTM, modulo a few minor comments in previous set of comments.

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/304#pullrequestreview-1577791481

From wkemper at openjdk.org  Tue Aug 15 16:13:59 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 15 Aug 2023 16:13:59 GMT
Subject: RFR: 8313574: GenShen: Completing a global mark should also
 complete an old mark [v2]
In-Reply-To: <QqIa6DIZ57-NHeLgq1i_K_QX_Bstug-JZ1b5pd5b3mw=.ced61261-5850-4563-9385-65d0f4367b83@github.com>
References: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>
 <CHef0YpeYMXjVHC7QYk75nGhLUFIVprRAE-YQho9PNM=.d7387a48-15a8-4362-8184-8bacc2d4d94b@github.com>
 <QqIa6DIZ57-NHeLgq1i_K_QX_Bstug-JZ1b5pd5b3mw=.ced61261-5850-4563-9385-65d0f4367b83@github.com>
Message-ID: <HuKuLQxH00oeGZiVpFkuBkh5PSiuSfC3IoM2W8z4HPo=.edb78a3b-1905-4241-baba-2fd27e948922@github.com>

On Mon, 14 Aug 2023 22:35:46 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
>> 
>>  - Merge branch 'shenandoah-master' into global-completes-old-mark
>>  - Fix assertions and update comments
>>  - Prepare for mixed collections after global cycle
>>  - Merge branch upstream into global-completes-old-mark
>>  - Have global mark also complete young and old marks
>>    
>>    This will have the remembered set scan use the mark bitmap following a global mark and eliminates the need to coalesce and fill during a global collection.
>
> src/hotspot/share/gc/shenandoah/shenandoahDegeneratedGC.cpp line 105:
> 
>> 103:       assert(!heap->is_old_gc_active(), "Old GC should not be active during global cycle");
>> 104:     }
>> 105: 
> 
> Is this not true?
> 
> You could rearrange the test into the assertion check though:
> 
> 
>     assert(!_generation->is_global() || !heap->is_old_gc_active, "Old GC should not be active during global cycle");
> 
> (or its de-Morganization, if you prefer).

With this change, if the global cycle degenerates after final mark there could now be regions waiting to be coalesced and filled, so the old gc state is not idle.

> src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 417:
> 
>> 415: //           |   |        |     +-----------------+
>> 416: //           |   |        |     |    WAITING FOR  |
>> 417: //           |   +--------|---> |    EVACUATIONS  |
> 
> This transition has arrows at both ends (IDLE and WAITING FOR EVACUATIONS).

That is intentional, I shall also add an arrow at both ends for IDLE and WAITING FOR FILL. Global cycles do not progress the old generation through marking/bootstrapping so they are allowed to move the old generation from IDLE to one of these states at the end of global mark.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/304#discussion_r1294825052
PR Review Comment: https://git.openjdk.org/shenandoah/pull/304#discussion_r1294828183

From wkemper at openjdk.org  Tue Aug 15 16:36:02 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 15 Aug 2023 16:36:02 GMT
Subject: RFR: 8313574: GenShen: Completing a global mark should also
 complete an old mark [v2]
In-Reply-To: <QqIa6DIZ57-NHeLgq1i_K_QX_Bstug-JZ1b5pd5b3mw=.ced61261-5850-4563-9385-65d0f4367b83@github.com>
References: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>
 <CHef0YpeYMXjVHC7QYk75nGhLUFIVprRAE-YQho9PNM=.d7387a48-15a8-4362-8184-8bacc2d4d94b@github.com>
 <QqIa6DIZ57-NHeLgq1i_K_QX_Bstug-JZ1b5pd5b3mw=.ced61261-5850-4563-9385-65d0f4367b83@github.com>
Message-ID: <BTXBL6cOU9vQGhHLJwPRs8qMJdHbSQNkQ6pJq1Cm3c8=.d8e65e04-91bd-4b0a-b9f0-c9a8019622b2@github.com>

On Mon, 14 Aug 2023 22:30:54 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
>> 
>>  - Merge branch 'shenandoah-master' into global-completes-old-mark
>>  - Fix assertions and update comments
>>  - Prepare for mixed collections after global cycle
>>  - Merge branch upstream into global-completes-old-mark
>>  - Have global mark also complete young and old marks
>>    
>>    This will have the remembered set scan use the mark bitmap following a global mark and eliminates the need to coalesce and fill during a global collection.
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.hpp line 149:
> 
>> 147:   unsigned int get_coalesce_and_fill_candidates(ShenandoahHeapRegion** buffer);
>> 148:   bool has_coalesce_and_fill_candidates() const { return _last_old_region > 0; }
>> 149:   size_t coalesce_and_fill_candidates() const { return _last_old_region; }
> 
> The return value from `coalesce_and_fill_candidates()` (which by the way could be confused for an imperative action verb) lready available in `last_old_region_index()` although might be a good idea to rename appropriately? Also fine to leave as is, but it sounds to me like one name might work for both uses.

I agree, one (good) name for both is best.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/304#discussion_r1294849669

From ysr at openjdk.org  Tue Aug 15 16:36:06 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 15 Aug 2023 16:36:06 GMT
Subject: RFR: 8313574: GenShen: Completing a global mark should also
 complete an old mark [v2]
In-Reply-To: <HuKuLQxH00oeGZiVpFkuBkh5PSiuSfC3IoM2W8z4HPo=.edb78a3b-1905-4241-baba-2fd27e948922@github.com>
References: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>
 <CHef0YpeYMXjVHC7QYk75nGhLUFIVprRAE-YQho9PNM=.d7387a48-15a8-4362-8184-8bacc2d4d94b@github.com>
 <QqIa6DIZ57-NHeLgq1i_K_QX_Bstug-JZ1b5pd5b3mw=.ced61261-5850-4563-9385-65d0f4367b83@github.com>
 <HuKuLQxH00oeGZiVpFkuBkh5PSiuSfC3IoM2W8z4HPo=.edb78a3b-1905-4241-baba-2fd27e948922@github.com>
Message-ID: <koSa9gF1ZrYPXy1xILom4wfsrw00Kr2Fo6eEhWur8qU=.db3faddd-9e13-40bc-840b-7e1237bc8eb0@github.com>

On Tue, 15 Aug 2023 16:10:42 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 417:
>> 
>>> 415: //           |   |        |     +-----------------+
>>> 416: //           |   |        |     |    WAITING FOR  |
>>> 417: //           |   +--------|---> |    EVACUATIONS  |
>> 
>> This transition has arrows at both ends (IDLE and WAITING FOR EVACUATIONS).
>
> That is intentional, I shall also add an arrow at both ends for IDLE and WAITING FOR FILL. Global cycles do not progress the old generation through marking/bootstrapping so they are allowed to move the old generation from IDLE to one of these states at the end of global mark.

I see, so a bidirectional arrow?

It would of course be ideal (in the fullness of time) if there were events labeling the transitions, even if this required a table for the event alphabet.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/304#discussion_r1294853115

From ysr at openjdk.org  Tue Aug 15 17:54:51 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 15 Aug 2023 17:54:51 GMT
Subject: RFR: 8311883: [Genshen] Adaptive tenuring threshold [v11]
In-Reply-To: <ZbF82OtcrsNnycozj7G0lfK2tLj6VgvFrWTZqiPMStM=.664618c7-5060-418b-bd92-b7dc48b3443d@github.com>
References: <JtI01IJXIhWHSX4VI7paTeI2CnBmKQ5eqqJznX0RbGM=.d2535a64-074c-45bb-b3df-c77d2473f369@github.com>
 <ZbF82OtcrsNnycozj7G0lfK2tLj6VgvFrWTZqiPMStM=.664618c7-5060-418b-bd92-b7dc48b3443d@github.com>
Message-ID: <SYifFVF5pALJfYEvVJtEJAY2A2UP7IdBiWWmXINCVlA=.d6ee5a00-c323-43d4-ae82-d310364434e0@github.com>

On Mon, 14 Aug 2023 20:58:25 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> JDK-8311883 [GenShen] Adaptive tenuring
>> 
>> I am opening this previously draft PR for formal preliminary review. It has already benefited from review feedback from a code walkthrough of an earlier version of the code. Most of that feedback and the corrections thereof are to be found in the comments in this PR. I have addressed a large majority of those comments, and am working on the last one that I plan to address as part of this PR. For the ones that I don't plan to address in this PR, I will create follow up tickets. Those will be added in the responses for the remaining feedback comments recorded in this PR's conversation.
>> 
>> Preliminary testing w/SPECjbb didn't yield reliable performance data from which to infer any performance improvements stemming from enabling adaptive tenuring. I believe that was because of the way SPECjbb is run, which causes excessive degenerate and full gc's. I plan to collect SPECjbb numbers with a fixed lower max HBIR so as to be able to discern performance differences from this change, as well as Extremem workloads. Those will be added here once ready over the next few days.
>
> Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 82 commits:
> 
>  - Merge branch 'master' into adaptive_tenuring
>  - Merge branch 'master' into adaptive_tenuring
>  - Reposition a mis-positioned assert as well as surrounding if-test order.
>  - Fix global flags to make adaptive tenuring with census at mark the
>    default, along with ignoring mortality rates of cohorts older than the
>    selected tenuring age at previous epoch.
>  - Flag to ignore mortality rate of cohorts older than the tenuring age of
>    most recent preious cycle when computing new tenuring threshold.
>  - Clear partial census data before a new marking cycle.
>    Allow consideration of mortality rate of youngest cohort for tenuring
>    decisions. (It was previously excluded because census data didn't
>    include the youngest cohort allocated in the current epoch.)
>    
>    Still to do: avoid tenuring revisionism a la Kelvin's suggestion.
>  - jcheck whitespace.
>  - Merge branch 'master' into adaptive_tenuring
>  - Separate out clearing and testing of local and global portions of census
>    data, towards expunging partial/interrupted census data as a next step.
>  - Merge branch 'master' into adaptive_tenuring
>  - ... and 72 more: https://git.openjdk.org/shenandoah/compare/a4f6425e...13fa9ac9

> JDK-8311883 [GenShen] Adaptive tenuring
> 
...
> Preliminary testing w/SPECjbb didn't yield reliable performance data from which to infer any performance improvements stemming from enabling adaptive tenuring. I believe that was because of the way SPECjbb is run, which causes excessive degenerate and full gc's. I plan to collect SPECjbb numbers with a fixed lower max HBIR so as to be able to discern performance differences from this change, as well as Extremem workloads. Those will be added here once ready over the next few days.


Update on performance efforts:

I've run SPECjbb and Extremem under various configurations of the benchmarks and of Generational Shenandoah in order to separate the performance of reference and specimen. Unfortunately, it appears as if the triggering mechanisms often cause degenerate collections which has been a bit of a challenge to completely eliminate. This appears to inject enough variability into the results that any differences in performance between the two is difficult to discern. 

SPECjbb configurations included fixed preset IR among others, and Extremem configurations (thanks to Kelvin for advice) included various load characteristics. Generational Shenandoah configurations included guaranteed collection interval triggers (variously and in combination for young & old), and disabling garbage density as a criterion for inclusion in the collection set.

The only configuration in which I was able to fully eliminate degenerate GCs for these workloads was Extremem "hefty load" with a 45 g heap. This produced exactly one degenerate GC in each case during the early part of the run. Setting a guaranteed collection interval eliminated this degenerate collection.

A comparison was then made for the time in each of the concurrent phases, both total and average over all cycles. The differences were a wash.

The performance numbers for adaptive tenuring were based on the default settings to be found in
src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp.

While no positive or negative performance impacts have yet been measured, the framework is now in place to allow further experimentation in the future in order to investigate and potentially extract any potential benefits from adaptive tenuring.

I'm happy to integrate this now, or do further future experiments, including coupling this with the size budgeting for generations to demonstrate impact before this goes in. I am open to any and all suggestions.

I'll attach to the ticket the design document upon which the current adaptive tenuring is based, and where we discuss some alternatives and some pitfalls we encountered along the way.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/289#issuecomment-1679357909

From kdnilsen at openjdk.org  Tue Aug 15 18:15:44 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 15 Aug 2023 18:15:44 GMT
Subject: RFR: 8311883: [Genshen] Adaptive tenuring threshold [v11]
In-Reply-To: <ZbF82OtcrsNnycozj7G0lfK2tLj6VgvFrWTZqiPMStM=.664618c7-5060-418b-bd92-b7dc48b3443d@github.com>
References: <JtI01IJXIhWHSX4VI7paTeI2CnBmKQ5eqqJznX0RbGM=.d2535a64-074c-45bb-b3df-c77d2473f369@github.com>
 <ZbF82OtcrsNnycozj7G0lfK2tLj6VgvFrWTZqiPMStM=.664618c7-5060-418b-bd92-b7dc48b3443d@github.com>
Message-ID: <xVHM4bXgk7Qy1hJSWozNAZQ7iBGR0YWivZ3C8gEcyrc=.66246b2e-2889-4db1-ae2d-a626e5e294d1@github.com>

On Mon, 14 Aug 2023 20:58:25 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> JDK-8311883 [GenShen] Adaptive tenuring
>> 
>> I am opening this previously draft PR for formal preliminary review. It has already benefited from review feedback from a code walkthrough of an earlier version of the code. Most of that feedback and the corrections thereof are to be found in the comments in this PR. I have addressed a large majority of those comments, and am working on the last one that I plan to address as part of this PR. For the ones that I don't plan to address in this PR, I will create follow up tickets. Those will be added in the responses for the remaining feedback comments recorded in this PR's conversation.
>> 
>> Preliminary testing w/SPECjbb didn't yield reliable performance data from which to infer any performance improvements stemming from enabling adaptive tenuring. I believe that was because of the way SPECjbb is run, which causes excessive degenerate and full gc's. I plan to collect SPECjbb numbers with a fixed lower max HBIR so as to be able to discern performance differences from this change, as well as Extremem workloads. Those will be added here once ready over the next few days.
>
> Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 82 commits:
> 
>  - Merge branch 'master' into adaptive_tenuring
>  - Merge branch 'master' into adaptive_tenuring
>  - Reposition a mis-positioned assert as well as surrounding if-test order.
>  - Fix global flags to make adaptive tenuring with census at mark the
>    default, along with ignoring mortality rates of cohorts older than the
>    selected tenuring age at previous epoch.
>  - Flag to ignore mortality rate of cohorts older than the tenuring age of
>    most recent preious cycle when computing new tenuring threshold.
>  - Clear partial census data before a new marking cycle.
>    Allow consideration of mortality rate of youngest cohort for tenuring
>    decisions. (It was previously excluded because census data didn't
>    include the youngest cohort allocated in the current epoch.)
>    
>    Still to do: avoid tenuring revisionism a la Kelvin's suggestion.
>  - jcheck whitespace.
>  - Merge branch 'master' into adaptive_tenuring
>  - Separate out clearing and testing of local and global portions of census
>    data, towards expunging partial/interrupted census data as a next step.
>  - Merge branch 'master' into adaptive_tenuring
>  - ... and 72 more: https://git.openjdk.org/shenandoah/compare/a4f6425e...13fa9ac9

Thanks.  This was a lot of work.  Good to see it integrated.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/289#pullrequestreview-1579132899

From wkemper at openjdk.org  Tue Aug 15 22:11:20 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 15 Aug 2023 22:11:20 GMT
Subject: RFR: 8313574: GenShen: Completing a global mark should also
 complete an old mark [v3]
In-Reply-To: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>
References: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>
Message-ID: <Z3m61u3Kz6Ca3XUR5Xr8G5n74scQiUXvY8n4zsMUgj0=.52a48fe7-9a4c-4110-8ccc-8f96d89d3541@github.com>

> The mark context is shared, but each generation maintains a separate flag (`ShenandoahGeneration::_is_marking_complete`) to record the status of the mark. The remembered set scan will use the mark bitmap only if the old generation mark is complete. Otherwise, it walks the old regions and requires them to be parseable.
> 
> Presently, we make the heap parseable after global marking finishes, but this work can be deferred to the start of an old mark if we had global mark record completion of old marking.

William Kemper has updated the pull request incrementally with two additional commits since the last revision:

 - Cleanup duplicate methods to return number of coalesce and fill candidates
 - Restore assertion about which states old generation is allowed to be in when a global cycle degenerates

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/304/files
  - new: https://git.openjdk.org/shenandoah/pull/304/files/2bf50f3f..3351329d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=304&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=304&range=01-02

  Stats: 68 lines in 7 files changed: 22 ins; 8 del; 38 mod
  Patch: https://git.openjdk.org/shenandoah/pull/304.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/304/head:pull/304

PR: https://git.openjdk.org/shenandoah/pull/304

From ysr at openjdk.org  Tue Aug 15 22:53:41 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 15 Aug 2023 22:53:41 GMT
Subject: RFR: 8311883: [Genshen] Adaptive tenuring threshold [v11]
In-Reply-To: <GddWKGuB4c-E1MkgwCAQdYVSFqcXjb2GEB2BDlgmMcg=.777cef3f-4c31-44ba-ba2e-5f4ccf41e9c5@github.com>
References: <JtI01IJXIhWHSX4VI7paTeI2CnBmKQ5eqqJznX0RbGM=.d2535a64-074c-45bb-b3df-c77d2473f369@github.com>
 <GddWKGuB4c-E1MkgwCAQdYVSFqcXjb2GEB2BDlgmMcg=.777cef3f-4c31-44ba-ba2e-5f4ccf41e9c5@github.com>
Message-ID: <0eLw94J2z0993j9Jek28P65-bok9Etu2VoK58T1NVbg=.d782a214-4d77-4a12-9633-b2c066e65613@github.com>

On Mon, 3 Jul 2023 23:38:51 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 82 commits:
>> 
>>  - Merge branch 'master' into adaptive_tenuring
>>  - Merge branch 'master' into adaptive_tenuring
>>  - Reposition a mis-positioned assert as well as surrounding if-test order.
>>  - Fix global flags to make adaptive tenuring with census at mark the
>>    default, along with ignoring mortality rates of cohorts older than the
>>    selected tenuring age at previous epoch.
>>  - Flag to ignore mortality rate of cohorts older than the tenuring age of
>>    most recent preious cycle when computing new tenuring threshold.
>>  - Clear partial census data before a new marking cycle.
>>    Allow consideration of mortality rate of youngest cohort for tenuring
>>    decisions. (It was previously excluded because census data didn't
>>    include the youngest cohort allocated in the current epoch.)
>>    
>>    Still to do: avoid tenuring revisionism a la Kelvin's suggestion.
>>  - jcheck whitespace.
>>  - Merge branch 'master' into adaptive_tenuring
>>  - Separate out clearing and testing of local and global portions of census
>>    data, towards expunging partial/interrupted census data as a next step.
>>  - Merge branch 'master' into adaptive_tenuring
>>  - ... and 72 more: https://git.openjdk.org/shenandoah/compare/a4f6425e...13fa9ac9
>
> src/hotspot/share/gc/shared/ageTable.hpp line 60:
> 
>> 58: 
>> 59:   void add(uint age, size_t oop_size) {
>> 60:     assert(age > 0 && age < table_size, "invalid age of object");
> 
> We now also include age 0 objects in the census for the case of concurrent collectors in the young generation. This slot would have `sizes[0] == 0` for our traditional use cases for generational stop-world collectors.
> 
> I'll upstream this change separately in a sibling PR/JBS. TBD.

https://bugs.openjdk.org/browse/JDK-8314329 will track this and other changes to the age table that need to be separately udstreamed.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/289#discussion_r1295219544

From ysr at openjdk.org  Tue Aug 15 22:58:53 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 15 Aug 2023 22:58:53 GMT
Subject: RFR: 8311883: [Genshen] Adaptive tenuring threshold [v11]
In-Reply-To: <uhAgR6hVAeEsTYVPhbtRQNsllfhgV69diMRyG4R7R-s=.2cde5e87-bd26-4bc2-bc2a-fd015f398601@github.com>
References: <JtI01IJXIhWHSX4VI7paTeI2CnBmKQ5eqqJznX0RbGM=.d2535a64-074c-45bb-b3df-c77d2473f369@github.com>
 <GddWKGuB4c-E1MkgwCAQdYVSFqcXjb2GEB2BDlgmMcg=.777cef3f-4c31-44ba-ba2e-5f4ccf41e9c5@github.com>
 <WWRMu2VfH0mb-J5BnZY_xrvPOPeNTuJT7utn4Ew7Lng=.8fff288f-ee31-4ef8-8d35-0c591254a390@github.com>
 <uhAgR6hVAeEsTYVPhbtRQNsllfhgV69diMRyG4R7R-s=.2cde5e87-bd26-4bc2-bc2a-fd015f398601@github.com>
Message-ID: <yksXse3HC2pdwM4npimsQzG8sbVtNTjRG1sBbgdBUKk=.1cb06cfb-451b-4889-a6d1-7ca56e14262f@github.com>

On Tue, 11 Jul 2023 17:04:03 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> It is true that cancellations could lead to bogus partial censuses being used. I'll locate the right place in the cancellation path(s) where partial censuses will  need to be expunged. Essentially, these would be the points where concurrent marking is canceled (when censuses are conducted during the marking phase) and where concurrent evacuation is canceled (when censuses are conducted during the evacuation phase). All other cancellations are fine because the census data will be valid despite cancellation in those cases.
>
> In the event of degeneration, if a census-collection phase is interrupted mid-stream and restarted afresh, the partial census will need to be expunged. It'll then be replaced by the new census (in e.g. degenerate GC's mark or evac). In the case of a degenerate STW GC being cancelled and degenerating further to a full STW GC, during _its_ census-collection phase the partial census will be expunged. Any completed censuses from any census taking phase that persisted will be used as before. Currently, all calculations are independent of the (start and end) time(s) of the (completed) censuses being used, but a future enhancement might potentially consider a temporal dimension. We'll capture the times in the census today, but those are not used by the algorithm today.

The time tracking of the census is a potential future enhancement.

Today each new census clears any old partial or interrupted censuses, but we do not track the start and end times of completed censuses.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/289#discussion_r1295221746

From ysr at openjdk.org  Tue Aug 15 23:07:11 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 15 Aug 2023 23:07:11 GMT
Subject: RFR: 8311883: [Genshen] Adaptive tenuring threshold [v11]
In-Reply-To: <WWRMu2VfH0mb-J5BnZY_xrvPOPeNTuJT7utn4Ew7Lng=.8fff288f-ee31-4ef8-8d35-0c591254a390@github.com>
References: <JtI01IJXIhWHSX4VI7paTeI2CnBmKQ5eqqJznX0RbGM=.d2535a64-074c-45bb-b3df-c77d2473f369@github.com>
 <GddWKGuB4c-E1MkgwCAQdYVSFqcXjb2GEB2BDlgmMcg=.777cef3f-4c31-44ba-ba2e-5f4ccf41e9c5@github.com>
 <WWRMu2VfH0mb-J5BnZY_xrvPOPeNTuJT7utn4Ew7Lng=.8fff288f-ee31-4ef8-8d35-0c591254a390@github.com>
Message-ID: <b_9EXl9Pwpa2NMCnBYJUj8a5ftV-mhIOyaKDYm3XjLg=.a96a1f55-db06-459d-90f8-c6a71431462f@github.com>

On Tue, 4 Jul 2023 00:44:16 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahMark.inline.hpp line 60:
>> 
>>> 58: }
>>> 59: 
>>> 60: template <class T, ShenandoahGenerationType GENERATION, StringDedupMode STRING_DEDUP>
>> 
>> Consider extracting the GENERATION type from T (closure's type).
>
> This might be possible via the suggestion/comment further down at line 102. However, this might require a bit more refactoring of other code, which I'd prefer to do not here in this ticket, but separately.
> 
> I'll defer that to a separate PR for which I'll open a JBS in anticipation and link it here.

Tracked in task ticket https://bugs.openjdk.org/browse/JDK-8314331

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/289#discussion_r1295226171

From kdnilsen at openjdk.org  Wed Aug 16 14:11:03 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 16 Aug 2023 14:11:03 GMT
Subject: RFR: 8313574: GenShen: Completing a global mark should also
 complete an old mark [v3]
In-Reply-To: <Z3m61u3Kz6Ca3XUR5Xr8G5n74scQiUXvY8n4zsMUgj0=.52a48fe7-9a4c-4110-8ccc-8f96d89d3541@github.com>
References: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>
 <Z3m61u3Kz6Ca3XUR5Xr8G5n74scQiUXvY8n4zsMUgj0=.52a48fe7-9a4c-4110-8ccc-8f96d89d3541@github.com>
Message-ID: <kuHygC-lojfYKqZ0NqL4XOqnrh9ruztKRPxDBrG1v5M=.5ea4e904-104d-415f-84b2-369ceb9fe582@github.com>

On Tue, 15 Aug 2023 22:11:20 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> The mark context is shared, but each generation maintains a separate flag (`ShenandoahGeneration::_is_marking_complete`) to record the status of the mark. The remembered set scan will use the mark bitmap only if the old generation mark is complete. Otherwise, it walks the old regions and requires them to be parseable.
>> 
>> Presently, we make the heap parseable after global marking finishes, but this work can be deferred to the start of an old mark if we had global mark record completion of old marking.
>
> William Kemper has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Cleanup duplicate methods to return number of coalesce and fill candidates
>  - Restore assertion about which states old generation is allowed to be in when a global cycle degenerates

Still looks good to me...

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/304#pullrequestreview-1580723336

From wkemper at openjdk.org  Wed Aug 16 15:44:59 2023
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 16 Aug 2023 15:44:59 GMT
Subject: Integrated: 8313574: GenShen: Completing a global mark should also
 complete an old mark
In-Reply-To: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>
References: <J6YK3fp0cvz3gywZuQPJZXrBNF0eUMvzuLSrNNXkJdk=.0f8394da-223f-4439-bb46-48e9e2c2ae5a@github.com>
Message-ID: <T53o1NXt1XBh8fw30JBdb9JYMJK49gqPCD0KdUIpZuU=.00cd7d96-17af-446c-9da2-9ff6dd4d9e7a@github.com>

On Fri, 4 Aug 2023 18:17:29 GMT, William Kemper <wkemper at openjdk.org> wrote:

> The mark context is shared, but each generation maintains a separate flag (`ShenandoahGeneration::_is_marking_complete`) to record the status of the mark. The remembered set scan will use the mark bitmap only if the old generation mark is complete. Otherwise, it walks the old regions and requires them to be parseable.
> 
> Presently, we make the heap parseable after global marking finishes, but this work can be deferred to the start of an old mark if we had global mark record completion of old marking.

This pull request has now been integrated.

Changeset: 2c06240d
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/2c06240dd55a79c6c64bd4dad35952db15e05d56
Stats:     236 lines in 15 files changed: 67 ins; 77 del; 92 mod

8313574: GenShen: Completing a global mark should also complete an old mark

Reviewed-by: kdnilsen, ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/304

From wkemper at openjdk.org  Wed Aug 16 17:18:22 2023
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 16 Aug 2023 17:18:22 GMT
Subject: RFR: 8312120: GenShen: Update OLD Live at end of Previous Marking
 after Full GC
Message-ID: <o-AbhCqE13h3wHuNyIe2WcUIQEa2Xe4zSaNWQDwLu0c=.13c9b069-5f7b-4b0f-a7ba-a19f8e7b3818@github.com>

Record total live memory for old generation following Full GC marking phase.

-------------

Commit messages:
 - Fix merge error
 - Update live bytes in old generation after full gc STW mark
 - Fix assertions and update comments
 - Prepare for mixed collections after global cycle

Changes: https://git.openjdk.org/shenandoah/pull/306/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=306&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8312120
  Stats: 10 lines in 1 file changed: 10 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/306.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/306/head:pull/306

PR: https://git.openjdk.org/shenandoah/pull/306

From wkemper at openjdk.org  Wed Aug 16 23:38:07 2023
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 16 Aug 2023 23:38:07 GMT
Subject: RFR: 8309639: [Shenandoah] Regression in LRU cache benchmark
Message-ID: <etMdLuujwjeAxrbXAxuxN3VwiPuq2zxwDJl9T75Wr8Y=.24a6821f-ba55-4c4f-88cf-7f4835a8874f@github.com>

Shenandoah's adaptive heuristic begins in a state where it is "learning" the application. In this state, it triggers collections somewhat more aggressively in an effort to "learn" the allocation rate and cycle times for the application.

Shenandoah also has the ability to "skip" evacuation phases when marking finds sufficient garbage in regions with no live objects. We call these "abbreviated" cycles. During the course of development for the generational mode, we found that such cycles tend to decrease the average cycle time tracked by the heuristic. This in turn leads to the heuristic triggering a cycle "too late" - because it expects the cycle to take less time than a normal (i.e., not abbreviated) cycle requires. To address this, we modified the heuristic to not include abbreviated cycle times in the average cycle time.

Unfortunately, this change also made the heuristic not count abbreviated cycles as "learning" cycles. Ordinarily, this is not a concern because the application will usually run plenty of "normal" cycles. However, we've discovered that the certain benchmarks and jtreg tests only run abbreviated cycles. This causes the heuristic to stay in the "learning" state for the duration of the benchmark. In this state, the GC is run more aggressively and performance suffers (15-20%).

-------------

Commit messages:
 - Do not ignore learning cycles

Changes: https://git.openjdk.org/shenandoah/pull/307/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=307&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8309639
  Stats: 7 lines in 1 file changed: 3 ins; 3 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/307.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/307/head:pull/307

PR: https://git.openjdk.org/shenandoah/pull/307

From rkennke at openjdk.org  Thu Aug 17 00:25:00 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 17 Aug 2023 00:25:00 GMT
Subject: RFR: 8309639: [Shenandoah] Regression in LRU cache benchmark
In-Reply-To: <etMdLuujwjeAxrbXAxuxN3VwiPuq2zxwDJl9T75Wr8Y=.24a6821f-ba55-4c4f-88cf-7f4835a8874f@github.com>
References: <etMdLuujwjeAxrbXAxuxN3VwiPuq2zxwDJl9T75Wr8Y=.24a6821f-ba55-4c4f-88cf-7f4835a8874f@github.com>
Message-ID: <c-FzV7nEkYYpiY0AxULfL-h-A7n2yVz_3fJN6iK95q4=.2755c2dc-a66c-479a-bacf-c51e261997b7@github.com>

On Wed, 16 Aug 2023 23:31:33 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Shenandoah's adaptive heuristic begins in a state where it is "learning" the application. In this state, it triggers collections somewhat more aggressively in an effort to "learn" the allocation rate and cycle times for the application.
> 
> Shenandoah also has the ability to "skip" evacuation phases when marking finds sufficient garbage in regions with no live objects. We call these "abbreviated" cycles. During the course of development for the generational mode, we found that such cycles tend to decrease the average cycle time tracked by the heuristic. This in turn leads to the heuristic triggering a cycle "too late" - because it expects the cycle to take less time than a normal (i.e., not abbreviated) cycle requires. To address this, we modified the heuristic to not include abbreviated cycle times in the average cycle time.
> 
> Unfortunately, this change also made the heuristic not count abbreviated cycles as "learning" cycles. Ordinarily, this is not a concern because the application will usually run plenty of "normal" cycles. However, we've discovered that the certain benchmarks and jtreg tests only run abbreviated cycles. This causes the heuristic to stay in the "learning" state for the duration of the benchmark. In this state, the GC is run more aggressively and performance suffers (15-20%).

Should this go straight to upstream jdk repo?

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/307#issuecomment-1681429323

From lkorinth at openjdk.org  Thu Aug 17 16:03:42 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Thu, 17 Aug 2023 16:03:42 GMT
Subject: RFR: 8311639: Replace currentTimeMillis() with nanoTime() in jtreg/gc
Message-ID: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>

I have removed usage of `currentTimeMillis()` in jtreg/gc as `currentTimeMillis()` is not monotonic. It is mostly changing from milliseconds to nanoseconds. In certain places, I have changed the code from something like `(instant1 < instant2)` to `(instant1 - instant2 < 0)` It might look like I am removing instant2 from both sides of the inequality and the result ought to be the same, but due to overflow arithmetic, the change should be better if nanoseconds where to overflow. I have also removed some loops where the loop is doing nothing except sleeping. I somewhat shortened the string in the key of a hash map because the nanosecond value made the string too long for the heap size.

All tests within jtreg/gc passes.

-------------

Commit messages:
 - 8311639: Replace currentTimeMillis() with nanoTime() in jtreg/gc

Changes: https://git.openjdk.org/jdk/pull/15331/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=15331&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8311639
  Stats: 128 lines in 17 files changed: 2 ins; 15 del; 111 mod
  Patch: https://git.openjdk.org/jdk/pull/15331.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15331/head:pull/15331

PR: https://git.openjdk.org/jdk/pull/15331

From wkemper at openjdk.org  Thu Aug 17 16:37:18 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 17 Aug 2023 16:37:18 GMT
Subject: RFR: 8309639: GenShen: Regression in LRU cache benchmark
In-Reply-To: <c-FzV7nEkYYpiY0AxULfL-h-A7n2yVz_3fJN6iK95q4=.2755c2dc-a66c-479a-bacf-c51e261997b7@github.com>
References: <etMdLuujwjeAxrbXAxuxN3VwiPuq2zxwDJl9T75Wr8Y=.24a6821f-ba55-4c4f-88cf-7f4835a8874f@github.com>
 <c-FzV7nEkYYpiY0AxULfL-h-A7n2yVz_3fJN6iK95q4=.2755c2dc-a66c-479a-bacf-c51e261997b7@github.com>
Message-ID: <M4lWap4kKjiIMxVz2KzXvibQvnqPNh2_TggLSZeCh1A=.291e5936-e514-442a-a3b8-197cc41e3c9f@github.com>

On Thu, 17 Aug 2023 00:21:47 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> Should this go straight to upstream jdk repo?

No, this is only in the shenandoah repo. I will change the tag in the title to `GenShen`.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/307#issuecomment-1682607827

From ayang at openjdk.org  Thu Aug 17 17:49:30 2023
From: ayang at openjdk.org (Albert Mingkun Yang)
Date: Thu, 17 Aug 2023 17:49:30 GMT
Subject: RFR: 8311639: Replace currentTimeMillis() with nanoTime() in
 jtreg/gc
In-Reply-To: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
References: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
Message-ID: <3lUt9SMeH24aNsi_vj1X8cwF9W_ul_RG4oROF_yFj74=.b1af3d19-09db-4f4d-8fb4-58e8a9c74cc5@github.com>

On Thu, 17 Aug 2023 15:56:04 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

> I have removed usage of `currentTimeMillis()` in jtreg/gc as `currentTimeMillis()` is not monotonic. It is mostly changing from milliseconds to nanoseconds. In certain places, I have changed the code from something like `(instant1 < instant2)` to `(instant1 - instant2 < 0)` It might look like I am removing instant2 from both sides of the inequality and the result ought to be the same, but due to overflow arithmetic, the change should be better if nanoseconds where to overflow. I have also removed some loops where the loop is doing nothing except sleeping. I somewhat shortened the string in the key of a hash map because the nanosecond value made the string too long for the heap size.
> 
> All tests within jtreg/gc passes.

test/hotspot/jtreg/gc/stress/systemgc/TestSystemGC.java line 80:

> 78:     public void run() {
> 79:         while (!shouldExit()) {
> 80:             String prefix = "long" + (System.nanoTime() % 10000); // limit to 4 digits after changing from milliseconds to nanoseconds, else the key will use more memory

Is this used to calculate elapsed time? If not, I think using `currentTimeMillis` is fine.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15331#discussion_r1297552208

From kdnilsen at openjdk.org  Thu Aug 17 19:01:23 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 17 Aug 2023 19:01:23 GMT
Subject: RFR: 8312120: GenShen: Update OLD Live at end of Previous Marking
 after Full GC
In-Reply-To: <o-AbhCqE13h3wHuNyIe2WcUIQEa2Xe4zSaNWQDwLu0c=.13c9b069-5f7b-4b0f-a7ba-a19f8e7b3818@github.com>
References: <o-AbhCqE13h3wHuNyIe2WcUIQEa2Xe4zSaNWQDwLu0c=.13c9b069-5f7b-4b0f-a7ba-a19f8e7b3818@github.com>
Message-ID: <1f2Jr1k3_yB4REzrnZC4mjg52QXfQoF1wEyzwSBHEKY=.19844572-9727-4e79-b595-abbfb09707cc@github.com>

On Wed, 16 Aug 2023 17:10:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Record total live memory for old generation following Full GC marking phase.

Thanks.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/306#pullrequestreview-1583278005

From kdnilsen at openjdk.org  Thu Aug 17 19:07:15 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 17 Aug 2023 19:07:15 GMT
Subject: RFR: 8309639: GenShen: Regression in LRU cache benchmark
In-Reply-To: <etMdLuujwjeAxrbXAxuxN3VwiPuq2zxwDJl9T75Wr8Y=.24a6821f-ba55-4c4f-88cf-7f4835a8874f@github.com>
References: <etMdLuujwjeAxrbXAxuxN3VwiPuq2zxwDJl9T75Wr8Y=.24a6821f-ba55-4c4f-88cf-7f4835a8874f@github.com>
Message-ID: <nani3G82LRBVJId_F7avhKtLqO2v4BKRvPZ0Zje2q_M=.2f34dbcd-b43a-47ac-8ca0-8cb427e64bae@github.com>

On Wed, 16 Aug 2023 23:31:33 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Shenandoah's adaptive heuristic begins in a state where it is "learning" the application. In this state, it triggers collections somewhat more aggressively in an effort to "learn" the allocation rate and cycle times for the application.
> 
> Shenandoah also has the ability to "skip" evacuation phases when marking finds sufficient garbage in regions with no live objects. We call these "abbreviated" cycles. During the course of development for the generational mode, we found that such cycles tend to decrease the average cycle time tracked by the heuristic. This in turn leads to the heuristic triggering a cycle "too late" - because it expects the cycle to take less time than a normal (i.e., not abbreviated) cycle requires. To address this, we modified the heuristic to not include abbreviated cycle times in the average cycle time.
> 
> Unfortunately, this change also made the heuristic not count abbreviated cycles as "learning" cycles. Ordinarily, this is not a concern because the application will usually run plenty of "normal" cycles. However, we've discovered that the certain benchmarks and jtreg tests only run abbreviated cycles. This causes the heuristic to stay in the "learning" state for the duration of the benchmark. In this state, the GC is run more aggressively and performance suffers (15-20%).

This looks like a good compromise.  Since this new code isn't in the "isolation audit", I don't want to lose track of the change.  Should we make this conditional upon is_generational, and revert the non-generational mode to original behavior?

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/307#pullrequestreview-1583287453

From wkemper at openjdk.org  Thu Aug 17 21:01:08 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 17 Aug 2023 21:01:08 GMT
Subject: Integrated: 8312120: GenShen: Update OLD Live at end of Previous
 Marking after Full GC
In-Reply-To: <o-AbhCqE13h3wHuNyIe2WcUIQEa2Xe4zSaNWQDwLu0c=.13c9b069-5f7b-4b0f-a7ba-a19f8e7b3818@github.com>
References: <o-AbhCqE13h3wHuNyIe2WcUIQEa2Xe4zSaNWQDwLu0c=.13c9b069-5f7b-4b0f-a7ba-a19f8e7b3818@github.com>
Message-ID: <kGYIKS5WCxPSh3q6-AyEC2kfSS9hyIhZSk6THGj8EYk=.8dad07e2-5b4b-41d3-b0bd-52b7f452e129@github.com>

On Wed, 16 Aug 2023 17:10:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Record total live memory for old generation following Full GC marking phase.

This pull request has now been integrated.

Changeset: 529a3797
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/529a3797f2b01a1116db2af110f4cbc289a50755
Stats:     10 lines in 1 file changed: 10 ins; 0 del; 0 mod

8312120: GenShen: Update OLD Live at end of Previous Marking after Full GC

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/306

From ysr at openjdk.org  Thu Aug 17 21:01:05 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 17 Aug 2023 21:01:05 GMT
Subject: RFR: 8309639: GenShen: Regression in LRU cache benchmark
In-Reply-To: <etMdLuujwjeAxrbXAxuxN3VwiPuq2zxwDJl9T75Wr8Y=.24a6821f-ba55-4c4f-88cf-7f4835a8874f@github.com>
References: <etMdLuujwjeAxrbXAxuxN3VwiPuq2zxwDJl9T75Wr8Y=.24a6821f-ba55-4c4f-88cf-7f4835a8874f@github.com>
Message-ID: <D0LnNG4WfQ8M1uEjq4LDpepxXa24-r4IzSErjip3Yrs=.886b7a97-97c7-4ded-9456-e99b5d9471ac@github.com>

On Wed, 16 Aug 2023 23:31:33 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Shenandoah's adaptive heuristic begins in a state where it is "learning" the application. In this state, it triggers collections somewhat more aggressively in an effort to "learn" the allocation rate and cycle times for the application.
> 
> Shenandoah also has the ability to "skip" evacuation phases when marking finds sufficient garbage in regions with no live objects. We call these "abbreviated" cycles. During the course of development for the generational mode, we found that such cycles tend to decrease the average cycle time tracked by the heuristic. This in turn leads to the heuristic triggering a cycle "too late" - because it expects the cycle to take less time than a normal (i.e., not abbreviated) cycle requires. To address this, we modified the heuristic to not include abbreviated cycle times in the average cycle time.
> 
> Unfortunately, this change also made the heuristic not count abbreviated cycles as "learning" cycles. Ordinarily, this is not a concern because the application will usually run plenty of "normal" cycles. However, we've discovered that the certain benchmarks and jtreg tests only run abbreviated cycles. This causes the heuristic to stay in the "learning" state for the duration of the benchmark. In this state, the GC is run more aggressively and performance suffers (15-20%).

LGTM. In addition to the code documentation remark, can I also request that the observed change in performance with your favorite benchmark or test be included either here in the PR or in the JBS ticket?

I am just curious how the learning data is used. If that is entirely abbreviated cycles, say, wouldn't you have the same issue? And skipping abbreviated cycle counting after the learning steps would leave you in the same state of under-predicting when a normal cycle does occur. Does the small number of recorded samples then save you because of the potentially larger padding because of the potentially larger standard deviation than if you counted all of the abbreviated cycles?

Adversarial workload scenarios are easy to imagine, but may be real applications don't behave like I was thinking of adversarial scenarios, so the performance data that you share would be useful to allay such fears.

Allow me to do some thinking out loud here. May be there's a flaw in my thinking below.

In general, my instinct would be that including abbreviated cycles would be fine; if they are frequent then the infrequent normal cycles increase the standard deviation (even if it might subsequently decay). Conversely, if abbreviated cycles are infrequent then we should be fine. May be thinking of the benchmarks/tests that gained from the change in heuristic would allow us to determine why the scenarios in which this helped and why. In particular, was the load presented by the benchmark very spiky, and this allowed us to tread in the direction of being conservative. Still, why doesn't a degenerate collection swing us back into the right spot? Does it do that but do we swing back again to complacency? What was the difference between the durations for abbreviated and normal cycles in the cases where we saw this improvement? I'd have expected the standard deviation to have provided a buffer/cushion.

Thanks!

src/hotspot/share/gc/shenandoah/heuristics/shenandoahHeuristics.cpp line 241:

> 239:   adjust_penalty(Concurrent_Adjust);
> 240: 
> 241:   if (_gc_times_learned <= ShenandoahLearningSteps || !(abbreviated && ShenandoahAdaptiveIgnoreShortCycles)) {

Can you leave a documentation comment here recording the intent in English?

As I read it, you include the time in the recorded history if any of the following hold:
1. we haven't reached the required number of learning steps, or
2. we are not an abbreviated cycle, or
3. we haven't been asked to ignore short cycles

To the extent that this is so, I think the test would read easier if you de-Morganized the conditional into:


if (_gc_times_learned <= ShenandoahLearningSteps || !abbreviated || !ShenandoahAdaptiveIgnoreShortCycles)) {


Anyway, something for us to remember why we have this condition would be useful as well; your PR explanation is excellent. May be a condensed form of that would be appropriate as a documentation comment here?

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/307#pullrequestreview-1583439262
PR Review Comment: https://git.openjdk.org/shenandoah/pull/307#discussion_r1297718868

From wkemper at openjdk.org  Thu Aug 17 21:01:05 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 17 Aug 2023 21:01:05 GMT
Subject: RFR: 8309639: GenShen: Regression in LRU cache benchmark
In-Reply-To: <nani3G82LRBVJId_F7avhKtLqO2v4BKRvPZ0Zje2q_M=.2f34dbcd-b43a-47ac-8ca0-8cb427e64bae@github.com>
References: <etMdLuujwjeAxrbXAxuxN3VwiPuq2zxwDJl9T75Wr8Y=.24a6821f-ba55-4c4f-88cf-7f4835a8874f@github.com>
 <nani3G82LRBVJId_F7avhKtLqO2v4BKRvPZ0Zje2q_M=.2f34dbcd-b43a-47ac-8ca0-8cb427e64bae@github.com>
Message-ID: <liqlTObms0DxMjRFYzu8BdhcMAVIwWQX0x4P5Tu7bso=.4307ba1b-8581-474a-bb96-85f68fc2f552@github.com>

On Thu, 17 Aug 2023 19:04:20 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Should we make this conditional upon is_generational, and revert the non-generational mode to original behavior?

I'm more inclined to test if this feature (i.e., heuristic ignores abbreviated cycles) improves performance on the upstream branch. I will test for a measurable benefit for this feature. For now, I think we should fix this bug. If we can show the performance benefit of this feature, I think we should upstream it. Otherwise, we should just back it out altogether.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/307#issuecomment-1682968105

From wkemper at openjdk.org  Thu Aug 17 21:01:06 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 17 Aug 2023 21:01:06 GMT
Subject: Integrated: 8309639: GenShen: Regression in LRU cache benchmark
In-Reply-To: <etMdLuujwjeAxrbXAxuxN3VwiPuq2zxwDJl9T75Wr8Y=.24a6821f-ba55-4c4f-88cf-7f4835a8874f@github.com>
References: <etMdLuujwjeAxrbXAxuxN3VwiPuq2zxwDJl9T75Wr8Y=.24a6821f-ba55-4c4f-88cf-7f4835a8874f@github.com>
Message-ID: <9jUYIHiOuQKThYKu4fCnqgb9AVhRGX1ubAeNKZZYrYc=.8bb6ce1d-c5d0-40b5-81e7-0728faf7cfd6@github.com>

On Wed, 16 Aug 2023 23:31:33 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Shenandoah's adaptive heuristic begins in a state where it is "learning" the application. In this state, it triggers collections somewhat more aggressively in an effort to "learn" the allocation rate and cycle times for the application.
> 
> Shenandoah also has the ability to "skip" evacuation phases when marking finds sufficient garbage in regions with no live objects. We call these "abbreviated" cycles. During the course of development for the generational mode, we found that such cycles tend to decrease the average cycle time tracked by the heuristic. This in turn leads to the heuristic triggering a cycle "too late" - because it expects the cycle to take less time than a normal (i.e., not abbreviated) cycle requires. To address this, we modified the heuristic to not include abbreviated cycle times in the average cycle time.
> 
> Unfortunately, this change also made the heuristic not count abbreviated cycles as "learning" cycles. Ordinarily, this is not a concern because the application will usually run plenty of "normal" cycles. However, we've discovered that the certain benchmarks and jtreg tests only run abbreviated cycles. This causes the heuristic to stay in the "learning" state for the duration of the benchmark. In this state, the GC is run more aggressively and performance suffers (15-20%).

This pull request has now been integrated.

Changeset: 867f558d
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/867f558dc4249bdf6e7bf6a26c4a3d0dd235f7b6
Stats:     7 lines in 1 file changed: 3 ins; 3 del; 1 mod

8309639: GenShen: Regression in LRU cache benchmark

Reviewed-by: kdnilsen, ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/307

From ysr at openjdk.org  Fri Aug 18 03:05:36 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 18 Aug 2023 03:05:36 GMT
Subject: RFR: 8311883: [Genshen] Adaptive tenuring threshold [v12]
In-Reply-To: <JtI01IJXIhWHSX4VI7paTeI2CnBmKQ5eqqJznX0RbGM=.d2535a64-074c-45bb-b3df-c77d2473f369@github.com>
References: <JtI01IJXIhWHSX4VI7paTeI2CnBmKQ5eqqJznX0RbGM=.d2535a64-074c-45bb-b3df-c77d2473f369@github.com>
Message-ID: <agLdEgcZK8_dUrqTpNfr7Pi9eWP5y71UtOg8Vik61x4=.22ac3c8b-e358-4a2c-ba30-218f4a2959af@github.com>

> JDK-8311883 [GenShen] Adaptive tenuring
> 
> I am opening this previously draft PR for formal preliminary review. It has already benefited from review feedback from a code walkthrough of an earlier version of the code. Most of that feedback and the corrections thereof are to be found in the comments in this PR. I have addressed a large majority of those comments, and am working on the last one that I plan to address as part of this PR. For the ones that I don't plan to address in this PR, I will create follow up tickets. Those will be added in the responses for the remaining feedback comments recorded in this PR's conversation.
> 
> Preliminary testing w/SPECjbb didn't yield reliable performance data from which to infer any performance improvements stemming from enabling adaptive tenuring. I believe that was because of the way SPECjbb is run, which causes excessive degenerate and full gc's. I plan to collect SPECjbb numbers with a fixed lower max HBIR so as to be able to discern performance differences from this change, as well as Extremem workloads. Those will be added here once ready over the next few days.

Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 83 commits:

 - Merge branch 'master' into adaptive_tenuring
 - Merge branch 'master' into adaptive_tenuring
 - Merge branch 'master' into adaptive_tenuring
 - Reposition a mis-positioned assert as well as surrounding if-test order.
 - Fix global flags to make adaptive tenuring with census at mark the
   default, along with ignoring mortality rates of cohorts older than the
   selected tenuring age at previous epoch.
 - Flag to ignore mortality rate of cohorts older than the tenuring age of
   most recent preious cycle when computing new tenuring threshold.
 - Clear partial census data before a new marking cycle.
   Allow consideration of mortality rate of youngest cohort for tenuring
   decisions. (It was previously excluded because census data didn't
   include the youngest cohort allocated in the current epoch.)
   
   Still to do: avoid tenuring revisionism a la Kelvin's suggestion.
 - jcheck whitespace.
 - Merge branch 'master' into adaptive_tenuring
 - Separate out clearing and testing of local and global portions of census
   data, towards expunging partial/interrupted census data as a next step.
 - ... and 73 more: https://git.openjdk.org/shenandoah/compare/867f558d...2604d78a

-------------

Changes: https://git.openjdk.org/shenandoah/pull/289/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=289&range=11
  Stats: 994 lines in 30 files changed: 876 ins; 45 del; 73 mod
  Patch: https://git.openjdk.org/shenandoah/pull/289.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/289/head:pull/289

PR: https://git.openjdk.org/shenandoah/pull/289

From stefank at openjdk.org  Fri Aug 18 07:18:26 2023
From: stefank at openjdk.org (Stefan Karlsson)
Date: Fri, 18 Aug 2023 07:18:26 GMT
Subject: RFR: 8311639: Replace currentTimeMillis() with nanoTime() in
 jtreg/gc
In-Reply-To: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
References: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
Message-ID: <AxbXUg0sifjPnCRX7un5CfYG_u3GcN9H-46_iKayWig=.24685f12-ae6a-4906-8438-c768aa7c93aa@github.com>

On Thu, 17 Aug 2023 15:56:04 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

> I have removed usage of `currentTimeMillis()` in jtreg/gc as `currentTimeMillis()` is not monotonic. It is mostly changing from milliseconds to nanoseconds. In certain places, I have changed the code from something like `(instant1 < instant2)` to `(instant1 - instant2 < 0)` It might look like I am removing instant2 from both sides of the inequality and the result ought to be the same, but due to overflow arithmetic, the change should be better if nanoseconds where to overflow. I have also removed some loops where the loop is doing nothing except sleeping. I somewhat shortened the string in the key of a hash map because the nanosecond value made the string too long for the heap size.
> 
> All tests within jtreg/gc passes.

Looks good. I'd also go with Albert's suggestion and keep timeout logging in TestSystemGC.

test/hotspot/jtreg/gc/stress/systemgc/TestSystemGC.java line 195:

> 193:         }
> 194:         long timeoutNanos = Integer.parseInt(args[0]) * 1_000_000_000L;
> 195:         System.out.println("Running with timeout of " + timeoutNanos + "ns");

The precision here is in seconds. Maybe just leave this print out to be in ms?

-------------

Marked as reviewed by stefank (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/15331#pullrequestreview-1583987479
PR Review Comment: https://git.openjdk.org/jdk/pull/15331#discussion_r1298093911

From ysr at openjdk.org  Fri Aug 18 07:20:06 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 18 Aug 2023 07:20:06 GMT
Subject: RFR: 8311883: [Genshen] Adaptive tenuring threshold [v12]
In-Reply-To: <agLdEgcZK8_dUrqTpNfr7Pi9eWP5y71UtOg8Vik61x4=.22ac3c8b-e358-4a2c-ba30-218f4a2959af@github.com>
References: <JtI01IJXIhWHSX4VI7paTeI2CnBmKQ5eqqJznX0RbGM=.d2535a64-074c-45bb-b3df-c77d2473f369@github.com>
 <agLdEgcZK8_dUrqTpNfr7Pi9eWP5y71UtOg8Vik61x4=.22ac3c8b-e358-4a2c-ba30-218f4a2959af@github.com>
Message-ID: <ZJ1adABX1ggWQ-LetdMyjHnuugCxcCCibCs_nK4vYJI=.8a7b17de-3532-4736-8e2f-049f0c917f2d@github.com>

On Fri, 18 Aug 2023 03:05:36 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> JDK-8311883 [GenShen] Adaptive tenuring
>> 
>> I am opening this previously draft PR for formal preliminary review. It has already benefited from review feedback from a code walkthrough of an earlier version of the code. Most of that feedback and the corrections thereof are to be found in the comments in this PR. I have addressed a large majority of those comments, and am working on the last one that I plan to address as part of this PR. For the ones that I don't plan to address in this PR, I will create follow up tickets. Those will be added in the responses for the remaining feedback comments recorded in this PR's conversation.
>> 
>> Preliminary testing w/SPECjbb didn't yield reliable performance data from which to infer any performance improvements stemming from enabling adaptive tenuring. I believe that was because of the way SPECjbb is run, which causes excessive degenerate and full gc's. I plan to collect SPECjbb numbers with a fixed lower max HBIR so as to be able to discern performance differences from this change, as well as Extremem workloads. Those will be added here once ready over the next few days.
>
> Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 83 commits:
> 
>  - Merge branch 'master' into adaptive_tenuring
>  - Merge branch 'master' into adaptive_tenuring
>  - Merge branch 'master' into adaptive_tenuring
>  - Reposition a mis-positioned assert as well as surrounding if-test order.
>  - Fix global flags to make adaptive tenuring with census at mark the
>    default, along with ignoring mortality rates of cohorts older than the
>    selected tenuring age at previous epoch.
>  - Flag to ignore mortality rate of cohorts older than the tenuring age of
>    most recent preious cycle when computing new tenuring threshold.
>  - Clear partial census data before a new marking cycle.
>    Allow consideration of mortality rate of youngest cohort for tenuring
>    decisions. (It was previously excluded because census data didn't
>    include the youngest cohort allocated in the current epoch.)
>    
>    Still to do: avoid tenuring revisionism a la Kelvin's suggestion.
>  - jcheck whitespace.
>  - Merge branch 'master' into adaptive_tenuring
>  - Separate out clearing and testing of local and global portions of census
>    data, towards expunging partial/interrupted census data as a next step.
>  - ... and 73 more: https://git.openjdk.org/shenandoah/compare/867f558d...2604d78a

I am integrating these changes with the GHA failures https://github.com/ysramakrishna/shenandoah/actions/runs/5898305273 that are tracked in https://bugs.openjdk.org/browse/JDK-8311843

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/289#issuecomment-1683463179

From ysr at openjdk.org  Fri Aug 18 07:20:08 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 18 Aug 2023 07:20:08 GMT
Subject: Integrated: 8311883: [Genshen] Adaptive tenuring threshold
In-Reply-To: <JtI01IJXIhWHSX4VI7paTeI2CnBmKQ5eqqJznX0RbGM=.d2535a64-074c-45bb-b3df-c77d2473f369@github.com>
References: <JtI01IJXIhWHSX4VI7paTeI2CnBmKQ5eqqJznX0RbGM=.d2535a64-074c-45bb-b3df-c77d2473f369@github.com>
Message-ID: <lBn27Ws5d2cKFWOjexaZM_IQJyD-mA_CNm2aFitnae4=.8ca060ba-5645-4d81-a599-ea21b67a1fed@github.com>

On Thu, 15 Jun 2023 20:08:58 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> JDK-8311883 [GenShen] Adaptive tenuring
> 
> I am opening this previously draft PR for formal preliminary review. It has already benefited from review feedback from a code walkthrough of an earlier version of the code. Most of that feedback and the corrections thereof are to be found in the comments in this PR. I have addressed a large majority of those comments, and am working on the last one that I plan to address as part of this PR. For the ones that I don't plan to address in this PR, I will create follow up tickets. Those will be added in the responses for the remaining feedback comments recorded in this PR's conversation.
> 
> Preliminary testing w/SPECjbb didn't yield reliable performance data from which to infer any performance improvements stemming from enabling adaptive tenuring. I believe that was because of the way SPECjbb is run, which causes excessive degenerate and full gc's. I plan to collect SPECjbb numbers with a fixed lower max HBIR so as to be able to discern performance differences from this change, as well as Extremem workloads. Those will be added here once ready over the next few days.

This pull request has now been integrated.

Changeset: ef4b4532
Author:    Y. Srinivas Ramakrishna <ysr at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/ef4b4532e509412fd0a21e2af36a0708d3696094
Stats:     994 lines in 30 files changed: 876 ins; 45 del; 73 mod

8311883: [Genshen] Adaptive tenuring threshold

Generational Shenandoah currently has the notion of a tenuring threshold but it isn't dynamically adapted, but rather kept fixed at 7.

We now adapt the tenuring threshold based on object demographics as determined by a recent GC that visits objects in the young generation.

We keep track of age-cohort populations at each minor GC epoch and use the historical data to determine if it would be a good idea to tenure or not based on measured mortality rates. A few tunable (experimental) knobs are exposed to play with these to determine some good settings in the future.

The object census is conducted by default at marking, and is subject to noise on account of objects whose age could not be determined because of displaced header. In this case, the computed tenuring threshold is used in the following evacuation of the same cycle. Optionally, the census can be conducted at evacuation time, but sees only objects that are in the collection set. In this case, the tenuring threshold that is computed is used for tenuring decisions in the next evacuation cycle.

Other variants are possible, and may be implemented / tested in the future as opportunity and data permit.

The computed tenuring threshold has not yet been coupled with size budgeting, but will be in a followup PR.

At this time, performance measurements have not shown any benefit, but we believe that with the framework now in place, we may be able to find an adaptive tenuring algorithm that works better than the current one and provide performance benefits. Adaptive tenuring is enabled by default, but can be optionally disabled to mimic previous behavior.

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/289

From lkorinth at openjdk.org  Fri Aug 18 08:59:25 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Fri, 18 Aug 2023 08:59:25 GMT
Subject: RFR: 8311639: Replace currentTimeMillis() with nanoTime() in
 jtreg/gc
In-Reply-To: <3lUt9SMeH24aNsi_vj1X8cwF9W_ul_RG4oROF_yFj74=.b1af3d19-09db-4f4d-8fb4-58e8a9c74cc5@github.com>
References: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
 <3lUt9SMeH24aNsi_vj1X8cwF9W_ul_RG4oROF_yFj74=.b1af3d19-09db-4f4d-8fb4-58e8a9c74cc5@github.com>
Message-ID: <ktTRrrseSndhXVa8I0Qm7cmnRViqA2T0HhKurTzwGHI=.06b48e59-e064-4e8a-90c4-c3bfbf8ba620@github.com>

On Thu, 17 Aug 2023 17:46:28 GMT, Albert Mingkun Yang <ayang at openjdk.org> wrote:

>> I have removed usage of `currentTimeMillis()` in jtreg/gc as `currentTimeMillis()` is not monotonic. It is mostly changing from milliseconds to nanoseconds. In certain places, I have changed the code from something like `(instant1 < instant2)` to `(instant1 - instant2 < 0)` It might look like I am removing instant2 from both sides of the inequality and the result ought to be the same, but due to overflow arithmetic, the change should be better if nanoseconds where to overflow. I have also removed some loops where the loop is doing nothing except sleeping. I somewhat shortened the string in the key of a hash map because the nanosecond value made the string too long for the heap size.
>> 
>> All tests within jtreg/gc passes.
>
> test/hotspot/jtreg/gc/stress/systemgc/TestSystemGC.java line 80:
> 
>> 78:     public void run() {
>> 79:         while (!shouldExit()) {
>> 80:             String prefix = "long" + (System.nanoTime() % 10000); // limit to 4 digits after changing from milliseconds to nanoseconds, else the key will use more memory
> 
> Is this used to calculate elapsed time? If not, I think using `currentTimeMillis` is fine.

It is not to calculate elapsed time. It is to create a unique(ish) key for a hash table. The new code does not guarantee a unique value, but then neither did the old.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15331#discussion_r1298196377

From lkorinth at openjdk.org  Fri Aug 18 08:59:27 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Fri, 18 Aug 2023 08:59:27 GMT
Subject: RFR: 8311639: Replace currentTimeMillis() with nanoTime() in
 jtreg/gc
In-Reply-To: <AxbXUg0sifjPnCRX7un5CfYG_u3GcN9H-46_iKayWig=.24685f12-ae6a-4906-8438-c768aa7c93aa@github.com>
References: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
 <AxbXUg0sifjPnCRX7un5CfYG_u3GcN9H-46_iKayWig=.24685f12-ae6a-4906-8438-c768aa7c93aa@github.com>
Message-ID: <_jFP3gxGUDORlTHpUVaRb899Nh8584p8YjUACC9MGIc=.b3b4adc8-b0fd-415f-9f99-95313d073b0d@github.com>

On Fri, 18 Aug 2023 07:14:41 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> I have removed usage of `currentTimeMillis()` in jtreg/gc as `currentTimeMillis()` is not monotonic. It is mostly changing from milliseconds to nanoseconds. In certain places, I have changed the code from something like `(instant1 < instant2)` to `(instant1 - instant2 < 0)` It might look like I am removing instant2 from both sides of the inequality and the result ought to be the same, but due to overflow arithmetic, the change should be better if nanoseconds where to overflow. I have also removed some loops where the loop is doing nothing except sleeping. I somewhat shortened the string in the key of a hash map because the nanosecond value made the string too long for the heap size.
>> 
>> All tests within jtreg/gc passes.
>
> test/hotspot/jtreg/gc/stress/systemgc/TestSystemGC.java line 195:
> 
>> 193:         }
>> 194:         long timeoutNanos = Integer.parseInt(args[0]) * 1_000_000_000L;
>> 195:         System.out.println("Running with timeout of " + timeoutNanos + "ns");
> 
> The precision here is in seconds. Maybe just leave this print out to be in ms?

Okay.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15331#discussion_r1298197464

From lkorinth at openjdk.org  Fri Aug 18 09:09:29 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Fri, 18 Aug 2023 09:09:29 GMT
Subject: RFR: 8311639: Replace currentTimeMillis() with nanoTime() in
 jtreg/gc
In-Reply-To: <ktTRrrseSndhXVa8I0Qm7cmnRViqA2T0HhKurTzwGHI=.06b48e59-e064-4e8a-90c4-c3bfbf8ba620@github.com>
References: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
 <3lUt9SMeH24aNsi_vj1X8cwF9W_ul_RG4oROF_yFj74=.b1af3d19-09db-4f4d-8fb4-58e8a9c74cc5@github.com>
 <ktTRrrseSndhXVa8I0Qm7cmnRViqA2T0HhKurTzwGHI=.06b48e59-e064-4e8a-90c4-c3bfbf8ba620@github.com>
Message-ID: <S8HWsmipRvuR6DGaTF9v_0G3puXKOKkpW_Ja2ISbWKk=.046686bf-036b-4b4f-8339-152d2e3ae31d@github.com>

On Fri, 18 Aug 2023 08:56:01 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> test/hotspot/jtreg/gc/stress/systemgc/TestSystemGC.java line 80:
>> 
>>> 78:     public void run() {
>>> 79:         while (!shouldExit()) {
>>> 80:             String prefix = "long" + (System.nanoTime() % 10000); // limit to 4 digits after changing from milliseconds to nanoseconds, else the key will use more memory
>> 
>> Is this used to calculate elapsed time? If not, I think using `currentTimeMillis` is fine.
>
> It is not to calculate elapsed time. It is to create a unique(ish) key for a hash table. The new code does not guarantee a unique value, but then neither did the old.

I could possibly add some characters to the key if that is what bothers you. I could of course also use `currentTimeMillis` as it is nothing wrong with it. But I would prefer removing `currentTimeMillis` in all gc testing.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15331#discussion_r1298206509

From lkorinth at openjdk.org  Fri Aug 18 09:23:27 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Fri, 18 Aug 2023 09:23:27 GMT
Subject: RFR: 8311639: Replace currentTimeMillis() with nanoTime() in
 jtreg/gc
In-Reply-To: <S8HWsmipRvuR6DGaTF9v_0G3puXKOKkpW_Ja2ISbWKk=.046686bf-036b-4b4f-8339-152d2e3ae31d@github.com>
References: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
 <3lUt9SMeH24aNsi_vj1X8cwF9W_ul_RG4oROF_yFj74=.b1af3d19-09db-4f4d-8fb4-58e8a9c74cc5@github.com>
 <ktTRrrseSndhXVa8I0Qm7cmnRViqA2T0HhKurTzwGHI=.06b48e59-e064-4e8a-90c4-c3bfbf8ba620@github.com>
 <S8HWsmipRvuR6DGaTF9v_0G3puXKOKkpW_Ja2ISbWKk=.046686bf-036b-4b4f-8339-152d2e3ae31d@github.com>
Message-ID: <YL7NIhNhZpVuXr8FzhpxeE1Ic7y4zhX-wWzrN5ZWhrY=.b7909e98-8e3c-4ad3-bde8-844f24b1649c@github.com>

On Fri, 18 Aug 2023 09:06:15 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> It is not to calculate elapsed time. It is to create a unique(ish) key for a hash table. The new code does not guarantee a unique value, but then neither did the old.
>
> I could possibly add some characters to the key if that is what bothers you. I could of course also use `currentTimeMillis` as it is nothing wrong with it. But I would prefer removing `currentTimeMillis` in all gc testing.

I will try to see if I can do it without using time at all.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15331#discussion_r1298220026

From lkorinth at openjdk.org  Fri Aug 18 09:23:29 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Fri, 18 Aug 2023 09:23:29 GMT
Subject: RFR: 8311639: Replace currentTimeMillis() with nanoTime() in
 jtreg/gc
In-Reply-To: <_jFP3gxGUDORlTHpUVaRb899Nh8584p8YjUACC9MGIc=.b3b4adc8-b0fd-415f-9f99-95313d073b0d@github.com>
References: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
 <AxbXUg0sifjPnCRX7un5CfYG_u3GcN9H-46_iKayWig=.24685f12-ae6a-4906-8438-c768aa7c93aa@github.com>
 <_jFP3gxGUDORlTHpUVaRb899Nh8584p8YjUACC9MGIc=.b3b4adc8-b0fd-415f-9f99-95313d073b0d@github.com>
Message-ID: <TrJQjK3P_5IyQOnorjMO8gXw2Zvw7-Xk67Pv5EjJN_Y=.8e773446-c773-4ce6-9786-619557ba80de@github.com>

On Fri, 18 Aug 2023 08:57:08 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> test/hotspot/jtreg/gc/stress/systemgc/TestSystemGC.java line 195:
>> 
>>> 193:         }
>>> 194:         long timeoutNanos = Integer.parseInt(args[0]) * 1_000_000_000L;
>>> 195:         System.out.println("Running with timeout of " + timeoutNanos + "ns");
>> 
>> The precision here is in seconds. Maybe just leave this print out to be in ms?
>
> Okay.

I will change it to seconds.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15331#discussion_r1298221614

From ayang at openjdk.org  Fri Aug 18 11:19:25 2023
From: ayang at openjdk.org (Albert Mingkun Yang)
Date: Fri, 18 Aug 2023 11:19:25 GMT
Subject: RFR: 8311639: Replace currentTimeMillis() with nanoTime() in
 jtreg/gc
In-Reply-To: <YL7NIhNhZpVuXr8FzhpxeE1Ic7y4zhX-wWzrN5ZWhrY=.b7909e98-8e3c-4ad3-bde8-844f24b1649c@github.com>
References: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
 <3lUt9SMeH24aNsi_vj1X8cwF9W_ul_RG4oROF_yFj74=.b1af3d19-09db-4f4d-8fb4-58e8a9c74cc5@github.com>
 <ktTRrrseSndhXVa8I0Qm7cmnRViqA2T0HhKurTzwGHI=.06b48e59-e064-4e8a-90c4-c3bfbf8ba620@github.com>
 <S8HWsmipRvuR6DGaTF9v_0G3puXKOKkpW_Ja2ISbWKk=.046686bf-036b-4b4f-8339-152d2e3ae31d@github.com>
 <YL7NIhNhZpVuXr8FzhpxeE1Ic7y4zhX-wWzrN5ZWhrY=.b7909e98-8e3c-4ad3-bde8-844f24b1649c@github.com>
Message-ID: <j-Ltr25x4LKUw7__BGGCpZ1agEzic8lWPldnCwzrGcM=.6cf578f7-58b0-46dc-8d20-209463beb66c@github.com>

On Fri, 18 Aug 2023 09:19:08 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> I could possibly add some characters to the key if that is what bothers you. I could of course also use `currentTimeMillis` as it is nothing wrong with it. But I would prefer removing `currentTimeMillis` in all gc testing.
>
> I will try to see if I can do it without using time at all.

Not using time is probably the best. Not obvious if #digits here is critical. Maybe this is useful -- e.g. to generate fixed 6-digit int:


Random rnd = new Random();
int n = 100_000 + rnd.nextInt(900_000);

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15331#discussion_r1298333710

From lkorinth at openjdk.org  Fri Aug 18 12:05:07 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Fri, 18 Aug 2023 12:05:07 GMT
Subject: RFR: 8311639: Replace currentTimeMillis() with nanoTime() in
 jtreg/gc [v2]
In-Reply-To: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
References: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
Message-ID: <LaOhLPX0jJSastmJCjwUGp8BMPLCSxVGf-wSEH037-Q=.15efd63b-9d09-4a0f-80e7-c11d815f9521@github.com>

> I have removed usage of `currentTimeMillis()` in jtreg/gc as `currentTimeMillis()` is not monotonic. It is mostly changing from milliseconds to nanoseconds. In certain places, I have changed the code from something like `(instant1 < instant2)` to `(instant1 - instant2 < 0)` It might look like I am removing instant2 from both sides of the inequality and the result ought to be the same, but due to overflow arithmetic, the change should be better if nanoseconds where to overflow. I have also removed some loops where the loop is doing nothing except sleeping. I somewhat shortened the string in the key of a hash map because the nanosecond value made the string too long for the heap size.
> 
> All tests within jtreg/gc passes.

Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:

  fix after comments from Albert and StefanK

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15331/files
  - new: https://git.openjdk.org/jdk/pull/15331/files/6d2fe207..c2e26a85

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15331&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15331&range=00-01

  Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/jdk/pull/15331.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15331/head:pull/15331

PR: https://git.openjdk.org/jdk/pull/15331

From lkorinth at openjdk.org  Fri Aug 18 12:13:29 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Fri, 18 Aug 2023 12:13:29 GMT
Subject: RFR: 8311639: Replace currentTimeMillis() with nanoTime() in
 jtreg/gc [v2]
In-Reply-To: <LaOhLPX0jJSastmJCjwUGp8BMPLCSxVGf-wSEH037-Q=.15efd63b-9d09-4a0f-80e7-c11d815f9521@github.com>
References: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
 <LaOhLPX0jJSastmJCjwUGp8BMPLCSxVGf-wSEH037-Q=.15efd63b-9d09-4a0f-80e7-c11d815f9521@github.com>
Message-ID: <KLaDF_Yvx6s7I9wLgDI7-mE4aL_dwLlXLYSGYMeIn9I=.7271e799-f35b-47e9-b34c-b769888437f9@github.com>

On Fri, 18 Aug 2023 12:05:07 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> I have removed usage of `currentTimeMillis()` in jtreg/gc as `currentTimeMillis()` is not monotonic. It is mostly changing from milliseconds to nanoseconds. In certain places, I have changed the code from something like `(instant1 < instant2)` to `(instant1 - instant2 < 0)` It might look like I am removing instant2 from both sides of the inequality and the result ought to be the same, but due to overflow arithmetic, the change should be better if nanoseconds where to overflow. I have also removed some loops where the loop is doing nothing except sleeping. I somewhat shortened the string in the key of a hash map because the nanosecond value made the string too long for the heap size.
>> 
>> All tests within jtreg/gc passes.
>
> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix after comments from Albert and StefanK

I will refactor a bit more...

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15331#issuecomment-1683827689

From lkorinth at openjdk.org  Fri Aug 18 12:38:03 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Fri, 18 Aug 2023 12:38:03 GMT
Subject: RFR: 8311639: Replace currentTimeMillis() with nanoTime() in
 jtreg/gc [v3]
In-Reply-To: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
References: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
Message-ID: <idzoZ6c3vZoUUqVEtfAss1dgOam8uDOGx4IS3Qx5i5o=.eeaf110a-6471-4153-8d9e-5b023943603c@github.com>

> I have removed usage of `currentTimeMillis()` in jtreg/gc as `currentTimeMillis()` is not monotonic. It is mostly changing from milliseconds to nanoseconds. In certain places, I have changed the code from something like `(instant1 < instant2)` to `(instant1 - instant2 < 0)` It might look like I am removing instant2 from both sides of the inequality and the result ought to be the same, but due to overflow arithmetic, the change should be better if nanoseconds where to overflow. I have also removed some loops where the loop is doing nothing except sleeping. I somewhat shortened the string in the key of a hash map because the nanosecond value made the string too long for the heap size.
> 
> All tests within jtreg/gc passes.

Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:

  printing unparsed values

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15331/files
  - new: https://git.openjdk.org/jdk/pull/15331/files/c2e26a85..926a7f31

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15331&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15331&range=01-02

  Stats: 3 lines in 2 files changed: 0 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/15331.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15331/head:pull/15331

PR: https://git.openjdk.org/jdk/pull/15331

From ayang at openjdk.org  Fri Aug 18 12:38:23 2023
From: ayang at openjdk.org (Albert Mingkun Yang)
Date: Fri, 18 Aug 2023 12:38:23 GMT
Subject: RFR: 8311639: Replace currentTimeMillis() with nanoTime() in
 jtreg/gc [v3]
In-Reply-To: <idzoZ6c3vZoUUqVEtfAss1dgOam8uDOGx4IS3Qx5i5o=.eeaf110a-6471-4153-8d9e-5b023943603c@github.com>
References: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
 <idzoZ6c3vZoUUqVEtfAss1dgOam8uDOGx4IS3Qx5i5o=.eeaf110a-6471-4153-8d9e-5b023943603c@github.com>
Message-ID: <t0DrrL5oY8fuIuERx0MsoJNSOrMFzLnUleyjOIKm4ZA=.bd0e6562-a460-41ed-84ce-daaf8bd6cc52@github.com>

On Fri, 18 Aug 2023 12:38:03 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> I have removed usage of `currentTimeMillis()` in jtreg/gc as `currentTimeMillis()` is not monotonic. It is mostly changing from milliseconds to nanoseconds. In certain places, I have changed the code from something like `(instant1 < instant2)` to `(instant1 - instant2 < 0)` It might look like I am removing instant2 from both sides of the inequality and the result ought to be the same, but due to overflow arithmetic, the change should be better if nanoseconds where to overflow. I have also removed some loops where the loop is doing nothing except sleeping. I somewhat shortened the string in the key of a hash map because the nanosecond value made the string too long for the heap size.
>> 
>> All tests within jtreg/gc passes.
>
> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
> 
>   printing unparsed values

Marked as reviewed by ayang (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/15331#pullrequestreview-1584491156

From lkorinth at openjdk.org  Fri Aug 18 12:49:27 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Fri, 18 Aug 2023 12:49:27 GMT
Subject: RFR: 8311639: Replace currentTimeMillis() with nanoTime() in
 jtreg/gc [v3]
In-Reply-To: <j-Ltr25x4LKUw7__BGGCpZ1agEzic8lWPldnCwzrGcM=.6cf578f7-58b0-46dc-8d20-209463beb66c@github.com>
References: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
 <3lUt9SMeH24aNsi_vj1X8cwF9W_ul_RG4oROF_yFj74=.b1af3d19-09db-4f4d-8fb4-58e8a9c74cc5@github.com>
 <ktTRrrseSndhXVa8I0Qm7cmnRViqA2T0HhKurTzwGHI=.06b48e59-e064-4e8a-90c4-c3bfbf8ba620@github.com>
 <S8HWsmipRvuR6DGaTF9v_0G3puXKOKkpW_Ja2ISbWKk=.046686bf-036b-4b4f-8339-152d2e3ae31d@github.com>
 <YL7NIhNhZpVuXr8FzhpxeE1Ic7y4zhX-wWzrN5ZWhrY=.b7909e98-8e3c-4ad3-bde8-844f24b1649c@github.com>
 <j-Ltr25x4LKUw7__BGGCpZ1agEzic8lWPldnCwzrGcM=.6cf578f7-58b0-46dc-8d20-209463beb66c@github.com>
Message-ID: <m_wTMN72fKLzra1-hNCw8Rvpgxp0aOi103gMkZTjUmY=.6b814094-d381-4d93-a6f7-2351efa5d8eb@github.com>

On Fri, 18 Aug 2023 11:16:30 GMT, Albert Mingkun Yang <ayang at openjdk.org> wrote:

>> I will try to see if I can do it without using time at all.
>
> Not using time is probably the best. Not obvious if #digits here is critical. Maybe this is useful -- e.g. to generate fixed 6-digit int:
> 
> 
> Random rnd = new Random();
> int n = 100_000 + rnd.nextInt(900_000);

I reverted to the original implementation as I did not understand the idea behind it.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15331#discussion_r1298412049

From stefank at openjdk.org  Fri Aug 18 13:13:27 2023
From: stefank at openjdk.org (Stefan Karlsson)
Date: Fri, 18 Aug 2023 13:13:27 GMT
Subject: RFR: 8311639: Replace currentTimeMillis() with nanoTime() in
 jtreg/gc [v3]
In-Reply-To: <idzoZ6c3vZoUUqVEtfAss1dgOam8uDOGx4IS3Qx5i5o=.eeaf110a-6471-4153-8d9e-5b023943603c@github.com>
References: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
 <idzoZ6c3vZoUUqVEtfAss1dgOam8uDOGx4IS3Qx5i5o=.eeaf110a-6471-4153-8d9e-5b023943603c@github.com>
Message-ID: <j4BydCbFTycGIa3x8pU_8EUjVP8Vdn3C4hQjL3OGAXY=.e96d5d7d-463b-435b-8a71-768359dc6032@github.com>

On Fri, 18 Aug 2023 12:38:03 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> I have removed usage of `currentTimeMillis()` in jtreg/gc as `currentTimeMillis()` is not monotonic. It is mostly changing from milliseconds to nanoseconds. In certain places, I have changed the code from something like `(instant1 < instant2)` to `(instant1 - instant2 < 0)` It might look like I am removing instant2 from both sides of the inequality and the result ought to be the same, but due to overflow arithmetic, the change should be better if nanoseconds where to overflow. I have also removed some loops where the loop is doing nothing except sleeping. I somewhat shortened the string in the key of a hash map because the nanosecond value made the string too long for the heap size.
>> 
>> All tests within jtreg/gc passes.
>
> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
> 
>   printing unparsed values

Marked as reviewed by stefank (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/15331#pullrequestreview-1584544357

From lkorinth at openjdk.org  Fri Aug 18 13:13:28 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Fri, 18 Aug 2023 13:13:28 GMT
Subject: RFR: 8311639: Replace currentTimeMillis() with nanoTime() in
 jtreg/gc [v3]
In-Reply-To: <idzoZ6c3vZoUUqVEtfAss1dgOam8uDOGx4IS3Qx5i5o=.eeaf110a-6471-4153-8d9e-5b023943603c@github.com>
References: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
 <idzoZ6c3vZoUUqVEtfAss1dgOam8uDOGx4IS3Qx5i5o=.eeaf110a-6471-4153-8d9e-5b023943603c@github.com>
Message-ID: <c90DJ4c-m0dEMPOuy7z5Hp8M3BGByBS1FQEbjCJA7ds=.1e7da271-603b-48ac-8d07-4ed3e9592169@github.com>

On Fri, 18 Aug 2023 12:38:03 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> I have removed usage of `currentTimeMillis()` in jtreg/gc as `currentTimeMillis()` is not monotonic. It is mostly changing from milliseconds to nanoseconds. In certain places, I have changed the code from something like `(instant1 < instant2)` to `(instant1 - instant2 < 0)` It might look like I am removing instant2 from both sides of the inequality and the result ought to be the same, but due to overflow arithmetic, the change should be better if nanoseconds where to overflow. I have also removed some loops where the loop is doing nothing except sleeping. I somewhat shortened the string in the key of a hash map because the nanosecond value made the string too long for the heap size.
>> 
>> All tests within jtreg/gc passes.
>
> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
> 
>   printing unparsed values

Thanks Albert and StefanK, I will integrate next week.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15331#issuecomment-1683900138

From lkorinth at openjdk.org  Mon Aug 21 12:22:40 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Mon, 21 Aug 2023 12:22:40 GMT
Subject: Integrated: 8311639: Replace currentTimeMillis() with nanoTime() in
 jtreg/gc
In-Reply-To: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
References: <tosnZvgq7GIZXcmWK2keYyhwN8JqgQNUQ-1St_oloq8=.4bd54d69-6f90-4ff5-8960-1e2dfcbfa18c@github.com>
Message-ID: <LaaS7QTkX5AdvxkPIdEK5s8RS-c5I_NSoORGTH8udNc=.7d41e539-acbc-4280-86eb-dd87d1b6b523@github.com>

On Thu, 17 Aug 2023 15:56:04 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

> I have removed usage of `currentTimeMillis()` in jtreg/gc as `currentTimeMillis()` is not monotonic. It is mostly changing from milliseconds to nanoseconds. In certain places, I have changed the code from something like `(instant1 < instant2)` to `(instant1 - instant2 < 0)` It might look like I am removing instant2 from both sides of the inequality and the result ought to be the same, but due to overflow arithmetic, the change should be better if nanoseconds where to overflow. I have also removed some loops where the loop is doing nothing except sleeping. I somewhat shortened the string in the key of a hash map because the nanosecond value made the string too long for the heap size.
> 
> All tests within jtreg/gc passes.

This pull request has now been integrated.

Changeset: 17a19dc0
Author:    Leo Korinth <lkorinth at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/17a19dc06062331c954fa519a4f668664b1533c9
Stats:     127 lines in 17 files changed: 2 ins; 15 del; 110 mod

8311639: Replace currentTimeMillis() with nanoTime() in jtreg/gc

Reviewed-by: stefank, ayang

-------------

PR: https://git.openjdk.org/jdk/pull/15331

From wkemper at openjdk.org  Mon Aug 21 17:21:19 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 21 Aug 2023 17:21:19 GMT
Subject: Integrated: Merge openjdk/jdk:master
Message-ID: <1TpuHbVNKlzphy8Gky_i1O244TNWDdeCRmVMZwzldKE=.e21dd07d-4d6a-4198-b89f-f909878da352@github.com>

Merges tag jdk-22+11

-------------

Commit messages:
 - Merge tag 'jdk-22+11' into merge-jdk-22-11
 - 8311943: Cleanup usages of toLowerCase() and toUpperCase() in java.base
 - 8313765: Invalid CEN header (invalid zip64 extra data field size)
 - 8310275: Bug in assignment operator of ReservedMemoryRegion
 - 8314144: gc/g1/ihop/TestIHOPStatic.java fails due to extra concurrent mark with -Xcomp
 - 8314209: Wrong @since tag for RandomGenerator::equiDoubles
 - 8314211: Add NativeLibraryUnload event
 - 8313760: [REDO] Enhance AES performance
 - 8313720: C2 SuperWord: wrong result with -XX:+UseVectorCmov -XX:+UseCMoveUnconditionally
 - 8314262: GHA: Cut down cross-compilation sysroots deeper
 - ... and 69 more: https://git.openjdk.org/shenandoah/compare/867f558d...aa98fd01

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah/pull/308/files
  Stats: 9165 lines in 330 files changed: 5446 ins; 693 del; 3026 mod
  Patch: https://git.openjdk.org/shenandoah/pull/308.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/308/head:pull/308

PR: https://git.openjdk.org/shenandoah/pull/308

From wkemper at openjdk.org  Mon Aug 21 17:21:21 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 21 Aug 2023 17:21:21 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <1TpuHbVNKlzphy8Gky_i1O244TNWDdeCRmVMZwzldKE=.e21dd07d-4d6a-4198-b89f-f909878da352@github.com>
References: <1TpuHbVNKlzphy8Gky_i1O244TNWDdeCRmVMZwzldKE=.e21dd07d-4d6a-4198-b89f-f909878da352@github.com>
Message-ID: <Z6IcsLHRmiD_DDoKD6V3JBH_O_dYVXsI_shqBLEN4f8=.be9ea713-9e1c-49dd-b2a7-555c06cc52c4@github.com>

On Mon, 21 Aug 2023 17:11:55 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-22+11

This pull request has now been integrated.

Changeset: 8607f8f4
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/8607f8f48654defecad18fb8a6eac36063e45bb3
Stats:     9165 lines in 330 files changed: 5446 ins; 693 del; 3026 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/308

From wkemper at openjdk.org  Tue Aug 22 18:31:20 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 22 Aug 2023 18:31:20 GMT
Subject: RFR: JDK-8314777: GenShen: Alias young and old marking bits to
 legacy Shenandoah marking bit in gc state
In-Reply-To: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
References: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
Message-ID: <s9_RcuQoG1PgtyOF2Yk32mOP4xHtEA93puAVYckyjD0=.c4b50384-969c-412d-8651-a535b4a0ff66@github.com>

On Tue, 22 Aug 2023 00:10:42 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This change sets the MARKING bit of _gc_state whenever either Young marking or Old marking is active.  With this change, barrier code more closely resembles the code in the original single-generation Shenandoah.  The performance impact is negligible.  The primary benefit is to simplify code reviews and clarify that the addition of generational mode to Shenandoah does not negatively impact performance of single-generation Shenandoah.

Changes requested by wkemper (Committer).

src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.cpp line 944:

> 942:     int flags = ShenandoahHeap::HAS_FORWARDED;
> 943:     if (ShenandoahIUBarrier) {
> 944:       assert(!ShenandoahHeap::heap()->mode()->is_generational(), "Generational mode does not support IU barrier");

Not sure we need an assert here, we could check this condition in `shenandoahGenerationalMode.cpp` or `shenandoahArguments.cpp` to minimize deltas from upstream.

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2460:

> 2458:   if (!in_progress && is_concurrent_old_mark_in_progress()) {
> 2459:     assert(mode()->is_generational(), "Only generational GC has old marking");
> 2460:     // If old-marking is in progress when we turn off YOUNG_MARKING, leave MARKING (and OLD_MARKING) on

Can we assert that gc state mask has `MARKING` set here?

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2468:

> 2466: }
> 2467: 
> 2468: void ShenandoahHeap::set_concurrent_old_mark_in_progress(bool in_progress) {

Should we assert that if `has_forwarded_objects` that the gc state mask has `UPDATEREFS` set?

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2470:

> 2468: void ShenandoahHeap::set_concurrent_old_mark_in_progress(bool in_progress) {
> 2469:   if (!in_progress && is_concurrent_young_mark_in_progress()) {
> 2470:     // If young-marking is in progress when we turn off OLD_MARKING, leave MARKING (and YOUNG_MARKING) on

Similarly, assert that `MARKING` is set here?

src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 661:

> 659: 
> 660: inline bool ShenandoahHeap::is_idle() const {
> 661:   return _gc_state.is_unset(MARKING | YOUNG_MARKING | OLD_MARKING | EVACUATION | UPDATEREFS);

We should just need to check `MARKING` now, right?

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/309#pullrequestreview-1589843000
PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1301892881
PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1301885170
PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1301883842
PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1301887335
PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1301888882

From kdnilsen at openjdk.org  Tue Aug 22 18:30:47 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 22 Aug 2023 18:30:47 GMT
Subject: RFR: JDK-8314777: GenShen: Alias young and old marking bits to legacy
 Shenandoah marking bit in gc state
Message-ID: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>

This change sets the MARKING bit of _gc_state whenever either Young marking or Old marking is active.  With this change, barrier code more closely resembles the code in the original single-generation Shenandoah.  The performance impact is negligible.  The primary benefit is to simplify code reviews and clarify that the addition of generational mode to Shenandoah does not negatively impact performance of single-generation Shenandoah.

-------------

Commit messages:
 - Fix whitespace
 - More reviewer requests
 - Fix white space error
 - Fiddle with format to fix jcheck
 - Respond to reviewer feedback
 - Fixup bit-manipulation errors for multi-bit gc state
 - Make MARKING bit an alias for YOUNG_MARKING | OLD_MARKING

Changes: https://git.openjdk.org/shenandoah/pull/309/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=309&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8314777
  Stats: 65 lines in 11 files changed: 27 ins; 8 del; 30 mod
  Patch: https://git.openjdk.org/shenandoah/pull/309.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/309/head:pull/309

PR: https://git.openjdk.org/shenandoah/pull/309

From kdnilsen at openjdk.org  Tue Aug 22 18:31:20 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 22 Aug 2023 18:31:20 GMT
Subject: RFR: JDK-8314777: GenShen: Alias young and old marking bits to
 legacy Shenandoah marking bit in gc state
In-Reply-To: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
References: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
Message-ID: <lnEv0iY0ZTfx3v5stZaekmzWrrLmr1_VMs3ZRcfq-ZY=.690fff26-77cd-49f4-baf7-2eabbc541574@github.com>

On Tue, 22 Aug 2023 00:10:42 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This change sets the MARKING bit of _gc_state whenever either Young marking or Old marking is active.  With this change, barrier code more closely resembles the code in the original single-generation Shenandoah.  The performance impact is negligible.  The primary benefit is to simplify code reviews and clarify that the addition of generational mode to Shenandoah does not negatively impact performance of single-generation Shenandoah.

Seems to be a bug in jcheck white-space error reports.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/309#issuecomment-1688617455

From kdnilsen at openjdk.org  Tue Aug 22 18:31:22 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 22 Aug 2023 18:31:22 GMT
Subject: RFR: JDK-8314777: GenShen: Alias young and old marking bits to
 legacy Shenandoah marking bit in gc state
In-Reply-To: <s9_RcuQoG1PgtyOF2Yk32mOP4xHtEA93puAVYckyjD0=.c4b50384-969c-412d-8651-a535b4a0ff66@github.com>
References: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
 <s9_RcuQoG1PgtyOF2Yk32mOP4xHtEA93puAVYckyjD0=.c4b50384-969c-412d-8651-a535b4a0ff66@github.com>
Message-ID: <9G1I7FP6ZML4XZXuYfppWsTk8HncPOHMw3XkEZyVsX0=.597ee6e4-ac9a-4b40-9dd4-c1731d2f0bd5@github.com>

On Tue, 22 Aug 2023 16:20:54 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> This change sets the MARKING bit of _gc_state whenever either Young marking or Old marking is active.  With this change, barrier code more closely resembles the code in the original single-generation Shenandoah.  The performance impact is negligible.  The primary benefit is to simplify code reviews and clarify that the addition of generational mode to Shenandoah does not negatively impact performance of single-generation Shenandoah.
>
> src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.cpp line 944:
> 
>> 942:     int flags = ShenandoahHeap::HAS_FORWARDED;
>> 943:     if (ShenandoahIUBarrier) {
>> 944:       assert(!ShenandoahHeap::heap()->mode()->is_generational(), "Generational mode does not support IU barrier");
> 
> Not sure we need an assert here, we could check this condition in `shenandoahGenerationalMode.cpp` or `shenandoahArguments.cpp` to minimize deltas from upstream.

Thanks.  Moving this check to shenandoahArguments.cpp.

> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2460:
> 
>> 2458:   if (!in_progress && is_concurrent_old_mark_in_progress()) {
>> 2459:     assert(mode()->is_generational(), "Only generational GC has old marking");
>> 2460:     // If old-marking is in progress when we turn off YOUNG_MARKING, leave MARKING (and OLD_MARKING) on
> 
> Can we assert that gc state mask has `MARKING` set here?

adding this

> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2468:
> 
>> 2466: }
>> 2467: 
>> 2468: void ShenandoahHeap::set_concurrent_old_mark_in_progress(bool in_progress) {
> 
> Should we assert that if `has_forwarded_objects` that the gc state mask has `UPDATEREFS` set?

I think original code may have been wrong.  When we set_gc_state_mask(), this sets the bits specified in the argument and leaves other bits alone.  So UPDATEREFS or EVACUATION bit should have already been on.  I'll go ahead and add this assert.

> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2470:
> 
>> 2468: void ShenandoahHeap::set_concurrent_old_mark_in_progress(bool in_progress) {
>> 2469:   if (!in_progress && is_concurrent_young_mark_in_progress()) {
>> 2470:     // If young-marking is in progress when we turn off OLD_MARKING, leave MARKING (and YOUNG_MARKING) on
> 
> Similarly, assert that `MARKING` is set here?

and adding this

> src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 661:
> 
>> 659: 
>> 660: inline bool ShenandoahHeap::is_idle() const {
>> 661:   return _gc_state.is_unset(MARKING | YOUNG_MARKING | OLD_MARKING | EVACUATION | UPDATEREFS);
> 
> We should just need to check `MARKING` now, right?

true.  (I guess I can also add assert (!MARKING || (YOUNG_MARKING || OLD_MARKING)).  I'll make this change.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1301926924
PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1301900108
PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1301990462
PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1301900332
PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1301905689

From shade at openjdk.org  Tue Aug 22 18:31:22 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 22 Aug 2023 18:31:22 GMT
Subject: RFR: JDK-8314777: GenShen: Alias young and old marking bits to
 legacy Shenandoah marking bit in gc state
In-Reply-To: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
References: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
Message-ID: <P46T3RBluEr8XC3Kdv5WwIZt30vJWu8eoTBT3dmkVMc=.8fd52845-9e64-4a34-afd4-abc96a03a78a@github.com>

On Tue, 22 Aug 2023 00:10:42 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This change sets the MARKING bit of _gc_state whenever either Young marking or Old marking is active.  With this change, barrier code more closely resembles the code in the original single-generation Shenandoah.  The performance impact is negligible.  The primary benefit is to simplify code reviews and clarify that the addition of generational mode to Shenandoah does not negatively impact performance of single-generation Shenandoah.

src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 317:

> 315:     HAS_FORWARDED_BITPOS   = 0,
> 316: 
> 317:     // Young or OLD regions are under marking: needs SATB barriers.

To match upstream better, it would be proper to do:


// Heap is under marking: needs SATB barriers.
// For generational mode, it means either young or old marking, or both.
MARKING_BITPOS    = 1,

src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 665:

> 663:   assert((_gc_state.is_unset(MARKING) && _gc_state.is_unset(YOUNG_MARKING) && _gc_state.is_unset(OLD_MARKING)) ||
> 664: 	 (_gc_state.is_set(MARKING) && (_gc_state.is_set(YOUNG_MARKING) || _gc_state.is_set(OLD_MARKING))),
> 665: 	 "Inconsistent gc marking state");

I think the proper place for checks like these are in `ShenandoahVerifier::verify_at_safepoint`

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1301996305
PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1302004828

From kdnilsen at openjdk.org  Tue Aug 22 19:41:47 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 22 Aug 2023 19:41:47 GMT
Subject: RFR: JDK-8314777: GenShen: Alias young and old marking bits to
 legacy Shenandoah marking bit in gc state
In-Reply-To: <P46T3RBluEr8XC3Kdv5WwIZt30vJWu8eoTBT3dmkVMc=.8fd52845-9e64-4a34-afd4-abc96a03a78a@github.com>
References: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
 <P46T3RBluEr8XC3Kdv5WwIZt30vJWu8eoTBT3dmkVMc=.8fd52845-9e64-4a34-afd4-abc96a03a78a@github.com>
Message-ID: <8-MaDk5icnsfqi4h_F0e6wlik4FtA4Su8NFHeoQPcv0=.faa6e2d7-2a59-4917-84e3-ac776fe1b90f@github.com>

On Tue, 22 Aug 2023 17:52:11 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> This change sets the MARKING bit of _gc_state whenever either Young marking or Old marking is active.  With this change, barrier code more closely resembles the code in the original single-generation Shenandoah.  The performance impact is negligible.  The primary benefit is to simplify code reviews and clarify that the addition of generational mode to Shenandoah does not negatively impact performance of single-generation Shenandoah.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 317:
> 
>> 315:     HAS_FORWARDED_BITPOS   = 0,
>> 316: 
>> 317:     // Young or OLD regions are under marking: needs SATB barriers.
> 
> To match upstream better, it would be proper to do:
> 
> 
> // Heap is under marking: needs SATB barriers.
> // For generational mode, it means either young or old marking, or both.
> MARKING_BITPOS    = 1,

Changing this comment.

> src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 665:
> 
>> 663:   assert((_gc_state.is_unset(MARKING) && _gc_state.is_unset(YOUNG_MARKING) && _gc_state.is_unset(OLD_MARKING)) ||
>> 664: 	 (_gc_state.is_set(MARKING) && (_gc_state.is_set(YOUNG_MARKING) || _gc_state.is_set(OLD_MARKING))),
>> 665: 	 "Inconsistent gc marking state");
> 
> I think the proper place for checks like these are in `ShenandoahVerifier::verify_at_safepoint`

Moving these to +ShenandoahVerify

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1302101067
PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1302101326

From kdnilsen at openjdk.org  Tue Aug 22 20:47:47 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 22 Aug 2023 20:47:47 GMT
Subject: RFR: JDK-8314777: GenShen: Alias young and old marking bits to
 legacy Shenandoah marking bit in gc state [v2]
In-Reply-To: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
References: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
Message-ID: <Jfua5Mra8GcoYczbaGI7Y7J6uMAmlgd_yhd__zJWDLw=.41956667-46a2-4259-af83-e98cfc24b0bc@github.com>

> This change sets the MARKING bit of _gc_state whenever either Young marking or Old marking is active.  With this change, barrier code more closely resembles the code in the original single-generation Shenandoah.  The performance impact is negligible.  The primary benefit is to simplify code reviews and clarify that the addition of generational mode to Shenandoah does not negatively impact performance of single-generation Shenandoah.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Tidy up asserts

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/309/files
  - new: https://git.openjdk.org/shenandoah/pull/309/files/cf4a02ad..2aaf20b5

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=309&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=309&range=00-01

  Stats: 19 lines in 4 files changed: 11 ins; 5 del; 3 mod
  Patch: https://git.openjdk.org/shenandoah/pull/309.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/309/head:pull/309

PR: https://git.openjdk.org/shenandoah/pull/309

From ysr at openjdk.org  Tue Aug 22 22:13:17 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 22 Aug 2023 22:13:17 GMT
Subject: RFR: JDK-8314777: GenShen: Alias young and old marking bits to
 legacy Shenandoah marking bit in gc state [v2]
In-Reply-To: <Jfua5Mra8GcoYczbaGI7Y7J6uMAmlgd_yhd__zJWDLw=.41956667-46a2-4259-af83-e98cfc24b0bc@github.com>
References: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
 <Jfua5Mra8GcoYczbaGI7Y7J6uMAmlgd_yhd__zJWDLw=.41956667-46a2-4259-af83-e98cfc24b0bc@github.com>
Message-ID: <KkcYzjMdUk2TKCbEZ7DkTYh66zvgczUZfwAXxZKWonc=.935c652d-f5d3-4c70-9fd7-696bb1aed50e@github.com>

On Tue, 22 Aug 2023 20:47:47 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> This change sets the MARKING bit of _gc_state whenever either Young marking or Old marking is active.  With this change, barrier code more closely resembles the code in the original single-generation Shenandoah.  The performance impact is negligible.  The primary benefit is to simplify code reviews and clarify that the addition of generational mode to Shenandoah does not negatively impact performance of single-generation Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Tidy up asserts

Some minor comments and a small cleanup/refactor suggestion that is separate and would need to upstream separately, and would simplify some code.

src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp line 842:

> 840:              ((actual & ShenandoahHeap::MARKING) && (actual & (ShenandoahHeap::YOUNG_MARKING | ShenandoahHeap::OLD_MARKING))),
> 841:              "Inconsistent gc marking state");
> 842: 

Would this read easier?


#ifdef ASSERT
    const ShenandoahSharedValue val = actual & (MARKING|YOUNG_MARKING|OLD_MARKING);
    assert((val & MARKING) == ((val & YOUNG_MARKING) || (val & OLD_MARKING)), "Error");
#endif

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/309#pullrequestreview-1590367114
PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1302233167

From ysr at openjdk.org  Tue Aug 22 22:14:17 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 22 Aug 2023 22:14:17 GMT
Subject: RFR: JDK-8314777: GenShen: Alias young and old marking bits to
 legacy Shenandoah marking bit in gc state [v2]
In-Reply-To: <9G1I7FP6ZML4XZXuYfppWsTk8HncPOHMw3XkEZyVsX0=.597ee6e4-ac9a-4b40-9dd4-c1731d2f0bd5@github.com>
References: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
 <s9_RcuQoG1PgtyOF2Yk32mOP4xHtEA93puAVYckyjD0=.c4b50384-969c-412d-8651-a535b4a0ff66@github.com>
 <9G1I7FP6ZML4XZXuYfppWsTk8HncPOHMw3XkEZyVsX0=.597ee6e4-ac9a-4b40-9dd4-c1731d2f0bd5@github.com>
Message-ID: <ksDCvbooZzMk7h39UOfXAenQDRxziMeBQAMnXQCcAi0=.d3b965ed-d5a0-4962-9edb-46455c2e3ead@github.com>

On Tue, 22 Aug 2023 16:26:07 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2460:
>> 
>>> 2458:   if (!in_progress && is_concurrent_old_mark_in_progress()) {
>>> 2459:     assert(mode()->is_generational(), "Only generational GC has old marking");
>>> 2460:     // If old-marking is in progress when we turn off YOUNG_MARKING, leave MARKING (and OLD_MARKING) on
>> 
>> Can we assert that gc state mask has `MARKING` set here?
>
> adding this

Would this be more succinct and easier to read?


  if (mode()->is_generational() && is_concurrent_old_mark_in_progress()) {
     assert(_gc_state.is_set(MARKING), "Marking flag should be set");
     mask = YOUNG_MARKING;
  } else {
     assert(_gc_state.is_clear(MARKING|OLD_MARKING|YOUNG_MARKING), "All marking flags should be clear");
     mask = MARKING | YOUNG_MARKING;
  }
  set_gc_state_mask(mask, in_progress);


where `is_set()` and `is_clear()` check if all flag bits in argument are either all set or all clear (which is different from their current sense). See comment below.

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2470:
>> 
>>> 2468: void ShenandoahHeap::set_concurrent_old_mark_in_progress(bool in_progress) {
>>> 2469:   if (!in_progress && is_concurrent_young_mark_in_progress()) {
>>> 2470:     // If young-marking is in progress when we turn off OLD_MARKING, leave MARKING (and YOUNG_MARKING) on
>> 
>> Similarly, assert that `MARKING` is set here?
>
> and adding this

This could also be simplified along the lines of my previous suggestion above.

Also the "if and only if" for state at the beginning here can be simplified to (but continue reading below):


   assert(has_forwarded_objects() == (_gc_state.is_set(UPDATEREFS | EVACUATION), "Error");


provided `is_set()` and `is_unset()` (or ideally called `is_clear()`)  methods are defined symmetrically to test all of the bits passed in the mask argument; i.e. `is_set()` answers true iff they are _all_ set, and `is_unset()` returns true iff they are _all_ unset. This will simplify many of the asserts that need to test the state of multiple bits at once.

That change might want to go in separately as it would affect a bunch of existing code/tests. And, yes, it is possible that that extent of change would be considered unacceptable upstream (but I am guessing it should be fine if made on tip now as a clean-up/refactor).

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1302212292
PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1302235992

From wkemper at openjdk.org  Tue Aug 22 23:57:47 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 22 Aug 2023 23:57:47 GMT
Subject: RFR: JDK-8314777: GenShen: Alias young and old marking bits to
 legacy Shenandoah marking bit in gc state [v2]
In-Reply-To: <Jfua5Mra8GcoYczbaGI7Y7J6uMAmlgd_yhd__zJWDLw=.41956667-46a2-4259-af83-e98cfc24b0bc@github.com>
References: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
 <Jfua5Mra8GcoYczbaGI7Y7J6uMAmlgd_yhd__zJWDLw=.41956667-46a2-4259-af83-e98cfc24b0bc@github.com>
Message-ID: <hAAAFX-N54vEqJeCQVO_YZgNgIZ_8-JCY2AfyKcZ31E=.8961cc0f-49d9-4579-a4a6-04e4f1803215@github.com>

On Tue, 22 Aug 2023 20:47:47 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> This change sets the MARKING bit of _gc_state whenever either Young marking or Old marking is active.  With this change, barrier code more closely resembles the code in the original single-generation Shenandoah.  The performance impact is negligible.  The primary benefit is to simplify code reviews and clarify that the addition of generational mode to Shenandoah does not negatively impact performance of single-generation Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Tidy up asserts

Marked as reviewed by wkemper (Committer).

Looks like new assert is causing build problems and test failures.

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/309#pullrequestreview-1590497751
Changes requested by wkemper (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/309#pullrequestreview-1590499434

From kdnilsen at openjdk.org  Wed Aug 23 09:30:13 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 23 Aug 2023 09:30:13 GMT
Subject: RFR: JDK-8314777: GenShen: Alias young and old marking bits to
 legacy Shenandoah marking bit in gc state [v3]
In-Reply-To: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
References: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
Message-ID: <_sNDhbL7t_koO3PLmjxywSCtkn3rXDT-JrMHfZwSfkw=.91c09c0d-5149-4f63-b3fc-775c478e7787@github.com>

> This change sets the MARKING bit of _gc_state whenever either Young marking or Old marking is active.  With this change, barrier code more closely resembles the code in the original single-generation Shenandoah.  The performance impact is negligible.  The primary benefit is to simplify code reviews and clarify that the addition of generational mode to Shenandoah does not negatively impact performance of single-generation Shenandoah.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Add parens to assert statement

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/309/files
  - new: https://git.openjdk.org/shenandoah/pull/309/files/2aaf20b5..d3ecbc91

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=309&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=309&range=01-02

  Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/shenandoah/pull/309.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/309/head:pull/309

PR: https://git.openjdk.org/shenandoah/pull/309

From kdnilsen at openjdk.org  Wed Aug 23 14:46:14 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 23 Aug 2023 14:46:14 GMT
Subject: RFR: JDK-8314777: GenShen: Alias young and old marking bits to
 legacy Shenandoah marking bit in gc state [v2]
In-Reply-To: <KkcYzjMdUk2TKCbEZ7DkTYh66zvgczUZfwAXxZKWonc=.935c652d-f5d3-4c70-9fd7-696bb1aed50e@github.com>
References: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
 <Jfua5Mra8GcoYczbaGI7Y7J6uMAmlgd_yhd__zJWDLw=.41956667-46a2-4259-af83-e98cfc24b0bc@github.com>
 <KkcYzjMdUk2TKCbEZ7DkTYh66zvgczUZfwAXxZKWonc=.935c652d-f5d3-4c70-9fd7-696bb1aed50e@github.com>
Message-ID: <2Sbcz3RuDJmY3eRkY1cOuhhCnhedOyeX8GHUSmqcvKc=.6b38b608-2763-47de-ab66-8a8259e6d683@github.com>

On Tue, 22 Aug 2023 21:54:57 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Tidy up asserts
>
> src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp line 842:
> 
>> 840:              ((actual & ShenandoahHeap::MARKING) && (actual & (ShenandoahHeap::YOUNG_MARKING | ShenandoahHeap::OLD_MARKING))),
>> 841:              "Inconsistent gc marking state");
>> 842: 
> 
> Would this read easier?
> 
> 
> #ifdef ASSERT
>     const ShenandoahSharedValue val = actual & (MARKING|YOUNG_MARKING|OLD_MARKING);
>     assert((val & MARKING) == ((val & YOUNG_MARKING) || (val & OLD_MARKING)), "Error");
> #endif

Thanks.  I like this improvement, slightly refined.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1303131797

From kdnilsen at openjdk.org  Wed Aug 23 19:39:54 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 23 Aug 2023 19:39:54 GMT
Subject: RFR: JDK-8314777: GenShen: Alias young and old marking bits to
 legacy Shenandoah marking bit in gc state [v3]
In-Reply-To: <ksDCvbooZzMk7h39UOfXAenQDRxziMeBQAMnXQCcAi0=.d3b965ed-d5a0-4962-9edb-46455c2e3ead@github.com>
References: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
 <s9_RcuQoG1PgtyOF2Yk32mOP4xHtEA93puAVYckyjD0=.c4b50384-969c-412d-8651-a535b4a0ff66@github.com>
 <9G1I7FP6ZML4XZXuYfppWsTk8HncPOHMw3XkEZyVsX0=.597ee6e4-ac9a-4b40-9dd4-c1731d2f0bd5@github.com>
 <ksDCvbooZzMk7h39UOfXAenQDRxziMeBQAMnXQCcAi0=.d3b965ed-d5a0-4962-9edb-46455c2e3ead@github.com>
Message-ID: <e132aBT0ClEbxEtuWDQVPBsGgElZ1OiSY5zN7ouE4Jg=.67f58bce-9204-4417-8a59-a4ea92a0949e@github.com>

On Tue, 22 Aug 2023 21:25:49 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> adding this
>
> Would this be more succinct and easier to read?
> 
> 
>   if (mode()->is_generational() && is_concurrent_old_mark_in_progress()) {
>      assert(_gc_state.is_set(MARKING), "Marking flag should be set");
>      mask = YOUNG_MARKING;
>   } else {
>      assert(_gc_state.is_clear(MARKING|OLD_MARKING|YOUNG_MARKING), "All marking flags should be clear");
>      mask = MARKING | YOUNG_MARKING;
>   }
>   set_gc_state_mask(mask, in_progress);
> 
> 
> where `is_set()` and `is_clear()` check if all flag bits in argument are either all set or all clear (which is different from their current sense). See comment below.

pursuing this, but not adding redundant call to is_generational() along the non-ASSERT control path.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1303460395

From kdnilsen at openjdk.org  Wed Aug 23 19:54:52 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 23 Aug 2023 19:54:52 GMT
Subject: RFR: JDK-8314777: GenShen: Alias young and old marking bits to
 legacy Shenandoah marking bit in gc state [v3]
In-Reply-To: <ksDCvbooZzMk7h39UOfXAenQDRxziMeBQAMnXQCcAi0=.d3b965ed-d5a0-4962-9edb-46455c2e3ead@github.com>
References: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
 <s9_RcuQoG1PgtyOF2Yk32mOP4xHtEA93puAVYckyjD0=.c4b50384-969c-412d-8651-a535b4a0ff66@github.com>
 <9G1I7FP6ZML4XZXuYfppWsTk8HncPOHMw3XkEZyVsX0=.597ee6e4-ac9a-4b40-9dd4-c1731d2f0bd5@github.com>
 <ksDCvbooZzMk7h39UOfXAenQDRxziMeBQAMnXQCcAi0=.d3b965ed-d5a0-4962-9edb-46455c2e3ead@github.com>
Message-ID: <U_8-aMZar_IKkQpkp1rf2sPAAA_ArftOy1nwj2OSinY=.5c2eaa78-b2bc-43d2-865e-266bfcdfe253@github.com>

On Tue, 22 Aug 2023 21:58:57 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> and adding this
>
> This could also be simplified along the lines of my previous suggestion above.
> 
> Also the "if and only if" for state at the beginning here can be simplified to (but continue reading below):
> 
> 
>    assert(has_forwarded_objects() == (_gc_state.is_set(UPDATEREFS | EVACUATION), "Error");
> 
> 
> provided `is_set()` and `is_unset()` (or ideally called `is_clear()`)  methods are defined symmetrically to test all of the bits passed in the mask argument; i.e. `is_set()` answers true iff they are _all_ set, and `is_unset()` returns true iff they are _all_ unset. This will simplify many of the asserts that need to test the state of multiple bits at once.
> 
> That change might want to go in separately as it would affect a bunch of existing code/tests. And, yes, it is possible that that extent of change would be considered unacceptable upstream (but I am guessing it should be fine if made on tip now as a clean-up/refactor).

I'm making this refinement also.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1303476102

From kdnilsen at openjdk.org  Wed Aug 23 20:43:19 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 23 Aug 2023 20:43:19 GMT
Subject: RFR: JDK-8314777: GenShen: Alias young and old marking bits to
 legacy Shenandoah marking bit in gc state [v4]
In-Reply-To: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
References: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
Message-ID: <dMdnmIkoTdGLwQDAtREzl0zGkAm2TRjYiFOssHvdiJY=.e0e45826-b134-4acb-9745-75050bda9bd2@github.com>

> This change sets the MARKING bit of _gc_state whenever either Young marking or Old marking is active.  With this change, barrier code more closely resembles the code in the original single-generation Shenandoah.  The performance impact is negligible.  The primary benefit is to simplify code reviews and clarify that the addition of generational mode to Shenandoah does not negatively impact performance of single-generation Shenandoah.

Kelvin Nilsen has updated the pull request incrementally with two additional commits since the last revision:

 - Fix up some assert syntax errors
 - Clean up assertions to make them more readable and maintainable

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/309/files
  - new: https://git.openjdk.org/shenandoah/pull/309/files/d3ecbc91..4faf7f67

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=309&range=03
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=309&range=02-03

  Stats: 40 lines in 5 files changed: 21 ins; 10 del; 9 mod
  Patch: https://git.openjdk.org/shenandoah/pull/309.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/309/head:pull/309

PR: https://git.openjdk.org/shenandoah/pull/309

From kdnilsen at openjdk.org  Wed Aug 23 20:54:02 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 23 Aug 2023 20:54:02 GMT
Subject: RFR: JDK-8314777: GenShen: Alias young and old marking bits to
 legacy Shenandoah marking bit in gc state [v4]
In-Reply-To: <e132aBT0ClEbxEtuWDQVPBsGgElZ1OiSY5zN7ouE4Jg=.67f58bce-9204-4417-8a59-a4ea92a0949e@github.com>
References: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
 <s9_RcuQoG1PgtyOF2Yk32mOP4xHtEA93puAVYckyjD0=.c4b50384-969c-412d-8651-a535b4a0ff66@github.com>
 <9G1I7FP6ZML4XZXuYfppWsTk8HncPOHMw3XkEZyVsX0=.597ee6e4-ac9a-4b40-9dd4-c1731d2f0bd5@github.com>
 <ksDCvbooZzMk7h39UOfXAenQDRxziMeBQAMnXQCcAi0=.d3b965ed-d5a0-4962-9edb-46455c2e3ead@github.com>
 <e132aBT0ClEbxEtuWDQVPBsGgElZ1OiSY5zN7ouE4Jg=.67f58bce-9204-4417-8a59-a4ea92a0949e@github.com>
Message-ID: <_-JON6Z7SV5sHMpx5e5Me95zOldy_2BgVJfWh2dy9Uo=.bfa5e4a3-24ba-4099-8673-7fe387713cd0@github.com>

On Wed, 23 Aug 2023 19:34:35 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Would this be more succinct and easier to read?
>> 
>> 
>>   if (mode()->is_generational() && is_concurrent_old_mark_in_progress()) {
>>      assert(_gc_state.is_set(MARKING), "Marking flag should be set");
>>      mask = YOUNG_MARKING;
>>   } else {
>>      assert(_gc_state.is_clear(MARKING|OLD_MARKING|YOUNG_MARKING), "All marking flags should be clear");
>>      mask = MARKING | YOUNG_MARKING;
>>   }
>>   set_gc_state_mask(mask, in_progress);
>> 
>> 
>> where `is_set()` and `is_clear()` check if all flag bits in argument are either all set or all clear (which is different from their current sense). See comment below.
>
> pursuing this, but not adding redundant call to is_generational() along the non-ASSERT control path.

Actually, this is too strong.  The suggested assert fails because we may already be doing OLD_MARKING when we start to do the YOUNG_MARKING.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/309#discussion_r1303528339

From kdnilsen at openjdk.org  Wed Aug 23 21:08:18 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 23 Aug 2023 21:08:18 GMT
Subject: RFR: JDK-8314777: GenShen: Alias young and old marking bits to
 legacy Shenandoah marking bit in gc state [v5]
In-Reply-To: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
References: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
Message-ID: <4YqbowYyEnN2fXg5-uBwNppAfQFN1vFr8dc1sbnYuKM=.3d6ec275-b824-43f0-b6e4-008619dd6004@github.com>

> This change sets the MARKING bit of _gc_state whenever either Young marking or Old marking is active.  With this change, barrier code more closely resembles the code in the original single-generation Shenandoah.  The performance impact is negligible.  The primary benefit is to simplify code reviews and clarify that the addition of generational mode to Shenandoah does not negatively impact performance of single-generation Shenandoah.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Remove over-zealous assertion

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/309/files
  - new: https://git.openjdk.org/shenandoah/pull/309/files/4faf7f67..cdae96c7

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=309&range=04
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=309&range=03-04

  Stats: 4 lines in 1 file changed: 0 ins; 4 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/309.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/309/head:pull/309

PR: https://git.openjdk.org/shenandoah/pull/309

From kdnilsen at openjdk.org  Thu Aug 24 21:25:38 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 24 Aug 2023 21:25:38 GMT
Subject: RFR: JDK-8314777: GenShen: Alias young and old marking bits to
 legacy Shenandoah marking bit in gc state [v6]
In-Reply-To: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
References: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
Message-ID: <AaQi5nvVoeBGK7w-pqJrRcoU8tjfGsFTKZGk5GTWOS4=.dc150bc2-afb1-4728-922c-7be8d3ddbc02@github.com>

> This change sets the MARKING bit of _gc_state whenever either Young marking or Old marking is active.  With this change, barrier code more closely resembles the code in the original single-generation Shenandoah.  The performance impact is negligible.  The primary benefit is to simplify code reviews and clarify that the addition of generational mode to Shenandoah does not negatively impact performance of single-generation Shenandoah.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Remove over-zealous assertion

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/309/files
  - new: https://git.openjdk.org/shenandoah/pull/309/files/cdae96c7..fe9a3655

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=309&range=05
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=309&range=04-05

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/309.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/309/head:pull/309

PR: https://git.openjdk.org/shenandoah/pull/309

From rkennke at openjdk.org  Fri Aug 25 15:00:44 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 25 Aug 2023 15:00:44 GMT
Subject: RFR: 8305896: Alternative full GC forwarding [v52]
In-Reply-To: <yFs73BPFgB89xCPY4k63X4VTz13N8ELJf1lGQlE6UpA=.85ee4f25-a4ee-41d0-b4c2-ed9ff2cb1bb1@github.com>
References: <yFs73BPFgB89xCPY4k63X4VTz13N8ELJf1lGQlE6UpA=.85ee4f25-a4ee-41d0-b4c2-ed9ff2cb1bb1@github.com>
Message-ID: <tfdi101e-Gix-uGMuPySQg-LAXQJVhO7KwCkeAhDasI=.5aff95cb-6982-4416-9674-a4d53a96bbcf@github.com>

> Currently, the full-GC modes of Serial, Shenandoah and G1 GCs are forwarding objects by over-writing the object header with the new object location. Unfortunately, for compact object headers ([JDK-8294992](https://bugs.openjdk.org/browse/JDK-8294992)) this would not work, because the crucial class information is also stored in the header, and we could no longer iterate over objects until the headers would be restored. Also, the preserved-headers tables would grow quite large.
> 
> I propose to use an alternative algorithm for full-GC (sliding-GC) forwarding that uses a special encoding so that the forwarding information fits into the lowest 32 bits of the header.
> 
> It exploits the insight that, with sliding GCs, objects from one region will only ever be forwarded to one of two possible target regions. For this to work, we need to divide the heap into equal-sized regions. This is already the case for Shenandoah and G1, and can easily be overlaid for Serial GC, by assuming either the whole heap as a single region (if it fits) or by using SpaceAlignment-sized virtual regions.
> 
> We also build and maintain a table that has N elements, where N is the number of regions. Each entry is two addresses, which are the start-address of the possible target regions for each source region.
> 
> With this, forwarding information would be encoded like this:
>  - Bits 0 and 1: same as before, we put in '11' to indicate that the object is forwarded.
>  - Bit 2: Used for 'fallback'-forwarding (see below)
>  - Bit 3: Selects the target region 0 or 1. Look up the base address in the table (see above)
>  - Bits 4..31 The number of heap words from the target base address
> 
> This works well for all sliding GCs in Serial, G1 and Shenandoah. The exception is in G1, there is a special mode called 'serial compaction' which acts as a last-last-ditch effort to squeeze more space out of the heap by re-forwarding the tails of the compaction chains. Unfortunately, this breaks the assumption of the sliding-forwarding-table. When that happens, we initialize a fallback table, which is a simple open hash-table, and set the Bit 2 in the forwarding to indicate that we shall look up the forwardee in the fallback-table.
> 
> All the table accesses can be done unsynchronized because:
> - Serial GC is single-threaded anyway
> - In G1 and Shenandoah, GC worker threads divide up the work such that each worker does disjoint sets of regions.
> - G1 serial compaction is single-threaded
> 
> The change introduces a new (experimental) flag -...

Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 119 commits:

 - Merge remote-tracking branch 'upstream/master' into JDK-8305896
 - Merge branch 'master' into JDK-8305896
 - Various cleanups
 - Some @shipilev comments from downstream review
 - Further templatize Serial GC's adjust_pointers()
 - Merge branch 'master' into JDK-8305896
 - Specialize full-GC loops to get UseAltGCForwarding flag check out of hot paths
 - Remove G1-only assert for fallback forwarding, and comment with explanation
 - Merge branch 'master' into JDK-8305896
 - Replace homegrown FallbackTable with a ResourceHashtable based impl
 - ... and 109 more: https://git.openjdk.org/jdk/compare/f2383b3c...bc155699

-------------

Changes: https://git.openjdk.org/jdk/pull/13582/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=13582&range=51
  Stats: 1201 lines in 39 files changed: 1077 ins; 10 del; 114 mod
  Patch: https://git.openjdk.org/jdk/pull/13582.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/13582/head:pull/13582

PR: https://git.openjdk.org/jdk/pull/13582

From kdnilsen at openjdk.org  Fri Aug 25 16:28:09 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 25 Aug 2023 16:28:09 GMT
Subject: RFR: 8314972: GenShen: promote_in_place needs to prepare remembered
 set before it enables old allocations within region
Message-ID: <YXrZciMUEs6p4RuDksC20sqZ2Kc0WMc7wDjE7V34Jik=.fb271a79-c288-4dc2-b259-0eb189db6891@github.com>

An error in the original implementation left us vulnerable to a race between a worker thread which is promoting a region in place and another worker or mutator thread who is allocating memory for old objects within the same region.

This PR prevents this race by requiring the worker thread that promotes a region in place to finish the preparation of the region's remembered set before it makes the region available to other threads for allocation of old-gen memory.

-------------

Commit messages:
 - Use top_before_promote() to prepare remembered set
 - Fix object registry before promoting young region in place

Changes: https://git.openjdk.org/shenandoah/pull/310/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=310&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8314972
  Stats: 59 lines in 1 file changed: 29 ins; 30 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/310.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/310/head:pull/310

PR: https://git.openjdk.org/shenandoah/pull/310

From kdnilsen at openjdk.org  Fri Aug 25 16:28:10 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 25 Aug 2023 16:28:10 GMT
Subject: RFR: 8314972: GenShen: promote_in_place needs to prepare
 remembered set before it enables old allocations within region
In-Reply-To: <YXrZciMUEs6p4RuDksC20sqZ2Kc0WMc7wDjE7V34Jik=.fb271a79-c288-4dc2-b259-0eb189db6891@github.com>
References: <YXrZciMUEs6p4RuDksC20sqZ2Kc0WMc7wDjE7V34Jik=.fb271a79-c288-4dc2-b259-0eb189db6891@github.com>
Message-ID: <d9J00duyhD6wPjQDKE73AT_n4UUcEveXpjWAT7eo5Mk=.83204e20-ddfe-4750-ab0b-d87792e95e64@github.com>

On Thu, 24 Aug 2023 21:44:29 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> An error in the original implementation left us vulnerable to a race between a worker thread which is promoting a region in place and another worker or mutator thread who is allocating memory for old objects within the same region.
> 
> This PR prevents this race by requiring the worker thread that promotes a region in place to finish the preparation of the region's remembered set before it makes the region available to other threads for allocation of old-gen memory.

I've got some debugging to complete on this PR.

Failed pre-submit tests are due to:

1. riscv64 not currently supported by GenShen
2. https://bugs.openjdk.org/browse/JDK-8311843

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/310#issuecomment-1692503254
PR Comment: https://git.openjdk.org/shenandoah/pull/310#issuecomment-1693492334

From ysr at openjdk.org  Fri Aug 25 17:19:55 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 25 Aug 2023 17:19:55 GMT
Subject: RFR: 8314972: GenShen: promote_in_place needs to prepare
 remembered set before it enables old allocations within region
In-Reply-To: <YXrZciMUEs6p4RuDksC20sqZ2Kc0WMc7wDjE7V34Jik=.fb271a79-c288-4dc2-b259-0eb189db6891@github.com>
References: <YXrZciMUEs6p4RuDksC20sqZ2Kc0WMc7wDjE7V34Jik=.fb271a79-c288-4dc2-b259-0eb189db6891@github.com>
Message-ID: <EeVkx9t-fdq9OErgpsKZZEJfLbinxXh_PC2PnqPx2co=.7e04fa97-c081-4579-bc92-bb764680a4bd@github.com>

On Thu, 24 Aug 2023 21:44:29 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> An error in the original implementation left us vulnerable to a race between a worker thread which is promoting a region in place and another worker or mutator thread who is allocating memory for old objects within the same region.
> 
> This PR prevents this race by requiring the worker thread that promotes a region in place to finish the preparation of the region's remembered set before it makes the region available to other threads for allocation of old-gen memory.

LGTM. ?

src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 1020:

> 1018:   assert(get_top_before_promote() == tams, "Cannot promote regions in place if top has advanced beyond TAMS");
> 1019: 
> 1020:   // Since this region may have served previously as OLD, it may hold obsolete object range info.

Why don't we clear the range when an old region is freed, rather than waiting for it to become old? Then we could assert here that the range is clear. Young regions wouldn't register objects so nothing would need to be done for young regions.

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/310#pullrequestreview-1596144705
PR Review Comment: https://git.openjdk.org/shenandoah/pull/310#discussion_r1305917208

From ysr at openjdk.org  Fri Aug 25 17:19:57 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 25 Aug 2023 17:19:57 GMT
Subject: RFR: 8314972: GenShen: promote_in_place needs to prepare
 remembered set before it enables old allocations within region
In-Reply-To: <EeVkx9t-fdq9OErgpsKZZEJfLbinxXh_PC2PnqPx2co=.7e04fa97-c081-4579-bc92-bb764680a4bd@github.com>
References: <YXrZciMUEs6p4RuDksC20sqZ2Kc0WMc7wDjE7V34Jik=.fb271a79-c288-4dc2-b259-0eb189db6891@github.com>
 <EeVkx9t-fdq9OErgpsKZZEJfLbinxXh_PC2PnqPx2co=.7e04fa97-c081-4579-bc92-bb764680a4bd@github.com>
Message-ID: <Ouuh4snz0mg8MeF4gEKw2N7fz9acZThMZeNdFvYU2RA=.90666eb7-f77a-4456-8a3e-c53479e7c89c@github.com>

On Fri, 25 Aug 2023 16:57:05 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> An error in the original implementation left us vulnerable to a race between a worker thread which is promoting a region in place and another worker or mutator thread who is allocating memory for old objects within the same region.
>> 
>> This PR prevents this race by requiring the worker thread that promotes a region in place to finish the preparation of the region's remembered set before it makes the region available to other threads for allocation of old-gen memory.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 1020:
> 
>> 1018:   assert(get_top_before_promote() == tams, "Cannot promote regions in place if top has advanced beyond TAMS");
>> 1019: 
>> 1020:   // Since this region may have served previously as OLD, it may hold obsolete object range info.
> 
> Why don't we clear the range when an old region is freed, rather than waiting for it to become old? Then we could assert here that the range is clear. Young regions wouldn't register objects so nothing would need to be done for young regions.

May be modify the comment to state something like:
// Clear any obsolete object start information; we'll register the live objects in the loop below,
// as well as registering and stamping free block fillers over dead objects.
// We also dirty the card table for the entire allocated range; it'll be updated/refined when we
// scan it next time for young pointers.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/310#discussion_r1305930519

From kdnilsen at openjdk.org  Fri Aug 25 17:30:47 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 25 Aug 2023 17:30:47 GMT
Subject: RFR: 8314972: GenShen: promote_in_place needs to prepare
 remembered set before it enables old allocations within region
In-Reply-To: <Ouuh4snz0mg8MeF4gEKw2N7fz9acZThMZeNdFvYU2RA=.90666eb7-f77a-4456-8a3e-c53479e7c89c@github.com>
References: <YXrZciMUEs6p4RuDksC20sqZ2Kc0WMc7wDjE7V34Jik=.fb271a79-c288-4dc2-b259-0eb189db6891@github.com>
 <EeVkx9t-fdq9OErgpsKZZEJfLbinxXh_PC2PnqPx2co=.7e04fa97-c081-4579-bc92-bb764680a4bd@github.com>
 <Ouuh4snz0mg8MeF4gEKw2N7fz9acZThMZeNdFvYU2RA=.90666eb7-f77a-4456-8a3e-c53479e7c89c@github.com>
Message-ID: <PH_YjHXypq9PnVlfKKFBPsDGNsC1R0QUk-bGdHg5wAk=.04fa8516-3768-4e9b-bf8a-8ef999c6883a@github.com>

On Fri, 25 Aug 2023 17:11:35 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 1020:
>> 
>>> 1018:   assert(get_top_before_promote() == tams, "Cannot promote regions in place if top has advanced beyond TAMS");
>>> 1019: 
>>> 1020:   // Since this region may have served previously as OLD, it may hold obsolete object range info.
>> 
>> Why don't we clear the range when an old region is freed, rather than waiting for it to become old? Then we could assert here that the range is clear. Young regions wouldn't register objects so nothing would need to be done for young regions.
>
> May be modify the comment to state something like:
> // Clear any obsolete object start information; we'll register the live objects in the loop below,
> // as well as registering and stamping free block fillers over dead objects.
> // We also dirty the card table for the entire allocated range; it'll be updated/refined when we
> // scan it next time for young pointers.

We don't clear object registrations when an old region is freed because often times, the free region may next be put into service as a young-region, where the object registrations are "don't care" values.  I'll make the adjustment to the comment.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/310#discussion_r1305947540

From ysr at openjdk.org  Fri Aug 25 17:30:47 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 25 Aug 2023 17:30:47 GMT
Subject: RFR: 8314972: GenShen: promote_in_place needs to prepare
 remembered set before it enables old allocations within region
In-Reply-To: <YXrZciMUEs6p4RuDksC20sqZ2Kc0WMc7wDjE7V34Jik=.fb271a79-c288-4dc2-b259-0eb189db6891@github.com>
References: <YXrZciMUEs6p4RuDksC20sqZ2Kc0WMc7wDjE7V34Jik=.fb271a79-c288-4dc2-b259-0eb189db6891@github.com>
Message-ID: <lRC2JwT2UYDxXOKEuwsfgB3jNkuVAE6PtZeIWtBSJBQ=.8d90174b-2b1d-4695-9c2d-8a2325e86871@github.com>

On Thu, 24 Aug 2023 21:44:29 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> An error in the original implementation left us vulnerable to a race between a worker thread which is promoting a region in place and another worker or mutator thread who is allocating memory for old objects within the same region.
> 
> This PR prevents this race by requiring the worker thread that promotes a region in place to finish the preparation of the region's remembered set before it makes the region available to other threads for allocation of old-gen memory.

src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 1040:

> 1038:       assert(fill_size >= ShenandoahHeap::min_fill_size(), "previously allocated objects known to be larger than min_size");
> 1039:       ShenandoahHeap::fill_with_object(obj_addr, fill_size);
> 1040:       heap->card_scan()->register_object_without_lock(obj_addr);

On second thoughts, do we really need to register the dead objects? Does it help at all, since the preceding and succeeding live objects will be registered anyway. Not wrong, but I do wonder if it helps at all or not. Probably not a big deal either way since dead objects are likely very few in regions promoted in place.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/310#discussion_r1305945666

From wkemper at openjdk.org  Fri Aug 25 17:52:53 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 25 Aug 2023 17:52:53 GMT
Subject: RFR: 8314972: GenShen: promote_in_place needs to prepare
 remembered set before it enables old allocations within region
In-Reply-To: <YXrZciMUEs6p4RuDksC20sqZ2Kc0WMc7wDjE7V34Jik=.fb271a79-c288-4dc2-b259-0eb189db6891@github.com>
References: <YXrZciMUEs6p4RuDksC20sqZ2Kc0WMc7wDjE7V34Jik=.fb271a79-c288-4dc2-b259-0eb189db6891@github.com>
Message-ID: <2uRfhU9DDSe4DzypwffSZ9KM9xg6onwetTURSXVPqYM=.97c159c1-986d-4897-b25d-dc70c255f173@github.com>

On Thu, 24 Aug 2023 21:44:29 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> An error in the original implementation left us vulnerable to a race between a worker thread which is promoting a region in place and another worker or mutator thread who is allocating memory for old objects within the same region.
> 
> This PR prevents this race by requiring the worker thread that promotes a region in place to finish the preparation of the region's remembered set before it makes the region available to other threads for allocation of old-gen memory.

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/310#pullrequestreview-1596229319

From wkemper at openjdk.org  Fri Aug 25 17:53:22 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 25 Aug 2023 17:53:22 GMT
Subject: RFR: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity"
Message-ID: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>

There are a few fixes here:

During a Full GC, the compaction process is not constrained by the maximum capacity for young and old generations. We must, therefore, reestablish these constraints after the Full GC completes by _forcing_ the transfer of regions to old, if necessary.

When a region is "flipped" from the Mutator's free set to the Collector's, we cannot _force_ the transfer of the region because the young generation may not have any free regions.

A logic error in the loop which rebuilds the free set could prevent transferring free regions from the mutator to the collector reserves.

-------------

Commit messages:
 - Do not prematurely exit when reserving regions
 - 8314610: hotspot can't compile with the latest of gtest because of <iomanip>
 - Force transfer of old regions to old generation capacity after full GC
 - Remove vestigial member and methods

Changes: https://git.openjdk.org/shenandoah/pull/311/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=311&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8311843
  Stats: 93 lines in 7 files changed: 27 ins; 46 del; 20 mod
  Patch: https://git.openjdk.org/shenandoah/pull/311.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/311/head:pull/311

PR: https://git.openjdk.org/shenandoah/pull/311

From wkemper at openjdk.org  Fri Aug 25 17:54:48 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 25 Aug 2023 17:54:48 GMT
Subject: RFR: JDK-8314777: GenShen: Alias young and old marking bits to
 legacy Shenandoah marking bit in gc state [v6]
In-Reply-To: <AaQi5nvVoeBGK7w-pqJrRcoU8tjfGsFTKZGk5GTWOS4=.dc150bc2-afb1-4728-922c-7be8d3ddbc02@github.com>
References: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
 <AaQi5nvVoeBGK7w-pqJrRcoU8tjfGsFTKZGk5GTWOS4=.dc150bc2-afb1-4728-922c-7be8d3ddbc02@github.com>
Message-ID: <D48GPdJrftp-SdIrlHfdhhduLNhsWzJLYZDu0I9Biac=.3aadfa59-290d-4401-9904-c04f59812e18@github.com>

On Thu, 24 Aug 2023 21:25:38 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> This change sets the MARKING bit of _gc_state whenever either Young marking or Old marking is active.  With this change, barrier code more closely resembles the code in the original single-generation Shenandoah.  The performance impact is negligible.  The primary benefit is to simplify code reviews and clarify that the addition of generational mode to Shenandoah does not negatively impact performance of single-generation Shenandoah.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Remove over-zealous assertion

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/309#pullrequestreview-1596231692

From kdnilsen at openjdk.org  Fri Aug 25 17:55:44 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 25 Aug 2023 17:55:44 GMT
Subject: RFR: 8314972: GenShen: promote_in_place needs to prepare
 remembered set before it enables old allocations within region
In-Reply-To: <lRC2JwT2UYDxXOKEuwsfgB3jNkuVAE6PtZeIWtBSJBQ=.8d90174b-2b1d-4695-9c2d-8a2325e86871@github.com>
References: <YXrZciMUEs6p4RuDksC20sqZ2Kc0WMc7wDjE7V34Jik=.fb271a79-c288-4dc2-b259-0eb189db6891@github.com>
 <lRC2JwT2UYDxXOKEuwsfgB3jNkuVAE6PtZeIWtBSJBQ=.8d90174b-2b1d-4695-9c2d-8a2325e86871@github.com>
Message-ID: <lwIJ2oBMx828yaA0qfpp7GCjFSCddzCVsg8PrpiP_VM=.97880417-950b-437f-97df-320a5fe006b0@github.com>

On Fri, 25 Aug 2023 17:26:28 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> An error in the original implementation left us vulnerable to a race between a worker thread which is promoting a region in place and another worker or mutator thread who is allocating memory for old objects within the same region.
>> 
>> This PR prevents this race by requiring the worker thread that promotes a region in place to finish the preparation of the region's remembered set before it makes the region available to other threads for allocation of old-gen memory.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 1040:
> 
>> 1038:       assert(fill_size >= ShenandoahHeap::min_fill_size(), "previously allocated objects known to be larger than min_size");
>> 1039:       ShenandoahHeap::fill_with_object(obj_addr, fill_size);
>> 1040:       heap->card_scan()->register_object_without_lock(obj_addr);
> 
> On second thoughts, do we really need to register the dead objects? Does it help at all, since the preceding and succeeding live objects will be registered anyway. Not wrong, but I do wonder if it helps at all or not. Probably not a big deal either way since dead objects are likely very few in regions promoted in place.

Existing code (and some existing assertions in ShenandoahVerifier) expect the filler objects to be registered.  You make a good point though.  We don't really need these to be registered.  If the first or only object in a card range is a filler, remembered set scanning can just ignore that.  The remembered set scanner wants to find the first non-filler object in the range.  We might want to address this in a separate ticket later, if we decide to keep the current remembered set representation longer term.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/310#discussion_r1305972752

From kdnilsen at openjdk.org  Fri Aug 25 18:02:42 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 25 Aug 2023 18:02:42 GMT
Subject: RFR: 8314972: GenShen: promote_in_place needs to prepare
 remembered set before it enables old allocations within region
In-Reply-To: <lwIJ2oBMx828yaA0qfpp7GCjFSCddzCVsg8PrpiP_VM=.97880417-950b-437f-97df-320a5fe006b0@github.com>
References: <YXrZciMUEs6p4RuDksC20sqZ2Kc0WMc7wDjE7V34Jik=.fb271a79-c288-4dc2-b259-0eb189db6891@github.com>
 <lRC2JwT2UYDxXOKEuwsfgB3jNkuVAE6PtZeIWtBSJBQ=.8d90174b-2b1d-4695-9c2d-8a2325e86871@github.com>
 <lwIJ2oBMx828yaA0qfpp7GCjFSCddzCVsg8PrpiP_VM=.97880417-950b-437f-97df-320a5fe006b0@github.com>
Message-ID: <wXBfty5vaD6XohILwXHHTqxagW2KYI_q5v8BNYRdz4A=.a8c368d5-445f-487b-994e-fd169ca04a14@github.com>

On Fri, 25 Aug 2023 17:53:06 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 1040:
>> 
>>> 1038:       assert(fill_size >= ShenandoahHeap::min_fill_size(), "previously allocated objects known to be larger than min_size");
>>> 1039:       ShenandoahHeap::fill_with_object(obj_addr, fill_size);
>>> 1040:       heap->card_scan()->register_object_without_lock(obj_addr);
>> 
>> On second thoughts, do we really need to register the dead objects? Does it help at all, since the preceding and succeeding live objects will be registered anyway. Not wrong, but I do wonder if it helps at all or not. Probably not a big deal either way since dead objects are likely very few in regions promoted in place.
>
> Existing code (and some existing assertions in ShenandoahVerifier) expect the filler objects to be registered.  You make a good point though.  We don't really need these to be registered.  If the first or only object in a card range is a filler, remembered set scanning can just ignore that.  The remembered set scanner wants to find the first non-filler object in the range.  We might want to address this in a separate ticket later, if we decide to keep the current remembered set representation longer term.

Note that the cost of registering filler objects is minimal because there are never multiple contiguous filler objects residing within memory.  We always coalesce them together.  At most, the cost during remembered set scanning is that we look at one extra filler object per DIRTY card.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/310#discussion_r1305982797

From kdnilsen at openjdk.org  Fri Aug 25 18:15:08 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 25 Aug 2023 18:15:08 GMT
Subject: RFR: 8314972: GenShen: promote_in_place needs to prepare
 remembered set before it enables old allocations within region [v2]
In-Reply-To: <YXrZciMUEs6p4RuDksC20sqZ2Kc0WMc7wDjE7V34Jik=.fb271a79-c288-4dc2-b259-0eb189db6891@github.com>
References: <YXrZciMUEs6p4RuDksC20sqZ2Kc0WMc7wDjE7V34Jik=.fb271a79-c288-4dc2-b259-0eb189db6891@github.com>
Message-ID: <VxSdAHPnFeOyCWJAWKZ3-NprVmtDpppsAXrermcVH2M=.97c43000-ac30-41bd-a76b-5421a34f93c0@github.com>

> An error in the original implementation left us vulnerable to a race between a worker thread which is promoting a region in place and another worker or mutator thread who is allocating memory for old objects within the same region.
> 
> This PR prevents this race by requiring the worker thread that promotes a region in place to finish the preparation of the region's remembered set before it makes the region available to other threads for allocation of old-gen memory.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Improve comments

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/310/files
  - new: https://git.openjdk.org/shenandoah/pull/310/files/96ca2bea..98469ffa

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=310&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=310&range=00-01

  Stats: 9 lines in 1 file changed: 6 ins; 1 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/310.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/310/head:pull/310

PR: https://git.openjdk.org/shenandoah/pull/310

From kdnilsen at openjdk.org  Fri Aug 25 18:28:44 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 25 Aug 2023 18:28:44 GMT
Subject: Integrated: 8314972: GenShen: promote_in_place needs to prepare
 remembered set before it enables old allocations within region
In-Reply-To: <YXrZciMUEs6p4RuDksC20sqZ2Kc0WMc7wDjE7V34Jik=.fb271a79-c288-4dc2-b259-0eb189db6891@github.com>
References: <YXrZciMUEs6p4RuDksC20sqZ2Kc0WMc7wDjE7V34Jik=.fb271a79-c288-4dc2-b259-0eb189db6891@github.com>
Message-ID: <FQuyghrMVu6B0dDh4vP-gYTXKtjeskwxllaQ24oMZjw=.e3d9129a-426a-4a5e-96af-27d45d58bff8@github.com>

On Thu, 24 Aug 2023 21:44:29 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> An error in the original implementation left us vulnerable to a race between a worker thread which is promoting a region in place and another worker or mutator thread who is allocating memory for old objects within the same region.
> 
> This PR prevents this race by requiring the worker thread that promotes a region in place to finish the preparation of the region's remembered set before it makes the region available to other threads for allocation of old-gen memory.

This pull request has now been integrated.

Changeset: 045ae688
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/045ae6885ce33c7b718defabfc3f0736cc58e281
Stats:     64 lines in 1 file changed: 34 ins; 30 del; 0 mod

8314972: GenShen: promote_in_place needs to prepare remembered set before it enables old allocations within region

Reviewed-by: ysr, wkemper

-------------

PR: https://git.openjdk.org/shenandoah/pull/310

From wkemper at openjdk.org  Fri Aug 25 22:55:31 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 25 Aug 2023 22:55:31 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <GDgOFpUMkjoyIR2MrLavdSxk359SMaA6yGZ3-GhGtyc=.059c436e-1836-4ee8-9f82-46daac7ccbae@github.com>

Merges tag jdk-22+12

-------------

Commit messages:
 - Use new ptrQueue API
 - Merge tag 'jdk-22+12' into merge-jdk-22-12
 - 8314734: Remove unused field TypeVariableImpl.EMPTY_ANNOTATION_ARRAY
 - 8308042: [macos] Developer ID Application Certificate not picked up by jpackage if it contains UNICODE characters
 - 8314679: SA fails to properly attach to JVM after having just detached from a different JVM
 - 8280743: HSDB "Monitor Cache Dump" command might throw NPE
 - 8175874: Update Security.insertProviderAt to specify behavior when requested position is out of range.
 - 8294535: Add screen capture functionality to PassFailJFrame
 - 8314746: Remove unused private put* methods from DirectByteBufferR
 - 8314810: (fs) java/nio/file/Files/CopyInterference.java should use TestUtil::supportsLinks
 - ... and 75 more: https://git.openjdk.org/shenandoah/compare/8607f8f4...da46497d

The webrevs contain the adjustments done while merging with regards to each parent branch:
 - master: https://webrevs.openjdk.org/?repo=shenandoah&pr=312&range=00.0
 - openjdk/jdk:master: https://webrevs.openjdk.org/?repo=shenandoah&pr=312&range=00.1

Changes: https://git.openjdk.org/shenandoah/pull/312/files
  Stats: 6127 lines in 348 files changed: 4249 ins; 734 del; 1144 mod
  Patch: https://git.openjdk.org/shenandoah/pull/312.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/312/head:pull/312

PR: https://git.openjdk.org/shenandoah/pull/312

From wkemper at openjdk.org  Fri Aug 25 23:04:55 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 25 Aug 2023 23:04:55 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <GDgOFpUMkjoyIR2MrLavdSxk359SMaA6yGZ3-GhGtyc=.059c436e-1836-4ee8-9f82-46daac7ccbae@github.com>
References: <GDgOFpUMkjoyIR2MrLavdSxk359SMaA6yGZ3-GhGtyc=.059c436e-1836-4ee8-9f82-46daac7ccbae@github.com>
Message-ID: <-md7GZLcpGCwKb7_uNClhzN_t805vI_6RpvcgGKFOT0=.0aed0b89-a8a6-46b8-bcf0-9f5950cdf220@github.com>

> Merges tag jdk-22+12

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 313 commits:

 - Use new ptrQueue API
 - Merge tag 'jdk-22+12' into merge-jdk-22-12
   
   Added tag jdk-22+12 for changeset 68815d54
 - Merge
 - 8311883: [Genshen] Adaptive tenuring threshold
   
   Generational Shenandoah currently has the notion of a tenuring threshold but it isn't dynamically adapted, but rather kept fixed at 7.
   
   We now adapt the tenuring threshold based on object demographics as determined by a recent GC that visits objects in the young generation.
   
   We keep track of age-cohort populations at each minor GC epoch and use the historical data to determine if it would be a good idea to tenure or not based on measured mortality rates. A few tunable (experimental) knobs are exposed to play with these to determine some good settings in the future.
   
   The object census is conducted by default at marking, and is subject to noise on account of objects whose age could not be determined because of displaced header. In this case, the computed tenuring threshold is used in the following evacuation of the same cycle. Optionally, the census can be conducted at evacuation time, but sees only objects that are in the collection set. In this case, the tenuring threshold that is computed is used for tenuring decisions in the next evacuation cycle.
   
   Other variants are possible, and may be implemented / tested in the future as opportunity and data permit.
   
   The computed tenuring threshold has not yet been coupled with size budgeting, but will be in a followup PR.
   
   At this time, performance measurements have not shown any benefit, but we believe that with the framework now in place, we may be able to find an adaptive tenuring algorithm that works better than the current one and provide performance benefits. Adaptive tenuring is enabled by default, but can be optionally disabled to mimic previous behavior.
   
   Reviewed-by: kdnilsen
 - 8309639: GenShen: Regression in LRU cache benchmark
   
   Reviewed-by: kdnilsen, ysr
 - 8312120: GenShen: Update OLD Live at end of Previous Marking after Full GC
   
   Reviewed-by: kdnilsen
 - 8313574: GenShen: Completing a global mark should also complete an old mark
   
   Reviewed-by: kdnilsen, ysr
 - Merge
 - Merge
 - 8312610: GenShen: Old generation available is unintentionally restricted by mutator's available memory
   
   Reviewed-by: kdnilsen, ysr
 - ... and 303 more: https://git.openjdk.org/shenandoah/compare/68815d54...da46497d

-------------

Changes: https://git.openjdk.org/shenandoah/pull/312/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=312&range=01
  Stats: 21412 lines in 216 files changed: 19522 ins; 909 del; 981 mod
  Patch: https://git.openjdk.org/shenandoah/pull/312.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/312/head:pull/312

PR: https://git.openjdk.org/shenandoah/pull/312

From wkemper at openjdk.org  Fri Aug 25 23:04:57 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 25 Aug 2023 23:04:57 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <GDgOFpUMkjoyIR2MrLavdSxk359SMaA6yGZ3-GhGtyc=.059c436e-1836-4ee8-9f82-46daac7ccbae@github.com>
References: <GDgOFpUMkjoyIR2MrLavdSxk359SMaA6yGZ3-GhGtyc=.059c436e-1836-4ee8-9f82-46daac7ccbae@github.com>
Message-ID: <QGIGbICP8XUFjhE1OSZWAOjDdsnV7euQ9YiJXBC_Stw=.8bc2f5cb-9959-41ab-9e3e-c0f06635f78a@github.com>

On Fri, 25 Aug 2023 22:47:36 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-22+12

This pull request has now been integrated.

Changeset: c31a1357
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/c31a135760945c5fd5c182c6c531001286554643
Stats:     6127 lines in 348 files changed: 4249 ins; 734 del; 1144 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/312

From wkemper at openjdk.org  Fri Aug 25 23:31:21 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 25 Aug 2023 23:31:21 GMT
Subject: RFR: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity" [v2]
In-Reply-To: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
References: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
Message-ID: <OhyEgq1lfNSl1XD5us3Wt3j7IyJNOsEz6LpBm9eNO3w=.344323e2-4329-4fd2-bcd9-07b015dc5763@github.com>

> There are a few fixes here:
> 
> During a Full GC, the compaction process is not constrained by the maximum capacity for young and old generations. We must, therefore, reestablish these constraints after the Full GC completes by _forcing_ the transfer of regions to old, if necessary.
> 
> When a region is "flipped" from the Mutator's free set to the Collector's, we cannot _force_ the transfer of the region because the young generation may not have any free regions.
> 
> A logic error in the loop which rebuilds the free set could prevent transferring free regions from the mutator to the collector reserves.

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:

 - Merge remote-tracking branch 'shenandoah/master' into full-gc-max-capacity
 - Do not prematurely exit when reserving regions
 - 8314610: hotspot can't compile with the latest of gtest because of <iomanip>
   
   Reviewed-by: jiefu, stuefe
 - Force transfer of old regions to old generation capacity after full GC
 - Remove vestigial member and methods

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/311/files
  - new: https://git.openjdk.org/shenandoah/pull/311/files/8ecfc06b..3af6309e

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=311&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=311&range=00-01

  Stats: 15309 lines in 643 files changed: 9721 ins; 1450 del; 4138 mod
  Patch: https://git.openjdk.org/shenandoah/pull/311.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/311/head:pull/311

PR: https://git.openjdk.org/shenandoah/pull/311

From kdnilsen at openjdk.org  Fri Aug 25 23:37:44 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 25 Aug 2023 23:37:44 GMT
Subject: RFR: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity" [v2]
In-Reply-To: <OhyEgq1lfNSl1XD5us3Wt3j7IyJNOsEz6LpBm9eNO3w=.344323e2-4329-4fd2-bcd9-07b015dc5763@github.com>
References: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
 <OhyEgq1lfNSl1XD5us3Wt3j7IyJNOsEz6LpBm9eNO3w=.344323e2-4329-4fd2-bcd9-07b015dc5763@github.com>
Message-ID: <sbClzhk2YT0CDoAOqNHqcezHA5VyALZuIbW-cc4Uxdk=.f25078e1-0b03-4df8-82ad-e69b1cf502de@github.com>

On Fri, 25 Aug 2023 23:31:21 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> There are a few fixes here:
>> 
>> During a Full GC, the compaction process is not constrained by the maximum capacity for young and old generations. We must, therefore, reestablish these constraints after the Full GC completes by _forcing_ the transfer of regions to old, if necessary.
>> 
>> When a region is "flipped" from the Mutator's free set to the Collector's, we cannot _force_ the transfer of the region because the young generation may not have any free regions.
>> 
>> A logic error in the loop which rebuilds the free set could prevent transferring free regions from the mutator to the collector reserves.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into full-gc-max-capacity
>  - Do not prematurely exit when reserving regions
>  - 8314610: hotspot can't compile with the latest of gtest because of <iomanip>
>    
>    Reviewed-by: jiefu, stuefe
>  - Force transfer of old regions to old generation capacity after full GC
>  - Remove vestigial member and methods

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1013:

> 1011:   _free_sets.move_to_set(idx, OldCollector, region_capacity);
> 1012:   _free_sets.assert_bounds();
> 1013: 

IIUC, the new code will fail to transfer if the transfer makes YOUNG smaller than min_capacity or makes OLD larger than max_capacity.  It will never fail because unaffiliated_regions < 1.

I have two concerns with the code as written:
1. I don't think we should flip the region if we don't flip the capacity.  That will create future assertion failures.
2. If we fail to flip_to_old(), we need to return a status code to so indicate.  The caller needs to know that the requested flip cannot be achieved and thus the precipitating "allocation request" must fail.

Alternatively, we could adopt the convention that min/max generation sizes are "preferences" but are not absolutely enforced.  Maybe it is better to bend the rules to allow pause-free operation rather than forcing an old evacuation failure that would require a Full GC.

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1228:

> 1226:     }
> 1227:   }
> 1228:   if (old_reserve > _free_sets.capacity_of(OldCollector)) {

Good catch here.  Sorry for the code as originally written.

src/hotspot/share/gc/shenandoah/shenandoahFullGC.cpp line 1539:

> 1537:         size_t old_regions_deficit = (old_usage - old_capacity) / ShenandoahHeapRegion::region_size_bytes();
> 1538:         heap->generation_sizer()->force_transfer_to_old(old_regions_deficit);
> 1539:       }

So here, we may end up violating max/min sizes of young generation.  I'm inclined to allow this if that is required to make the just-completed full gc valid.

src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 982:

> 980:   ShenandoahHeap::heap()->shenandoah_policy()->record_success_degenerated();
> 981: }
> 982: 

I gather that this code was redundant/not needed.  I'm not sure removal of this code was highlighted in the commit comment.  Maybe we should also mention some code cleanup regarding generation collection times.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/311#discussion_r1306206629
PR Review Comment: https://git.openjdk.org/shenandoah/pull/311#discussion_r1306210708
PR Review Comment: https://git.openjdk.org/shenandoah/pull/311#discussion_r1306214404
PR Review Comment: https://git.openjdk.org/shenandoah/pull/311#discussion_r1306215596

From kdnilsen at openjdk.org  Fri Aug 25 23:37:44 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 25 Aug 2023 23:37:44 GMT
Subject: RFR: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity" [v2]
In-Reply-To: <sbClzhk2YT0CDoAOqNHqcezHA5VyALZuIbW-cc4Uxdk=.f25078e1-0b03-4df8-82ad-e69b1cf502de@github.com>
References: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
 <OhyEgq1lfNSl1XD5us3Wt3j7IyJNOsEz6LpBm9eNO3w=.344323e2-4329-4fd2-bcd9-07b015dc5763@github.com>
 <sbClzhk2YT0CDoAOqNHqcezHA5VyALZuIbW-cc4Uxdk=.f25078e1-0b03-4df8-82ad-e69b1cf502de@github.com>
Message-ID: <79eeGPZ0O-ujnLgaw1MpFyTrpIJyI-Zggko9L12bv5Y=.a19cfe43-4e6e-476f-b3dc-c408da85d91d@github.com>

On Fri, 25 Aug 2023 23:08:12 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
>> 
>>  - Merge remote-tracking branch 'shenandoah/master' into full-gc-max-capacity
>>  - Do not prematurely exit when reserving regions
>>  - 8314610: hotspot can't compile with the latest of gtest because of <iomanip>
>>    
>>    Reviewed-by: jiefu, stuefe
>>  - Force transfer of old regions to old generation capacity after full GC
>>  - Remove vestigial member and methods
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1013:
> 
>> 1011:   _free_sets.move_to_set(idx, OldCollector, region_capacity);
>> 1012:   _free_sets.assert_bounds();
>> 1013: 
> 
> IIUC, the new code will fail to transfer if the transfer makes YOUNG smaller than min_capacity or makes OLD larger than max_capacity.  It will never fail because unaffiliated_regions < 1.
> 
> I have two concerns with the code as written:
> 1. I don't think we should flip the region if we don't flip the capacity.  That will create future assertion failures.
> 2. If we fail to flip_to_old(), we need to return a status code to so indicate.  The caller needs to know that the requested flip cannot be achieved and thus the precipitating "allocation request" must fail.
> 
> Alternatively, we could adopt the convention that min/max generation sizes are "preferences" but are not absolutely enforced.  Maybe it is better to bend the rules to allow pause-free operation rather than forcing an old evacuation failure that would require a Full GC.

If we adopt this convention, we should get rid of any asserts that endeavor to enforce max/min generation capacities  (or change these to enforce that capacity never shrinks below 0 and never grows above heap->capacity())

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/311#discussion_r1306212389

From wkemper at openjdk.org  Fri Aug 25 23:55:38 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 25 Aug 2023 23:55:38 GMT
Subject: RFR: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity" [v2]
In-Reply-To: <79eeGPZ0O-ujnLgaw1MpFyTrpIJyI-Zggko9L12bv5Y=.a19cfe43-4e6e-476f-b3dc-c408da85d91d@github.com>
References: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
 <OhyEgq1lfNSl1XD5us3Wt3j7IyJNOsEz6LpBm9eNO3w=.344323e2-4329-4fd2-bcd9-07b015dc5763@github.com>
 <sbClzhk2YT0CDoAOqNHqcezHA5VyALZuIbW-cc4Uxdk=.f25078e1-0b03-4df8-82ad-e69b1cf502de@github.com>
 <79eeGPZ0O-ujnLgaw1MpFyTrpIJyI-Zggko9L12bv5Y=.a19cfe43-4e6e-476f-b3dc-c408da85d91d@github.com>
Message-ID: <9fxTtSwE30Ok5oP-03fUjxAPvAnKIC4NTl4jz6XMZsc=.88ba5b7d-24aa-466e-b42b-7ffaa1ff7087@github.com>

On Fri, 25 Aug 2023 23:26:11 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1013:
>> 
>>> 1011:   _free_sets.move_to_set(idx, OldCollector, region_capacity);
>>> 1012:   _free_sets.assert_bounds();
>>> 1013: 
>> 
>> IIUC, the new code will fail to transfer if the transfer makes YOUNG smaller than min_capacity or makes OLD larger than max_capacity.  It will never fail because unaffiliated_regions < 1.
>> 
>> I have two concerns with the code as written:
>> 1. I don't think we should flip the region if we don't flip the capacity.  That will create future assertion failures.
>> 2. If we fail to flip_to_old(), we need to return a status code to so indicate.  The caller needs to know that the requested flip cannot be achieved and thus the precipitating "allocation request" must fail.
>> 
>> Alternatively, we could adopt the convention that min/max generation sizes are "preferences" but are not absolutely enforced.  Maybe it is better to bend the rules to allow pause-free operation rather than forcing an old evacuation failure that would require a Full GC.
>
> If we adopt this convention, we should get rid of any asserts that endeavor to enforce max/min generation capacities  (or change these to enforce that capacity never shrinks below 0 and never grows above heap->capacity())

`transfer_to_old` does check if young has enough affiliated regions to transfer:

  if (young_gen->free_unaffiliated_regions() < regions) {
    return false;


I think the original error here was caused by the Mutator free set holding many more regions than were available to the young generation (the issue with `reserve_regions`). I don't expect that to happen again, but I did add a log message if the transfer fails.

We are already somewhat loose when it comes to enforcing generational capacities. Are you proposing we have `shFreeSet` be the sole arbiter of capacity? That sounds like a good simplification - also sounds like a different PR?

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/311#discussion_r1306224321

From kdnilsen at openjdk.org  Sat Aug 26 00:03:40 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Sat, 26 Aug 2023 00:03:40 GMT
Subject: RFR: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity" [v2]
In-Reply-To: <9fxTtSwE30Ok5oP-03fUjxAPvAnKIC4NTl4jz6XMZsc=.88ba5b7d-24aa-466e-b42b-7ffaa1ff7087@github.com>
References: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
 <OhyEgq1lfNSl1XD5us3Wt3j7IyJNOsEz6LpBm9eNO3w=.344323e2-4329-4fd2-bcd9-07b015dc5763@github.com>
 <sbClzhk2YT0CDoAOqNHqcezHA5VyALZuIbW-cc4Uxdk=.f25078e1-0b03-4df8-82ad-e69b1cf502de@github.com>
 <79eeGPZ0O-ujnLgaw1MpFyTrpIJyI-Zggko9L12bv5Y=.a19cfe43-4e6e-476f-b3dc-c408da85d91d@github.com>
 <9fxTtSwE30Ok5oP-03fUjxAPvAnKIC4NTl4jz6XMZsc=.88ba5b7d-24aa-466e-b42b-7ffaa1ff7087@github.com>
Message-ID: <INMrC617J77StGT28mPv77DYSnV4vNOkJpSkCmCyDBw=.3286d296-4f01-40f7-8e45-06e6154f6603@github.com>

On Fri, 25 Aug 2023 23:52:59 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> If we adopt this convention, we should get rid of any asserts that endeavor to enforce max/min generation capacities  (or change these to enforce that capacity never shrinks below 0 and never grows above heap->capacity())
>
> `transfer_to_old` does check if young has enough affiliated regions to transfer:
> 
>   if (young_gen->free_unaffiliated_regions() < regions) {
>     return false;
> 
> 
> I think the original error here was caused by the Mutator free set holding many more regions than were available to the young generation (the issue with `reserve_regions`). I don't expect that to happen again, but I did add a log message if the transfer fails.
> 
> We are already somewhat loose when it comes to enforcing generational capacities. Are you proposing we have `shFreeSet` be the sole arbiter of capacity? That sounds like a good simplification - also sounds like a different PR?

I understand that it checks, but the check should never fail, because the two asserts that precede the call (and did not need to be removed) confirm that the region to be flipped is in the Mutator free set (i.e. is part of young capacity) and is entirely empty (so must be unaffiliated).

I need to study the reserve_regions problem more.  I saw that you refactored a bit.  I didn't catch that you were fixing a bug.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/311#discussion_r1306235144

From kdnilsen at openjdk.org  Sat Aug 26 00:03:40 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Sat, 26 Aug 2023 00:03:40 GMT
Subject: RFR: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity" [v2]
In-Reply-To: <INMrC617J77StGT28mPv77DYSnV4vNOkJpSkCmCyDBw=.3286d296-4f01-40f7-8e45-06e6154f6603@github.com>
References: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
 <OhyEgq1lfNSl1XD5us3Wt3j7IyJNOsEz6LpBm9eNO3w=.344323e2-4329-4fd2-bcd9-07b015dc5763@github.com>
 <sbClzhk2YT0CDoAOqNHqcezHA5VyALZuIbW-cc4Uxdk=.f25078e1-0b03-4df8-82ad-e69b1cf502de@github.com>
 <79eeGPZ0O-ujnLgaw1MpFyTrpIJyI-Zggko9L12bv5Y=.a19cfe43-4e6e-476f-b3dc-c408da85d91d@github.com>
 <9fxTtSwE30Ok5oP-03fUjxAPvAnKIC4NTl4jz6XMZsc=.88ba5b7d-24aa-466e-b42b-7ffaa1ff7087@github.com>
 <INMrC617J77StGT28mPv77DYSnV4vNOkJpSkCmCyDBw=.3286d296-4f01-40f7-8e45-06e6154f6603@github.com>
Message-ID: <Posf7wErDbTpQPpP43Z89mllD3FeQbcJwRTXc4PCsHQ=.732bacb1-3780-4e4f-bb04-06790b170b9b@github.com>

On Fri, 25 Aug 2023 23:58:25 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> `transfer_to_old` does check if young has enough affiliated regions to transfer:
>> 
>>   if (young_gen->free_unaffiliated_regions() < regions) {
>>     return false;
>> 
>> 
>> I think the original error here was caused by the Mutator free set holding many more regions than were available to the young generation (the issue with `reserve_regions`). I don't expect that to happen again, but I did add a log message if the transfer fails.
>> 
>> We are already somewhat loose when it comes to enforcing generational capacities. Are you proposing we have `shFreeSet` be the sole arbiter of capacity? That sounds like a good simplification - also sounds like a different PR?
>
> I understand that it checks, but the check should never fail, because the two asserts that precede the call (and did not need to be removed) confirm that the region to be flipped is in the Mutator free set (i.e. is part of young capacity) and is entirely empty (so must be unaffiliated).
> 
> I need to study the reserve_regions problem more.  I saw that you refactored a bit.  I didn't catch that you were fixing a bug.

Agree that changing shFreeSet to sole arbiter is probably a separate PR, that might be a good improvement.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/311#discussion_r1306235325

From wkemper at openjdk.org  Sat Aug 26 00:03:42 2023
From: wkemper at openjdk.org (William Kemper)
Date: Sat, 26 Aug 2023 00:03:42 GMT
Subject: RFR: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity" [v2]
In-Reply-To: <sbClzhk2YT0CDoAOqNHqcezHA5VyALZuIbW-cc4Uxdk=.f25078e1-0b03-4df8-82ad-e69b1cf502de@github.com>
References: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
 <OhyEgq1lfNSl1XD5us3Wt3j7IyJNOsEz6LpBm9eNO3w=.344323e2-4329-4fd2-bcd9-07b015dc5763@github.com>
 <sbClzhk2YT0CDoAOqNHqcezHA5VyALZuIbW-cc4Uxdk=.f25078e1-0b03-4df8-82ad-e69b1cf502de@github.com>
Message-ID: <Q1EAQ3nxxn40ThGH_woelmNibfW9sSU_p1Sv6Vg1WME=.f53acee3-89e1-413e-8bc7-54a4b96d0a9f@github.com>

On Fri, 25 Aug 2023 23:31:00 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
>> 
>>  - Merge remote-tracking branch 'shenandoah/master' into full-gc-max-capacity
>>  - Do not prematurely exit when reserving regions
>>  - 8314610: hotspot can't compile with the latest of gtest because of <iomanip>
>>    
>>    Reviewed-by: jiefu, stuefe
>>  - Force transfer of old regions to old generation capacity after full GC
>>  - Remove vestigial member and methods
>
> src/hotspot/share/gc/shenandoah/shenandoahFullGC.cpp line 1539:
> 
>> 1537:         size_t old_regions_deficit = (old_usage - old_capacity) / ShenandoahHeapRegion::region_size_bytes();
>> 1538:         heap->generation_sizer()->force_transfer_to_old(old_regions_deficit);
>> 1539:       }
> 
> So here, we may end up violating max/min sizes of young generation.  I'm inclined to allow this if that is required to make the just-completed full gc valid.

We may violate a configured constraint, but the usage accounting should be correct now.

> src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 982:
> 
>> 980:   ShenandoahHeap::heap()->shenandoah_policy()->record_success_degenerated();
>> 981: }
>> 982: 
> 
> I gather that this code was redundant/not needed.  I'm not sure removal of this code was highlighted in the commit comment.  Maybe we should also mention some code cleanup regarding generation collection times.

Added this cleanup to the PR description. Yes, this code was a vestige of the old mmu based resizer.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/311#discussion_r1306236442
PR Review Comment: https://git.openjdk.org/shenandoah/pull/311#discussion_r1306236688

From kdnilsen at openjdk.org  Sat Aug 26 00:15:37 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Sat, 26 Aug 2023 00:15:37 GMT
Subject: RFR: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity" [v2]
In-Reply-To: <Posf7wErDbTpQPpP43Z89mllD3FeQbcJwRTXc4PCsHQ=.732bacb1-3780-4e4f-bb04-06790b170b9b@github.com>
References: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
 <OhyEgq1lfNSl1XD5us3Wt3j7IyJNOsEz6LpBm9eNO3w=.344323e2-4329-4fd2-bcd9-07b015dc5763@github.com>
 <sbClzhk2YT0CDoAOqNHqcezHA5VyALZuIbW-cc4Uxdk=.f25078e1-0b03-4df8-82ad-e69b1cf502de@github.com>
 <79eeGPZ0O-ujnLgaw1MpFyTrpIJyI-Zggko9L12bv5Y=.a19cfe43-4e6e-476f-b3dc-c408da85d91d@github.com>
 <9fxTtSwE30Ok5oP-03fUjxAPvAnKIC4NTl4jz6XMZsc=.88ba5b7d-24aa-466e-b42b-7ffaa1ff7087@github.com>
 <INMrC617J77StGT28mPv77DYSnV4vNOkJpSkCmCyDBw=.3286d296-4f01-40f7-8e45-06e6154f6603@github.com>
 <Posf7wErDbTpQPpP43Z89mllD3FeQbcJwRTXc4PCsHQ=.732bacb1-3780-4e4f-bb04-06790b170b9b@github.com>
Message-ID: <nqeQGPNVtinu0RAICshVF6X0Zw78xexoEGEsBhCLt4c=.d7daa689-4cf5-4a96-8430-ac7ab505c7c8@github.com>

On Fri, 25 Aug 2023 23:59:03 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> I understand that it checks, but the check should never fail, because the two asserts that precede the call (and did not need to be removed) confirm that the region to be flipped is in the Mutator free set (i.e. is part of young capacity) and is entirely empty (so must be unaffiliated).
>> 
>> I need to study the reserve_regions problem more.  I saw that you refactored a bit.  I didn't catch that you were fixing a bug.
>
> Agree that changing shFreeSet to sole arbiter is probably a separate PR, that might be a good improvement.

Given that full-gc is allowed to violate the min/max constraints on generation sizes, I'm inclined to allow the same here, and revert this to force_transfer_to_old().  If that doesn't work, maybe a comment to explain why.  Because I'm not understanding this entirely.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/311#discussion_r1306239408

From kdnilsen at openjdk.org  Sat Aug 26 00:15:37 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Sat, 26 Aug 2023 00:15:37 GMT
Subject: RFR: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity" [v2]
In-Reply-To: <OhyEgq1lfNSl1XD5us3Wt3j7IyJNOsEz6LpBm9eNO3w=.344323e2-4329-4fd2-bcd9-07b015dc5763@github.com>
References: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
 <OhyEgq1lfNSl1XD5us3Wt3j7IyJNOsEz6LpBm9eNO3w=.344323e2-4329-4fd2-bcd9-07b015dc5763@github.com>
Message-ID: <wUAV2bH3St09ueegx4iEZhzDC2UYRl4bEQw38tNqDVw=.291d4eb8-1541-4c17-bfb8-795e03bad7ef@github.com>

On Fri, 25 Aug 2023 23:31:21 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> There are a few fixes here:
>> 
>> During a Full GC, the compaction process is not constrained by the maximum capacity for young and old generations. We must, therefore, reestablish these constraints after the Full GC completes by _forcing_ the transfer of regions to old, if necessary.
>> 
>> When a region is "flipped" from the Mutator's free set to the Collector's, we cannot _force_ the transfer of the region because the young generation may not have any free regions.
>> 
>> A logic error in the loop which rebuilds the free set could prevent transferring free regions from the mutator to the collector reserves.
>> 
>> Some unused methods and fields were removed.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into full-gc-max-capacity
>  - Do not prematurely exit when reserving regions
>  - 8314610: hotspot can't compile with the latest of gtest because of <iomanip>
>    
>    Reviewed-by: jiefu, stuefe
>  - Force transfer of old regions to old generation capacity after full GC
>  - Remove vestigial member and methods

Marked as reviewed by kdnilsen (Committer).

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1295:

> 1293:   }
> 1294: }
> 1295: 

When we rebuild, we compute target amounts of memory to_reserve_old and to_reserve_young.  We may not satisfy our targets if the Mutator.is_free memory is "too fragmented".  This would be rare, I think, but maybe deserving of a log_gc(info) message whenever we get to the end of rebuild and we are short.  I think the rest of the implementation is supposed to be robust to this scenario.  (i.e. if we did not reserve as much as we wanted to, then we will target a smaller collection set (in old or young)).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/311#pullrequestreview-1596629532
PR Review Comment: https://git.openjdk.org/shenandoah/pull/311#discussion_r1306238648

From kdnilsen at openjdk.org  Sat Aug 26 00:15:37 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Sat, 26 Aug 2023 00:15:37 GMT
Subject: RFR: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity" [v2]
In-Reply-To: <Q1EAQ3nxxn40ThGH_woelmNibfW9sSU_p1Sv6Vg1WME=.f53acee3-89e1-413e-8bc7-54a4b96d0a9f@github.com>
References: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
 <OhyEgq1lfNSl1XD5us3Wt3j7IyJNOsEz6LpBm9eNO3w=.344323e2-4329-4fd2-bcd9-07b015dc5763@github.com>
 <sbClzhk2YT0CDoAOqNHqcezHA5VyALZuIbW-cc4Uxdk=.f25078e1-0b03-4df8-82ad-e69b1cf502de@github.com>
 <Q1EAQ3nxxn40ThGH_woelmNibfW9sSU_p1Sv6Vg1WME=.f53acee3-89e1-413e-8bc7-54a4b96d0a9f@github.com>
Message-ID: <RhFJGvm3mxOl5qns4WVs5aH0ONnuGj3cUwwuTLMpOlE=.525c4f34-2506-4c5a-9be8-41aac67de0c4@github.com>

On Fri, 25 Aug 2023 23:59:51 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahFullGC.cpp line 1539:
>> 
>>> 1537:         size_t old_regions_deficit = (old_usage - old_capacity) / ShenandoahHeapRegion::region_size_bytes();
>>> 1538:         heap->generation_sizer()->force_transfer_to_old(old_regions_deficit);
>>> 1539:       }
>> 
>> So here, we may end up violating max/min sizes of young generation.  I'm inclined to allow this if that is required to make the just-completed full gc valid.
>
> We may violate a configured constraint, but the usage accounting should be correct now.

good.  and i assume that the configured constraint violation no longer manifests as a failed assertion.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/311#discussion_r1306238943

From kdnilsen at openjdk.org  Sat Aug 26 12:41:37 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Sat, 26 Aug 2023 12:41:37 GMT
Subject: RFR: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity" [v2]
In-Reply-To: <nqeQGPNVtinu0RAICshVF6X0Zw78xexoEGEsBhCLt4c=.d7daa689-4cf5-4a96-8430-ac7ab505c7c8@github.com>
References: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
 <OhyEgq1lfNSl1XD5us3Wt3j7IyJNOsEz6LpBm9eNO3w=.344323e2-4329-4fd2-bcd9-07b015dc5763@github.com>
 <sbClzhk2YT0CDoAOqNHqcezHA5VyALZuIbW-cc4Uxdk=.f25078e1-0b03-4df8-82ad-e69b1cf502de@github.com>
 <79eeGPZ0O-ujnLgaw1MpFyTrpIJyI-Zggko9L12bv5Y=.a19cfe43-4e6e-476f-b3dc-c408da85d91d@github.com>
 <9fxTtSwE30Ok5oP-03fUjxAPvAnKIC4NTl4jz6XMZsc=.88ba5b7d-24aa-466e-b42b-7ffaa1ff7087@github.com>
 <INMrC617J77StGT28mPv77DYSnV4vNOkJpSkCmCyDBw=.3286d296-4f01-40f7-8e45-06e6154f6603@github.com>
 <Posf7wErDbTpQPpP43Z89mllD3FeQbcJwRTXc4PCsHQ=.732bacb1-3780-4e4f-bb04-06790b170b9b@github.com>
 <nqeQGPNVtinu0RAICshVF6X0Zw78xexoEGEsBhCLt4c=.d7daa689-4cf5-4a96-8430-ac7ab505c7c8@github.com>
Message-ID: <vZeyYj2ZhF7qZxG-ql-1A8ota3gx1_XwyZu9mJENcc4=.0b222cf9-599e-4466-a2ba-2ecddbc80f4d@github.com>

On Sat, 26 Aug 2023 00:12:28 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Agree that changing shFreeSet to sole arbiter is probably a separate PR, that might be a good improvement.
>
> Given that full-gc is allowed to violate the min/max constraints on generation sizes, I'm inclined to allow the same here, and revert this to force_transfer_to_old().  If that doesn't work, maybe a comment to explain why.  Because I'm not understanding this entirely.

Specifically, it seems if we ever flip without adjusting capacity, we will end up violating assertions of the form old_gen->usage() <= old_gen->capacity().

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/311#discussion_r1306432510

From ysr at openjdk.org  Sun Aug 27 05:27:37 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sun, 27 Aug 2023 05:27:37 GMT
Subject: RFR: 8314972: GenShen: promote_in_place needs to prepare
 remembered set before it enables old allocations within region [v2]
In-Reply-To: <PH_YjHXypq9PnVlfKKFBPsDGNsC1R0QUk-bGdHg5wAk=.04fa8516-3768-4e9b-bf8a-8ef999c6883a@github.com>
References: <YXrZciMUEs6p4RuDksC20sqZ2Kc0WMc7wDjE7V34Jik=.fb271a79-c288-4dc2-b259-0eb189db6891@github.com>
 <EeVkx9t-fdq9OErgpsKZZEJfLbinxXh_PC2PnqPx2co=.7e04fa97-c081-4579-bc92-bb764680a4bd@github.com>
 <Ouuh4snz0mg8MeF4gEKw2N7fz9acZThMZeNdFvYU2RA=.90666eb7-f77a-4456-8a3e-c53479e7c89c@github.com>
 <PH_YjHXypq9PnVlfKKFBPsDGNsC1R0QUk-bGdHg5wAk=.04fa8516-3768-4e9b-bf8a-8ef999c6883a@github.com>
Message-ID: <4zZWm9tddpbotnIPsfoXul9TOt65OyG88pdZyJ031aU=.42f943bc-4443-4706-8459-8a2fe8d11cae@github.com>

On Fri, 25 Aug 2023 17:28:29 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> May be modify the comment to state something like:
>> // Clear any obsolete object start information; we'll register the live objects in the loop below,
>> // as well as registering and stamping free block fillers over dead objects.
>> // We also dirty the card table for the entire allocated range; it'll be updated/refined when we
>> // scan it next time for young pointers.
>
> We don't clear object registrations when an old region is freed because often times, the free region may next be put into service as a young-region, where the object registrations are "don't care" values.  I'll make the adjustment to the comment.

In any run, the number of clearings is basically the number old region births if cleared at promotion or allocation of an old generation, or deaths if cleared at freeing of an old generation. In a non-terminating run,  the rate of births equals rate of deaths. So the difference is just _when_ we clear, not how much. The amount of clearing work in the long run is no different in either case. So the difference is cache pollution, mutator latency, and code maintenance. I'm guessing there's really no difference, since it all happens on concurrent GC threads. So code maintenance would probably trump everything, and thinking some more about it, I am guessing there's not much difference in that either in when the clearing is done.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/310#discussion_r1306601922

From ysr at openjdk.org  Sun Aug 27 05:32:38 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Sun, 27 Aug 2023 05:32:38 GMT
Subject: RFR: 8314972: GenShen: promote_in_place needs to prepare
 remembered set before it enables old allocations within region [v2]
In-Reply-To: <wXBfty5vaD6XohILwXHHTqxagW2KYI_q5v8BNYRdz4A=.a8c368d5-445f-487b-994e-fd169ca04a14@github.com>
References: <YXrZciMUEs6p4RuDksC20sqZ2Kc0WMc7wDjE7V34Jik=.fb271a79-c288-4dc2-b259-0eb189db6891@github.com>
 <lRC2JwT2UYDxXOKEuwsfgB3jNkuVAE6PtZeIWtBSJBQ=.8d90174b-2b1d-4695-9c2d-8a2325e86871@github.com>
 <lwIJ2oBMx828yaA0qfpp7GCjFSCddzCVsg8PrpiP_VM=.97880417-950b-437f-97df-320a5fe006b0@github.com>
 <wXBfty5vaD6XohILwXHHTqxagW2KYI_q5v8BNYRdz4A=.a8c368d5-445f-487b-994e-fd169ca04a14@github.com>
Message-ID: <h4vZdY7-Ph-Df_SgUUBmAkGBkQQhbqN8oT96vTjvxhg=.43c89272-73e8-40f9-8d6f-201c4ce5956c@github.com>

On Fri, 25 Aug 2023 17:59:52 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Existing code (and some existing assertions in ShenandoahVerifier) expect the filler objects to be registered.  You make a good point though.  We don't really need these to be registered.  If the first or only object in a card range is a filler, remembered set scanning can just ignore that.  The remembered set scanner wants to find the first non-filler object in the range.  We might want to address this in a separate ticket later, if we decide to keep the current remembered set representation longer term.
>
> Note that the cost of registering filler objects is minimal because there are never multiple contiguous filler objects residing within memory.  We always coalesce them together.  At most, the cost during remembered set scanning is that we look at one extra filler object per DIRTY card.

I agree, I don't think it helps one way or other. In fact, the current card scanning code looks for the last object on the _previous_ card from which to start walking objects to scan. In the case that the filler object is entirely on a card, this does not make any difference. However, in the case that the filler object is straddling a card, registering it shortens the walk by one object, the one preceding the filler. I think any performance difference is a wash, and I'd just scratch my thought above.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/310#discussion_r1306602225

From lkorinth at openjdk.org  Mon Aug 28 16:01:53 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Mon, 28 Aug 2023 16:01:53 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder
Message-ID: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>

Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.

I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`

Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:

   /**
     * Create ProcessBuilder using the java launcher from the jdk to
     * be tested.
     *
     * <p><b> Please observe that you likely should use
     * createTestJvm() instead of this method because createTestJvm()
     * will add JVM options from "test.vm.opts" and "test.java.opts"
     * </b> and this method will not do that.
     *
     * @param command Arguments to pass to the java command.
     * @return The ProcessBuilder instance representing the java command.
     */


I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...

I have run tier 1 testing, and I have started more exhaustive testing.

-------------

Commit messages:
 - 8315097: Rename createJavaProcessBuilder

Changes: https://git.openjdk.org/jdk/pull/15452/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=15452&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8315097
  Stats: 756 lines in 462 files changed: 22 ins; 10 del; 724 mod
  Patch: https://git.openjdk.org/jdk/pull/15452.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15452/head:pull/15452

PR: https://git.openjdk.org/jdk/pull/15452

From wkemper at openjdk.org  Mon Aug 28 16:40:01 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 28 Aug 2023 16:40:01 GMT
Subject: RFR: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity" [v2]
In-Reply-To: <wUAV2bH3St09ueegx4iEZhzDC2UYRl4bEQw38tNqDVw=.291d4eb8-1541-4c17-bfb8-795e03bad7ef@github.com>
References: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
 <OhyEgq1lfNSl1XD5us3Wt3j7IyJNOsEz6LpBm9eNO3w=.344323e2-4329-4fd2-bcd9-07b015dc5763@github.com>
 <wUAV2bH3St09ueegx4iEZhzDC2UYRl4bEQw38tNqDVw=.291d4eb8-1541-4c17-bfb8-795e03bad7ef@github.com>
Message-ID: <K5F7Ws0q9Ykbn1ur4yddTA0l1cWQAq6ERFD9IOUkyoU=.db345923-f63d-49df-8226-fb54c48a1c8b@github.com>

On Sat, 26 Aug 2023 00:09:19 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
>> 
>>  - Merge remote-tracking branch 'shenandoah/master' into full-gc-max-capacity
>>  - Do not prematurely exit when reserving regions
>>  - 8314610: hotspot can't compile with the latest of gtest because of <iomanip>
>>    
>>    Reviewed-by: jiefu, stuefe
>>  - Force transfer of old regions to old generation capacity after full GC
>>  - Remove vestigial member and methods
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 1295:
> 
>> 1293:   }
>> 1294: }
>> 1295: 
> 
> When we rebuild, we compute target amounts of memory to_reserve_old and to_reserve_young.  We may not satisfy our targets if the Mutator.is_free memory is "too fragmented".  This would be rare, I think, but maybe deserving of a log_gc(info) message whenever we get to the end of rebuild and we are short.  I think the rest of the implementation is supposed to be robust to this scenario.  (i.e. if we did not reserve as much as we wanted to, then we will target a smaller collection set (in old or young)).

I'll have it log when it can't satisfy the collector's reserves.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/311#discussion_r1307663073

From wkemper at openjdk.org  Mon Aug 28 16:44:54 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 28 Aug 2023 16:44:54 GMT
Subject: RFR: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity" [v2]
In-Reply-To: <OhyEgq1lfNSl1XD5us3Wt3j7IyJNOsEz6LpBm9eNO3w=.344323e2-4329-4fd2-bcd9-07b015dc5763@github.com>
References: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
 <OhyEgq1lfNSl1XD5us3Wt3j7IyJNOsEz6LpBm9eNO3w=.344323e2-4329-4fd2-bcd9-07b015dc5763@github.com>
Message-ID: <f3UMIFJtigRw4OgRA8baOBbGCL8wS7bt2pESgscsCKk=.bc33182d-2e5f-4801-93e4-b136a1bc50ac@github.com>

On Fri, 25 Aug 2023 23:31:21 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> There are a few fixes here:
>> 
>> During a Full GC, the compaction process is not constrained by the maximum capacity for young and old generations. We must, therefore, reestablish these constraints after the Full GC completes by _forcing_ the transfer of regions to old, if necessary.
>> 
>> When a region is "flipped" from the Mutator's free set to the Collector's, we cannot _force_ the transfer of the region because the young generation may not have any free regions.
>> 
>> A logic error in the loop which rebuilds the free set could prevent transferring free regions from the mutator to the collector reserves.
>> 
>> Some unused methods and fields were removed.
>
> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision:
> 
>  - Merge remote-tracking branch 'shenandoah/master' into full-gc-max-capacity
>  - Do not prematurely exit when reserving regions
>  - 8314610: hotspot can't compile with the latest of gtest because of <iomanip>
>    
>    Reviewed-by: jiefu, stuefe
>  - Force transfer of old regions to old generation capacity after full GC
>  - Remove vestigial member and methods

I'll try letting `ShenandoahFreeSet::flip_to_old_gc` force transferring the region to old. I think it only caused problems because the free regions in the mutator set had not been properly distributed when the free sets were `rebuilt` (that is, it tried to force a young generation with no unaffiliated regions to give one up).

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/311#issuecomment-1696010692

From wkemper at openjdk.org  Mon Aug 28 16:52:01 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 28 Aug 2023 16:52:01 GMT
Subject: RFR: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity" [v2]
In-Reply-To: <vZeyYj2ZhF7qZxG-ql-1A8ota3gx1_XwyZu9mJENcc4=.0b222cf9-599e-4466-a2ba-2ecddbc80f4d@github.com>
References: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
 <OhyEgq1lfNSl1XD5us3Wt3j7IyJNOsEz6LpBm9eNO3w=.344323e2-4329-4fd2-bcd9-07b015dc5763@github.com>
 <sbClzhk2YT0CDoAOqNHqcezHA5VyALZuIbW-cc4Uxdk=.f25078e1-0b03-4df8-82ad-e69b1cf502de@github.com>
 <79eeGPZ0O-ujnLgaw1MpFyTrpIJyI-Zggko9L12bv5Y=.a19cfe43-4e6e-476f-b3dc-c408da85d91d@github.com>
 <9fxTtSwE30Ok5oP-03fUjxAPvAnKIC4NTl4jz6XMZsc=.88ba5b7d-24aa-466e-b42b-7ffaa1ff7087@github.com>
 <INMrC617J77StGT28mPv77DYSnV4vNOkJpSkCmCyDBw=.3286d296-4f01-40f7-8e45-06e6154f6603@github.com>
 <Posf7wErDbTpQPpP43Z89mllD3FeQbcJwRTXc4PCsHQ=.732bacb1-3780-4e4f-bb04-06790b170b9b@github.com>
 <nqeQGPNVtinu0RAICshVF6X0Zw78xexoEGEsBhCLt4c=.d7daa689-4cf5-4a96-8430-ac7ab505c7c8@github.com>
 <vZeyYj2ZhF7qZxG-ql-1A8ota3gx1_XwyZu9mJENcc4=.0b222cf9-599e-4466-a2ba-2ecddbc80f4d@github.com>
Message-ID: <KBIdDocY_Zu4KMtKHdhPDob17eLS-_9OYlUfgP7yDoE=.d526dc9b-35cd-49f7-af3d-4fc03acd283c@github.com>

On Sat, 26 Aug 2023 12:38:53 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Given that full-gc is allowed to violate the min/max constraints on generation sizes, I'm inclined to allow the same here, and revert this to force_transfer_to_old().  If that doesn't work, maybe a comment to explain why.  Because I'm not understanding this entirely.
>
> Specifically, it seems if we ever flip without adjusting capacity, we will end up violating assertions of the form old_gen->usage() <= old_gen->capacity().

Yes, there was a logic error in `rebuild` that caused the loop to exit well before any regions were transferred to the collector's reserves. This left it in a state where `capacity_of(Mutator)` was well higher than it ought to have been (i.e., when `young->available() == 0`).

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/311#discussion_r1307674739

From wkemper at openjdk.org  Mon Aug 28 16:59:26 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 28 Aug 2023 16:59:26 GMT
Subject: RFR: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity" [v3]
In-Reply-To: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
References: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
Message-ID: <QSec95zjgDEmQJ9Ckl12mNjjVpHO1GNCt-MJpPoZJo0=.71c32191-3f23-4ab0-aac1-0cdac3602984@github.com>

> There are a few fixes here:
> 
> During a Full GC, the compaction process is not constrained by the maximum capacity for young and old generations. We must, therefore, reestablish these constraints after the Full GC completes by _forcing_ the transfer of regions to old, if necessary.
> 
> When a region is "flipped" from the Mutator's free set to the Collector's, we cannot _force_ the transfer of the region because the young generation may not have any free regions.
> 
> A logic error in the loop which rebuilds the free set could prevent transferring free regions from the mutator to the collector reserves.
> 
> Some unused methods and fields were removed.

William Kemper has updated the pull request incrementally with two additional commits since the last revision:

 - Force transfer of flipped region after logging a warning
 - Log when collector reserves cannot be satisfied.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/311/files
  - new: https://git.openjdk.org/shenandoah/pull/311/files/3af6309e..2b3380d9

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=311&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=311&range=01-02

  Stats: 16 lines in 1 file changed: 14 ins; 1 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/311.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/311/head:pull/311

PR: https://git.openjdk.org/shenandoah/pull/311

From lmesnik at openjdk.org  Mon Aug 28 19:06:11 2023
From: lmesnik at openjdk.org (Leonid Mesnik)
Date: Mon, 28 Aug 2023 19:06:11 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder
In-Reply-To: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
Message-ID: <kTqMYPi97NeiCYDUN5NRbc7Z3eUlU8h4KxhgxYmRlTA=.06936fb1-58f7-4779-9814-55872d529630@github.com>

On Mon, 28 Aug 2023 15:54:08 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
> 
> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
> 
> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
> 
>    /**
>      * Create ProcessBuilder using the java launcher from the jdk to
>      * be tested.
>      *
>      * <p><b> Please observe that you likely should use
>      * createTestJvm() instead of this method because createTestJvm()
>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>      * </b> and this method will not do that.
>      *
>      * @param command Arguments to pass to the java command.
>      * @return The ProcessBuilder instance representing the java command.
>      */
> 
> 
> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
> 
> I have run tier 1 testing, and I have started more exhaustive testing.

The. changes looks good. Please update copyrights for changed filed.
I expect that you completed execution of all tests to ensure that nothing is broken.

-------------

Marked as reviewed by lmesnik (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/15452#pullrequestreview-1598884378

From wkemper at openjdk.org  Mon Aug 28 20:58:30 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 28 Aug 2023 20:58:30 GMT
Subject: RFR: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity" [v4]
In-Reply-To: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
References: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
Message-ID: <pjBJGougwpZOUdjX4D376KNs0fBnoXq4sx_p5DQRXjQ=.b59a7695-f468-46a8-bc57-6be67cb8d3c9@github.com>

> There are a few fixes here:
> 
> During a Full GC, the compaction process is not constrained by the maximum capacity for young and old generations. We must, therefore, reestablish these constraints after the Full GC completes by _forcing_ the transfer of regions to old, if necessary.
> 
> When a region is "flipped" from the Mutator's free set to the Collector's, we cannot _force_ the transfer of the region because the young generation may not have any free regions.
> 
> A logic error in the loop which rebuilds the free set could prevent transferring free regions from the mutator to the collector reserves.
> 
> Some unused methods and fields were removed.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Improve assertion

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/311/files
  - new: https://git.openjdk.org/shenandoah/pull/311/files/2b3380d9..8048c611

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=311&range=03
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=311&range=02-03

  Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/311.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/311/head:pull/311

PR: https://git.openjdk.org/shenandoah/pull/311

From kdnilsen at openjdk.org  Mon Aug 28 23:33:00 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 28 Aug 2023 23:33:00 GMT
Subject: git: openjdk/shenandoah: master: 8314777: GenShen: Alias young and old
 marking bits to legacy Shenandoah marking bit in gc state
Message-ID: <de504090-a510-44ca-a2b4-637e7e94d6aa@openjdk.org>

Changeset: 8d0b9009
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
Date:      2023-08-28 23:31:24 +0000
URL:       https://git.openjdk.org/shenandoah/commit/8d0b9009044d3b7478a6ca76fca9a123e1d28e40

8314777: GenShen: Alias young and old marking bits to legacy Shenandoah marking bit in gc state

Reviewed-by: wkemper, ysr

! src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp
! src/hotspot/cpu/ppc/gc/shenandoah/shenandoahBarrierSetAssembler_ppc.cpp
! src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp
! src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.cpp
! src/hotspot/share/gc/shenandoah/c2/shenandoahSupport.cpp
! src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp
! src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp
! src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp
! src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp
! src/hotspot/share/gc/shenandoah/shenandoahSharedVariables.hpp
! src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp


From kdnilsen at openjdk.org  Mon Aug 28 23:34:48 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 28 Aug 2023 23:34:48 GMT
Subject: Integrated: JDK-8314777: GenShen: Alias young and old marking bits to
 legacy Shenandoah marking bit in gc state
In-Reply-To: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
References: <6d6DG_Dhp1kD2FbyIRfWjM8rcZRlp7XE5kGKYNwC1lI=.e146113f-0787-4ba1-bdb3-560d18e5ba59@github.com>
Message-ID: <b5U-rmwXghePvii87_kQUCVEENX16RR-8sPaXA-I_o4=.09bd6990-6427-4646-8daa-55e6a33d97d8@github.com>

On Tue, 22 Aug 2023 00:10:42 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This change sets the MARKING bit of _gc_state whenever either Young marking or Old marking is active.  With this change, barrier code more closely resembles the code in the original single-generation Shenandoah.  The performance impact is negligible.  The primary benefit is to simplify code reviews and clarify that the addition of generational mode to Shenandoah does not negatively impact performance of single-generation Shenandoah.

This pull request has now been integrated.

Changeset: 8d0b9009
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/8d0b9009044d3b7478a6ca76fca9a123e1d28e40
Stats:     79 lines in 11 files changed: 40 ins; 8 del; 31 mod

8314777: GenShen: Alias young and old marking bits to legacy Shenandoah marking bit in gc state

Reviewed-by: wkemper, ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/309

From dholmes at openjdk.org  Tue Aug 29 05:06:08 2023
From: dholmes at openjdk.org (David Holmes)
Date: Tue, 29 Aug 2023 05:06:08 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder
In-Reply-To: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
Message-ID: <na05dDa2ZtCK2sfdXwnvzPgInmAMd9J9qUwSGVjhw_Q=.e63bc687-31af-47ac-ac0a-1a874693fff5@github.com>

On Mon, 28 Aug 2023 15:54:08 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
> 
> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
> 
> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
> 
>    /**
>      * Create ProcessBuilder using the java launcher from the jdk to
>      * be tested.
>      *
>      * <p><b> Please observe that you likely should use
>      * createTestJvm() instead of this method because createTestJvm()
>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>      * </b> and this method will not do that.
>      *
>      * @param command Arguments to pass to the java command.
>      * @return The ProcessBuilder instance representing the java command.
>      */
> 
> 
> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
> 
> I have run tier 1 testing, and I have started more exhaustive testing.

I think the admonition about using this method is a bit strong in that it is natural to use this plain process builder method when a test is going to set its own specific flags for the exec'd process. But I'm okay with renaming to avoid copy'n'paste errors that accidentally use the wrong version.

Thanks

-------------

Marked as reviewed by dholmes (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/15452#pullrequestreview-1599483514

From lkorinth at openjdk.org  Tue Aug 29 09:12:51 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Tue, 29 Aug 2023 09:12:51 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v2]
In-Reply-To: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
Message-ID: <ISuejqIr1InBH-hXEzgOt7iBXrhH4iv93glC3woOewQ=.3a7357d0-a911-4dfa-834b-2c5123915ce7@github.com>

> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
> 
> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
> 
> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
> 
>    /**
>      * Create ProcessBuilder using the java launcher from the jdk to
>      * be tested.
>      *
>      * <p><b> Please observe that you likely should use
>      * createTestJvm() instead of this method because createTestJvm()
>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>      * </b> and this method will not do that.
>      *
>      * @param command Arguments to pass to the java command.
>      * @return The ProcessBuilder instance representing the java command.
>      */
> 
> 
> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
> 
> I have run tier 1 testing, and I have started more exhaustive testing.

Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:

  copyright

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15452/files
  - new: https://git.openjdk.org/jdk/pull/15452/files/4b2d1711..f3418c80

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15452&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15452&range=00-01

  Stats: 344 lines in 344 files changed: 0 ins; 0 del; 344 mod
  Patch: https://git.openjdk.org/jdk/pull/15452.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15452/head:pull/15452

PR: https://git.openjdk.org/jdk/pull/15452

From rriggs at openjdk.org  Tue Aug 29 14:09:12 2023
From: rriggs at openjdk.org (Roger Riggs)
Date: Tue, 29 Aug 2023 14:09:12 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v2]
In-Reply-To: <ISuejqIr1InBH-hXEzgOt7iBXrhH4iv93glC3woOewQ=.3a7357d0-a911-4dfa-834b-2c5123915ce7@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <ISuejqIr1InBH-hXEzgOt7iBXrhH4iv93glC3woOewQ=.3a7357d0-a911-4dfa-834b-2c5123915ce7@github.com>
Message-ID: <pnAnjv8HLt1sykmB9jRamtiVRoTbmwQi9xGuaSnCi10=.34427fd4-7b88-436e-946b-04c7364a1853@github.com>

On Tue, 29 Aug 2023 09:12:51 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
>> 
>> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
>> 
>> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
>> 
>>    /**
>>      * Create ProcessBuilder using the java launcher from the jdk to
>>      * be tested.
>>      *
>>      * <p><b> Please observe that you likely should use
>>      * createTestJvm() instead of this method because createTestJvm()
>>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>>      * </b> and this method will not do that.
>>      *
>>      * @param command Arguments to pass to the java command.
>>      * @return The ProcessBuilder instance representing the java command.
>>      */
>> 
>> 
>> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
>> 
>> I have run tier 1 testing, and I have started more exhaustive testing.
>
> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
> 
>   copyright

I don't think this is the best change across so many files.
It gives a very ugly name to a common test function and affects a very large number of tests.

-------------

Changes requested by rriggs (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/15452#pullrequestreview-1600512718

From lkorinth at openjdk.org  Tue Aug 29 15:13:11 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Tue, 29 Aug 2023 15:13:11 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v2]
In-Reply-To: <pnAnjv8HLt1sykmB9jRamtiVRoTbmwQi9xGuaSnCi10=.34427fd4-7b88-436e-946b-04c7364a1853@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <ISuejqIr1InBH-hXEzgOt7iBXrhH4iv93glC3woOewQ=.3a7357d0-a911-4dfa-834b-2c5123915ce7@github.com>
 <pnAnjv8HLt1sykmB9jRamtiVRoTbmwQi9xGuaSnCi10=.34427fd4-7b88-436e-946b-04c7364a1853@github.com>
Message-ID: <HCFLeG3EfMrzIj9IjcMM4NeSOpSu6cC3Nr4Ws85M4lU=.00c1992b-c510-4349-a998-24a06b2b31a5@github.com>

On Tue, 29 Aug 2023 14:06:01 GMT, Roger Riggs <rriggs at openjdk.org> wrote:

>> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   copyright
>
> I don't think this is the best change across so many files.
> It gives a very ugly name to a common test function and affects a very large number of tests.

@RogerRiggs If it is only the name you want changed, maybe you can offer a better name?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1697634872

From wkemper at openjdk.org  Tue Aug 29 16:18:24 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 29 Aug 2023 16:18:24 GMT
Subject: RFR: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity" [v5]
In-Reply-To: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
References: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
Message-ID: <-Ga63qBr8HY4vwatXJt8eVIOJclU0rSe1mFcno6DSVw=.4d42ce8a-801a-4fb9-a8f1-5933d76438ee@github.com>

> There are a few fixes here:
> 
> During a Full GC, the compaction process is not constrained by the maximum capacity for young and old generations. We must, therefore, reestablish these constraints after the Full GC completes by _forcing_ the transfer of regions to old, if necessary.
> 
> When a region is "flipped" from the Mutator's free set to the Collector's, we cannot _force_ the transfer of the region because the young generation may not have any free regions.
> 
> A logic error in the loop which rebuilds the free set could prevent transferring free regions from the mutator to the collector reserves.
> 
> Some unused methods and fields were removed.

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Remove restriction on young generation size

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/311/files
  - new: https://git.openjdk.org/shenandoah/pull/311/files/8048c611..bafdbf06

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=311&range=04
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=311&range=03-04

  Stats: 2 lines in 1 file changed: 0 ins; 2 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/311.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/311/head:pull/311

PR: https://git.openjdk.org/shenandoah/pull/311

From rriggs at openjdk.org  Tue Aug 29 16:48:15 2023
From: rriggs at openjdk.org (Roger Riggs)
Date: Tue, 29 Aug 2023 16:48:15 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v2]
In-Reply-To: <pnAnjv8HLt1sykmB9jRamtiVRoTbmwQi9xGuaSnCi10=.34427fd4-7b88-436e-946b-04c7364a1853@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <ISuejqIr1InBH-hXEzgOt7iBXrhH4iv93glC3woOewQ=.3a7357d0-a911-4dfa-834b-2c5123915ce7@github.com>
 <pnAnjv8HLt1sykmB9jRamtiVRoTbmwQi9xGuaSnCi10=.34427fd4-7b88-436e-946b-04c7364a1853@github.com>
Message-ID: <5AjL4PbmGc-9jQdiCK1jd4LDdC-B7NgJ_dwIZW1U-aA=.0ad22596-a1f9-4e87-946f-dceedf0a37e5@github.com>

On Tue, 29 Aug 2023 14:06:01 GMT, Roger Riggs <rriggs at openjdk.org> wrote:

>> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   copyright
>
> I don't think this is the best change across so many files.
> It gives a very ugly name to a common test function and affects a very large number of tests.

> @RogerRiggs If it is only the name you want changed, maybe you can offer a better name?
@lkorinth 

Sorry for the too short comment; I wanted to make sure it wasn't integrated before I could look at it more closely.
Neither the bug report or the PR describe the problem and its ramifications, only the solution.
Can you elaborate on the conditions that lead to this. (and include them in the bug report).

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1697803842

From kdnilsen at openjdk.org  Tue Aug 29 17:30:02 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 29 Aug 2023 17:30:02 GMT
Subject: RFR: 8315044: GenShen: Verifier detects clean card should be dirty
In-Reply-To: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
References: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
Message-ID: <LH_sngBizi5fhAEnqUNK2HSdHYoADzHdfK4ILqkENsw=.d430b2f8-43a5-4160-87f7-6f64ab78b8e7@github.com>

On Mon, 28 Aug 2023 23:35:46 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> When a Reference object is newly discovered, it is placed onto the worker's thread-local discovered list.  This sometimes results in a reference from an old object to a young object, requiring that the remembered set card-table entry be marked as dirty.  This patch causes the marking to be performed.

I'm going to leave this in draft mode until I see and analyze results of integration testing.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/314#issuecomment-1696563480

From kdnilsen at openjdk.org  Tue Aug 29 17:30:02 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 29 Aug 2023 17:30:02 GMT
Subject: RFR: 8315044: GenShen: Verifier detects clean card should be dirty
Message-ID: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>

When a Reference object is newly discovered, it is placed onto the worker's thread-local discovered list.  This sometimes results in a reference from an old object to a young object, requiring that the remembered set card-table entry be marked as dirty.  This patch causes the marking to be performed.

-------------

Commit messages:
 - Fix whitespace
 - Mark card as dirty if discovered reference list has interesting pointer

Changes: https://git.openjdk.org/shenandoah/pull/314/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=314&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8315044
  Stats: 17 lines in 1 file changed: 17 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/314.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/314/head:pull/314

PR: https://git.openjdk.org/shenandoah/pull/314

From ysr at openjdk.org  Tue Aug 29 18:48:13 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 29 Aug 2023 18:48:13 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag
Message-ID: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>

Protect the card barrier code generation by a new global `ShenandoahCardBarrier`, which is disabled by default and enabled for GenShen (generational mode Shenandoah) only. If the user forces the barrier flag to a value inconsistent with mode, we exit with an appropriate error message.

The changes have been made for x86, aarch, and ppc. I have tested on x86 on a cloud host running jtreg1-4 and SPECjbb. A round of aarch64 and x86 pipeline tests have run (but ran into existing assertion that William is fixing). Will request PPC testing from partners able to test. RISC-V & S390 don't currently support generational mode Shenandoah.

-------------

Commit messages:
 - Check and exit if user specifies a setting of ShenandoahCardBarrier that
 - check that ShenandoahCardBarrier is off by default for most
 - Changes from prelim review by William & Kelvin.
 - Take care of some ppc changes
 - Merge branch 'master' into card_barrier
 - First cut of changes, checking in for testing:

Changes: https://git.openjdk.org/shenandoah/pull/313/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=313&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8315247
  Stats: 105 lines in 15 files changed: 47 ins; 21 del; 37 mod
  Patch: https://git.openjdk.org/shenandoah/pull/313.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/313/head:pull/313

PR: https://git.openjdk.org/shenandoah/pull/313

From wkemper at openjdk.org  Tue Aug 29 18:48:14 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 29 Aug 2023 18:48:14 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag
In-Reply-To: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
Message-ID: <IvsBiSW8yJY3MbaOr-VYGodhPXIFKwGJo7y_eGohV40=.c177a21a-9a53-45cf-9a4a-a4bad56caed3@github.com>

On Mon, 28 Aug 2023 19:48:58 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> Protect the card barrier code generation by a new global `ShenandoahCardBarrier`, which is disabled by default and enabled for GenShen (generational mode Shenandoah) only. If the user forces the barrier flag to a value inconsistent with mode, we exit with an appropriate error message.
> 
> The changes have been made for x86, aarch, and ppc. I have tested on x86 on a cloud host running jtreg1-4 and SPECjbb. A round of aarch64 and x86 pipeline tests have run (but ran into existing assertion that William is fixing). Will request PPC testing from partners able to test. RISC-V & S390 don't currently support generational mode Shenandoah.

Minor nit - can we have a consistent, more helpful message for the assertions?


  assert(ShenandoahHeap::heap()->mode()->is_generational(), "Not needed");

Maybe:

  assert(ShenandoahHeap::heap()->mode()->is_generational(), "Expected generational mode here");

-------------

Changes requested by wkemper (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/313#pullrequestreview-1599001416

From kdnilsen at openjdk.org  Tue Aug 29 18:48:22 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 29 Aug 2023 18:48:22 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag
In-Reply-To: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
Message-ID: <DjRqqEAOlCF2tQAusnQ8DeQ3S6dV8i2kiu6sHhUle1U=.5c85f179-9f8c-428c-8de9-16f7c4f467c5@github.com>

On Mon, 28 Aug 2023 19:48:58 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> Protect the card barrier code generation by a new global `ShenandoahCardBarrier`, which is disabled by default and enabled for GenShen (generational mode Shenandoah) only. If the user forces the barrier flag to a value inconsistent with mode, we exit with an appropriate error message.
> 
> The changes have been made for x86, aarch, and ppc. I have tested on x86 on a cloud host running jtreg1-4 and SPECjbb. A round of aarch64 and x86 pipeline tests have run (but ran into existing assertion that William is fixing). Will request PPC testing from partners able to test. RISC-V & S390 don't currently support generational mode Shenandoah.

This looks pretty good.  Have you sent it through the test pipeline?

If we have this ShenandoahCardBarrier flag, I'd be inclined to use it everywhere we currently have the query heap->mode()->is_generational().  It ought to be more efficient.  Before you make that global change, maybe we should confirm that William (and other interested reviewers) like that approach.

src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp line 390:

> 388:   // This will need to be changed if we ever use a card-marking post-barrier w/a non-generational
> 389:   // Shenandoah in the future.
> 390:   assert(ShenandoahHeap::heap()->mode()->is_generational(), "Only use case for Shenandoah today");

I'd be ok to assert ShenandoahCardBarrier since it ought to be a bit faster than code as written.  Though I understand this is perhaps a bit more rigorous.

src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp line 636:

> 634:   // This will need to be changed if we ever use a card-marking post-barrier w/a non-generational
> 635:   // Shenandoah in the future.
> 636:   assert(ShenandoahHeap::heap()->mode()->is_generational(), "Only use case for Shenandoah today");

same here: ok to use ShenandoahCardBarrier.

src/hotspot/cpu/ppc/gc/shenandoah/shenandoahBarrierSetAssembler_ppc.cpp line 600:

> 598: 
> 599: void ShenandoahBarrierSetAssembler::store_check(MacroAssembler* masm, Register base, RegisterOrConstant ind_or_offs, Register tmp) {
> 600:   assert(ShenandoahCardBarrier && ShenandoahHeap::heap()->mode()->is_generational(), "Error");)

And here...

src/hotspot/cpu/ppc/gc/shenandoah/shenandoahBarrierSetAssembler_ppc.cpp line 796:

> 794: void ShenandoahBarrierSetAssembler::gen_write_ref_array_post_barrier(MacroAssembler* masm, DecoratorSet decorators,
> 795:                                                                      Register addr, Register count, Register preserve) {
> 796:   assert(ShenandoahCardBarrier && ShenandoahHeap::heap()->mode()->is_generational(), "Error");

Here too.

src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp line 213:

> 211:                                                        Register src, Register dst, Register count) {
> 212: 
> 213:   if (!ShenandoahCardBarrier) {

Why wouldn't we handle this the same as other barriers.  assert(ShenandoahCardBarrier) here and check for ShenandoahCardBarrier before calling arraycopy_epilogue()?

src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 523:

> 521:   product(bool, ShenandoahCardBarrier, false, DIAGNOSTIC,                   \
> 522:           "Turn on/off card-marking post-write barrier in Shenandoah")      \
> 523:                                                                             \

I was thinking this would not be visible on the command line.  If we make it visible, then we would need to assure consistency with mode == generational.

My thought was we would just set this global variable in, for example, ShenandoahArguments::initialize() if !strcmp(ShenandoahGCMode, "generational")

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/313#pullrequestreview-1599099341
PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1307943153
PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1307943562
PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1307944197
PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1307944658
PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1307946501
PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1307956536

From ysr at openjdk.org  Tue Aug 29 18:48:23 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 29 Aug 2023 18:48:23 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag
In-Reply-To: <IvsBiSW8yJY3MbaOr-VYGodhPXIFKwGJo7y_eGohV40=.c177a21a-9a53-45cf-9a4a-a4bad56caed3@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
 <IvsBiSW8yJY3MbaOr-VYGodhPXIFKwGJo7y_eGohV40=.c177a21a-9a53-45cf-9a4a-a4bad56caed3@github.com>
Message-ID: <7MikyfYgotd6XOqof0bxaK_TquScOhCbDQEGCyNI3-g=.a63ac297-f76b-4a3b-b82a-a26976d54856@github.com>

On Mon, 28 Aug 2023 20:00:23 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Minor nit - can we have a consistent, more helpful message for the assertions?
> 
> ```
>   assert(ShenandoahHeap::heap()->mode()->is_generational(), "Not needed");
> ```
> 
> Maybe:
> 
> ```
>   assert(ShenandoahHeap::heap()->mode()->is_generational(), "Expected generational mode here");
> ```

Following Kelvin's suggestion, I removed any reference to generational mode in the barrier/generayion code itself, and instead just assert that the `ShenandoahCardBarrier` flag should be set. I am going to look for a suitable place during initialization of non-generational mode to assert that it isn't set. We do check and bail at generational mode initialization if the flag isn't set.

I used the message `Did you mean to enable ShenandoahCardBarrier?` if it isn't enabled, but I could replace it with `Expecting ShenandoahCardBarrier set here` in case that reads better in the error message, or `ShenandoahCardBarrier should be set`.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/313#issuecomment-1696599490

From ysr at openjdk.org  Tue Aug 29 18:48:24 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 29 Aug 2023 18:48:24 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag
In-Reply-To: <DjRqqEAOlCF2tQAusnQ8DeQ3S6dV8i2kiu6sHhUle1U=.5c85f179-9f8c-428c-8de9-16f7c4f467c5@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
 <DjRqqEAOlCF2tQAusnQ8DeQ3S6dV8i2kiu6sHhUle1U=.5c85f179-9f8c-428c-8de9-16f7c4f467c5@github.com>
Message-ID: <aDtKwRg3xdzILfP42LPyH_E1N9MxpN0PeXhNLUeXMS8=.04f1a775-73c3-429e-bb17-fa26d4f710aa@github.com>

On Mon, 28 Aug 2023 21:16:00 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This looks pretty good. Have you sent it through the test pipeline?
> 
> If we have this ShenandoahCardBarrier flag, I'd be inclined to use it everywhere we currently have the query heap->mode()->is_generational(). It ought to be more efficient. Before you make that global change, maybe we should confirm that William (and other interested reviewers) like that approach.

Did a spin through test pipeline and will run another following the latest changes. All looks good from what I can tell from tests, including jtreg's & SPECjbb as well.

Opened official PR from draft and will link JBS ticket momentarily.

> src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp line 390:
> 
>> 388:   // This will need to be changed if we ever use a card-marking post-barrier w/a non-generational
>> 389:   // Shenandoah in the future.
>> 390:   assert(ShenandoahHeap::heap()->mode()->is_generational(), "Only use case for Shenandoah today");
> 
> I'd be ok to assert ShenandoahCardBarrier since it ought to be a bit faster than code as written.  Though I understand this is perhaps a bit more rigorous.

Done.

> src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp line 636:
> 
>> 634:   // This will need to be changed if we ever use a card-marking post-barrier w/a non-generational
>> 635:   // Shenandoah in the future.
>> 636:   assert(ShenandoahHeap::heap()->mode()->is_generational(), "Only use case for Shenandoah today");
> 
> same here: ok to use ShenandoahCardBarrier.

Done

> src/hotspot/cpu/ppc/gc/shenandoah/shenandoahBarrierSetAssembler_ppc.cpp line 600:
> 
>> 598: 
>> 599: void ShenandoahBarrierSetAssembler::store_check(MacroAssembler* masm, Register base, RegisterOrConstant ind_or_offs, Register tmp) {
>> 600:   assert(ShenandoahCardBarrier && ShenandoahHeap::heap()->mode()->is_generational(), "Error");)
> 
> And here...

Done

> src/hotspot/cpu/ppc/gc/shenandoah/shenandoahBarrierSetAssembler_ppc.cpp line 796:
> 
>> 794: void ShenandoahBarrierSetAssembler::gen_write_ref_array_post_barrier(MacroAssembler* masm, DecoratorSet decorators,
>> 795:                                                                      Register addr, Register count, Register preserve) {
>> 796:   assert(ShenandoahCardBarrier && ShenandoahHeap::heap()->mode()->is_generational(), "Error");
> 
> Here too.

Done.

> src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp line 213:
> 
>> 211:                                                        Register src, Register dst, Register count) {
>> 212: 
>> 213:   if (!ShenandoahCardBarrier) {
> 
> Why wouldn't we handle this the same as other barriers.  assert(ShenandoahCardBarrier) here and check for ShenandoahCardBarrier before calling arraycopy_epilogue()?

Today we call arraycopy_epilogue() unconditionally from common code (such as stub generators) irrespective of collector. We'll probably want a better barrier set abstraction across all collectors to do what you suggest in the future. For now, I'll just massage this to conform to the other platforms where we test and branch on the flag here (rather than test and return at the top, or do the job in the body that follows).

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/313#issuecomment-1697923735
PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1308013930
PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1308030882
PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1308014029
PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1308031191
PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1308024348

From ysr at openjdk.org  Tue Aug 29 18:48:25 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 29 Aug 2023 18:48:25 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag
In-Reply-To: <aDtKwRg3xdzILfP42LPyH_E1N9MxpN0PeXhNLUeXMS8=.04f1a775-73c3-429e-bb17-fa26d4f710aa@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
 <DjRqqEAOlCF2tQAusnQ8DeQ3S6dV8i2kiu6sHhUle1U=.5c85f179-9f8c-428c-8de9-16f7c4f467c5@github.com>
 <aDtKwRg3xdzILfP42LPyH_E1N9MxpN0PeXhNLUeXMS8=.04f1a775-73c3-429e-bb17-fa26d4f710aa@github.com>
Message-ID: <RRZul1sR9Ed1rVLs2SaPXA_QYLDDqpVX6D0hb3BNlSY=.38ec6140-0fce-4aee-91ba-443846b6de80@github.com>

On Mon, 28 Aug 2023 22:37:07 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp line 213:
>> 
>>> 211:                                                        Register src, Register dst, Register count) {
>>> 212: 
>>> 213:   if (!ShenandoahCardBarrier) {
>> 
>> Why wouldn't we handle this the same as other barriers.  assert(ShenandoahCardBarrier) here and check for ShenandoahCardBarrier before calling arraycopy_epilogue()?
>
> Today we call arraycopy_epilogue() unconditionally from common code (such as stub generators) irrespective of collector. We'll probably want a better barrier set abstraction across all collectors to do what you suggest in the future. For now, I'll just massage this to conform to the other platforms where we test and branch on the flag here (rather than test and return at the top, or do the job in the body that follows).

Updated.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1309194701

From wkemper at openjdk.org  Tue Aug 29 18:48:27 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 29 Aug 2023 18:48:27 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag
In-Reply-To: <DjRqqEAOlCF2tQAusnQ8DeQ3S6dV8i2kiu6sHhUle1U=.5c85f179-9f8c-428c-8de9-16f7c4f467c5@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
 <DjRqqEAOlCF2tQAusnQ8DeQ3S6dV8i2kiu6sHhUle1U=.5c85f179-9f8c-428c-8de9-16f7c4f467c5@github.com>
Message-ID: <zogp1slntTEG-27hztiOx7qSO471Qv2wH_5ga9xXZyQ=.6cbbdf13-0224-435b-bb11-88ff280c6baa@github.com>

On Mon, 28 Aug 2023 21:13:22 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Protect the card barrier code generation by a new global `ShenandoahCardBarrier`, which is disabled by default and enabled for GenShen (generational mode Shenandoah) only. If the user forces the barrier flag to a value inconsistent with mode, we exit with an appropriate error message.
>> 
>> The changes have been made for x86, aarch, and ppc. I have tested on x86 on a cloud host running jtreg1-4 and SPECjbb. A round of aarch64 and x86 pipeline tests have run (but ran into existing assertion that William is fixing). Will request PPC testing from partners able to test. RISC-V & S390 don't currently support generational mode Shenandoah.
>
> src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 523:
> 
>> 521:   product(bool, ShenandoahCardBarrier, false, DIAGNOSTIC,                   \
>> 522:           "Turn on/off card-marking post-write barrier in Shenandoah")      \
>> 523:                                                                             \
> 
> I was thinking this would not be visible on the command line.  If we make it visible, then we would need to assure consistency with mode == generational.
> 
> My thought was we would just set this global variable in, for example, ShenandoahArguments::initialize() if !strcmp(ShenandoahGCMode, "generational")

This approach has parity with the other barrier flags. `shenandoahGenerationalMode.cpp` asserts that the flag is on. 

It shouldn't be strictly necessary with the default being `false`, but we may want to have `shenandoahPassiveMode.cpp` explicitly turn this flag off as it does the other barrier flags.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1307974222

From kdnilsen at openjdk.org  Tue Aug 29 18:48:27 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 29 Aug 2023 18:48:27 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag
In-Reply-To: <zogp1slntTEG-27hztiOx7qSO471Qv2wH_5ga9xXZyQ=.6cbbdf13-0224-435b-bb11-88ff280c6baa@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
 <DjRqqEAOlCF2tQAusnQ8DeQ3S6dV8i2kiu6sHhUle1U=.5c85f179-9f8c-428c-8de9-16f7c4f467c5@github.com>
 <zogp1slntTEG-27hztiOx7qSO471Qv2wH_5ga9xXZyQ=.6cbbdf13-0224-435b-bb11-88ff280c6baa@github.com>
Message-ID: <ykz_vXcq1tq6rU9DWxu_1KAJHK8zaGyiuRHdihunFQA=.0c4385bb-6ab0-4c73-b124-2c2230adc65b@github.com>

On Mon, 28 Aug 2023 21:28:21 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 523:
>> 
>>> 521:   product(bool, ShenandoahCardBarrier, false, DIAGNOSTIC,                   \
>>> 522:           "Turn on/off card-marking post-write barrier in Shenandoah")      \
>>> 523:                                                                             \
>> 
>> I was thinking this would not be visible on the command line.  If we make it visible, then we would need to assure consistency with mode == generational.
>> 
>> My thought was we would just set this global variable in, for example, ShenandoahArguments::initialize() if !strcmp(ShenandoahGCMode, "generational")
>
> This approach has parity with the other barrier flags. `shenandoahGenerationalMode.cpp` asserts that the flag is on. 
> 
> It shouldn't be strictly necessary with the default being `false`, but we may want to have `shenandoahPassiveMode.cpp` explicitly turn this flag off as it does the other barrier flags.

If we make this visible on the command line, we will want to make sure it is consistent with mode->is_generational().  Right?

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1307995770

From kdnilsen at openjdk.org  Tue Aug 29 18:48:28 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 29 Aug 2023 18:48:28 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag
In-Reply-To: <ykz_vXcq1tq6rU9DWxu_1KAJHK8zaGyiuRHdihunFQA=.0c4385bb-6ab0-4c73-b124-2c2230adc65b@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
 <DjRqqEAOlCF2tQAusnQ8DeQ3S6dV8i2kiu6sHhUle1U=.5c85f179-9f8c-428c-8de9-16f7c4f467c5@github.com>
 <zogp1slntTEG-27hztiOx7qSO471Qv2wH_5ga9xXZyQ=.6cbbdf13-0224-435b-bb11-88ff280c6baa@github.com>
 <ykz_vXcq1tq6rU9DWxu_1KAJHK8zaGyiuRHdihunFQA=.0c4385bb-6ab0-4c73-b124-2c2230adc65b@github.com>
Message-ID: <-8I0PBBTvtRkXM8G4Z-FObmhbtyvvV45yDBJ5x_19FY=.2d119786-c5ef-4f57-b7ec-f6fd92ffb6f2@github.com>

On Mon, 28 Aug 2023 21:51:31 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> This approach has parity with the other barrier flags. `shenandoahGenerationalMode.cpp` asserts that the flag is on. 
>> 
>> It shouldn't be strictly necessary with the default being `false`, but we may want to have `shenandoahPassiveMode.cpp` explicitly turn this flag off as it does the other barrier flags.
>
> If we make this visible on the command line, we will want to make sure it is consistent with mode->is_generational().  Right?

(or you could say, if they turn off ShenandoahCardBarrier, that would mean we have no remembered set and always scan all of old-gen.  but that would be a lot more work for us, and of no value, I think...)

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1307996693

From ysr at openjdk.org  Tue Aug 29 18:48:30 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 29 Aug 2023 18:48:30 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag
In-Reply-To: <0DPNN_ydQevNha9CxMkxtInJONoaKg25gJ3D0g1kGaw=.d6129e57-0325-4d3d-a125-025682e73b83@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
 <DjRqqEAOlCF2tQAusnQ8DeQ3S6dV8i2kiu6sHhUle1U=.5c85f179-9f8c-428c-8de9-16f7c4f467c5@github.com>
 <zogp1slntTEG-27hztiOx7qSO471Qv2wH_5ga9xXZyQ=.6cbbdf13-0224-435b-bb11-88ff280c6baa@github.com>
 <ykz_vXcq1tq6rU9DWxu_1KAJHK8zaGyiuRHdihunFQA=.0c4385bb-6ab0-4c73-b124-2c2230adc65b@github.com>
 <-8I0PBBTvtRkXM8G4Z-FObmhbtyvvV45yDBJ5x_19FY=.2d119786-c5ef-4f57-b7ec-f6fd92ffb6f2@github.com>
 <0DPNN_ydQevNha9CxMkxtInJONoaKg25gJ3D0g1kGaw=.d6129e57-0325-4d3d-a125-025682e73b83@github.com>
Message-ID: <PEeKLkhUTUmVcwC0r2e-Nw6xaFdOw-YootF6Yh7c-FE=.816fe6f8-06e8-4981-8b24-aad193bab93e@github.com>

On Mon, 28 Aug 2023 22:00:09 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> (or you could say, if they turn off ShenandoahCardBarrier, that would mean we have no remembered set and always scan all of old-gen.  but that would be a lot more work for us, and of no value, I think...)
>
> maybe just fatal out in ShenandoahArguments if mode->is_generational() and !ShenandoahCardBarrier.  So the user has to specify ShenandoahCardBarrier if they specify ShenandoahGCMode == generational.

I'm going to leave it visible on command-line like the other `Shenandoah*Barrier` flags, but force it off in all modes except generational where we force it on. Since these aren't manageable flags,, jcmd should not be able to enable them after start up either. 

If the concern is that we would be silently ignoring user request to explicitly enable it in non-generational mode, or disable it in generational mode, we can log a warning upon such an attempt and still do the right thing.

Let me test these various flag combos and ensure that they do the right thing. I'll update the PR once finished w/the changes and testing.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1308019915

From kdnilsen at openjdk.org  Tue Aug 29 18:48:29 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 29 Aug 2023 18:48:29 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag
In-Reply-To: <-8I0PBBTvtRkXM8G4Z-FObmhbtyvvV45yDBJ5x_19FY=.2d119786-c5ef-4f57-b7ec-f6fd92ffb6f2@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
 <DjRqqEAOlCF2tQAusnQ8DeQ3S6dV8i2kiu6sHhUle1U=.5c85f179-9f8c-428c-8de9-16f7c4f467c5@github.com>
 <zogp1slntTEG-27hztiOx7qSO471Qv2wH_5ga9xXZyQ=.6cbbdf13-0224-435b-bb11-88ff280c6baa@github.com>
 <ykz_vXcq1tq6rU9DWxu_1KAJHK8zaGyiuRHdihunFQA=.0c4385bb-6ab0-4c73-b124-2c2230adc65b@github.com>
 <-8I0PBBTvtRkXM8G4Z-FObmhbtyvvV45yDBJ5x_19FY=.2d119786-c5ef-4f57-b7ec-f6fd92ffb6f2@github.com>
Message-ID: <0DPNN_ydQevNha9CxMkxtInJONoaKg25gJ3D0g1kGaw=.d6129e57-0325-4d3d-a125-025682e73b83@github.com>

On Mon, 28 Aug 2023 21:52:55 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> If we make this visible on the command line, we will want to make sure it is consistent with mode->is_generational().  Right?
>
> (or you could say, if they turn off ShenandoahCardBarrier, that would mean we have no remembered set and always scan all of old-gen.  but that would be a lot more work for us, and of no value, I think...)

maybe just fatal out in ShenandoahArguments if mode->is_generational() and !ShenandoahCardBarrier.  So the user has to specify ShenandoahCardBarrier if they specify ShenandoahGCMode == generational.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1308001950

From kdnilsen at openjdk.org  Tue Aug 29 18:48:31 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 29 Aug 2023 18:48:31 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag
In-Reply-To: <PEeKLkhUTUmVcwC0r2e-Nw6xaFdOw-YootF6Yh7c-FE=.816fe6f8-06e8-4981-8b24-aad193bab93e@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
 <DjRqqEAOlCF2tQAusnQ8DeQ3S6dV8i2kiu6sHhUle1U=.5c85f179-9f8c-428c-8de9-16f7c4f467c5@github.com>
 <zogp1slntTEG-27hztiOx7qSO471Qv2wH_5ga9xXZyQ=.6cbbdf13-0224-435b-bb11-88ff280c6baa@github.com>
 <ykz_vXcq1tq6rU9DWxu_1KAJHK8zaGyiuRHdihunFQA=.0c4385bb-6ab0-4c73-b124-2c2230adc65b@github.com>
 <-8I0PBBTvtRkXM8G4Z-FObmhbtyvvV45yDBJ5x_19FY=.2d119786-c5ef-4f57-b7ec-f6fd92ffb6f2@github.com>
 <0DPNN_ydQevNha9CxMkxtInJONoaKg25gJ3D0g1kGaw=.d6129e57-0325-4d3d-a125-025682e73b83@github.com>
 <PEeKLkhUTUmVcwC0r2e-Nw6xaFdOw-YootF6Yh7c-FE=.816fe6f8-06e8-4981-8b24-aad193bab93e@github.com>
Message-ID: <02vFGiRQULtbyuZMY9u37n7REmIqSa3ioTtUdr4rMI0=.86f047a8-bb7c-4302-aab6-31ea0a1cc626@github.com>

On Mon, 28 Aug 2023 22:29:32 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> maybe just fatal out in ShenandoahArguments if mode->is_generational() and !ShenandoahCardBarrier.  So the user has to specify ShenandoahCardBarrier if they specify ShenandoahGCMode == generational.
>
> I'm going to leave it visible on command-line like the other `Shenandoah*Barrier` flags, but force it off in all modes except generational where we force it on. Since these aren't manageable flags,, jcmd should not be able to enable them after start up either. 
> 
> If the concern is that we would be silently ignoring user request to explicitly enable it in non-generational mode, or disable it in generational mode, we can log a warning upon such an attempt and still do the right thing.
> 
> Let me test these various flag combos and ensure that they do the right thing. I'll update the PR once finished w/the changes and testing.

Maybe the comment can say: "ShenandoahCardBarrier" is true by default iff ShenandoahGCMode is generational.

Otherwise, I think we would need to change all of the existing test configurations that have generational mode to add -XX:ShenandoahCardBarrier=true

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1309121504

From ysr at openjdk.org  Tue Aug 29 18:48:31 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 29 Aug 2023 18:48:31 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag
In-Reply-To: <02vFGiRQULtbyuZMY9u37n7REmIqSa3ioTtUdr4rMI0=.86f047a8-bb7c-4302-aab6-31ea0a1cc626@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
 <DjRqqEAOlCF2tQAusnQ8DeQ3S6dV8i2kiu6sHhUle1U=.5c85f179-9f8c-428c-8de9-16f7c4f467c5@github.com>
 <zogp1slntTEG-27hztiOx7qSO471Qv2wH_5ga9xXZyQ=.6cbbdf13-0224-435b-bb11-88ff280c6baa@github.com>
 <ykz_vXcq1tq6rU9DWxu_1KAJHK8zaGyiuRHdihunFQA=.0c4385bb-6ab0-4c73-b124-2c2230adc65b@github.com>
 <-8I0PBBTvtRkXM8G4Z-FObmhbtyvvV45yDBJ5x_19FY=.2d119786-c5ef-4f57-b7ec-f6fd92ffb6f2@github.com>
 <0DPNN_ydQevNha9CxMkxtInJONoaKg25gJ3D0g1kGaw=.d6129e57-0325-4d3d-a125-025682e73b83@github.com>
 <PEeKLkhUTUmVcwC0r2e-Nw6xaFdOw-YootF6Yh7c-FE=.816fe6f8-06e8-4981-8b24-aad193bab93e@github.com>
 <02vFGiRQULtbyuZMY9u37n7REmIqSa3ioTtUdr4rMI0=.86f047a8-bb7c-4302-aab6-31ea0a1cc626@github.com>
Message-ID: <15csLEjyZ0vVG-a-hFs5OdkjGflhkECB9hd6NZ15SPI=.691e547a-099b-40d7-ac72-03c52c47f625@github.com>

On Tue, 29 Aug 2023 17:03:02 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> I'm going to leave it visible on command-line like the other `Shenandoah*Barrier` flags, but force it off in all modes except generational where we force it on. Since these aren't manageable flags,, jcmd should not be able to enable them after start up either. 
>> 
>> If the concern is that we would be silently ignoring user request to explicitly enable it in non-generational mode, or disable it in generational mode, we can log a warning upon such an attempt and still do the right thing.
>> 
>> Let me test these various flag combos and ensure that they do the right thing. I'll update the PR once finished w/the changes and testing.
>
> Maybe the comment can say: "ShenandoahCardBarrier" is true by default iff ShenandoahGCMode is generational.
> 
> Otherwise, I think we would need to change all of the existing test configurations that have generational mode to add -XX:ShenandoahCardBarrier=true

I updated with an attempt to address the comments above. Take a look and see how it looks.

I'll also take this out of draft form to a regular PR.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1309182791

From ysr at openjdk.org  Tue Aug 29 18:48:32 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 29 Aug 2023 18:48:32 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag
In-Reply-To: <15csLEjyZ0vVG-a-hFs5OdkjGflhkECB9hd6NZ15SPI=.691e547a-099b-40d7-ac72-03c52c47f625@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
 <DjRqqEAOlCF2tQAusnQ8DeQ3S6dV8i2kiu6sHhUle1U=.5c85f179-9f8c-428c-8de9-16f7c4f467c5@github.com>
 <zogp1slntTEG-27hztiOx7qSO471Qv2wH_5ga9xXZyQ=.6cbbdf13-0224-435b-bb11-88ff280c6baa@github.com>
 <ykz_vXcq1tq6rU9DWxu_1KAJHK8zaGyiuRHdihunFQA=.0c4385bb-6ab0-4c73-b124-2c2230adc65b@github.com>
 <-8I0PBBTvtRkXM8G4Z-FObmhbtyvvV45yDBJ5x_19FY=.2d119786-c5ef-4f57-b7ec-f6fd92ffb6f2@github.com>
 <0DPNN_ydQevNha9CxMkxtInJONoaKg25gJ3D0g1kGaw=.d6129e57-0325-4d3d-a125-025682e73b83@github.com>
 <PEeKLkhUTUmVcwC0r2e-Nw6xaFdOw-YootF6Yh7c-FE=.816fe6f8-06e8-4981-8b24-aad193bab93e@github.com>
 <02vFGiRQULtbyuZMY9u37n7REmIqSa3ioTtUdr4rMI0=.86f047a8-bb7c-4302-aab6-31ea0a1cc626@github.com>
 <15csLEjyZ0vVG-a-hFs5OdkjGflhkECB9hd6NZ15SPI=.691e547a-099b-40d7-ac72-03c52c47f625@github.com>
Message-ID: <wOJy0B4dSHkRgu99tFKW5Nzj9msjmQjizQryW3Q-iHI=.c0a3edde-97c3-4728-a4c2-f9d1bd19d501@github.com>

On Tue, 29 Aug 2023 18:04:11 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Maybe the comment can say: "ShenandoahCardBarrier" is true by default iff ShenandoahGCMode is generational.
>> 
>> Otherwise, I think we would need to change all of the existing test configurations that have generational mode to add -XX:ShenandoahCardBarrier=true
>
> I updated with an attempt to address the comments above. Take a look and see how it looks.
> 
> I'll also take this out of draft form to a regular PR.

So we'll now exit with an appropriate error message if the user changes `ShenandoahCardBarrier` to a value that conflicts with the `ShenandoahGCMode` they selected:

**1. No user selection of barrier flag:**

(23-08-29 17:54:47) <0> [~/workplace/ysr/genshen-ysr/shenandoah]  
dev-dsk-ysr-2a-87502c94 % build/linux-x86_64-server-slowdebug/jdk/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:+PrintFlagsFinal -version | grep ShenandoahCardBarrier
     bool ShenandoahCardBarrier                    = false                                  {diagnostic} {default}
openjdk version "22-internal" 2024-03-19
OpenJDK Runtime Environment (slowdebug build 22-internal-adhoc.ysr.shenandoah)
OpenJDK 64-Bit Server VM (slowdebug build 22-internal-adhoc.ysr.shenandoah, mixed mode)

(23-08-29 18:11:09) <0> [~/workplace/ysr/genshen-ysr/shenandoah]  
dev-dsk-ysr-2a-87502c94 % build/linux-x86_64-server-slowdebug/jdk/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:ShenandoahGCMode=generational -XX:+PrintFlagsFinal -version | grep ShenandoahCardBarrier
     bool ShenandoahCardBarrier                    = true                                   {diagnostic} {default}
openjdk version "22-internal" 2024-03-19
OpenJDK Runtime Environment (slowdebug build 22-internal-adhoc.ysr.shenandoah)
OpenJDK 64-Bit Server VM (slowdebug build 22-internal-adhoc.ysr.shenandoah, mixed mode)


**2. User sets barrier flag to value that conflicts with gc mode:**

(23-08-29 18:11:36) <0> [~/workplace/ysr/genshen-ysr/shenandoah]  
dev-dsk-ysr-2a-87502c94 % build/linux-x86_64-server-slowdebug/jdk/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:+ShenandoahCardBarrier -XX:+PrintFlagsFinal -version | grep ShenandoahCardBarrier                                  
Error: GC mode needs -XX:-ShenandoahCardBarrier to work correctly

(23-08-29 18:11:55) <0> [~/workplace/ysr/genshen-ysr/shenandoah]  
dev-dsk-ysr-2a-87502c94 % build/linux-x86_64-server-slowdebug/jdk/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:ShenandoahGCMode=generational -XX:-ShenandoahCardBarrier -XX:+PrintFlagsFinal -version | grep ShenandoahCardBarrier
Error: GC mode needs -XX:+ShenandoahCardBarrier to work correctly


**3. Use sets barrier flag to value consistent with gc mode:**

(23-08-29 18:12:15) <0> [~/workplace/ysr/genshen-ysr/shenandoah]  
dev-dsk-ysr-2a-87502c94 % build/linux-x86_64-server-slowdebug/jdk/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:-ShenandoahCardBarrier -XX:+PrintFlagsFinal -version | grep ShenandoahCardBarrier
     bool ShenandoahCardBarrier                    = false                                  {diagnostic} {command line}
openjdk version "22-internal" 2024-03-19
OpenJDK Runtime Environment (slowdebug build 22-internal-adhoc.ysr.shenandoah)
OpenJDK 64-Bit Server VM (slowdebug build 22-internal-adhoc.ysr.shenandoah, mixed mode)

(23-08-29 18:12:43) <0> [~/workplace/ysr/genshen-ysr/shenandoah]  
dev-dsk-ysr-2a-87502c94 % build/linux-x86_64-server-slowdebug/jdk/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:ShenandoahGCMode=generational -XX:+ShenandoahCardBarrier -XX:+PrintFlagsFinal -version | grep ShenandoahCardBarrier
     bool ShenandoahCardBarrier                    = true                                   {diagnostic} {command line}
openjdk version "22-internal" 2024-03-19
OpenJDK Runtime Environment (slowdebug build 22-internal-adhoc.ysr.shenandoah)
OpenJDK 64-Bit Server VM (slowdebug build 22-internal-adhoc.ysr.shenandoah, mixed mode)

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1309186174

From ysr at openjdk.org  Tue Aug 29 19:13:47 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 29 Aug 2023 19:13:47 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag
In-Reply-To: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
Message-ID: <jFXs4lDAxyN4sL55rb7897O7ZXQgGGA4i_Yn4zFZp3Q=.75e3adb4-776d-4331-8054-758eee4e6bae@github.com>

On Mon, 28 Aug 2023 19:48:58 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> Protect the card barrier code generation by a new global `ShenandoahCardBarrier`, which is disabled by default and enabled for GenShen (generational mode Shenandoah) only. If the user forces the barrier flag to a value inconsistent with mode, we exit with an appropriate error message.
> 
> The changes have been made for x86, aarch, and ppc. I have tested on x86 on a cloud host running jtreg1-4 and SPECjbb. A round of aarch64 and x86 pipeline tests have run (but ran into existing assertion that William is fixing). Will request PPC testing from partners able to test. RISC-V & S390 don't currently support generational mode Shenandoah.

Made changes stemming from review feedback & discussion. Please take another look and let me know how this looks. Thanks!

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/313#issuecomment-1697990222

From cjplummer at openjdk.org  Tue Aug 29 19:25:12 2023
From: cjplummer at openjdk.org (Chris Plummer)
Date: Tue, 29 Aug 2023 19:25:12 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v2]
In-Reply-To: <ISuejqIr1InBH-hXEzgOt7iBXrhH4iv93glC3woOewQ=.3a7357d0-a911-4dfa-834b-2c5123915ce7@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <ISuejqIr1InBH-hXEzgOt7iBXrhH4iv93glC3woOewQ=.3a7357d0-a911-4dfa-834b-2c5123915ce7@github.com>
Message-ID: <SAf3vxbCOAXOTb9egSyc50Z7lt8fCMzE86nBLEHFyrw=.276f82b8-6889-4c50-93fe-5f69ef56f22f@github.com>

On Tue, 29 Aug 2023 09:12:51 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
>> 
>> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
>> 
>> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
>> 
>>    /**
>>      * Create ProcessBuilder using the java launcher from the jdk to
>>      * be tested.
>>      *
>>      * <p><b> Please observe that you likely should use
>>      * createTestJvm() instead of this method because createTestJvm()
>>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>>      * </b> and this method will not do that.
>>      *
>>      * @param command Arguments to pass to the java command.
>>      * @return The ProcessBuilder instance representing the java command.
>>      */
>> 
>> 
>> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
>> 
>> I have run tier 1 testing, and I have started more exhaustive testing.
>
> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
> 
>   copyright

> I kind of like that it is long and clumsy, that makes it harder to use...

...but it's used in 462 files. That implies it is commonly used. Are you suggesting nearly all those uses are incorrect and eventually should be converted to `createTestJvm()`?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1698003837

From wkemper at openjdk.org  Tue Aug 29 19:25:58 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 29 Aug 2023 19:25:58 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag
In-Reply-To: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
Message-ID: <cOHsADnyGlmSY5XY7hzAlR-9phsknrLkj0MZEXGQw_A=.939d29a2-f143-4203-9b2e-0530b7e59af6@github.com>

On Mon, 28 Aug 2023 19:48:58 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> Protect the card barrier code generation by a new global `ShenandoahCardBarrier`, which is disabled by default and enabled for GenShen (generational mode Shenandoah) only. If the user forces the barrier flag to a value inconsistent with mode, we exit with an appropriate error message.
> 
> The changes have been made for x86, aarch, and ppc. I have tested on x86 on a cloud host running jtreg1-4 and SPECjbb. A round of aarch64 and x86 pipeline tests have run (but ran into existing assertion that William is fixing). Will request PPC testing from partners able to test. RISC-V & S390 don't currently support generational mode Shenandoah.

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/313#pullrequestreview-1601131904

From kdnilsen at openjdk.org  Tue Aug 29 19:35:48 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 29 Aug 2023 19:35:48 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag
In-Reply-To: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
Message-ID: <uqX0-W1HsEPwQzSlSsm_wdREuPVr63w9jLZtEUnLZmA=.323e1f1e-e698-44da-8bea-8dfda25d00f0@github.com>

On Mon, 28 Aug 2023 19:48:58 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> Protect the card barrier code generation by a new global `ShenandoahCardBarrier`, which is disabled by default and enabled for GenShen (generational mode Shenandoah) only. If the user forces the barrier flag to a value inconsistent with mode, we exit with an appropriate error message.
> 
> The changes have been made for x86, aarch, and ppc. I have tested on x86 on a cloud host running jtreg1-4 and SPECjbb. A round of aarch64 and x86 pipeline tests have run (but ran into existing assertion that William is fixing). Will request PPC testing from partners able to test. RISC-V & S390 don't currently support generational mode Shenandoah.

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/313#pullrequestreview-1601147233

From ysr at openjdk.org  Tue Aug 29 19:58:46 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 29 Aug 2023 19:58:46 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag [v2]
In-Reply-To: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
Message-ID: <L7lSTE__nMaktMPXKnHEv9-kUAypM7VSDOv60DkY6_M=.1005f9cf-3a98-4911-adc5-0f51df49fe44@github.com>

> Protect the card barrier code generation by a new global `ShenandoahCardBarrier`, which is disabled by default and enabled for GenShen (generational mode Shenandoah) only. If the user forces the barrier flag to a value inconsistent with mode, we exit with an appropriate error message.
> 
> The changes have been made for x86, aarch, and ppc. I have tested on x86 on a cloud host running jtreg1-4 and SPECjbb. A round of aarch64 and x86 pipeline tests have run (but ran into existing assertion that William is fixing). Will request PPC testing from partners able to test. RISC-V & S390 don't currently support generational mode Shenandoah.

Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision:

 - Elaborate ShenandoahCardBarrier documentation a bit.
 - Merge branch 'master' into card_barrier
 - Check and exit if user specifies a setting of ShenandoahCardBarrier that
   conflicts with ShenandoahGCMode.
 - check that ShenandoahCardBarrier is off by default for most
   configurations. Exit with an error message if the user tries to change
   it.
 - Changes from prelim review by William & Kelvin.
 - Take care of some ppc changes
 - Merge branch 'master' into card_barrier
 - First cut of changes, checking in for testing:
   test at caller of post-write-barrier for ShenandoahCardBarrier, and
   assert in post-barrier/generation code that we are in generational mode.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/313/files
  - new: https://git.openjdk.org/shenandoah/pull/313/files/71bd4973..8fb05b2c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=313&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=313&range=00-01

  Stats: 81 lines in 12 files changed: 41 ins; 8 del; 32 mod
  Patch: https://git.openjdk.org/shenandoah/pull/313.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/313/head:pull/313

PR: https://git.openjdk.org/shenandoah/pull/313

From ysr at openjdk.org  Tue Aug 29 19:59:47 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 29 Aug 2023 19:59:47 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag [v2]
In-Reply-To: <wOJy0B4dSHkRgu99tFKW5Nzj9msjmQjizQryW3Q-iHI=.c0a3edde-97c3-4728-a4c2-f9d1bd19d501@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
 <DjRqqEAOlCF2tQAusnQ8DeQ3S6dV8i2kiu6sHhUle1U=.5c85f179-9f8c-428c-8de9-16f7c4f467c5@github.com>
 <zogp1slntTEG-27hztiOx7qSO471Qv2wH_5ga9xXZyQ=.6cbbdf13-0224-435b-bb11-88ff280c6baa@github.com>
 <ykz_vXcq1tq6rU9DWxu_1KAJHK8zaGyiuRHdihunFQA=.0c4385bb-6ab0-4c73-b124-2c2230adc65b@github.com>
 <-8I0PBBTvtRkXM8G4Z-FObmhbtyvvV45yDBJ5x_19FY=.2d119786-c5ef-4f57-b7ec-f6fd92ffb6f2@github.com>
 <0DPNN_ydQevNha9CxMkxtInJONoaKg25gJ3D0g1kGaw=.d6129e57-0325-4d3d-a125-025682e73b83@github.com>
 <PEeKLkhUTUmVcwC0r2e-Nw6xaFdOw-YootF6Yh7c-FE=.816fe6f8-06e8-4981-8b24-aad193bab93e@github.com>
 <02vFGiRQULtbyuZMY9u37n7REmIqSa3ioTtUdr4rMI0=.86f047a8-bb7c-4302-aab6-31ea0a1cc626@github.com>
 <15csLEjyZ0vVG-a-hFs5OdkjGflhkECB9hd6NZ15SPI=.691e547a-099b-40d7-ac72-03c52c47f625@github.com>
 <wOJy0B4dSHkRgu99tFKW5Nzj9msjmQjizQryW3Q-iHI=.c0a3edde-97c3-4728-a4c2-f9d1bd19d501@github.com>
Message-ID: <dGuRV1YtmKXWGiuxmTC5c3N1Sc23S9hTBb_Q6AOz8Z4=.45059e02-e1ab-4fe4-a39e-a1132f8a98d2@github.com>

On Tue, 29 Aug 2023 18:08:02 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> I updated with an attempt to address the comments above. Take a look and see how it looks.
>> 
>> I'll also take this out of draft form to a regular PR.
>
> So we'll now exit with an appropriate error message if the user changes `ShenandoahCardBarrier` to a value that conflicts with the `ShenandoahGCMode` they selected:
> 
> **1. No user selection of barrier flag:**
> 
> (23-08-29 17:54:47) <0> [~/workplace/ysr/genshen-ysr/shenandoah]  
> dev-dsk-ysr-2a-87502c94 % build/linux-x86_64-server-slowdebug/jdk/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:+PrintFlagsFinal -version | grep ShenandoahCardBarrier
>      bool ShenandoahCardBarrier                    = false                                  {diagnostic} {default}
> openjdk version "22-internal" 2024-03-19
> OpenJDK Runtime Environment (slowdebug build 22-internal-adhoc.ysr.shenandoah)
> OpenJDK 64-Bit Server VM (slowdebug build 22-internal-adhoc.ysr.shenandoah, mixed mode)
> 
> (23-08-29 18:11:09) <0> [~/workplace/ysr/genshen-ysr/shenandoah]  
> dev-dsk-ysr-2a-87502c94 % build/linux-x86_64-server-slowdebug/jdk/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:ShenandoahGCMode=generational -XX:+PrintFlagsFinal -version | grep ShenandoahCardBarrier
>      bool ShenandoahCardBarrier                    = true                                   {diagnostic} {default}
> openjdk version "22-internal" 2024-03-19
> OpenJDK Runtime Environment (slowdebug build 22-internal-adhoc.ysr.shenandoah)
> OpenJDK 64-Bit Server VM (slowdebug build 22-internal-adhoc.ysr.shenandoah, mixed mode)
> 
> 
> **2. User sets barrier flag to value that conflicts with gc mode:**
> 
> (23-08-29 18:11:36) <0> [~/workplace/ysr/genshen-ysr/shenandoah]  
> dev-dsk-ysr-2a-87502c94 % build/linux-x86_64-server-slowdebug/jdk/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:+ShenandoahCardBarrier -XX:+PrintFlagsFinal -version | grep ShenandoahCardBarrier                                  
> Error: GC mode needs -XX:-ShenandoahCardBarrier to work correctly
> 
> (23-08-29 18:11:55) <0> [~/workplace/ysr/genshen-ysr/shenandoah]  
> dev-dsk-ysr-2a-87502c94 % build/linux-x86_64-server-slowdebug/jdk/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:ShenandoahGCMode=generational -XX:-ShenandoahCardBarrier -XX:+PrintFlagsFinal -version | grep ShenandoahCardBarrier
> Error: GC mode needs -XX:+ShenandoahCardBarrier to work correctly
> 
> 
> **3. Use sets barrier flag to value consistent with gc mode:**
> 
> (23-08-29 18:12:15) <0> [~/workplace/ysr/genshen-ysr/shenandoah]  
> dev-dsk-ysr-2a-87502c94 % build/linux-x86_64-server-slowdebug/jdk/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -X...

Elaborated documentation so users aren't confused when they try and toggle settings to those that might conflict, and the JVM exits with a somewhat terse error message such as above.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/313#discussion_r1309292825

From kdnilsen at openjdk.org  Tue Aug 29 20:31:22 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 29 Aug 2023 20:31:22 GMT
Subject: RFR: 8315044: GenShen: Verifier detects clean card should be dirty
 [v2]
In-Reply-To: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
References: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
Message-ID: <sZd7LCiIqt67_1GuniZUT5Sg-g94bXCrmmXHGd_j7dY=.b3f5e73e-3b30-4e74-b50d-a13df3e45071@github.com>

> When a Reference object is newly discovered, it is placed onto the worker's thread-local discovered list.  This sometimes results in a reference from an old object to a young object, requiring that the remembered set card-table entry be marked as dirty.  This patch causes the marking to be performed.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Also update card table when moving discovered References to global list

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/314/files
  - new: https://git.openjdk.org/shenandoah/pull/314/files/dc95476c..8b898a56

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=314&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=314&range=00-01

  Stats: 22 lines in 1 file changed: 20 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/314.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/314/head:pull/314

PR: https://git.openjdk.org/shenandoah/pull/314

From wkemper at openjdk.org  Tue Aug 29 20:49:51 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 29 Aug 2023 20:49:51 GMT
Subject: git: openjdk/shenandoah: master: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity"
Message-ID: <cf125a87-5bf5-4147-a522-c5d43ed3c99c@openjdk.org>

Changeset: 72009f4f
Author:    William Kemper <wkemper at openjdk.org>
Date:      2023-08-29 20:49:06 +0000
URL:       https://git.openjdk.org/shenandoah/commit/72009f4f73afc4a4be0701554be1338380eeb816

8311843: GenShen: assertion failed "Old generation affiliated regions must be less than capacity"

Reviewed-by: kdnilsen

! src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp
! src/hotspot/share/gc/shenandoah/shenandoahFullGC.cpp
! src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp
! src/hotspot/share/gc/shenandoah/shenandoahGeneration.hpp
! src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp
! src/hotspot/share/gc/shenandoah/shenandoahYoungGeneration.cpp
! src/hotspot/share/gc/shenandoah/shenandoahYoungGeneration.hpp
! test/hotspot/jtreg/gc/stress/gclocker/TestGCLockerWithShenandoah.java


From wkemper at openjdk.org  Tue Aug 29 20:51:47 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 29 Aug 2023 20:51:47 GMT
Subject: Integrated: 8311843: GenShen: assertion failed
 "Old generation affiliated regions must be less than capacity"
In-Reply-To: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
References: <Yl9SdeOnyeG4Pl7zAmm2rC12eVFsw-k3ZvaKu6LN1cA=.64f8551f-3bd1-4d68-98be-4c3d28aad123@github.com>
Message-ID: <KZR2n9maGta3W2L2Kk84genk4KBPshl0LfAKcZaQkdM=.a3d9276a-c711-4803-83cc-97e618dde81c@github.com>

On Fri, 25 Aug 2023 17:45:44 GMT, William Kemper <wkemper at openjdk.org> wrote:

> There are a few fixes here:
> 
> During a Full GC, the compaction process is not constrained by the maximum capacity for young and old generations. We must, therefore, reestablish these constraints after the Full GC completes by _forcing_ the transfer of regions to old, if necessary.
> 
> When a region is "flipped" from the Mutator's free set to the Collector's, we cannot _force_ the transfer of the region because the young generation may not have any free regions.
> 
> A logic error in the loop which rebuilds the free set could prevent transferring free regions from the mutator to the collector reserves.
> 
> Some unused methods and fields were removed.

This pull request has now been integrated.

Changeset: 72009f4f
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/72009f4f73afc4a4be0701554be1338380eeb816
Stats:     110 lines in 8 files changed: 42 ins; 49 del; 19 mod

8311843: GenShen: assertion failed "Old generation affiliated regions must be less than capacity"

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/311

From kdnilsen at openjdk.org  Tue Aug 29 21:16:33 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 29 Aug 2023 21:16:33 GMT
Subject: RFR: 8315044: GenShen: Verifier detects clean card should be dirty
 [v3]
In-Reply-To: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
References: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
Message-ID: <4G_BaQ9TxjlGvbDoXpv9lhU_agKpE8gmblJxkuQ09uk=.d10a9938-a16e-4785-8530-c52f18d0d86a@github.com>

> When a Reference object is newly discovered, it is placed onto the worker's thread-local discovered list.  This sometimes results in a reference from an old object to a young object, requiring that the remembered set card-table entry be marked as dirty.  This patch causes the marking to be performed.

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:

 - Merge remote-tracking branch 'origin/master' into ref-processor-updates-remembered-set
 - Also update card table when moving discovered References to global list
 - Fix whitespace
 - Mark card as dirty if discovered reference list has interesting pointer

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/314/files
  - new: https://git.openjdk.org/shenandoah/pull/314/files/8b898a56..597a6e02

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=314&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=314&range=01-02

  Stats: 189 lines in 18 files changed: 82 ins; 57 del; 50 mod
  Patch: https://git.openjdk.org/shenandoah/pull/314.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/314/head:pull/314

PR: https://git.openjdk.org/shenandoah/pull/314

From ysr at openjdk.org  Tue Aug 29 23:00:08 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 29 Aug 2023 23:00:08 GMT
Subject: RFR: 8315247: GenShen: Condition calls to post-write barrier code
 generation by a flag [v3]
In-Reply-To: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
Message-ID: <wdXAE2rqrsg70ERoH8FzPRMNla1cbHwRF-5z2mztTGA=.9db2a6af-71f2-4a8c-8540-d3a0972ab017@github.com>

> Protect the card barrier code generation by a new global `ShenandoahCardBarrier`, which is disabled by default and enabled for GenShen (generational mode Shenandoah) only. If the user forces the barrier flag to a value inconsistent with mode, we exit with an appropriate error message.
> 
> The intention of the change is two-fold:
> 1. Make the card-barrier code in support of GenShen more idiomatic along the lines of existing barrier-gen code in Shenandoah
> 2. Reduce and simplify to the extent possible the impact of changes from GenShen on shared/legacy Shenandoah code, so as to make reviews/audits of shared/legacy code a bit easier
> 
> The changes have been made for x86, aarch, and ppc, and tested as follows:
> 1. I have tested on x86 on a cloud host running jtreg1-4 and SPECjbb.
> 3. A round of aarch64 and x86 pipeline tests have run (but ran into existing assertion that William is fixing).
> 4. Many thanks to Martin Doerr (SAP) for putting this through a "quick spin" on PPC
> 5. RISC-V & S390 don't currently support generational mode Shenandoah.

Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains ten additional commits since the last revision:

 - Merge branch 'master' into card_barrier
 - Elaborate ShenandoahCardBarrier documentation a bit.
 - Merge branch 'master' into card_barrier
 - Check and exit if user specifies a setting of ShenandoahCardBarrier that
   conflicts with ShenandoahGCMode.
 - check that ShenandoahCardBarrier is off by default for most
   configurations. Exit with an error message if the user tries to change
   it.
 - Changes from prelim review by William & Kelvin.
 - Take care of some ppc changes
 - Merge branch 'master' into card_barrier
 - First cut of changes, checking in for testing:
   test at caller of post-write-barrier for ShenandoahCardBarrier, and
   assert in post-barrier/generation code that we are in generational mode.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/313/files
  - new: https://git.openjdk.org/shenandoah/pull/313/files/8fb05b2c..61d264dc

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=313&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=313&range=01-02

  Stats: 110 lines in 8 files changed: 42 ins; 49 del; 19 mod
  Patch: https://git.openjdk.org/shenandoah/pull/313.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/313/head:pull/313

PR: https://git.openjdk.org/shenandoah/pull/313

From ysr at openjdk.org  Tue Aug 29 23:16:42 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 29 Aug 2023 23:16:42 GMT
Subject: Integrated: 8315247: GenShen: Condition calls to post-write barrier
 code generation by a flag
In-Reply-To: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
References: <NbmsByAOAxGj-gBlrc7CzHhkmaXUyEcky2DxgqTk3iM=.08b5b912-c099-449f-ae65-4cccbbf3117d@github.com>
Message-ID: <Cc8Job4mQvToUHn4RWTuuOiyfwFT9KpzL5zaHL0woyI=.396f93e9-d87f-446e-93fa-d2c2eed6a5ef@github.com>

On Mon, 28 Aug 2023 19:48:58 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> Protect the card barrier code generation by a new global `ShenandoahCardBarrier`, which is disabled by default and enabled for GenShen (generational mode Shenandoah) only. If the user forces the barrier flag to a value inconsistent with mode, we exit with an appropriate error message.
> 
> The intention of the change is two-fold:
> 1. Make the card-barrier code in support of GenShen more idiomatic along the lines of existing barrier-gen code in Shenandoah
> 2. Reduce and simplify to the extent possible the impact of changes from GenShen on shared/legacy Shenandoah code, so as to make reviews/audits of shared/legacy code a bit easier
> 
> The changes have been made for x86, aarch, and ppc, and tested as follows:
> 1. I have tested on x86 on a cloud host running jtreg1-4 and SPECjbb.
> 3. A round of aarch64 and x86 pipeline tests have run (but ran into existing assertion that William is fixing).
> 4. Many thanks to Martin Doerr (SAP) for putting this through a "quick spin" on PPC
> 5. RISC-V & S390 don't currently support generational mode Shenandoah.

This pull request has now been integrated.

Changeset: ade7796c
Author:    Y. Srinivas Ramakrishna <ysr at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/ade7796c9401c7348ebd771a11c88f31649e7509
Stats:     106 lines in 15 files changed: 48 ins; 21 del; 37 mod

8315247: GenShen: Condition calls to post-write barrier code generation by a flag

Reviewed-by: wkemper, kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/313

From wkemper at openjdk.org  Tue Aug 29 23:58:47 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 29 Aug 2023 23:58:47 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <Ozp7JRr3-Le9XiiTb5K79Ou-0uCf544j7_QaNQLwtiM=.78094627-c9fc-46c3-8838-8595a6559547@github.com>

Merges tag jdk-22+12

-------------

Commit messages:
 - 8314932: G1: Fix -Wconversion warnings for simple cases inside g1 folder
 - 8314573: G1: Heap resizing at Remark does not take existing eden regions into account
 - 8314753: Remove support for @beaninfo, @ToDo, @since.unbundled, and @Note
 - 8309463: IGV: Dynamic graph layout algorithm
 - 8314476: TestJstatdPortAndServer.java failed with "java.rmi.NoSuchObjectException: no such object in table"
 - 8314569: (fs) Improve normalization of UnixPath for input with trailing slashes
 - 8315087: G1: Use uint for G1 flags indicating percentage
 - 8315070: RISC-V: Clean up platform dependent inline headers
 - 8315073: Zero build on macOS fails after JDK-8303852
 - 8309697: [TESTBUG] Remove "@requires vm.flagless" from jtreg vectorization tests
 - ... and 47 more: https://git.openjdk.org/shenandoah/compare/68815d54...e22762c0

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah/pull/315/files
  Stats: 7257 lines in 249 files changed: 5301 ins; 777 del; 1179 mod
  Patch: https://git.openjdk.org/shenandoah/pull/315.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/315/head:pull/315

PR: https://git.openjdk.org/shenandoah/pull/315

From wkemper at openjdk.org  Wed Aug 30 00:01:40 2023
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 30 Aug 2023 00:01:40 GMT
Subject: RFR: Merge openjdk/jdk:master
In-Reply-To: <Ozp7JRr3-Le9XiiTb5K79Ou-0uCf544j7_QaNQLwtiM=.78094627-c9fc-46c3-8838-8595a6559547@github.com>
References: <Ozp7JRr3-Le9XiiTb5K79Ou-0uCf544j7_QaNQLwtiM=.78094627-c9fc-46c3-8838-8595a6559547@github.com>
Message-ID: <4G8g8zdsSeNiAbA0g1-c-q75vcghSkA6TPgR-HBNG8g=.5ab68d22-d869-4d14-a852-cb872c4ec621@github.com>

On Tue, 29 Aug 2023 23:53:12 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-22+12

This was a test for automation. I don't want to merge this PR.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/315#issuecomment-1698296534

From wkemper at openjdk.org  Wed Aug 30 00:01:40 2023
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 30 Aug 2023 00:01:40 GMT
Subject: Withdrawn: Merge openjdk/jdk:master
In-Reply-To: <Ozp7JRr3-Le9XiiTb5K79Ou-0uCf544j7_QaNQLwtiM=.78094627-c9fc-46c3-8838-8595a6559547@github.com>
References: <Ozp7JRr3-Le9XiiTb5K79Ou-0uCf544j7_QaNQLwtiM=.78094627-c9fc-46c3-8838-8595a6559547@github.com>
Message-ID: <SfV81mGiNmXpbia5_2KAbOmBjHi82DumiqKDSFVvw_Q=.db616243-d51c-41bb-86bc-fa3ccb91f31c@github.com>

On Tue, 29 Aug 2023 23:53:12 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-22+12

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/shenandoah/pull/315

From cslucas at openjdk.org  Wed Aug 30 03:20:38 2023
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Wed, 30 Aug 2023 03:20:38 GMT
Subject: RFR: JDK-8315279: Factor 'basic_plus_adr' out of PhaseMacroExpand and
 delete make_load/store
Message-ID: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>

I believe the factory methods for AddPNode should be in the AddPNode class. The make_load / make_store methods in PhaseMacroExpand can be refactored to instead just use the "make" methods from Load/Store classes.

Tested with tier1-3.

-------------

Commit messages:
 - Refactoring on PhaseMacroExpand

Changes: https://git.openjdk.org/jdk/pull/15480/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=15480&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8315279
  Stats: 92 lines in 11 files changed: 37 ins; 32 del; 23 mod
  Patch: https://git.openjdk.org/jdk/pull/15480.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15480/head:pull/15480

PR: https://git.openjdk.org/jdk/pull/15480

From dholmes at openjdk.org  Wed Aug 30 07:22:10 2023
From: dholmes at openjdk.org (David Holmes)
Date: Wed, 30 Aug 2023 07:22:10 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v2]
In-Reply-To: <5AjL4PbmGc-9jQdiCK1jd4LDdC-B7NgJ_dwIZW1U-aA=.0ad22596-a1f9-4e87-946f-dceedf0a37e5@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <ISuejqIr1InBH-hXEzgOt7iBXrhH4iv93glC3woOewQ=.3a7357d0-a911-4dfa-834b-2c5123915ce7@github.com>
 <pnAnjv8HLt1sykmB9jRamtiVRoTbmwQi9xGuaSnCi10=.34427fd4-7b88-436e-946b-04c7364a1853@github.com>
 <5AjL4PbmGc-9jQdiCK1jd4LDdC-B7NgJ_dwIZW1U-aA=.0ad22596-a1f9-4e87-946f-dceedf0a37e5@github.com>
Message-ID: <LC17h9qCmoHNn4rcN1kgdrvaYOrkTkPXn_BlKTXyN0s=.e1d9cbdc-8991-455b-be06-166476c6436f@github.com>

On Tue, 29 Aug 2023 16:45:12 GMT, Roger Riggs <rriggs at openjdk.org> wrote:

>> I don't think this is the best change across so many files.
>> It gives a very ugly name to a common test function and affects a very large number of tests.
>
>> @RogerRiggs If it is only the name you want changed, maybe you can offer a better name?
> @lkorinth 
> 
> Sorry for the too short comment; I wanted to make sure it wasn't integrated before I could look at it more closely.
> Neither the bug report or the PR describe the problem and its ramifications, only the solution.
> Can you elaborate on the conditions that lead to this. (and include them in the bug report).

@RogerRiggs , @plummercj , please see the additional discussion in the parent bug: https://bugs.openjdk.org/browse/JDK-8314823

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1698629345

From lkorinth at openjdk.org  Wed Aug 30 09:23:55 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Wed, 30 Aug 2023 09:23:55 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v3]
In-Reply-To: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
Message-ID: <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>

> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
> 
> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
> 
> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
> 
>    /**
>      * Create ProcessBuilder using the java launcher from the jdk to
>      * be tested.
>      *
>      * <p><b> Please observe that you likely should use
>      * createTestJvm() instead of this method because createTestJvm()
>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>      * </b> and this method will not do that.
>      *
>      * @param command Arguments to pass to the java command.
>      * @return The ProcessBuilder instance representing the java command.
>      */
> 
> 
> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
> 
> I have run tier 1 testing, and I have started more exhaustive testing.

Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:

  fix static import

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15452/files
  - new: https://git.openjdk.org/jdk/pull/15452/files/f3418c80..27da7150

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15452&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15452&range=01-02

  Stats: 2 lines in 1 file changed: 0 ins; 1 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/15452.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15452/head:pull/15452

PR: https://git.openjdk.org/jdk/pull/15452

From msheppar at openjdk.org  Wed Aug 30 11:34:12 2023
From: msheppar at openjdk.org (Mark Sheppard)
Date: Wed, 30 Aug 2023 11:34:12 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v3]
In-Reply-To: <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>
Message-ID: <wVUv1MM9_YCMellQujMDEORehLIrVzlkDgDgTiLM3f8=.0693ba3b-61bb-4af2-b680-dabb4ec099aa@github.com>

On Wed, 30 Aug 2023 09:23:55 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
>> 
>> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
>> 
>> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
>> 
>>    /**
>>      * Create ProcessBuilder using the java launcher from the jdk to
>>      * be tested.
>>      *
>>      * <p><b> Please observe that you likely should use
>>      * createTestJvm() instead of this method because createTestJvm()
>>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>>      * </b> and this method will not do that.
>>      *
>>      * @param command Arguments to pass to the java command.
>>      * @return The ProcessBuilder instance representing the java command.
>>      */
>> 
>> 
>> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
>> 
>> I have run tier 1 testing, and I have started more exhaustive testing.
>
> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix static import

I don't think  a name change is required. The method is createJavaProcessBuilder with a "list" of argurments and a builder is returned. As such, there is no inference, in the name, that test args will be propagated. Adding the additional java doc description should be sufficient to dispell any misconceptions. 
The createTestJvm is a misnomer as it returns a ProcessBulder rather than a Process, which is what I would expected from createTestJvm, without looking at its signature.

So you could create a single createJavaProcessBuilder with add an additional parameter boolean addTestOpts e.g.
createJavaProcessBuilder(List<String> command, boolean addTestOpts) { ... }

createProcessBuilderIgnoreJavaTestOpt(cmdArgs)  maps to createJavaProcessBuilder(cmdArgs, false)

createTestJvm(cmdArgs) maps to createJavaProcessBuilder(cmdArgs, true)

But this is a lot more work.

alternatively change createTestJvm to createTestJavaProcessBuilder  or createJavaProcessBuilderAddTestOpts

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1698985460

From rriggs at openjdk.org  Wed Aug 30 15:10:13 2023
From: rriggs at openjdk.org (Roger Riggs)
Date: Wed, 30 Aug 2023 15:10:13 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v3]
In-Reply-To: <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>
Message-ID: <QhskrdnUTvPoXQkFzEXv6cXGiyHNCtGZ4NiHqz5SO_s=.7d87a087-bf3a-4ae8-8aa4-999220667db7@github.com>

On Wed, 30 Aug 2023 09:23:55 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
>> 
>> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
>> 
>> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
>> 
>>    /**
>>      * Create ProcessBuilder using the java launcher from the jdk to
>>      * be tested.
>>      *
>>      * <p><b> Please observe that you likely should use
>>      * createTestJvm() instead of this method because createTestJvm()
>>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>>      * </b> and this method will not do that.
>>      *
>>      * @param command Arguments to pass to the java command.
>>      * @return The ProcessBuilder instance representing the java command.
>>      */
>> 
>> 
>> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
>> 
>> I have run tier 1 testing, and I have started more exhaustive testing.
>
> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix static import

In the context of the goal is to declaratively identify tests that do or do not benefit from additional test flags renaming the `createjavaProcessBuilder` method does not further that goal.

The method name and javadoc of `createjavaProcessBuilder` do not imply that the test options are consulted or used; it only says it creates a ProcessBuilder, and does not promise or document more than that.  The javadoc should probably describe the use of the the three properties that modify the way that the java is launched.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1699362435

From lkorinth at openjdk.org  Wed Aug 30 17:13:53 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Wed, 30 Aug 2023 17:13:53 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v3]
In-Reply-To: <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>
Message-ID: <3xqrbByJLBJTDN1-C_aG0wwe6PFy09-tMyL0TwEn4Gc=.88dbe2a2-ad5d-4545-ab30-4a9e5312641f@github.com>

On Wed, 30 Aug 2023 09:23:55 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
>> 
>> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
>> 
>> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
>> 
>>    /**
>>      * Create ProcessBuilder using the java launcher from the jdk to
>>      * be tested.
>>      *
>>      * <p><b> Please observe that you likely should use
>>      * createTestJvm() instead of this method because createTestJvm()
>>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>>      * </b> and this method will not do that.
>>      *
>>      * @param command Arguments to pass to the java command.
>>      * @return The ProcessBuilder instance representing the java command.
>>      */
>> 
>> 
>> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
>> 
>> I have run tier 1 testing, and I have started more exhaustive testing.
>
> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix static import

I have some ideas that I will work on that might lead to a new proposal.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1699542111

From stefank at openjdk.org  Wed Aug 30 17:40:33 2023
From: stefank at openjdk.org (Stefan Karlsson)
Date: Wed, 30 Aug 2023 17:40:33 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v3]
In-Reply-To: <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>
Message-ID: <VAAsfW_V1GEfQezmwBQwvXa3svN5cMzanUtpk4W4kv8=.6ff00b4d-01f4-4607-a7be-b17ae9669ae3@github.com>

On Wed, 30 Aug 2023 09:23:55 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
>> 
>> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
>> 
>> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
>> 
>>    /**
>>      * Create ProcessBuilder using the java launcher from the jdk to
>>      * be tested.
>>      *
>>      * <p><b> Please observe that you likely should use
>>      * createTestJvm() instead of this method because createTestJvm()
>>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>>      * </b> and this method will not do that.
>>      *
>>      * @param command Arguments to pass to the java command.
>>      * @return The ProcessBuilder instance representing the java command.
>>      */
>> 
>> 
>> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
>> 
>> I have run tier 1 testing, and I have started more exhaustive testing.
>
> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix static import

>From talking to other HotSpot devs it is quite clear that the different names lead to mistakes when writing (copy-n-pasting) tests, so I'm happy that we try to figure out some way to make it more obvious when we prepend the extra test options and when we don't.

I agree with @msheppar that `createTestJvm` isn't a good name and I wouldn't be opposed to changing that name instead of `createJavaProcessBuilder`. However, if we do rename that function I strongly urge us to also rename the corresponding `executeTestJvm` function.

I also think it is too obscure that functions with *Test* in the name prepend the extra test options, and those without *Test* don't, so I'd like to get rid of that convention. 

I wouldn't be opposed to a change that:
* Keeps the `createJavaProcessBuilder` name
* Renames `createTestJvm` to `createJavaProcessBuilderPrependTestOpts`
* Renames `executeTestJvm` to `executeJavaPrependTestOpts`
* Removes `createTestJava`
* Removes `executeTestJava`

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1699588853

From kdnilsen at openjdk.org  Wed Aug 30 18:51:46 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 30 Aug 2023 18:51:46 GMT
Subject: RFR: 8315044: GenShen: Verifier detects clean card should be dirty
 [v4]
In-Reply-To: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
References: <nWIm_qCq8vzSn61xjzuEyKVhd3u1pfjJeIuSnAkY0z8=.d2ba5e15-29fd-4714-a4e1-cf9445454f4e@github.com>
Message-ID: <bRtf0tMn5iUcB2UkfVwyrL6JoEf0BzQjy3mvM5vh3So=.329ff53b-0234-407f-a2d2-c2c618506b87@github.com>

> When a Reference object is newly discovered, it is placed onto the worker's thread-local discovered list.  This sometimes results in a reference from an old object to a young object, requiring that the remembered set card-table entry be marked as dirty.  This patch causes the marking to be performed.

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:

 - Replace is_generational with ShenandoahCardBarrier test
 - Merge remote-tracking branch 'origin/master' into ref-processor-updates-remembered-set
 - Merge remote-tracking branch 'origin/master' into ref-processor-updates-remembered-set
 - Also update card table when moving discovered References to global list
 - Fix whitespace
 - Mark card as dirty if discovered reference list has interesting pointer

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/314/files
  - new: https://git.openjdk.org/shenandoah/pull/314/files/597a6e02..f375d571

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=314&range=03
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=314&range=02-03

  Stats: 108 lines in 16 files changed: 48 ins; 21 del; 39 mod
  Patch: https://git.openjdk.org/shenandoah/pull/314.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/314/head:pull/314

PR: https://git.openjdk.org/shenandoah/pull/314

From kvn at openjdk.org  Wed Aug 30 21:13:40 2023
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Wed, 30 Aug 2023 21:13:40 GMT
Subject: RFR: JDK-8315279: Factor 'basic_plus_adr' out of PhaseMacroExpand
 and delete make_load/store
In-Reply-To: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
References: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
Message-ID: <fI4NzMkhC99E-Vlbi94he9ORBpfR19zF9d_0SzH6Z14=.beed141e-b3ea-4b2e-a76f-f34c9141545c@github.com>

On Wed, 30 Aug 2023 03:13:08 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

> I believe the factory methods for AddPNode should be in the AddPNode class. The make_load / make_store methods in PhaseMacroExpand can be refactored to instead just use the "make" methods from Load/Store classes.
> 
> Tested with tier1-3.

Good cleanup.
My only suggestion is to keep `AddPNode::make()` and `PhaseValues::makecon()` in header files .hpp - they are small and no need to put to .cpp files.

-------------

PR Review: https://git.openjdk.org/jdk/pull/15480#pullrequestreview-1603482700

From wkemper at openjdk.org  Wed Aug 30 23:40:19 2023
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 30 Aug 2023 23:40:19 GMT
Subject: RFR: 8314935: Shenandoah: Unable to throw OOME on back-to-back Full
 GCs
Message-ID: <n1hNJjSUcrwUmhgxO_1PbICgSNNWirib5tn4TbyoHGM=.c7e6fd10-32a9-4221-81af-446c9ae39d39@github.com>

Retry allocations until at least one full gc is complete, do not spin unnecessarily when gc is not making progress.

-------------

Commit messages:
 - Retry allocations until at least one full gc is complete

Changes: https://git.openjdk.org/jdk/pull/15500/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=15500&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8314935
  Stats: 22 lines in 2 files changed: 5 ins; 11 del; 6 mod
  Patch: https://git.openjdk.org/jdk/pull/15500.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15500/head:pull/15500

PR: https://git.openjdk.org/jdk/pull/15500

From cslucas at openjdk.org  Thu Aug 31 00:58:44 2023
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Thu, 31 Aug 2023 00:58:44 GMT
Subject: RFR: JDK-8315279: Factor 'basic_plus_adr' out of PhaseMacroExpand
 and delete make_load/store [v2]
In-Reply-To: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
References: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
Message-ID: <f1BXEuQxV9I2KYvh-3eVOGA4LYvzhF4Y1J2BkyHciCM=.ef7a1252-c3eb-4b6c-b904-cf1529c90bba@github.com>

> I believe the factory methods for AddPNode should be in the AddPNode class. The make_load / make_store methods in PhaseMacroExpand can be refactored to instead just use the "make" methods from Load/Store classes.
> 
> Tested with tier1-3.

Cesar Soares Lucas has updated the pull request incrementally with one additional commit since the last revision:

  move definitions to hpp files

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15480/files
  - new: https://git.openjdk.org/jdk/pull/15480/files/776b202c..5d988b01

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15480&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15480&range=00-01

  Stats: 42 lines in 4 files changed: 13 ins; 24 del; 5 mod
  Patch: https://git.openjdk.org/jdk/pull/15480.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15480/head:pull/15480

PR: https://git.openjdk.org/jdk/pull/15480

From kvn at openjdk.org  Thu Aug 31 02:55:01 2023
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Thu, 31 Aug 2023 02:55:01 GMT
Subject: RFR: JDK-8315279: Factor 'basic_plus_adr' out of PhaseMacroExpand
 and delete make_load/store [v2]
In-Reply-To: <f1BXEuQxV9I2KYvh-3eVOGA4LYvzhF4Y1J2BkyHciCM=.ef7a1252-c3eb-4b6c-b904-cf1529c90bba@github.com>
References: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
 <f1BXEuQxV9I2KYvh-3eVOGA4LYvzhF4Y1J2BkyHciCM=.ef7a1252-c3eb-4b6c-b904-cf1529c90bba@github.com>
Message-ID: <Jdna4S1oOvq7UcuFX_drXD1sdQbnMcKVGXoAXPSKZ3U=.5131c161-ab5a-4a2e-a6f7-5baa85030043@github.com>

On Thu, 31 Aug 2023 00:58:44 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

>> I believe the factory methods for AddPNode should be in the AddPNode class. The make_load / make_store methods in PhaseMacroExpand can be refactored to instead just use the "make" methods from Load/Store classes.
>> 
>> Tested with tier1-3.
>
> Cesar Soares Lucas has updated the pull request incrementally with one additional commit since the last revision:
> 
>   move definitions to hpp files

Good. You need second review.

-------------

Marked as reviewed by kvn (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/15480#pullrequestreview-1603781408

From dholmes at openjdk.org  Thu Aug 31 05:48:04 2023
From: dholmes at openjdk.org (David Holmes)
Date: Thu, 31 Aug 2023 05:48:04 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v3]
In-Reply-To: <wVUv1MM9_YCMellQujMDEORehLIrVzlkDgDgTiLM3f8=.0693ba3b-61bb-4af2-b680-dabb4ec099aa@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>
 <wVUv1MM9_YCMellQujMDEORehLIrVzlkDgDgTiLM3f8=.0693ba3b-61bb-4af2-b680-dabb4ec099aa@github.com>
Message-ID: <sXekzx-YhGFu-1-ONgzSYf1puehFR4QlJ2OaiCT_q98=.b2781fc0-3201-4a24-af79-143b94260469@github.com>

On Wed, 30 Aug 2023 11:31:20 GMT, Mark Sheppard <msheppar at openjdk.org> wrote:

> So you could create a single createJavaProcessBuilder with add an additional parameter boolean addTestOpts e.g.
createJavaProcessBuilder(List command, boolean addTestOpts) { ... }

@msheppar that is actually where we started, and it was then split into two differently named methods to "make it clear" which one included the test opts without having to remember the name of the parameter that the true/false argument was bound to.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1700402301

From alanb at openjdk.org  Thu Aug 31 06:51:00 2023
From: alanb at openjdk.org (Alan Bateman)
Date: Thu, 31 Aug 2023 06:51:00 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v3]
In-Reply-To: <VAAsfW_V1GEfQezmwBQwvXa3svN5cMzanUtpk4W4kv8=.6ff00b4d-01f4-4607-a7be-b17ae9669ae3@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>
 <VAAsfW_V1GEfQezmwBQwvXa3svN5cMzanUtpk4W4kv8=.6ff00b4d-01f4-4607-a7be-b17ae9669ae3@github.com>
Message-ID: <RdenHruw4C5Yd3ZAu4G-QXF00NV99-fdZUipQwrPJ4M=.4329acd6-8a3a-4aed-9384-35cd8e2a6246@github.com>

On Wed, 30 Aug 2023 17:38:01 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

> I wouldn't be opposed to a change that:
> 
> * Keeps the `createJavaProcessBuilder` name
> * Renames `createTestJvm` to `createJavaProcessBuilderPrependTestOpts`
> * Renames `executeTestJvm` to `executeJavaPrependTestOpts`
> * Removes `createTestJava`
> * Removes `executeTestJava`

The good thing with this is that it very explicit at the use-sites where the test opts are prepended. Also moving from "Jvm" to "Java" is probably right as it executes the "java" launcher, the options are a mix of java launcher options (that the VM doesn't know about) and VM options.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1700459277

From chagedorn at openjdk.org  Thu Aug 31 07:09:01 2023
From: chagedorn at openjdk.org (Christian Hagedorn)
Date: Thu, 31 Aug 2023 07:09:01 GMT
Subject: RFR: JDK-8315279: Factor 'basic_plus_adr' out of PhaseMacroExpand
 and delete make_load/store [v2]
In-Reply-To: <f1BXEuQxV9I2KYvh-3eVOGA4LYvzhF4Y1J2BkyHciCM=.ef7a1252-c3eb-4b6c-b904-cf1529c90bba@github.com>
References: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
 <f1BXEuQxV9I2KYvh-3eVOGA4LYvzhF4Y1J2BkyHciCM=.ef7a1252-c3eb-4b6c-b904-cf1529c90bba@github.com>
Message-ID: <A9AKP9FBOGXHzk38palxlKVwvF5QVLlxdL5v4mbsFOc=.7c01c304-31a6-495d-9017-df2e15cd1541@github.com>

On Thu, 31 Aug 2023 00:58:44 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

>> I believe the factory methods for AddPNode should be in the AddPNode class. The make_load / make_store methods in PhaseMacroExpand can be refactored to instead just use the "make" methods from Load/Store classes.
>> 
>> Tested with tier1-3.
>
> Cesar Soares Lucas has updated the pull request incrementally with one additional commit since the last revision:
> 
>   move definitions to hpp files

Otherwise, the clean-up looks good.

src/hotspot/share/opto/addnode.hpp line 177:

> 175: 
> 176:   // Helper methods roughly modeled after GraphKit:
> 177:   static Node* make(PhaseIterGVN* igvn, Node* base, Node* ptr, Node* offset) {

I suggest to pass `igvn` as reference here (i.e. `PhaseIterGVN&`) and below. Then you can avoid to use `&igvn()` in the code and just pass it as reference directly.

src/hotspot/share/opto/macro.cpp line 1683:

> 1681: 
> 1682:   Node* memadr = AddPNode::make(&_igvn, object, oopDesc::mark_offset_in_bytes());
> 1683:   rawmem = transform_later(StoreNode::make(_igvn, control, rawmem, memadr, nullptr, mark_node, TypeX_X->basic_type(), MemNode::unordered));

You could still think about keeping `make_load/store()` and just replace `basic_plus_adr()` with `AddPNode::make()` there which might make it easier to read the code here. But both is fine and I leave it up to you to decide.

-------------

Marked as reviewed by chagedorn (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/15480#pullrequestreview-1604052768
PR Review Comment: https://git.openjdk.org/jdk/pull/15480#discussion_r1311168043
PR Review Comment: https://git.openjdk.org/jdk/pull/15480#discussion_r1311173320

From wkemper at openjdk.org  Thu Aug 31 08:21:59 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 31 Aug 2023 08:21:59 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <EirN69aUv_pebFpI1EoV2kpR6hEWqDcocWN6CIXkYF8=.ce8854de-f1c6-4d7c-b4e0-5e9483d4e493@github.com>

Merges tag jdk-22+13

-------------

Commit messages:
 - 8314651: G1: Fix -Wconversion warnings in static fields of HeapRegion
 - 8315206: RISC-V: hwprobe query is_set return wrong value
 - 8314024: SIGSEGV in PhaseIdealLoop::build_loop_late_post_work due to bad immediate dominator info
 - 8315213: java/lang/ProcessHandle/TreeTest.java test enhance output of children
 - 8315214: Do not run sun/tools/jhsdb tests concurrently
 - 8314056: Remove runtime platform check from frem/drem
 - 8312428: PKCS11 tests fail with NSS 3.91
 - 8189685: need PerfMemory class update and a volatile_static_field support in VMStructs
 - 8314932: G1: Fix -Wconversion warnings for simple cases inside g1 folder
 - 8314573: G1: Heap resizing at Remark does not take existing eden regions into account
 - ... and 55 more: https://git.openjdk.org/shenandoah/compare/68815d54...cb3f9680

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah/pull/316/files
  Stats: 7855 lines in 278 files changed: 5615 ins; 927 del; 1313 mod
  Patch: https://git.openjdk.org/shenandoah/pull/316.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/316/head:pull/316

PR: https://git.openjdk.org/shenandoah/pull/316

From msheppar at openjdk.org  Thu Aug 31 10:11:02 2023
From: msheppar at openjdk.org (Mark Sheppard)
Date: Thu, 31 Aug 2023 10:11:02 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v3]
In-Reply-To: <sXekzx-YhGFu-1-ONgzSYf1puehFR4QlJ2OaiCT_q98=.b2781fc0-3201-4a24-af79-143b94260469@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>
 <wVUv1MM9_YCMellQujMDEORehLIrVzlkDgDgTiLM3f8=.0693ba3b-61bb-4af2-b680-dabb4ec099aa@github.com>
 <sXekzx-YhGFu-1-ONgzSYf1puehFR4QlJ2OaiCT_q98=.b2781fc0-3201-4a24-af79-143b94260469@github.com>
Message-ID: <vCLRZwJrzYryyifx7o1JmbNDvnCt9bPwJKPd3DqOR30=.447ed116-dbae-4192-8300-ac7eaba309b3@github.com>

On Thu, 31 Aug 2023 05:45:27 GMT, David Holmes <dholmes at openjdk.org> wrote:

> > So you could create a single createJavaProcessBuilder with add an additional parameter boolean addTestOpts e.g.
> > createJavaProcessBuilder(List command, boolean addTestOpts) { ... }
> 
> @msheppar that is actually where we started, and it was then split into two differently named methods to "make it clear" which one included the test opts without having to remember the name of the parameter that the true/false argument was bound to.

cheers David  thanks for the clarification on background

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1700749844

From cslucas at openjdk.org  Thu Aug 31 15:37:02 2023
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Thu, 31 Aug 2023 15:37:02 GMT
Subject: RFR: JDK-8315279: Factor 'basic_plus_adr' out of PhaseMacroExpand
 and delete make_load/store [v2]
In-Reply-To: <A9AKP9FBOGXHzk38palxlKVwvF5QVLlxdL5v4mbsFOc=.7c01c304-31a6-495d-9017-df2e15cd1541@github.com>
References: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
 <f1BXEuQxV9I2KYvh-3eVOGA4LYvzhF4Y1J2BkyHciCM=.ef7a1252-c3eb-4b6c-b904-cf1529c90bba@github.com>
 <A9AKP9FBOGXHzk38palxlKVwvF5QVLlxdL5v4mbsFOc=.7c01c304-31a6-495d-9017-df2e15cd1541@github.com>
Message-ID: <zOyUqu-4_8mXspIYde9p8bBon679HmqMjvHtj5v1zBc=.758e27c3-5b06-4444-8362-7465d71abbc2@github.com>

On Thu, 31 Aug 2023 06:47:10 GMT, Christian Hagedorn <chagedorn at openjdk.org> wrote:

>> Cesar Soares Lucas has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   move definitions to hpp files
>
> src/hotspot/share/opto/addnode.hpp line 177:
> 
>> 175: 
>> 176:   // Helper methods roughly modeled after GraphKit:
>> 177:   static Node* make(PhaseIterGVN* igvn, Node* base, Node* ptr, Node* offset) {
> 
> I suggest to pass `igvn` as reference here (i.e. `PhaseIterGVN&`) and below. Then you can avoid to use `&igvn()` in the code and just pass it as reference directly.

That's a good idea. Thanks!

> src/hotspot/share/opto/macro.cpp line 1683:
> 
>> 1681: 
>> 1682:   Node* memadr = AddPNode::make(&_igvn, object, oopDesc::mark_offset_in_bytes());
>> 1683:   rawmem = transform_later(StoreNode::make(_igvn, control, rawmem, memadr, nullptr, mark_node, TypeX_X->basic_type(), MemNode::unordered));
> 
> You could still think about keeping `make_load/store()` and just replace `basic_plus_adr()` with `AddPNode::make()` there which might make it easier to read the code here. But both is fine and I leave it up to you to decide.

Yeah, I actually tried that but the reuse of make_load/store wasn't justifiable without a much bigger refactor. I'll leave that refactor for another PR.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15480#discussion_r1311821426
PR Review Comment: https://git.openjdk.org/jdk/pull/15480#discussion_r1311823254

From cslucas at openjdk.org  Thu Aug 31 16:05:32 2023
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Thu, 31 Aug 2023 16:05:32 GMT
Subject: RFR: JDK-8315279: Factor 'basic_plus_adr' out of PhaseMacroExpand
 and delete make_load/store [v3]
In-Reply-To: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
References: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
Message-ID: <DH6qrky59jpys42HMOYlRosCntq04ZVBimtOy5bx914=.77795531-5495-44c0-b9e8-72a80cfdad70@github.com>

> I believe the factory methods for AddPNode should be in the AddPNode class. The make_load / make_store methods in PhaseMacroExpand can be refactored to instead just use the "make" methods from Load/Store classes.
> 
> Tested with tier1-3.

Cesar Soares Lucas has updated the pull request incrementally with one additional commit since the last revision:

  Use reference instead of pointer.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15480/files
  - new: https://git.openjdk.org/jdk/pull/15480/files/5d988b01..c4205846

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15480&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15480&range=01-02

  Stats: 30 lines in 7 files changed: 0 ins; 0 del; 30 mod
  Patch: https://git.openjdk.org/jdk/pull/15480.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15480/head:pull/15480

PR: https://git.openjdk.org/jdk/pull/15480

From wkemper at openjdk.org  Thu Aug 31 16:17:41 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 31 Aug 2023 16:17:41 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <EirN69aUv_pebFpI1EoV2kpR6hEWqDcocWN6CIXkYF8=.ce8854de-f1c6-4d7c-b4e0-5e9483d4e493@github.com>
References: <EirN69aUv_pebFpI1EoV2kpR6hEWqDcocWN6CIXkYF8=.ce8854de-f1c6-4d7c-b4e0-5e9483d4e493@github.com>
Message-ID: <PLCEEPCmSZr-jvenRCaClyWwNwtp5aWPWOYoHpUjso4=.155d2cce-df68-4351-889b-707f8c01a726@github.com>

On Thu, 31 Aug 2023 08:16:08 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-22+13

This pull request has now been integrated.

Changeset: 08518465
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/085184652f43e68cc91c057b9066d701773100df
Stats:     7855 lines in 278 files changed: 5615 ins; 927 del; 1313 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/316

From wkemper at openjdk.org  Thu Aug 31 16:17:41 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 31 Aug 2023 16:17:41 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <EirN69aUv_pebFpI1EoV2kpR6hEWqDcocWN6CIXkYF8=.ce8854de-f1c6-4d7c-b4e0-5e9483d4e493@github.com>
References: <EirN69aUv_pebFpI1EoV2kpR6hEWqDcocWN6CIXkYF8=.ce8854de-f1c6-4d7c-b4e0-5e9483d4e493@github.com>
Message-ID: <BYlMFqZuvhGbMONDmgSgLV23TvGhdYk1VdFiH4meUEY=.41caa135-e6ff-46b4-a491-ae14162bfb13@github.com>

> Merges tag jdk-22+13

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/316/files
  - new: https://git.openjdk.org/shenandoah/pull/316/files/cb3f9680..cb3f9680

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=316&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=316&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/316.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/316/head:pull/316

PR: https://git.openjdk.org/shenandoah/pull/316

From vlivanov at openjdk.org  Thu Aug 31 16:32:03 2023
From: vlivanov at openjdk.org (Vladimir Ivanov)
Date: Thu, 31 Aug 2023 16:32:03 GMT
Subject: RFR: JDK-8315279: Factor 'basic_plus_adr' out of PhaseMacroExpand
 and delete make_load/store [v3]
In-Reply-To: <DH6qrky59jpys42HMOYlRosCntq04ZVBimtOy5bx914=.77795531-5495-44c0-b9e8-72a80cfdad70@github.com>
References: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
 <DH6qrky59jpys42HMOYlRosCntq04ZVBimtOy5bx914=.77795531-5495-44c0-b9e8-72a80cfdad70@github.com>
Message-ID: <Rq1kaveO4WYpSxSAF-5xxzRpfEX4y7mDbHSw8ZbS2rM=.d7f41924-d4ce-440c-87ca-ef9697609ab8@github.com>

On Thu, 31 Aug 2023 16:05:32 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

>> I believe the factory methods for AddPNode should be in the AddPNode class. The make_load / make_store methods in PhaseMacroExpand can be refactored to instead just use the "make" methods from Load/Store classes.
>> 
>> Tested with tier1-3.
>
> Cesar Soares Lucas has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Use reference instead of pointer.

src/hotspot/share/opto/addnode.hpp line 181:

> 179:   }
> 180: 
> 181:   static Node* make(PhaseIterGVN& igvn, Node* base, int offset) {

The overloads do look confusing now. When seeing a call to a factory method, I expect to see all inputs to be separately initialized. With `basic_plus_adr` the intention was clearer.

I'm not against the refactoring, but it feels like it moves the code in the wrong direction. The root problem is that we can't use `GraphKit `during macro expansion, so we duplicated functionality there over time. (`PhaseIdealLoop` is another victim.)

A better way would be to use `GraphKit` directly. And when we discussed it last time [1], Roland mentioned it was already implemented in Valhalla repo. 

[1] https://mail.openjdk.org/pipermail/hotspot-compiler-dev/2020-August/039612.html

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15480#discussion_r1311898349

From cslucas at openjdk.org  Thu Aug 31 16:45:09 2023
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Thu, 31 Aug 2023 16:45:09 GMT
Subject: RFR: JDK-8315279: Factor 'basic_plus_adr' out of PhaseMacroExpand
 and delete make_load/store [v3]
In-Reply-To: <Rq1kaveO4WYpSxSAF-5xxzRpfEX4y7mDbHSw8ZbS2rM=.d7f41924-d4ce-440c-87ca-ef9697609ab8@github.com>
References: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
 <DH6qrky59jpys42HMOYlRosCntq04ZVBimtOy5bx914=.77795531-5495-44c0-b9e8-72a80cfdad70@github.com>
 <Rq1kaveO4WYpSxSAF-5xxzRpfEX4y7mDbHSw8ZbS2rM=.d7f41924-d4ce-440c-87ca-ef9697609ab8@github.com>
Message-ID: <kuG_lT0ADg561vFba6lSI3JvDVAELq0f1q1RSsIwLKQ=.24cdd64e-223c-40fd-8968-ceefa314905f@github.com>

On Thu, 31 Aug 2023 16:28:55 GMT, Vladimir Ivanov <vlivanov at openjdk.org> wrote:

> I expect to see all inputs to be separately initialized.

I'm not sure what you mean by this part. Can you elaborate a little here?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15480#discussion_r1311912885

From vlivanov at openjdk.org  Thu Aug 31 17:06:04 2023
From: vlivanov at openjdk.org (Vladimir Ivanov)
Date: Thu, 31 Aug 2023 17:06:04 GMT
Subject: RFR: JDK-8315279: Factor 'basic_plus_adr' out of PhaseMacroExpand
 and delete make_load/store [v3]
In-Reply-To: <kuG_lT0ADg561vFba6lSI3JvDVAELq0f1q1RSsIwLKQ=.24cdd64e-223c-40fd-8968-ceefa314905f@github.com>
References: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
 <DH6qrky59jpys42HMOYlRosCntq04ZVBimtOy5bx914=.77795531-5495-44c0-b9e8-72a80cfdad70@github.com>
 <Rq1kaveO4WYpSxSAF-5xxzRpfEX4y7mDbHSw8ZbS2rM=.d7f41924-d4ce-440c-87ca-ef9697609ab8@github.com>
 <kuG_lT0ADg561vFba6lSI3JvDVAELq0f1q1RSsIwLKQ=.24cdd64e-223c-40fd-8968-ceefa314905f@github.com>
Message-ID: <3_ThxcuU3e_hPvWi4lJBfXsyG4Ky_eyyifbkZ2izlKQ=.0070b59a-31ae-4ede-9625-a9e4bf3b7a16@github.com>

On Thu, 31 Aug 2023 16:42:12 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

>> src/hotspot/share/opto/addnode.hpp line 181:
>> 
>>> 179:   }
>>> 180: 
>>> 181:   static Node* make(PhaseIterGVN& igvn, Node* base, int offset) {
>> 
>> The overloads do look confusing now. When seeing a call to a factory method, I expect to see all inputs to be separately initialized. With `basic_plus_adr` the intention was clearer.
>> 
>> I'm not against the refactoring, but it feels like it moves the code in the wrong direction. The root problem is that we can't use `GraphKit `during macro expansion, so we duplicated functionality there over time. (`PhaseIdealLoop` is another victim.)
>> 
>> A better way would be to use `GraphKit` directly. And when we discussed it last time [1], Roland mentioned it was already implemented in Valhalla repo. 
>> 
>> [1] https://mail.openjdk.org/pipermail/hotspot-compiler-dev/2020-August/039612.html
>
>> I expect to see all inputs to be separately initialized.
> 
> I'm not sure what you mean by this part. Can you elaborate a little here?

static Node* make(PhaseIterGVN& igvn, Node* base, Node* ptr, Node* offset) {
    return igvn.register_new_node_with_optimizer(new AddPNode(base, ptr, offset));
  }

vs

  static Node* make(PhaseIterGVN& igvn, Node* base, int offset) {
    return (offset == 0) ? base : make(igvn, base, base, (Node*) igvn.makecon(offset));
  }


IMO latter case adds more confusion than improves readability. Why not  unconditionally call `AddPNode::make(base, base, offset)` instead?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15480#discussion_r1311936070

From kdnilsen at openjdk.org  Thu Aug 31 18:17:00 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 31 Aug 2023 18:17:00 GMT
Subject: RFR: 8314935: Shenandoah: Unable to throw OOME on back-to-back
 Full GCs
In-Reply-To: <n1hNJjSUcrwUmhgxO_1PbICgSNNWirib5tn4TbyoHGM=.c7e6fd10-32a9-4221-81af-446c9ae39d39@github.com>
References: <n1hNJjSUcrwUmhgxO_1PbICgSNNWirib5tn4TbyoHGM=.c7e6fd10-32a9-4221-81af-446c9ae39d39@github.com>
Message-ID: <TY1obxv3PLaRBgpfHonOcJRENURa-XJYUrHzTyYqYlg=.11f4f01c-9cac-4fd5-acbc-65a06db29950@github.com>

On Wed, 30 Aug 2023 23:33:57 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Retry allocations until at least one full gc is complete, do not spin unnecessarily when gc is not making progress.

Thanks.  Looks good to me.

-------------

Marked as reviewed by kdnilsen (no project role).

PR Review: https://git.openjdk.org/jdk/pull/15500#pullrequestreview-1605408982

From cslucas at openjdk.org  Thu Aug 31 20:36:04 2023
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Thu, 31 Aug 2023 20:36:04 GMT
Subject: RFR: JDK-8315279: Factor 'basic_plus_adr' out of PhaseMacroExpand
 and delete make_load/store [v3]
In-Reply-To: <3_ThxcuU3e_hPvWi4lJBfXsyG4Ky_eyyifbkZ2izlKQ=.0070b59a-31ae-4ede-9625-a9e4bf3b7a16@github.com>
References: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
 <DH6qrky59jpys42HMOYlRosCntq04ZVBimtOy5bx914=.77795531-5495-44c0-b9e8-72a80cfdad70@github.com>
 <Rq1kaveO4WYpSxSAF-5xxzRpfEX4y7mDbHSw8ZbS2rM=.d7f41924-d4ce-440c-87ca-ef9697609ab8@github.com>
 <kuG_lT0ADg561vFba6lSI3JvDVAELq0f1q1RSsIwLKQ=.24cdd64e-223c-40fd-8968-ceefa314905f@github.com>
 <3_ThxcuU3e_hPvWi4lJBfXsyG4Ky_eyyifbkZ2izlKQ=.0070b59a-31ae-4ede-9625-a9e4bf3b7a16@github.com>
Message-ID: <jqZc_VR69436ChXcaqwMMgYzIMYtiOFVLVG2QoQUAJc=.0ee1ff3b-0cfa-4594-84e8-508cde6703b5@github.com>

On Thu, 31 Aug 2023 17:02:49 GMT, Vladimir Ivanov <vlivanov at openjdk.org> wrote:

>>> I expect to see all inputs to be separately initialized.
>> 
>> I'm not sure what you mean by this part. Can you elaborate a little here?
>
> static Node* make(PhaseIterGVN& igvn, Node* base, Node* ptr, Node* offset) {
>     return igvn.register_new_node_with_optimizer(new AddPNode(base, ptr, offset));
>   }
> 
> vs
> 
>   static Node* make(PhaseIterGVN& igvn, Node* base, int offset) {
>     return (offset == 0) ? base : make(igvn, base, base, (Node*) igvn.makecon(offset));
>   }
> 
> 
> IMO latter case adds more confusion than improves readability. Why not  unconditionally call `AddPNode::make(base, base, offset)` instead?

Thanks for clarifying @iwanowww . I think I see your point now. My original intent was to just move these methods out of PhaseMacroExpand and not much else. 

I'm going to do some more refactoring and patch all users of these make methods to just use this single method: `static Node* make(PhaseIterGVN& igvn, Node* base, Node* ptr, Node* offset)`. What do you think?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15480#discussion_r1312197579

From vlivanov at openjdk.org  Thu Aug 31 20:59:09 2023
From: vlivanov at openjdk.org (Vladimir Ivanov)
Date: Thu, 31 Aug 2023 20:59:09 GMT
Subject: RFR: JDK-8315279: Factor 'basic_plus_adr' out of PhaseMacroExpand
 and delete make_load/store [v3]
In-Reply-To: <jqZc_VR69436ChXcaqwMMgYzIMYtiOFVLVG2QoQUAJc=.0ee1ff3b-0cfa-4594-84e8-508cde6703b5@github.com>
References: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
 <DH6qrky59jpys42HMOYlRosCntq04ZVBimtOy5bx914=.77795531-5495-44c0-b9e8-72a80cfdad70@github.com>
 <Rq1kaveO4WYpSxSAF-5xxzRpfEX4y7mDbHSw8ZbS2rM=.d7f41924-d4ce-440c-87ca-ef9697609ab8@github.com>
 <kuG_lT0ADg561vFba6lSI3JvDVAELq0f1q1RSsIwLKQ=.24cdd64e-223c-40fd-8968-ceefa314905f@github.com>
 <3_ThxcuU3e_hPvWi4lJBfXsyG4Ky_eyyifbkZ2izlKQ=.0070b59a-31ae-4ede-9625-a9e4bf3b7a16@github.com>
 <jqZc_VR69436ChXcaqwMMgYzIMYtiOFVLVG2QoQUAJc=.0ee1ff3b-0cfa-4594-84e8-508cde6703b5@github.com>
Message-ID: <GxzTnW0VOuKOfPP9vUaguO8cRjrhKDXLcduc0uHElG4=.34679448-aefc-4cfe-bf90-c0febfe617f5@github.com>

On Thu, 31 Aug 2023 20:32:56 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

>> static Node* make(PhaseIterGVN& igvn, Node* base, Node* ptr, Node* offset) {
>>     return igvn.register_new_node_with_optimizer(new AddPNode(base, ptr, offset));
>>   }
>> 
>> vs
>> 
>>   static Node* make(PhaseIterGVN& igvn, Node* base, int offset) {
>>     return (offset == 0) ? base : make(igvn, base, base, (Node*) igvn.makecon(offset));
>>   }
>> 
>> 
>> IMO latter case adds more confusion than improves readability. Why not  unconditionally call `AddPNode::make(base, base, offset)` instead?
>
> Thanks for clarifying @iwanowww . I think I see your point now. My original intent was to just move these methods out of PhaseMacroExpand and not much else. 
> 
> I'm going to do some more refactoring and patch all users of these make methods to just use this single method: `static Node* make(PhaseIterGVN& igvn, Node* base, Node* ptr, Node* offset)`. What do you think?

Sounds good. 

But in the future I'd like to see `PhaseMacroExpand` and `PhaseIdealLoop` migrated to `GraphKit` instead.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15480#discussion_r1312206706

From wkemper at openjdk.org  Thu Aug 31 21:02:56 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 31 Aug 2023 21:02:56 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <Ozp7JRr3-Le9XiiTb5K79Ou-0uCf544j7_QaNQLwtiM=.78094627-c9fc-46c3-8838-8595a6559547@github.com>
References: <Ozp7JRr3-Le9XiiTb5K79Ou-0uCf544j7_QaNQLwtiM=.78094627-c9fc-46c3-8838-8595a6559547@github.com>
Message-ID: <Xl7fc2naC5f81XhKKoDivxnHOEm5Z8T5xDPkV_9PlLo=.7a3d30e1-bfe0-4fe8-8bbc-faac24f810e5@github.com>

> Merges tag jdk-22+12

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/315/files
  - new: https://git.openjdk.org/shenandoah/pull/315/files/e22762c0..e22762c0

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=315&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=315&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/315.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/315/head:pull/315

PR: https://git.openjdk.org/shenandoah/pull/315

From cslucas at openjdk.org  Thu Aug 31 21:22:51 2023
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Thu, 31 Aug 2023 21:22:51 GMT
Subject: RFR: JDK-8315279: Factor 'basic_plus_adr' out of PhaseMacroExpand
 and delete make_load/store [v3]
In-Reply-To: <GxzTnW0VOuKOfPP9vUaguO8cRjrhKDXLcduc0uHElG4=.34679448-aefc-4cfe-bf90-c0febfe617f5@github.com>
References: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
 <DH6qrky59jpys42HMOYlRosCntq04ZVBimtOy5bx914=.77795531-5495-44c0-b9e8-72a80cfdad70@github.com>
 <Rq1kaveO4WYpSxSAF-5xxzRpfEX4y7mDbHSw8ZbS2rM=.d7f41924-d4ce-440c-87ca-ef9697609ab8@github.com>
 <kuG_lT0ADg561vFba6lSI3JvDVAELq0f1q1RSsIwLKQ=.24cdd64e-223c-40fd-8968-ceefa314905f@github.com>
 <3_ThxcuU3e_hPvWi4lJBfXsyG4Ky_eyyifbkZ2izlKQ=.0070b59a-31ae-4ede-9625-a9e4bf3b7a16@github.com>
 <jqZc_VR69436ChXcaqwMMgYzIMYtiOFVLVG2QoQUAJc=.0ee1ff3b-0cfa-4594-84e8-508cde6703b5@github.com>
 <GxzTnW0VOuKOfPP9vUaguO8cRjrhKDXLcduc0uHElG4=.34679448-aefc-4cfe-bf90-c0febfe617f5@github.com>
Message-ID: <YRWZmGa3yPyfa6j5zK0FeA1U1PvvFzAnC07q94eFTKo=.be5e6113-1187-48ff-9fa2-230a78df3167@github.com>

On Thu, 31 Aug 2023 20:42:39 GMT, Vladimir Ivanov <vlivanov at openjdk.org> wrote:

>> Thanks for clarifying @iwanowww . I think I see your point now. My original intent was to just move these methods out of PhaseMacroExpand and not much else. 
>> 
>> I'm going to do some more refactoring and patch all users of these make methods to just use this single method: `static Node* make(PhaseIterGVN& igvn, Node* base, Node* ptr, Node* offset)`. What do you think?
>
> Sounds good. 
> 
> But in the future I'd like to see `PhaseMacroExpand` and `PhaseIdealLoop` migrated to `GraphKit` instead.

Actually, I think it's silly to create a method to just do a "return ... new AddP":

static Node* make(PhaseIterGVN& igvn, Node* base, Node* ptr, Node* offset) {
    return igvn.register_new_node_with_optimizer(new AddPNode(base, ptr, offset));
}

I can just replace the uses of this `make` with a `new AddP...`.

> But in the future I'd like to see PhaseMacroExpand and PhaseIdealLoop migrated to GraphKit instead.

Roger that.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15480#discussion_r1312299405

From cslucas at openjdk.org  Thu Aug 31 21:56:07 2023
From: cslucas at openjdk.org (Cesar Soares Lucas)
Date: Thu, 31 Aug 2023 21:56:07 GMT
Subject: RFR: JDK-8315279: Factor 'basic_plus_adr' out of PhaseMacroExpand
 and delete make_load/store [v4]
In-Reply-To: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
References: <Fzbl7iQ9VXhegm-5nXsJ2fEhWd39KujBeb4WzRJQEMQ=.0ff89d71-5bbd-4015-9fa7-2b572251f39b@github.com>
Message-ID: <sNp-9t72PQRHZL_AjB8NqE8Ji5aJNubHqvtBRjBSunI=.d488ae9f-ff3e-453e-8851-1fd08dbd2526@github.com>

> I believe the factory methods for AddPNode should be in the AddPNode class. The make_load / make_store methods in PhaseMacroExpand can be refactored to instead just use the "make" methods from Load/Store classes.
> 
> Tested with tier1-3.

Cesar Soares Lucas has updated the pull request incrementally with one additional commit since the last revision:

  Convert uses of AddPNode::make to new AddPNode

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15480/files
  - new: https://git.openjdk.org/jdk/pull/15480/files/c4205846..943acfff

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15480&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15480&range=02-03

  Stats: 42 lines in 8 files changed: 1 ins; 18 del; 23 mod
  Patch: https://git.openjdk.org/jdk/pull/15480.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15480/head:pull/15480

PR: https://git.openjdk.org/jdk/pull/15480