From github.com+80801020+wattsun110 at openjdk.java.net  Sat May  1 11:38:08 2021
From: github.com+80801020+wattsun110 at openjdk.java.net (wattsun110)
Date: Sat, 1 May 2021 11:38:08 GMT
Subject: RFR: 8266018: Shenandoah: fix an incorrect assert
Message-ID: <-b7LvBwZVRgeB0LNtWPCcs7lm8Rac6l7tcZSvPTVu54=.87356765-2b10-48bb-bca4-cc103a6ec611@github.com>

May I get reviews for this one-line change which fixes an incorrect assert?

thanks
watt

-------------

Commit messages:
 - 8266018: Shenandoah: fix an incorrect assert

Changes: https://git.openjdk.java.net/jdk/pull/3710/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3710&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8266018
  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3710.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3710/head:pull/3710

PR: https://git.openjdk.java.net/jdk/pull/3710

From duke at openjdk.java.net  Sat May  1 14:40:29 2021
From: duke at openjdk.java.net (duke)
Date: Sat, 1 May 2021 14:40:29 GMT
Subject: git: openjdk/shenandoah: master: Fix jcheck conf to point to
 Shenandoah project, instead of JDK project
Message-ID: <5359ec0c-b9a0-4e51-97c3-001a414525aa@openjdk.java.net>

Changeset: 816c1a61
Author:    Roman Kennke <rkennke at redhat.com>
Date:      2021-05-01 16:39:41 +0000
URL:       https://git.openjdk.java.net/shenandoah/commit/816c1a611e6bd245904da56b67cb6432cb32b569

Fix jcheck conf to point to Shenandoah project, instead of JDK project

! .jcheck/conf


From rkennke at openjdk.java.net  Mon May  3 08:17:54 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 3 May 2021 08:17:54 GMT
Subject: RFR: 8266018: Shenandoah: fix an incorrect assert
In-Reply-To: <-b7LvBwZVRgeB0LNtWPCcs7lm8Rac6l7tcZSvPTVu54=.87356765-2b10-48bb-bca4-cc103a6ec611@github.com>
References: <-b7LvBwZVRgeB0LNtWPCcs7lm8Rac6l7tcZSvPTVu54=.87356765-2b10-48bb-bca4-cc103a6ec611@github.com>
Message-ID: <RRXmJFAoXPd16hRDpEYDbmxD6ytS2_8R2PE0c8D_S_c=.599d1860-0700-4f2e-87b8-64023440518b@github.com>

On Tue, 27 Apr 2021 03:10:52 GMT, wattsun110 <github.com+80801020+wattsun110 at openjdk.org> wrote:

> May I get reviews for this one-line change which fixes an incorrect assert?
> 
> thanks
> watt

Looks good! Thanks for the contribution!

-------------

Marked as reviewed by rkennke (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/3710

From rkennke at openjdk.java.net  Mon May  3 10:55:52 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 3 May 2021 10:55:52 GMT
Subject: RFR: 8234446: Post-CMS workgroup hierarchy cleanup [v3]
In-Reply-To: <IbwV1WM6_DtCQWpGYsj52hO1ZcXAiMfLaFxgZh2A7nk=.c0aa7545-60b9-4a4c-89b2-874b930975fb@github.com>
References: <nDYB1awSv8VTQx6_xcTguPcqC0-c3LzFHE2qDPOBq48=.52d85827-7783-4415-9280-dbe3b067e88f@github.com>
 <IbwV1WM6_DtCQWpGYsj52hO1ZcXAiMfLaFxgZh2A7nk=.c0aa7545-60b9-4a4c-89b2-874b930975fb@github.com>
Message-ID: <aZjUDMOcTWSGtCOEz-H4E2DnEacdzglOf0BroWoKyjI=.cbd0651f-993d-409d-8427-a12e1138a48f@github.com>

On Thu, 29 Apr 2021 09:07:18 GMT, Albert Mingkun Yang <ayang at openjdk.org> wrote:

>> Collapse workgroup hierarchy and some general cleanup.
>> 
>> Tested; hotspot_gc
>
> Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision:
> 
>   undo indentation change

Looks good to me. Thank you!

-------------

Marked as reviewed by rkennke (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/3663

From rkennke at openjdk.java.net  Mon May  3 10:59:49 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 3 May 2021 10:59:49 GMT
Subject: RFR: 8266083: Shenandoah: Consolidate dedup/no dedup oop closures
In-Reply-To: <ubBlgKxnMm5bigU1FcbEHheyQkNiibZ0et8H_lQ6n-Q=.50bb3ade-98d5-4acc-8b83-8ee6432bd337@github.com>
References: <ubBlgKxnMm5bigU1FcbEHheyQkNiibZ0et8H_lQ6n-Q=.50bb3ade-98d5-4acc-8b83-8ee6432bd337@github.com>
Message-ID: <IsqLR6uSzS8RN6hy9n7vKz4GSB3XBK12VXDk285fdFk=.675957ad-9575-43e2-9212-bc1f55ce9db8@github.com>

On Tue, 27 Apr 2021 18:37:17 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> We can use template to consolidate dedup/no dedup oop closures.

Looks good to me, thank you!

-------------

Marked as reviewed by rkennke (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/3724

From rkennke at openjdk.java.net  Mon May  3 11:06:53 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 3 May 2021 11:06:53 GMT
Subject: RFR: 8261527: Record page size used for underlying mapping in
 ReservedSpace
In-Reply-To: <Enix4NQ5pCJxvz_enr5SWYpgqKq_9YV91PApY0TDmrY=.34f394d4-ffe1-4b93-b3b4-500eb2c915f0@github.com>
References: <Enix4NQ5pCJxvz_enr5SWYpgqKq_9YV91PApY0TDmrY=.34f394d4-ffe1-4b93-b3b4-500eb2c915f0@github.com>
Message-ID: <or7XKtcDsSvxZqpYbT4FWw43rDzwkRfzz4UbBa0ejso=.ed162eb4-a97a-48f3-b83c-2d5f737c495d@github.com>

On Thu, 29 Apr 2021 20:38:19 GMT, Stefan Johansson <sjohanss at openjdk.org> wrote:

> Please review this change to use the actual page size rather than `bool large` when creating and initializing `ReservedSpace`. This allows us to then get rid of the helper `ReservedSpace::actual_reserved_page_size()` and instead use the "real" page size for logging and other use.
> 
> The PR consist of two commits, one that changes `ReservedSpace` to use and store a page size and one that removes `actual_reserved_page_size()` and instead uses `ReservedSpace::page_size()`. Hopefully this might help ease the review a bit. 
> 
> There are two changes in behavior:
> * In `JfrVirtualMemorySegment::initialize()` we now always pass down `os::vm_page_size()` which means never use large pages. This differs from the old condition where large pages were enabled if transparent huge pages were enabled. This change has been discussed with the JFR team and they will later investigate how to re-enable large page use.
> * In `ReservedSpace::reserve()`, if a file is used, the page size is always set to `os::vm_page_size()`. So when logging the page_size for such mapping it will not report a large page size. This was incorrectly done in the past when using `ReservedSpace::actual_reserved_page_size()` to figure out the page size. This makes the `runtime/os/TestTracePageSizes.java` test pass even if run with `-XX:AllocateHeapAt=/tmp`.
> 
> **Testing**
> Mach5 tier1-4 and a lot of local testing.

Looks good to me.

-------------

Marked as reviewed by rkennke (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/3802

From shade at openjdk.java.net  Mon May  3 11:41:27 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 3 May 2021 11:41:27 GMT
Subject: RFR: Fix build failures due to undeclared ShenandoahOldHeuristics
Message-ID: <ujEKNfDRLsHFbW3k8n5IGxtZRCgRHjbqdNTPvCnS0vk=.afe79681-c74b-4c97-a58b-bed8233b4f9f@github.com>

Some configs, notably Zero, does not build due to:


* For target hotspot_variant-zero_libjvm_objs_shenandoahAdaptiveHeuristics.o:
In file included from /home/buildbot/worker/build-shenandoah-jdkX-linux/build/src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.hpp:28:0,
                 from /home/buildbot/worker/build-shenandoah-jdkX-linux/build/src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp:27:

-------------

Commit messages:
 - Fix build failures due to undeclared ShenandoahOldHeuristics

Changes: https://git.openjdk.java.net/shenandoah/pull/34/files
 Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=34&range=00
  Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/34.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/34/head:pull/34

PR: https://git.openjdk.java.net/shenandoah/pull/34

From ayang at openjdk.java.net  Mon May  3 11:49:49 2021
From: ayang at openjdk.java.net (Albert Mingkun Yang)
Date: Mon, 3 May 2021 11:49:49 GMT
Subject: RFR: 8234446: Post-CMS workgroup hierarchy cleanup [v3]
In-Reply-To: <IbwV1WM6_DtCQWpGYsj52hO1ZcXAiMfLaFxgZh2A7nk=.c0aa7545-60b9-4a4c-89b2-874b930975fb@github.com>
References: <nDYB1awSv8VTQx6_xcTguPcqC0-c3LzFHE2qDPOBq48=.52d85827-7783-4415-9280-dbe3b067e88f@github.com>
 <IbwV1WM6_DtCQWpGYsj52hO1ZcXAiMfLaFxgZh2A7nk=.c0aa7545-60b9-4a4c-89b2-874b930975fb@github.com>
Message-ID: <bzhWZduXl2heBSVV2JFh18zGM_GbsIXG5dZowkR6OfQ=.964f6cef-e7f1-4eb9-b976-b80b05b95c46@github.com>

On Thu, 29 Apr 2021 09:07:18 GMT, Albert Mingkun Yang <ayang at openjdk.org> wrote:

>> Collapse workgroup hierarchy and some general cleanup.
>> 
>> Tested; hotspot_gc
>
> Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision:
> 
>   undo indentation change

Thanks for the review.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3663

From ayang at openjdk.java.net  Mon May  3 11:49:50 2021
From: ayang at openjdk.java.net (Albert Mingkun Yang)
Date: Mon, 3 May 2021 11:49:50 GMT
Subject: Integrated: 8234446: Post-CMS workgroup hierarchy cleanup
In-Reply-To: <nDYB1awSv8VTQx6_xcTguPcqC0-c3LzFHE2qDPOBq48=.52d85827-7783-4415-9280-dbe3b067e88f@github.com>
References: <nDYB1awSv8VTQx6_xcTguPcqC0-c3LzFHE2qDPOBq48=.52d85827-7783-4415-9280-dbe3b067e88f@github.com>
Message-ID: <vsRFck4EfofJHkyP5fXne8dLmG1YZVq2fVejEIz9_os=.fe8f7000-fc1d-403f-becb-e9483f5845e2@github.com>

On Fri, 23 Apr 2021 20:06:03 GMT, Albert Mingkun Yang <ayang at openjdk.org> wrote:

> Collapse workgroup hierarchy and some general cleanup.
> 
> Tested; hotspot_gc

This pull request has now been integrated.

Changeset: 80941f47
Author:    Albert Mingkun Yang <ayang at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/80941f475f7f3bd479f1ab75287f0ffe7935ad05
Stats:     293 lines in 5 files changed: 86 ins; 166 del; 41 mod

8234446: Post-CMS workgroup hierarchy cleanup

Reviewed-by: tschatzl, rkennke

-------------

PR: https://git.openjdk.java.net/jdk/pull/3663

From rkennke at openjdk.java.net  Mon May  3 12:23:26 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 3 May 2021 12:23:26 GMT
Subject: RFR: Fix build failures due to undeclared ShenandoahOldHeuristics
In-Reply-To: <ujEKNfDRLsHFbW3k8n5IGxtZRCgRHjbqdNTPvCnS0vk=.afe79681-c74b-4c97-a58b-bed8233b4f9f@github.com>
References: <ujEKNfDRLsHFbW3k8n5IGxtZRCgRHjbqdNTPvCnS0vk=.afe79681-c74b-4c97-a58b-bed8233b4f9f@github.com>
Message-ID: <A_8IVm0RXGjOLjaynkt_ROvTlgLf1fHiKmteQ1a_qi4=.dacd6114-9e62-41a2-bc3a-ba85b92a2383@github.com>

On Mon, 3 May 2021 11:35:58 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> Some configs, notably Zero, does not build due to:
> 
> 
> * For target hotspot_variant-zero_libjvm_objs_shenandoahAdaptiveHeuristics.o:
> In file included from /home/buildbot/worker/build-shenandoah-jdkX-linux/build/src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.hpp:28:0,
>                  from /home/buildbot/worker/build-shenandoah-jdkX-linux/build/src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp:27:

Looks good & trivial. Thanks!

-------------

Marked as reviewed by rkennke (Lead).

PR: https://git.openjdk.java.net/shenandoah/pull/34

From shade at openjdk.java.net  Mon May  3 12:23:27 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 3 May 2021 12:23:27 GMT
Subject: git: openjdk/shenandoah: master: Fix build failures due to undeclared
 ShenandoahOldHeuristics
Message-ID: <98e57e12-af83-47b5-b00b-9928e53a812f@openjdk.org>

Changeset: 9fc8ac20
Author:    Aleksey Shipilev <shade at openjdk.org>
Date:      2021-05-03 12:23:01 +0000
URL:       https://git.openjdk.java.net/shenandoah/commit/9fc8ac201745b21eab1ccf36a18e90f5f0a56af7

Fix build failures due to undeclared ShenandoahOldHeuristics

Reviewed-by: rkennke, zgu

! src/hotspot/share/gc/shenandoah/heuristics/shenandoahHeuristics.hpp


From zgu at openjdk.java.net  Mon May  3 12:23:26 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 3 May 2021 12:23:26 GMT
Subject: RFR: Fix build failures due to undeclared ShenandoahOldHeuristics
In-Reply-To: <ujEKNfDRLsHFbW3k8n5IGxtZRCgRHjbqdNTPvCnS0vk=.afe79681-c74b-4c97-a58b-bed8233b4f9f@github.com>
References: <ujEKNfDRLsHFbW3k8n5IGxtZRCgRHjbqdNTPvCnS0vk=.afe79681-c74b-4c97-a58b-bed8233b4f9f@github.com>
Message-ID: <vVOnxDcSmNNRGoXUEVAALrDXHM2Fc8wtfWB7YVMbCXo=.bc56f41e-95a6-4954-b898-20ed1252a059@github.com>

On Mon, 3 May 2021 11:35:58 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> Some configs, notably Zero, does not build due to:
> 
> 
> * For target hotspot_variant-zero_libjvm_objs_shenandoahAdaptiveHeuristics.o:
> In file included from /home/buildbot/worker/build-shenandoah-jdkX-linux/build/src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.hpp:28:0,
>                  from /home/buildbot/worker/build-shenandoah-jdkX-linux/build/src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp:27:

Looks good.

-------------

Marked as reviewed by zgu (Committer).

PR: https://git.openjdk.java.net/shenandoah/pull/34

From shade at openjdk.java.net  Mon May  3 12:26:14 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 3 May 2021 12:26:14 GMT
Subject: Integrated: Fix build failures due to undeclared
 ShenandoahOldHeuristics
In-Reply-To: <ujEKNfDRLsHFbW3k8n5IGxtZRCgRHjbqdNTPvCnS0vk=.afe79681-c74b-4c97-a58b-bed8233b4f9f@github.com>
References: <ujEKNfDRLsHFbW3k8n5IGxtZRCgRHjbqdNTPvCnS0vk=.afe79681-c74b-4c97-a58b-bed8233b4f9f@github.com>
Message-ID: <FMiOkxd2FvpKvab_RxfSe6dkdHsbyi34rm_4w14fnBU=.c827a3ce-05cc-484f-8ba6-81a8baec38cc@github.com>

On Mon, 3 May 2021 11:35:58 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> Some configs, notably Zero, does not build due to:
> 
> 
> * For target hotspot_variant-zero_libjvm_objs_shenandoahAdaptiveHeuristics.o:
> In file included from /home/buildbot/worker/build-shenandoah-jdkX-linux/build/src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.hpp:28:0,
>                  from /home/buildbot/worker/build-shenandoah-jdkX-linux/build/src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp:27:

This pull request has now been integrated.

Changeset: 9fc8ac20
Author:    Aleksey Shipilev <shade at openjdk.org>
URL:       https://git.openjdk.java.net/shenandoah/commit/9fc8ac201745b21eab1ccf36a18e90f5f0a56af7
Stats:     1 line in 1 file changed: 1 ins; 0 del; 0 mod

Fix build failures due to undeclared ShenandoahOldHeuristics

Reviewed-by: rkennke, zgu

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/34

From zgu at openjdk.java.net  Mon May  3 12:38:51 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 3 May 2021 12:38:51 GMT
Subject: Integrated: 8266083: Shenandoah: Consolidate dedup/no dedup oop
 closures
In-Reply-To: <ubBlgKxnMm5bigU1FcbEHheyQkNiibZ0et8H_lQ6n-Q=.50bb3ade-98d5-4acc-8b83-8ee6432bd337@github.com>
References: <ubBlgKxnMm5bigU1FcbEHheyQkNiibZ0et8H_lQ6n-Q=.50bb3ade-98d5-4acc-8b83-8ee6432bd337@github.com>
Message-ID: <Ohy4Xw7Az_nEuWwraEJYOeLk8AwxS6oG0xqL-UmM7-4=.b8630652-b626-4a4c-9182-33cc4f71561c@github.com>

On Tue, 27 Apr 2021 18:37:17 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> We can use template to consolidate dedup/no dedup oop closures.

This pull request has now been integrated.

Changeset: 1d9ea3ae
Author:    Zhengyu Gu <zgu at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/1d9ea3ae0ff6e5283781c8e088578c2e8df63aaa
Stats:     90 lines in 3 files changed: 13 ins; 53 del; 24 mod

8266083: Shenandoah: Consolidate dedup/no dedup oop closures

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/jdk/pull/3724

From iwalulya at openjdk.java.net  Mon May  3 13:08:51 2021
From: iwalulya at openjdk.java.net (Ivan Walulya)
Date: Mon, 3 May 2021 13:08:51 GMT
Subject: RFR: 8261527: Record page size used for underlying mapping in
 ReservedSpace
In-Reply-To: <Enix4NQ5pCJxvz_enr5SWYpgqKq_9YV91PApY0TDmrY=.34f394d4-ffe1-4b93-b3b4-500eb2c915f0@github.com>
References: <Enix4NQ5pCJxvz_enr5SWYpgqKq_9YV91PApY0TDmrY=.34f394d4-ffe1-4b93-b3b4-500eb2c915f0@github.com>
Message-ID: <N7qedosTDRS0wq7TnBBWFnpTFGNKFs6jn78ap9xn85E=.6e8ac6c0-d3a2-45b1-8daa-f4c78193e11c@github.com>

On Thu, 29 Apr 2021 20:38:19 GMT, Stefan Johansson <sjohanss at openjdk.org> wrote:

> Please review this change to use the actual page size rather than `bool large` when creating and initializing `ReservedSpace`. This allows us to then get rid of the helper `ReservedSpace::actual_reserved_page_size()` and instead use the "real" page size for logging and other use.
> 
> The PR consist of two commits, one that changes `ReservedSpace` to use and store a page size and one that removes `actual_reserved_page_size()` and instead uses `ReservedSpace::page_size()`. Hopefully this might help ease the review a bit. 
> 
> There are two changes in behavior:
> * In `JfrVirtualMemorySegment::initialize()` we now always pass down `os::vm_page_size()` which means never use large pages. This differs from the old condition where large pages were enabled if transparent huge pages were enabled. This change has been discussed with the JFR team and they will later investigate how to re-enable large page use.
> * In `ReservedSpace::reserve()`, if a file is used, the page size is always set to `os::vm_page_size()`. So when logging the page_size for such mapping it will not report a large page size. This was incorrectly done in the past when using `ReservedSpace::actual_reserved_page_size()` to figure out the page size. This makes the `runtime/os/TestTracePageSizes.java` test pass even if run with `-XX:AllocateHeapAt=/tmp`.
> 
> **Testing**
> Mach5 tier1-4 and a lot of local testing.

lgtm!

-------------

Marked as reviewed by iwalulya (Committer).

PR: https://git.openjdk.java.net/jdk/pull/3802

From github.com+16811675+linade at openjdk.java.net  Mon May  3 14:57:05 2021
From: github.com+16811675+linade at openjdk.java.net (linade)
Date: Mon, 3 May 2021 14:57:05 GMT
Subject: RFR: 8266185: Shenandoah: Fix incorrect comment/assertion messages
Message-ID: <PNfJ7_EcHVzwZiACKoZLu9Eyjh2G2-sxsQAlK-pPpqk=.ae4e27c0-ec9b-4c85-8183-0a2906be5983@github.com>

Fix incorrect comment and assertion messages in Shenandoah GC

-------------

Commit messages:
 - 8266185: Shenandoah: Fix incorrect comment/assertion messages

Changes: https://git.openjdk.java.net/jdk/pull/3750/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3750&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8266185
  Stats: 5 lines in 2 files changed: 0 ins; 0 del; 5 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3750.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3750/head:pull/3750

PR: https://git.openjdk.java.net/jdk/pull/3750

From shade at openjdk.java.net  Mon May  3 14:57:05 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 3 May 2021 14:57:05 GMT
Subject: RFR: 8266185: Shenandoah: Fix incorrect comment/assertion messages
In-Reply-To: <PNfJ7_EcHVzwZiACKoZLu9Eyjh2G2-sxsQAlK-pPpqk=.ae4e27c0-ec9b-4c85-8183-0a2906be5983@github.com>
References: <PNfJ7_EcHVzwZiACKoZLu9Eyjh2G2-sxsQAlK-pPpqk=.ae4e27c0-ec9b-4c85-8183-0a2906be5983@github.com>
Message-ID: <Rh3MfVhGeQMyFTR2lx-UdSrqGYfmYKKuPl5qHdbXeYg=.ddcd3f66-c4ce-44a9-9d68-3e2f9b458ea8@github.com>

On Wed, 28 Apr 2021 06:35:42 GMT, linade <github.com+16811675+linade at openjdk.org> wrote:

> Fix incorrect comment and assertion messages in Shenandoah GC

This looks fine to me. We need for OCA to be cleared before you can integrate.

-------------

Marked as reviewed by shade (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/3750

From github.com+16811675+linade at openjdk.java.net  Mon May  3 14:57:06 2021
From: github.com+16811675+linade at openjdk.java.net (linade)
Date: Mon, 3 May 2021 14:57:06 GMT
Subject: RFR: 8266185: Shenandoah: Fix incorrect comment/assertion messages
In-Reply-To: <Rh3MfVhGeQMyFTR2lx-UdSrqGYfmYKKuPl5qHdbXeYg=.ddcd3f66-c4ce-44a9-9d68-3e2f9b458ea8@github.com>
References: <PNfJ7_EcHVzwZiACKoZLu9Eyjh2G2-sxsQAlK-pPpqk=.ae4e27c0-ec9b-4c85-8183-0a2906be5983@github.com>
 <Rh3MfVhGeQMyFTR2lx-UdSrqGYfmYKKuPl5qHdbXeYg=.ddcd3f66-c4ce-44a9-9d68-3e2f9b458ea8@github.com>
Message-ID: <OkGbMCwp-G1UIs4QSlEoKCDRYsKijqmYilKnu3Zylq0=.982bffaa-1108-432a-b470-f4fe59863ef7@github.com>

On Wed, 28 Apr 2021 15:41:51 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> This looks fine to me. We need for OCA to be cleared before you can integrate.

Thanks for the review. Let's wait for OCA.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3750

From github.com+16811675+linade at openjdk.java.net  Mon May  3 15:26:52 2021
From: github.com+16811675+linade at openjdk.java.net (linade)
Date: Mon, 3 May 2021 15:26:52 GMT
Subject: Integrated: 8266185: Shenandoah: Fix incorrect comment/assertion
 messages
In-Reply-To: <PNfJ7_EcHVzwZiACKoZLu9Eyjh2G2-sxsQAlK-pPpqk=.ae4e27c0-ec9b-4c85-8183-0a2906be5983@github.com>
References: <PNfJ7_EcHVzwZiACKoZLu9Eyjh2G2-sxsQAlK-pPpqk=.ae4e27c0-ec9b-4c85-8183-0a2906be5983@github.com>
Message-ID: <xTknQTV8L37XSdDtck3eV2yyIWYK0eQwY1qVKG7nO80=.d252f361-18a8-4be6-96d7-ac25265d94b7@github.com>

On Wed, 28 Apr 2021 06:35:42 GMT, linade <github.com+16811675+linade at openjdk.org> wrote:

> Fix incorrect comment and assertion messages in Shenandoah GC

This pull request has now been integrated.

Changeset: 55cc0af4
Author:    Yude Lin <yude.lyd at alibaba-inc.com>
Committer: Aleksey Shipilev <shade at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/55cc0af40444262d0ba0f7dc01da44f1a431dc48
Stats:     5 lines in 2 files changed: 0 ins; 0 del; 5 mod

8266185: Shenandoah: Fix incorrect comment/assertion messages

Reviewed-by: shade

-------------

PR: https://git.openjdk.java.net/jdk/pull/3750

From wkemper at openjdk.java.net  Mon May  3 16:14:19 2021
From: wkemper at openjdk.java.net (William Kemper)
Date: Mon, 3 May 2021 16:14:19 GMT
Subject: Integrated: Fix barrier set cast for aarch64
In-Reply-To: <vuXgNEMwrVBErj704QQE1JFyUWeSkSczvCe1XVdEaWo=.dc46e827-9f1e-4d0c-b860-303b242514c6@github.com>
References: <vuXgNEMwrVBErj704QQE1JFyUWeSkSczvCe1XVdEaWo=.dc46e827-9f1e-4d0c-b860-303b242514c6@github.com>
Message-ID: <i2nNnFD8UFbnQIUz-LGUlGvttuyPZrcErYULzKapDGs=.898cc614-398b-47e1-b3fa-c9bfccd0ab15@github.com>

On Thu, 29 Apr 2021 22:00:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Simple fix for aarch64 (`ShenandoahBarrierSet` no longer extends from `CardTableBarrierSet`).

This pull request has now been integrated.

Changeset: fdd30f5c
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.java.net/shenandoah/commit/fdd30f5c605661d603374326619989def65865c7
Stats:     6 lines in 2 files changed: 0 ins; 2 del; 4 mod

Fix barrier set cast for aarch64

Reviewed-by: zgu, rkennke

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/33

From zgu at openjdk.java.net  Mon May  3 19:04:57 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 3 May 2021 19:04:57 GMT
Subject: RFR: 8266453: Shenandoah: Some tests failed on MacOSX/AArch64
Message-ID: <npyp-9OgbCwOlYvDN88kVEKqHliyqq2EUGWapW9hGno=.12ab1113-b1c3-40b0-9fbc-018f005298e8@github.com>

Some of tier1/tier2 tests failed on MacOSX/AArch64. The fix is similar to what JDK-8265984 does for ZGC.

Test:
  tier1 and tier2 with Shenandoah GC on MacOSX/AArch64

-------------

Commit messages:
 - v1

Changes: https://git.openjdk.java.net/jdk/pull/3843/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3843&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8266453
  Stats: 4 lines in 1 file changed: 3 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3843.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3843/head:pull/3843

PR: https://git.openjdk.java.net/jdk/pull/3843

From rkennke at openjdk.java.net  Mon May  3 19:26:50 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 3 May 2021 19:26:50 GMT
Subject: RFR: 8266453: Shenandoah: Some tests failed on MacOSX/AArch64
In-Reply-To: <npyp-9OgbCwOlYvDN88kVEKqHliyqq2EUGWapW9hGno=.12ab1113-b1c3-40b0-9fbc-018f005298e8@github.com>
References: <npyp-9OgbCwOlYvDN88kVEKqHliyqq2EUGWapW9hGno=.12ab1113-b1c3-40b0-9fbc-018f005298e8@github.com>
Message-ID: <ruS-mOIMjyW3fq0shusaw5JTA9dffpDpoNxWYACycMU=.1af602d7-7f28-40d7-a912-86fdb89ba5e1@github.com>

On Mon, 3 May 2021 18:58:43 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Some of tier1/tier2 tests failed on MacOSX/AArch64. The fix is similar to what JDK-8265984 does for ZGC.
> 
> Test:
>   tier1 and tier2 with Shenandoah GC on MacOSX/AArch64

Err, what does it do?
I'd prefer if the commit message describes what the fix does, not what the failure is.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3843

From zgu at openjdk.java.net  Mon May  3 19:40:49 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 3 May 2021 19:40:49 GMT
Subject: RFR: 8266453: Shenandoah: Some tests failed on MacOSX/AArch64
In-Reply-To: <ruS-mOIMjyW3fq0shusaw5JTA9dffpDpoNxWYACycMU=.1af602d7-7f28-40d7-a912-86fdb89ba5e1@github.com>
References: <npyp-9OgbCwOlYvDN88kVEKqHliyqq2EUGWapW9hGno=.12ab1113-b1c3-40b0-9fbc-018f005298e8@github.com>
 <ruS-mOIMjyW3fq0shusaw5JTA9dffpDpoNxWYACycMU=.1af602d7-7f28-40d7-a912-86fdb89ba5e1@github.com>
Message-ID: <w5j_RaWxWFHwMYr6auV6bN2M2y1wopsbrjWbkpezuK8=.5cf0703b-031f-48ea-a137-215b59502718@github.com>

On Mon, 3 May 2021 19:24:19 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> Err, what does it do?
> I'd prefer if the commit message describes what the fix does, not what the failure is.

Added line disables write protections, before nmethod barrier can patch nmethod code (see  https://developer.apple.com/documentation/apple-silicon/porting-just-in-time-compilers-to-apple-silicon for details)

-------------

PR: https://git.openjdk.java.net/jdk/pull/3843

From bmathiske at openjdk.java.net  Mon May  3 22:24:09 2021
From: bmathiske at openjdk.java.net (Bernd Mathiske)
Date: Mon, 3 May 2021 22:24:09 GMT
Subject: Integrated: Use PLABs for old gen allocations, including promotions.
In-Reply-To: <cUor0-NVIgdbW3DNC-1aOMxT41ldDICg_xCx-HM9Ux8=.63b5dc2b-f01b-4c50-8fe7-96285e93b168@github.com>
References: <cUor0-NVIgdbW3DNC-1aOMxT41ldDICg_xCx-HM9Ux8=.63b5dc2b-f01b-4c50-8fe7-96285e93b168@github.com>
Message-ID: <ApQxsONjp09GKvAG35sIVJZf1JB2FaxMJu1s1l8mdaM=.ca223c47-2ee8-43a4-b1e8-8bf23155262b@github.com>

On Mon, 12 Apr 2021 19:50:45 GMT, Bernd Mathiske <bmathiske at openjdk.org> wrote:

> Introducing a 3rd kind of LAB for generational mode: "PLAB".
> 
> Shenandoah already has TLABs and GCLABs. The latter pertain firmly to young gen and allocations in old gen are so far always shared allocations, without LAB. With this PR, old gen allocations go through LABs as well and we separate those from the other two kinds. In other words, each generation has its separate GC-decicated LAB. 
> 
> When not in generational mode, GCLABs work as before and PLABs are never used.
> 
> The new diagnostic flag -XX:-ShenandoahUsePLAB can be used to turn off PLAB use, so that all old gen allocations are shared as before this PR. By default PLABs will be used.
> 
> Note that In source code, the type of both GCLABs and PLABs is "PLAB*", sonce GCLABs already had that type, but variable references and other value-level identifiers are consistently named "gclab" and "plab".
> 
> In ShenandoahFreeSet::allocate_single() there is a notable additional change that can easily be reverted if needed. Instead of fitting the intended allocation into any free or occupied region that can hold it, we now search for an occupied region in the same generation first, before considering free regions. The idea is to pack regions more densely as long as they have space left. This provides more opportunity for region borrowing between generations, should the need arise. For single-generational Shenandoah, it should not make a huge difference, right? Or is it crucial to find any free slot hit in that search loop ASAP?
> 
> Full collections need to set empty/trash regions that they compact into to non-FREE. For now, they will be made "YOUNG". (See the second commit.)

This pull request has now been integrated.

Changeset: 234053e5
Author:    Bernd Mathiske <bmathiske at openjdk.org>
Committer: William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.java.net/shenandoah/commit/234053e5194462a8aaf324243f8954591ef934ff
Stats:     341 lines in 17 files changed: 246 ins; 45 del; 50 mod

Use PLABs for old gen allocations, including promotions.

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/30

From aph at openjdk.java.net  Tue May  4 08:11:53 2021
From: aph at openjdk.java.net (Andrew Haley)
Date: Tue, 4 May 2021 08:11:53 GMT
Subject: RFR: 8266453: Shenandoah: Some tests failed on MacOSX/AArch64
In-Reply-To: <w5j_RaWxWFHwMYr6auV6bN2M2y1wopsbrjWbkpezuK8=.5cf0703b-031f-48ea-a137-215b59502718@github.com>
References: <npyp-9OgbCwOlYvDN88kVEKqHliyqq2EUGWapW9hGno=.12ab1113-b1c3-40b0-9fbc-018f005298e8@github.com>
 <ruS-mOIMjyW3fq0shusaw5JTA9dffpDpoNxWYACycMU=.1af602d7-7f28-40d7-a912-86fdb89ba5e1@github.com>
 <w5j_RaWxWFHwMYr6auV6bN2M2y1wopsbrjWbkpezuK8=.5cf0703b-031f-48ea-a137-215b59502718@github.com>
Message-ID: <qpQrn_fUKDf37d0zEW1yBvmmSv78Mc6po0Tx3vUHvFM=.96a93ea0-f2ec-4f0d-a91d-779e3497c51e@github.com>

On Mon, 3 May 2021 19:37:36 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> > Err, what does it do?
> > I'd prefer if the commit message describes what the fix does, not what the failure is.
> 
> Added line disables write protections, before nmethod barrier can patch nmethod code (see https://developer.apple.com/documentation/apple-silicon/porting-just-in-time-compilers-to-apple-silicon for details)

So you need to change the title of this PR.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3843

From shade at openjdk.java.net  Tue May  4 08:42:29 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 4 May 2021 08:42:29 GMT
Subject: RFR: Fix Windows builds
Message-ID: <zJYjVUlOB2QlhznrqTlfcatGDvkCPF589F2gTHkp-Qw=.315b7590-d957-4f9b-995b-168aedf3558e@github.com>

Windows builds are currently failing with:


c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\shenandoahOldGC.cpp(149): error C2131: expression did not evaluate to a constant
c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\shenandoahOldGC.cpp(149): note: failure was caused by a read of a variable outside its lifetime
c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\shenandoahOldGC.cpp(149): note: see usage of 'coalesce_and_fill_regions_count'
make[3]: *** [lib/CompileJvm.gmk:143: /cygdrive/c/buildbot/worker/build-shenandoah-jdkx-windows/build/build/windows-x86_64-server-fastdebug/hotspot/variant-server/libjvm/objs/shenandoahOldGC.obj] Error 1


...and:


c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(239): error C2220: the following warning is treated as an error
c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(239): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(240): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(241): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(255): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
make[3]: *** [lib/CompileJvm.gmk:143: /cygdrive/c/buildbot/worker/build-shenandoah-jdkx-windows/build/build/windows-x86_64-server-fastdebug/hotspot/variant-server/libjvm/objs/shenandoahOldHeuristics.obj] Error 1
make[2]: *** [make/Main.gmk:252: hotspot-server-libs] Error 2


Additional testing:
 - [x] Windows x86_64 builds
 - [x] Linux x86_64 hotspot_gc_shenandoah

-------------

Commit messages:
 - Fix Windows builds

Changes: https://git.openjdk.java.net/shenandoah/pull/35/files
 Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=35&range=00
  Stats: 13 lines in 3 files changed: 4 ins; 1 del; 8 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/35.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/35/head:pull/35

PR: https://git.openjdk.java.net/shenandoah/pull/35

From rkennke at openjdk.java.net  Tue May  4 08:54:30 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 4 May 2021 08:54:30 GMT
Subject: RFR: Fix Windows builds
In-Reply-To: <zJYjVUlOB2QlhznrqTlfcatGDvkCPF589F2gTHkp-Qw=.315b7590-d957-4f9b-995b-168aedf3558e@github.com>
References: <zJYjVUlOB2QlhznrqTlfcatGDvkCPF589F2gTHkp-Qw=.315b7590-d957-4f9b-995b-168aedf3558e@github.com>
Message-ID: <s1el1g_lRVMsCi0WScmA919iwWTyvSEwLfpqqFw4YYA=.d573c19d-887f-48aa-be16-5cdf6d0025ca@github.com>

On Tue, 4 May 2021 08:36:30 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> Windows builds are currently failing with:
> 
> 
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\shenandoahOldGC.cpp(149): error C2131: expression did not evaluate to a constant
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\shenandoahOldGC.cpp(149): note: failure was caused by a read of a variable outside its lifetime
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\shenandoahOldGC.cpp(149): note: see usage of 'coalesce_and_fill_regions_count'
> make[3]: *** [lib/CompileJvm.gmk:143: /cygdrive/c/buildbot/worker/build-shenandoah-jdkx-windows/build/build/windows-x86_64-server-fastdebug/hotspot/variant-server/libjvm/objs/shenandoahOldGC.obj] Error 1
> 
> 
> ...and:
> 
> 
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(239): error C2220: the following warning is treated as an error
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(239): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(240): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(241): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(255): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
> make[3]: *** [lib/CompileJvm.gmk:143: /cygdrive/c/buildbot/worker/build-shenandoah-jdkx-windows/build/build/windows-x86_64-server-fastdebug/hotspot/variant-server/libjvm/objs/shenandoahOldHeuristics.obj] Error 1
> make[2]: *** [make/Main.gmk:252: hotspot-server-libs] Error 2
> 
> 
> Additional testing:
>  - [x] Windows x86_64 builds
>  - [x] Linux x86_64 hotspot_gc_shenandoah

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 241:

> 239:       _hidden_old_collection_candidates = (uint)i;
> 240:       _first_coalesce_and_fill_candidate = (uint)i;
> 241:       _old_coalesce_and_fill_candidates = (uint)(cand_idx - i);

I wonder if it would be more useful to change the type of i to uint, or maybe the fields to size_t instead?

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/35

From shade at openjdk.java.net  Tue May  4 09:00:20 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 4 May 2021 09:00:20 GMT
Subject: RFR: Fix Windows builds
In-Reply-To: <s1el1g_lRVMsCi0WScmA919iwWTyvSEwLfpqqFw4YYA=.d573c19d-887f-48aa-be16-5cdf6d0025ca@github.com>
References: <zJYjVUlOB2QlhznrqTlfcatGDvkCPF589F2gTHkp-Qw=.315b7590-d957-4f9b-995b-168aedf3558e@github.com>
 <s1el1g_lRVMsCi0WScmA919iwWTyvSEwLfpqqFw4YYA=.d573c19d-887f-48aa-be16-5cdf6d0025ca@github.com>
Message-ID: <81yiIzrIBeTjPBs-YiG8ckgoO_SD1F6czQvhYHzsFyM=.254205ec-133f-47ad-9120-fd1198b47590@github.com>

On Tue, 4 May 2021 08:50:57 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Windows builds are currently failing with:
>> 
>> 
>> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\shenandoahOldGC.cpp(149): error C2131: expression did not evaluate to a constant
>> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\shenandoahOldGC.cpp(149): note: failure was caused by a read of a variable outside its lifetime
>> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\shenandoahOldGC.cpp(149): note: see usage of 'coalesce_and_fill_regions_count'
>> make[3]: *** [lib/CompileJvm.gmk:143: /cygdrive/c/buildbot/worker/build-shenandoah-jdkx-windows/build/build/windows-x86_64-server-fastdebug/hotspot/variant-server/libjvm/objs/shenandoahOldGC.obj] Error 1
>> 
>> 
>> ...and:
>> 
>> 
>> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(239): error C2220: the following warning is treated as an error
>> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(239): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
>> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(240): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
>> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(241): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
>> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(255): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
>> make[3]: *** [lib/CompileJvm.gmk:143: /cygdrive/c/buildbot/worker/build-shenandoah-jdkx-windows/build/build/windows-x86_64-server-fastdebug/hotspot/variant-server/libjvm/objs/shenandoahOldHeuristics.obj] Error 1
>> make[2]: *** [make/Main.gmk:252: hotspot-server-libs] Error 2
>> 
>> 
>> Additional testing:
>>  - [x] Windows x86_64 builds
>>  - [x] Linux x86_64 hotspot_gc_shenandoah
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 241:
> 
>> 239:       _hidden_old_collection_candidates = (uint)i;
>> 240:       _first_coalesce_and_fill_candidate = (uint)i;
>> 241:       _old_coalesce_and_fill_candidates = (uint)(cand_idx - i);
> 
> I wonder if it would be more useful to change the type of i to uint, or maybe the fields to size_t instead?

Maybe. That code seems to be in flux, so I would just fix the builds at this time, and then reconsider changing the types more globally.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/35

From sjohanss at openjdk.java.net  Tue May  4 09:03:53 2021
From: sjohanss at openjdk.java.net (Stefan Johansson)
Date: Tue, 4 May 2021 09:03:53 GMT
Subject: RFR: 8261527: Record page size used for underlying mapping in
 ReservedSpace
In-Reply-To: <or7XKtcDsSvxZqpYbT4FWw43rDzwkRfzz4UbBa0ejso=.ed162eb4-a97a-48f3-b83c-2d5f737c495d@github.com>
References: <Enix4NQ5pCJxvz_enr5SWYpgqKq_9YV91PApY0TDmrY=.34f394d4-ffe1-4b93-b3b4-500eb2c915f0@github.com>
 <or7XKtcDsSvxZqpYbT4FWw43rDzwkRfzz4UbBa0ejso=.ed162eb4-a97a-48f3-b83c-2d5f737c495d@github.com>
Message-ID: <Xi5Aok_OoPOqo8nQQjpXw7jjUqkbUA9WVw21O2g-PIo=.3ef8f22e-8294-496b-9921-9abe73179b52@github.com>

On Mon, 3 May 2021 11:04:18 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Please review this change to use the actual page size rather than `bool large` when creating and initializing `ReservedSpace`. This allows us to then get rid of the helper `ReservedSpace::actual_reserved_page_size()` and instead use the "real" page size for logging and other use.
>> 
>> The PR consist of two commits, one that changes `ReservedSpace` to use and store a page size and one that removes `actual_reserved_page_size()` and instead uses `ReservedSpace::page_size()`. Hopefully this might help ease the review a bit. 
>> 
>> There are two changes in behavior:
>> * In `JfrVirtualMemorySegment::initialize()` we now always pass down `os::vm_page_size()` which means never use large pages. This differs from the old condition where large pages were enabled if transparent huge pages were enabled. This change has been discussed with the JFR team and they will later investigate how to re-enable large page use.
>> * In `ReservedSpace::reserve()`, if a file is used, the page size is always set to `os::vm_page_size()`. So when logging the page_size for such mapping it will not report a large page size. This was incorrectly done in the past when using `ReservedSpace::actual_reserved_page_size()` to figure out the page size. This makes the `runtime/os/TestTracePageSizes.java` test pass even if run with `-XX:AllocateHeapAt=/tmp`.
>> 
>> **Testing**
>> Mach5 tier1-4 and a lot of local testing.
>
> Looks good to me.

Thanks for the reviews @rkennke and @walulyai.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3802

From sjohanss at openjdk.java.net  Tue May  4 09:03:54 2021
From: sjohanss at openjdk.java.net (Stefan Johansson)
Date: Tue, 4 May 2021 09:03:54 GMT
Subject: Integrated: 8261527: Record page size used for underlying mapping in
 ReservedSpace
In-Reply-To: <Enix4NQ5pCJxvz_enr5SWYpgqKq_9YV91PApY0TDmrY=.34f394d4-ffe1-4b93-b3b4-500eb2c915f0@github.com>
References: <Enix4NQ5pCJxvz_enr5SWYpgqKq_9YV91PApY0TDmrY=.34f394d4-ffe1-4b93-b3b4-500eb2c915f0@github.com>
Message-ID: <G0y1_yD04nGH9gNR8BWujE0IAupNs_W4-0ZOczS8DI0=.5b3cd671-19e2-4173-8e81-c930e43a0899@github.com>

On Thu, 29 Apr 2021 20:38:19 GMT, Stefan Johansson <sjohanss at openjdk.org> wrote:

> Please review this change to use the actual page size rather than `bool large` when creating and initializing `ReservedSpace`. This allows us to then get rid of the helper `ReservedSpace::actual_reserved_page_size()` and instead use the "real" page size for logging and other use.
> 
> The PR consist of two commits, one that changes `ReservedSpace` to use and store a page size and one that removes `actual_reserved_page_size()` and instead uses `ReservedSpace::page_size()`. Hopefully this might help ease the review a bit. 
> 
> There are two changes in behavior:
> * In `JfrVirtualMemorySegment::initialize()` we now always pass down `os::vm_page_size()` which means never use large pages. This differs from the old condition where large pages were enabled if transparent huge pages were enabled. This change has been discussed with the JFR team and they will later investigate how to re-enable large page use.
> * In `ReservedSpace::reserve()`, if a file is used, the page size is always set to `os::vm_page_size()`. So when logging the page_size for such mapping it will not report a large page size. This was incorrectly done in the past when using `ReservedSpace::actual_reserved_page_size()` to figure out the page size. This makes the `runtime/os/TestTracePageSizes.java` test pass even if run with `-XX:AllocateHeapAt=/tmp`.
> 
> **Testing**
> Mach5 tier1-4 and a lot of local testing.

This pull request has now been integrated.

Changeset: 141cc2f2
Author:    Stefan Johansson <sjohanss at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/141cc2f2a35abdce48397071e2ce7ea862cf5755
Stats:     135 lines in 20 files changed: 18 ins; 26 del; 91 mod

8261527: Record page size used for underlying mapping in ReservedSpace

Reviewed-by: rkennke, iwalulya

-------------

PR: https://git.openjdk.java.net/jdk/pull/3802

From rkennke at openjdk.java.net  Tue May  4 09:10:31 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 4 May 2021 09:10:31 GMT
Subject: RFR: Fix Windows builds
In-Reply-To: <zJYjVUlOB2QlhznrqTlfcatGDvkCPF589F2gTHkp-Qw=.315b7590-d957-4f9b-995b-168aedf3558e@github.com>
References: <zJYjVUlOB2QlhznrqTlfcatGDvkCPF589F2gTHkp-Qw=.315b7590-d957-4f9b-995b-168aedf3558e@github.com>
Message-ID: <WwU_ZDH_cmNslE-ABeI4zGHUedmrpXAPHHih1XtMHCk=.2213909a-6671-4597-8767-a8cae8d44f18@github.com>

On Tue, 4 May 2021 08:36:30 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> Windows builds are currently failing with:
> 
> 
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\shenandoahOldGC.cpp(149): error C2131: expression did not evaluate to a constant
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\shenandoahOldGC.cpp(149): note: failure was caused by a read of a variable outside its lifetime
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\shenandoahOldGC.cpp(149): note: see usage of 'coalesce_and_fill_regions_count'
> make[3]: *** [lib/CompileJvm.gmk:143: /cygdrive/c/buildbot/worker/build-shenandoah-jdkx-windows/build/build/windows-x86_64-server-fastdebug/hotspot/variant-server/libjvm/objs/shenandoahOldGC.obj] Error 1
> 
> 
> ...and:
> 
> 
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(239): error C2220: the following warning is treated as an error
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(239): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(240): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(241): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(255): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
> make[3]: *** [lib/CompileJvm.gmk:143: /cygdrive/c/buildbot/worker/build-shenandoah-jdkx-windows/build/build/windows-x86_64-server-fastdebug/hotspot/variant-server/libjvm/objs/shenandoahOldHeuristics.obj] Error 1
> make[2]: *** [make/Main.gmk:252: hotspot-server-libs] Error 2
> 
> 
> Additional testing:
>  - [x] Windows x86_64 builds
>  - [x] Linux x86_64 hotspot_gc_shenandoah

Ok then.

-------------

Marked as reviewed by rkennke (Lead).

PR: https://git.openjdk.java.net/shenandoah/pull/35

From stuefe at openjdk.java.net  Tue May  4 09:54:58 2021
From: stuefe at openjdk.java.net (Thomas Stuefe)
Date: Tue, 4 May 2021 09:54:58 GMT
Subject: RFR: 8261527: Record page size used for underlying mapping in
 ReservedSpace
In-Reply-To: <Enix4NQ5pCJxvz_enr5SWYpgqKq_9YV91PApY0TDmrY=.34f394d4-ffe1-4b93-b3b4-500eb2c915f0@github.com>
References: <Enix4NQ5pCJxvz_enr5SWYpgqKq_9YV91PApY0TDmrY=.34f394d4-ffe1-4b93-b3b4-500eb2c915f0@github.com>
Message-ID: <OiCl2h4XPdr2rW3mnHY97SafElJHJx-2mOtvCWlLMO8=.716dec0d-b099-4dce-96bd-96a664efd089@github.com>

On Thu, 29 Apr 2021 20:38:19 GMT, Stefan Johansson <sjohanss at openjdk.org> wrote:

> Please review this change to use the actual page size rather than `bool large` when creating and initializing `ReservedSpace`. This allows us to then get rid of the helper `ReservedSpace::actual_reserved_page_size()` and instead use the "real" page size for logging and other use.
> 
> The PR consist of two commits, one that changes `ReservedSpace` to use and store a page size and one that removes `actual_reserved_page_size()` and instead uses `ReservedSpace::page_size()`. Hopefully this might help ease the review a bit. 
> 
> There are two changes in behavior:
> * In `JfrVirtualMemorySegment::initialize()` we now always pass down `os::vm_page_size()` which means never use large pages. This differs from the old condition where large pages were enabled if transparent huge pages were enabled. This change has been discussed with the JFR team and they will later investigate how to re-enable large page use.
> * In `ReservedSpace::reserve()`, if a file is used, the page size is always set to `os::vm_page_size()`. So when logging the page_size for such mapping it will not report a large page size. This was incorrectly done in the past when using `ReservedSpace::actual_reserved_page_size()` to figure out the page size. This makes the `runtime/os/TestTracePageSizes.java` test pass even if run with `-XX:AllocateHeapAt=/tmp`.
> 
> **Testing**
> Mach5 tier1-4 and a lot of local testing.

I see you were too fast for me :) 

Good patch. Much cleaner now.

Thanks, Thomas

-------------

PR: https://git.openjdk.java.net/jdk/pull/3802

From sjohanss at openjdk.java.net  Tue May  4 10:20:50 2021
From: sjohanss at openjdk.java.net (Stefan Johansson)
Date: Tue, 4 May 2021 10:20:50 GMT
Subject: RFR: 8261527: Record page size used for underlying mapping in
 ReservedSpace
In-Reply-To: <OiCl2h4XPdr2rW3mnHY97SafElJHJx-2mOtvCWlLMO8=.716dec0d-b099-4dce-96bd-96a664efd089@github.com>
References: <Enix4NQ5pCJxvz_enr5SWYpgqKq_9YV91PApY0TDmrY=.34f394d4-ffe1-4b93-b3b4-500eb2c915f0@github.com>
 <OiCl2h4XPdr2rW3mnHY97SafElJHJx-2mOtvCWlLMO8=.716dec0d-b099-4dce-96bd-96a664efd089@github.com>
Message-ID: <q0t0dxaSKZVAuCPb5AXPcGFYJcz_xA6jQGJ_tMbNpj8=.571b2a68-e97a-47c3-894d-c30f8f578063@github.com>

On Tue, 4 May 2021 09:52:53 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Please review this change to use the actual page size rather than `bool large` when creating and initializing `ReservedSpace`. This allows us to then get rid of the helper `ReservedSpace::actual_reserved_page_size()` and instead use the "real" page size for logging and other use.
>> 
>> The PR consist of two commits, one that changes `ReservedSpace` to use and store a page size and one that removes `actual_reserved_page_size()` and instead uses `ReservedSpace::page_size()`. Hopefully this might help ease the review a bit. 
>> 
>> There are two changes in behavior:
>> * In `JfrVirtualMemorySegment::initialize()` we now always pass down `os::vm_page_size()` which means never use large pages. This differs from the old condition where large pages were enabled if transparent huge pages were enabled. This change has been discussed with the JFR team and they will later investigate how to re-enable large page use.
>> * In `ReservedSpace::reserve()`, if a file is used, the page size is always set to `os::vm_page_size()`. So when logging the page_size for such mapping it will not report a large page size. This was incorrectly done in the past when using `ReservedSpace::actual_reserved_page_size()` to figure out the page size. This makes the `runtime/os/TestTracePageSizes.java` test pass even if run with `-XX:AllocateHeapAt=/tmp`.
>> 
>> **Testing**
>> Mach5 tier1-4 and a lot of local testing.
>
> I see you were too fast for me :) 
> 
> Good patch. Much cleaner now.
> 
> Thanks, Thomas

Thanks @tstuefe =)

-------------

PR: https://git.openjdk.java.net/jdk/pull/3802

From stuefe at openjdk.java.net  Tue May  4 11:28:35 2021
From: stuefe at openjdk.java.net (Thomas Stuefe)
Date: Tue, 4 May 2021 11:28:35 GMT
Subject: RFR: 8251392: Consolidate Metaspace Statistics
In-Reply-To: <GXWRwtpSZOjwEPKp5b-0G4euKgw1nPYXgbOX8diO-8o=.e07ccfe6-e6c7-4469-b7fe-ac24ae9d70a1@github.com>
References: <GXWRwtpSZOjwEPKp5b-0G4euKgw1nPYXgbOX8diO-8o=.e07ccfe6-e6c7-4469-b7fe-ac24ae9d70a1@github.com>
Message-ID: <Y8LxRWzGneJ2D4Kn7HQJHmGUjCwHjXzYjURTDGD0zEk=.640fb2ac-f8e7-4701-a850-6b1e77bac051@github.com>

On Thu, 29 Apr 2021 04:50:32 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> There is a lot of duplicate coding when it comes to the consumption of Metaspace Statistics.
> 
> Metaspace offers statistical APIs via `MetaspaceUtils::(reserved|committed|used)_(words|bytes)` for either the whole metaspace or non-class/class space separately. But many callers need some sort of combination of these values, which they keep in various data holder structures which all are named somewhat similarly and all do a very similar job. In particular, we have:
> 
> - MetaspaceSizesSnapshot (used for gc logs)
> - MetaspaceSnapshot (used in NMT)
> - MetaspaceSizes, MetaspaceSummary, JFRMetaspaceSummary (JFR)
> - MetaspaceCounters, CompressedClassSpaceCounters, MetaspacePerfCounters (jstat performance counters)
> - CompressedKlassSpacePool and MetaspacePool (used for MXBeans)
> 
> As much as possible coding should be unified.
> 
> In addition to that, all these callers share a common problem, in that retrieving individual statistical values via `MetaspaceUtils::(reserved|committed|used)_(words|bytes)` may yield inconsistent values. For example, "reserved" < "committed", or "committed" < "used". This is the cause of a variety of rare intermittent test errors in different areas, e.g. Performance counters (JDK-8163413, JDK-8153323), the gc log, the MemoryPoolMXBean and NMT (JDK-8237872).
> 
> -----------------------
> 
> This patch introduces two new data holder structures:
> - `MetaspaceStats` holds reserved/committed/used counter
> - `MetaspaceCombinedStats` holds an expression of these counters for class- and non-class metaspace each, as well as total counter.
> 
> Furthermore, the patch introduces two new APIs in MetaspaceUtils:
> - `MetaspaceStats get_statistics(type)`
> - `MetaspaceCombinedStats get_combined_statistics()`
> The former is used to get statistics for either non-class-space or class space; the latter gets statistics for the whole, including total values. Both APIs guarantee consistent values - reserved >= committed >= used - and never lock.
> 
> The patch uses these new data holders and these APIs to consolidate, clean up and simplify a lot of caller code, in addition to making that code resistant against inconsistent statistics:
> 
> - GC Log: class `MetaspaceSizesSnapshot` in *metaspace/metaspaceSizesSnapshot.cpp/hpp* had been used to hold metaspace statistics. Its default constructor sneakishly queried all values. `MetaspaceSizesSnapshot` has neem removed, caller code rewritten for MetaspaceCombinedStats and explicit value querying.
> 
> - NMT had class `MetaspaceSnapshot` to keep metaspace statistics and to print out baseline diffs. 
>    - `MetaspaceSizesSnapshot` was removed and replaced with `MetaspaceCombinedStats`.
>    - `MemSummaryDiffReporter::print_metaspace_diff()` has been modified: fixed a potential div-by-zero, removed the "free" statistics which is meaningless, and massaged the code a bit.
>    - Similar fixes have been done in `MemSummaryReporter::report_metadata()` (which, if backported, should fix JDK-8237872).
> 
> - jstat & co: Metaspace performance counters are realized by `MetaspaceCounters` and `CompressedClassSpaceCounters`, implementation resides in `MetaspacePerfCounters` (metaspace/metaspaceCounters.hpp/cpp). 
> 	- I completely removed `CompressedClassSpaceCounters` since there is no need for two APIs, class space is part of metaspace.
> 	- I simplified the counter coding a lot. I think the complexity was not needed.
> 	- The Metaspace counters are now retrieved in a consistent manner. This should take care of JDK-8163413, JDK-8153323 and similar bugs.
> 
> - `MetaspaceMemoryPool`, `CompressedClassSpaceMemoryPool` used in MxBeans: I changed the implementation to return consistent values. 
> 
> - JFR reports metaspace allocations (which, confusingly, uses a different terminology: "data_space" == non-class portion of metaspace). It used `MetaspaceSummary` to hold the statistics, which were composed of `MetaspaceSizes`.
> 	- I completely removed `MetaspaceSizes`
> 	- I rewrote `MetaspaceSummary` to use `MetaspaceCombinedStats`
> 
> - MetaspaceUtils::print_on() has been used to print metaspace statistics (by GCs). Function has been corrected to print consistent values.
> 
> - Added a simple gtest for the new APIs
> 
> Tests: 
> - manually executed hotspot tier1 tests
> - explicitly tested runtime/Metaspace, gc/metaspace, runtime/NMT
> - SAP nightlies ran on all our platforms

@zhengyu123 : could you take a look at the NMT part of this change? Thanks!

-------------

PR: https://git.openjdk.java.net/jdk/pull/3786

From stuefe at openjdk.java.net  Tue May  4 11:28:35 2021
From: stuefe at openjdk.java.net (Thomas Stuefe)
Date: Tue, 4 May 2021 11:28:35 GMT
Subject: RFR: 8251392: Consolidate Metaspace Statistics
Message-ID: <GXWRwtpSZOjwEPKp5b-0G4euKgw1nPYXgbOX8diO-8o=.e07ccfe6-e6c7-4469-b7fe-ac24ae9d70a1@github.com>

There is a lot of duplicate coding when it comes to the consumption of Metaspace Statistics.

Metaspace offers statistical APIs via `MetaspaceUtils::(reserved|committed|used)_(words|bytes)` for either the whole metaspace or non-class/class space separately. But many callers need some sort of combination of these values, which they keep in various data holder structures which all are named somewhat similarly and all do a very similar job. In particular, we have:

- MetaspaceSizesSnapshot (used for gc logs)
- MetaspaceSnapshot (used in NMT)
- MetaspaceSizes, MetaspaceSummary, JFRMetaspaceSummary (JFR)
- MetaspaceCounters, CompressedClassSpaceCounters, MetaspacePerfCounters (jstat performance counters)
- CompressedKlassSpacePool and MetaspacePool (used for MXBeans)

As much as possible coding should be unified.

In addition to that, all these callers share a common problem, in that retrieving individual statistical values via `MetaspaceUtils::(reserved|committed|used)_(words|bytes)` may yield inconsistent values. For example, "reserved" < "committed", or "committed" < "used". This is the cause of a variety of rare intermittent test errors in different areas, e.g. Performance counters (JDK-8163413, JDK-8153323), the gc log, the MemoryPoolMXBean and NMT (JDK-8237872).

-----------------------

This patch introduces two new data holder structures:
- `MetaspaceStats` holds reserved/committed/used counter
- `MetaspaceCombinedStats` holds an expression of these counters for class- and non-class metaspace each, as well as total counter.

Furthermore, the patch introduces two new APIs in MetaspaceUtils:
- `MetaspaceStats get_statistics(type)`
- `MetaspaceCombinedStats get_combined_statistics()`
The former is used to get statistics for either non-class-space or class space; the latter gets statistics for the whole, including total values. Both APIs guarantee consistent values - reserved >= committed >= used - and never lock.

The patch uses these new data holders and these APIs to consolidate, clean up and simplify a lot of caller code, in addition to making that code resistant against inconsistent statistics:

- GC Log: class `MetaspaceSizesSnapshot` in *metaspace/metaspaceSizesSnapshot.cpp/hpp* had been used to hold metaspace statistics. Its default constructor sneakishly queried all values. `MetaspaceSizesSnapshot` has neem removed, caller code rewritten for MetaspaceCombinedStats and explicit value querying.

- NMT had class `MetaspaceSnapshot` to keep metaspace statistics and to print out baseline diffs. 
   - `MetaspaceSizesSnapshot` was removed and replaced with `MetaspaceCombinedStats`.
   - `MemSummaryDiffReporter::print_metaspace_diff()` has been modified: fixed a potential div-by-zero, removed the "free" statistics which is meaningless, and massaged the code a bit.
   - Similar fixes have been done in `MemSummaryReporter::report_metadata()` (which, if backported, should fix JDK-8237872).

- jstat & co: Metaspace performance counters are realized by `MetaspaceCounters` and `CompressedClassSpaceCounters`, implementation resides in `MetaspacePerfCounters` (metaspace/metaspaceCounters.hpp/cpp). 
	- I completely removed `CompressedClassSpaceCounters` since there is no need for two APIs, class space is part of metaspace.
	- I simplified the counter coding a lot. I think the complexity was not needed.
	- The Metaspace counters are now retrieved in a consistent manner. This should take care of JDK-8163413, JDK-8153323 and similar bugs.

- `MetaspaceMemoryPool`, `CompressedClassSpaceMemoryPool` used in MxBeans: I changed the implementation to return consistent values. 

- JFR reports metaspace allocations (which, confusingly, uses a different terminology: "data_space" == non-class portion of metaspace). It used `MetaspaceSummary` to hold the statistics, which were composed of `MetaspaceSizes`.
	- I completely removed `MetaspaceSizes`
	- I rewrote `MetaspaceSummary` to use `MetaspaceCombinedStats`

- MetaspaceUtils::print_on() has been used to print metaspace statistics (by GCs). Function has been corrected to print consistent values.

- Added a simple gtest for the new APIs

Tests: 
- manually executed hotspot tier1 tests
- explicitly tested runtime/Metaspace, gc/metaspace, runtime/NMT
- SAP nightlies ran on all our platforms

-------------

Commit messages:
 - wip

Changes: https://git.openjdk.java.net/jdk/pull/3786/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3786&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8251392
  Stats: 593 lines in 33 files changed: 193 ins; 304 del; 96 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3786.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3786/head:pull/3786

PR: https://git.openjdk.java.net/jdk/pull/3786

From shade at openjdk.java.net  Tue May  4 11:31:14 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 4 May 2021 11:31:14 GMT
Subject: Integrated: Fix Windows builds
In-Reply-To: <zJYjVUlOB2QlhznrqTlfcatGDvkCPF589F2gTHkp-Qw=.315b7590-d957-4f9b-995b-168aedf3558e@github.com>
References: <zJYjVUlOB2QlhznrqTlfcatGDvkCPF589F2gTHkp-Qw=.315b7590-d957-4f9b-995b-168aedf3558e@github.com>
Message-ID: <l3rLnhhZTNiaILhTD2FxKBGRAs4N_npHl4iSKc-xh9I=.5a0ad3f3-9934-41e6-9784-77d873857880@github.com>

On Tue, 4 May 2021 08:36:30 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> Windows builds are currently failing with:
> 
> 
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\shenandoahOldGC.cpp(149): error C2131: expression did not evaluate to a constant
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\shenandoahOldGC.cpp(149): note: failure was caused by a read of a variable outside its lifetime
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\shenandoahOldGC.cpp(149): note: see usage of 'coalesce_and_fill_regions_count'
> make[3]: *** [lib/CompileJvm.gmk:143: /cygdrive/c/buildbot/worker/build-shenandoah-jdkx-windows/build/build/windows-x86_64-server-fastdebug/hotspot/variant-server/libjvm/objs/shenandoahOldGC.obj] Error 1
> 
> 
> ...and:
> 
> 
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(239): error C2220: the following warning is treated as an error
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(239): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(240): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(241): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
> c:\buildbot\worker\build-shenandoah-jdkx-windows\build\src\hotspot\share\gc\shenandoah\heuristics\shenandoahOldHeuristics.cpp(255): warning C4267: '=': conversion from 'size_t' to 'uint', possible loss of data
> make[3]: *** [lib/CompileJvm.gmk:143: /cygdrive/c/buildbot/worker/build-shenandoah-jdkx-windows/build/build/windows-x86_64-server-fastdebug/hotspot/variant-server/libjvm/objs/shenandoahOldHeuristics.obj] Error 1
> make[2]: *** [make/Main.gmk:252: hotspot-server-libs] Error 2
> 
> 
> Additional testing:
>  - [x] Windows x86_64 builds
>  - [x] Linux x86_64 hotspot_gc_shenandoah

This pull request has now been integrated.

Changeset: cbc9e448
Author:    Aleksey Shipilev <shade at openjdk.org>
URL:       https://git.openjdk.java.net/shenandoah/commit/cbc9e448f18848ab2211ea5d9a58146d5382d174
Stats:     13 lines in 3 files changed: 4 ins; 1 del; 8 mod

Fix Windows builds

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/35

From zgu at openjdk.java.net  Tue May  4 23:13:04 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 4 May 2021 23:13:04 GMT
Subject: RFR: 8266522: Shenandoah: Shenandoah LRB calls wrong runtime barrier
 on aarch64
Message-ID: <Husxaxmq8-iusB5dS8-0WE9zUwhoW1Z0XM8uuKW5PM0=.7ec7e170-7d78-478a-a57d-3e46e46c4367@github.com>

Shsenandoah LRB calls ShenandoahRuntime::load_reference_barrier_weak() on phantom access, which is wrong.

Should call ShenandoahRuntime::load_reference_barrier_phantom() instead.

Test:
  - [x] hotspot_gc_shenandoah on Linux aarch64

-------------

Commit messages:
 - v1

Changes: https://git.openjdk.java.net/jdk/pull/3864/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3864&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8266522
  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3864.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3864/head:pull/3864

PR: https://git.openjdk.java.net/jdk/pull/3864

From rkennke at openjdk.java.net  Wed May  5 09:43:01 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 5 May 2021 09:43:01 GMT
Subject: RFR: 8266522: Shenandoah: Shenandoah LRB calls wrong runtime
 barrier on aarch64
In-Reply-To: <Husxaxmq8-iusB5dS8-0WE9zUwhoW1Z0XM8uuKW5PM0=.7ec7e170-7d78-478a-a57d-3e46e46c4367@github.com>
References: <Husxaxmq8-iusB5dS8-0WE9zUwhoW1Z0XM8uuKW5PM0=.7ec7e170-7d78-478a-a57d-3e46e46c4367@github.com>
Message-ID: <gP7cdXZIhiPhEo3P7s_sW5MSDTPaM0ETLiwHZURzZMI=.520df569-9993-42f7-8014-d97503bb2a5e@github.com>

On Tue, 4 May 2021 19:09:59 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Shsenandoah LRB calls ShenandoahRuntime::load_reference_barrier_weak() on phantom access, which is wrong.
> 
> Should call ShenandoahRuntime::load_reference_barrier_phantom() instead.
> 
> Test:
>   - [x] hotspot_gc_shenandoah on Linux aarch64

Looks good! Thank you!

-------------

Marked as reviewed by rkennke (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/3864

From zgu at openjdk.java.net  Wed May  5 12:17:51 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 5 May 2021 12:17:51 GMT
Subject: Integrated: 8266522: Shenandoah: Shenandoah LRB calls wrong runtime
 barrier on aarch64
In-Reply-To: <Husxaxmq8-iusB5dS8-0WE9zUwhoW1Z0XM8uuKW5PM0=.7ec7e170-7d78-478a-a57d-3e46e46c4367@github.com>
References: <Husxaxmq8-iusB5dS8-0WE9zUwhoW1Z0XM8uuKW5PM0=.7ec7e170-7d78-478a-a57d-3e46e46c4367@github.com>
Message-ID: <yvSNm1BPXKm6vknMRJGA6M_r0Ing5hKoSMQ3fHg6H8Y=.92f23827-f464-4858-adfa-0703747e324e@github.com>

On Tue, 4 May 2021 19:09:59 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Shsenandoah LRB calls ShenandoahRuntime::load_reference_barrier_weak() on phantom access, which is wrong.
> 
> Should call ShenandoahRuntime::load_reference_barrier_phantom() instead.
> 
> Test:
>   - [x] hotspot_gc_shenandoah on Linux aarch64

This pull request has now been integrated.

Changeset: 71d0858b
Author:    Zhengyu Gu <zgu at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/71d0858b3216e6996e217707678680c602b4404d
Stats:     1 line in 1 file changed: 0 ins; 0 del; 1 mod

8266522: Shenandoah: Shenandoah LRB calls wrong runtime barrier on aarch64

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/jdk/pull/3864

From rkennke at openjdk.java.net  Wed May  5 12:56:52 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 5 May 2021 12:56:52 GMT
Subject: RFR: 8266453: Shenandoah: Disable write protections before
 patching nmethod in nmethod_barrier on MacOSX/AArch64
In-Reply-To: <npyp-9OgbCwOlYvDN88kVEKqHliyqq2EUGWapW9hGno=.12ab1113-b1c3-40b0-9fbc-018f005298e8@github.com>
References: <npyp-9OgbCwOlYvDN88kVEKqHliyqq2EUGWapW9hGno=.12ab1113-b1c3-40b0-9fbc-018f005298e8@github.com>
Message-ID: <pUIJRXRoj-8hMO3Cr1xQb4mfTP5G75ZS4JFFKabzs8o=.16c019aa-0629-436f-a9eb-fa64544de426@github.com>

On Mon, 3 May 2021 18:58:43 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Some of tier1/tier2 tests failed on MacOSX/AArch64. The fix is similar to what JDK-8265984 does for ZGC, that disable write protections when patching code.
> 
> Test:
>   tier1 and tier2 with Shenandoah GC on MacOSX/AArch64

Thanks, looks good!

-------------

Marked as reviewed by rkennke (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/3843

From zgu at openjdk.java.net  Wed May  5 13:04:52 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 5 May 2021 13:04:52 GMT
Subject: Integrated: 8266453: Shenandoah: Disable write protections before
 patching nmethod in nmethod_barrier on MacOSX/AArch64
In-Reply-To: <npyp-9OgbCwOlYvDN88kVEKqHliyqq2EUGWapW9hGno=.12ab1113-b1c3-40b0-9fbc-018f005298e8@github.com>
References: <npyp-9OgbCwOlYvDN88kVEKqHliyqq2EUGWapW9hGno=.12ab1113-b1c3-40b0-9fbc-018f005298e8@github.com>
Message-ID: <9k7rit76gvmRQ07wzOmw-SIlCqzJgoQQG1IKw5e97ug=.e3358d72-2842-4927-ba84-9094f66acaec@github.com>

On Mon, 3 May 2021 18:58:43 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Some of tier1/tier2 tests failed on MacOSX/AArch64. The fix is similar to what JDK-8265984 does for ZGC, that disable write protections when patching code.
> 
> Test:
>   tier1 and tier2 with Shenandoah GC on MacOSX/AArch64

This pull request has now been integrated.

Changeset: 61bb6eca
Author:    Zhengyu Gu <zgu at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/61bb6eca3e34b3f8382614edccd167f7ecefba65
Stats:     4 lines in 1 file changed: 3 ins; 0 del; 1 mod

8266453: Shenandoah: Disable write protections before patching nmethod in nmethod_barrier on MacOSX/AArch64

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/jdk/pull/3843

From wkemper at openjdk.java.net  Wed May  5 18:30:23 2021
From: wkemper at openjdk.java.net (William Kemper)
Date: Wed, 5 May 2021 18:30:23 GMT
Subject: RFR: Fix Region Promotion
Message-ID: <c0ceGxliYEOg3eJ3vqUtiH5p5ZMkudss519VeVZ6RPo=.4ebb64b0-ec72-482f-8bad-86ae0efd3a0b@github.com>

This is rebased pull request of https://github.com/openjdk/shenandoah/pull/32.

There are bugs in region promotion.

* Humongous object continuation region remainder object starts are not registered for remembered set scanning purposes.
* Also, iterating over humongous objects at promotion time to set card table values is inefficient. This is better delayed until card table scanning.
* Regions that are not either regular or humongous should not get promoted, in particular the cset region state needs to be excluded.
* We need to prevent regions that contain TLABs or GCLABs from getting promoted with these remaining active. One could retire those LABs once again at final update refs, but that would be inefficient. An alternative defensive and robust fix is to exclude regions that contain any TLAB or GCLAB from promotion. This should not restrict promotions much, because regions tend to fill up as they age and eventually they won't have LABs.

All of the above is fixed by the commits proposed here.
This PR is best applied after PR #30, the outcome of which it is rebased to at the moment.

-------------

Commit messages:
 - Prevent promotion of regions with active TLABs or GCLABs.
 - Fix humongous object promotion.
 - Remove remset object registration at a shared mere allocation.

Changes: https://git.openjdk.java.net/shenandoah/pull/36/files
 Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=36&range=00
  Stats: 140 lines in 7 files changed: 73 ins; 58 del; 9 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/36.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/36/head:pull/36

PR: https://git.openjdk.java.net/shenandoah/pull/36

From rkennke at openjdk.java.net  Wed May  5 19:33:13 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 5 May 2021 19:33:13 GMT
Subject: RFR: Fix Region Promotion
In-Reply-To: <c0ceGxliYEOg3eJ3vqUtiH5p5ZMkudss519VeVZ6RPo=.4ebb64b0-ec72-482f-8bad-86ae0efd3a0b@github.com>
References: <c0ceGxliYEOg3eJ3vqUtiH5p5ZMkudss519VeVZ6RPo=.4ebb64b0-ec72-482f-8bad-86ae0efd3a0b@github.com>
Message-ID: <j5itGtribQDASjjtjmflV0khX3JrTZi8ci5an50Tsv4=.e2704ce6-2525-4466-8fab-32ea6b983150@github.com>

On Wed, 5 May 2021 18:22:38 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This is rebased pull request of https://github.com/openjdk/shenandoah/pull/32.
> 
> There are bugs in region promotion.
> 
> * Humongous object continuation region remainder object starts are not registered for remembered set scanning purposes.
> * Also, iterating over humongous objects at promotion time to set card table values is inefficient. This is better delayed until card table scanning.
> * Regions that are not either regular or humongous should not get promoted, in particular the cset region state needs to be excluded.
> * We need to prevent regions that contain TLABs or GCLABs from getting promoted with these remaining active. One could retire those LABs once again at final update refs, but that would be inefficient. An alternative defensive and robust fix is to exclude regions that contain any TLAB or GCLAB from promotion. This should not restrict promotions much, because regions tend to fill up as they age and eventually they won't have LABs.
> 
> All of the above is fixed by the commits proposed here.
> This PR is best applied after PR #30, the outcome of which it is rebased to at the moment.

Looks good. Only minor comment/question.

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1274:

> 1272:       r->set_young_lab_flag();
> 1273:     }
> 1274:   }

Are you not interested in TLABs and PLABs here?

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1293:

> 1291:       r->set_young_lab_flag();
> 1292:     }
> 1293:   }

Ah ok I see you handle TLABs here. Maybe move this into the ShenandoahTagGCLABClosure? What about PLABs?

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/36

From wkemper at openjdk.java.net  Wed May  5 22:43:35 2021
From: wkemper at openjdk.java.net (William Kemper)
Date: Wed, 5 May 2021 22:43:35 GMT
Subject: RFR: Fix Region Promotion
In-Reply-To: <j5itGtribQDASjjtjmflV0khX3JrTZi8ci5an50Tsv4=.e2704ce6-2525-4466-8fab-32ea6b983150@github.com>
References: <c0ceGxliYEOg3eJ3vqUtiH5p5ZMkudss519VeVZ6RPo=.4ebb64b0-ec72-482f-8bad-86ae0efd3a0b@github.com>
 <j5itGtribQDASjjtjmflV0khX3JrTZi8ci5an50Tsv4=.e2704ce6-2525-4466-8fab-32ea6b983150@github.com>
Message-ID: <kZDyUipmvozZ2-NeKOMzpDo1Jw5I4W7J-0CC9wMExVk=.c1fea4f8-0a96-4b24-877e-ba5e1fc9e647@github.com>

On Wed, 5 May 2021 19:28:15 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This is rebased pull request of https://github.com/openjdk/shenandoah/pull/32.
>> 
>> There are bugs in region promotion.
>> 
>> * Humongous object continuation region remainder object starts are not registered for remembered set scanning purposes.
>> * Also, iterating over humongous objects at promotion time to set card table values is inefficient. This is better delayed until card table scanning.
>> * Regions that are not either regular or humongous should not get promoted, in particular the cset region state needs to be excluded.
>> * We need to prevent regions that contain TLABs or GCLABs from getting promoted with these remaining active. One could retire those LABs once again at final update refs, but that would be inefficient. An alternative defensive and robust fix is to exclude regions that contain any TLAB or GCLAB from promotion. This should not restrict promotions much, because regions tend to fill up as they age and eventually they won't have LABs.
>> 
>> All of the above is fixed by the commits proposed here.
>> This PR is best applied after PR #30, the outcome of which it is rebased to at the moment.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1293:
> 
>> 1291:       r->set_young_lab_flag();
>> 1292:     }
>> 1293:   }
> 
> Ah ok I see you handle TLABs here. Maybe move this into the ShenandoahTagGCLABClosure? What about PLABs?

I think we aren't worried about PLABs here because they should only exist in old regions (i.e., ones that have already been promoted). I'll refactor this to not loop over the threads twice and add an assert that the thread's plab exists in an old region.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/36

From wkemper at openjdk.java.net  Wed May  5 23:26:10 2021
From: wkemper at openjdk.java.net (William Kemper)
Date: Wed, 5 May 2021 23:26:10 GMT
Subject: RFR: Fix Region Promotion
In-Reply-To: <kZDyUipmvozZ2-NeKOMzpDo1Jw5I4W7J-0CC9wMExVk=.c1fea4f8-0a96-4b24-877e-ba5e1fc9e647@github.com>
References: <c0ceGxliYEOg3eJ3vqUtiH5p5ZMkudss519VeVZ6RPo=.4ebb64b0-ec72-482f-8bad-86ae0efd3a0b@github.com>
 <j5itGtribQDASjjtjmflV0khX3JrTZi8ci5an50Tsv4=.e2704ce6-2525-4466-8fab-32ea6b983150@github.com>
 <kZDyUipmvozZ2-NeKOMzpDo1Jw5I4W7J-0CC9wMExVk=.c1fea4f8-0a96-4b24-877e-ba5e1fc9e647@github.com>
Message-ID: <w6wW_jmJAVVvtxGhKmyzlZGGwaQ6Yrs6C53TbZTs284=.ba41ed5e-8da6-4849-8c78-0c50c8155482@github.com>

On Wed, 5 May 2021 22:39:17 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1293:
>> 
>>> 1291:       r->set_young_lab_flag();
>>> 1292:     }
>>> 1293:   }
>> 
>> Ah ok I see you handle TLABs here. Maybe move this into the ShenandoahTagGCLABClosure? What about PLABs?
>
> I think we aren't worried about PLABs here because they should only exist in old regions (i.e., ones that have already been promoted). I'll refactor this to not loop over the threads twice and add an assert that the thread's plab exists in an old region.

On second look, the `GCLABClosure` is only visiting the worker threads (they won't have TLABs).

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/36

From rkennke at openjdk.java.net  Thu May  6 09:11:25 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 6 May 2021 09:11:25 GMT
Subject: RFR: Fix Region Promotion
In-Reply-To: <c0ceGxliYEOg3eJ3vqUtiH5p5ZMkudss519VeVZ6RPo=.4ebb64b0-ec72-482f-8bad-86ae0efd3a0b@github.com>
References: <c0ceGxliYEOg3eJ3vqUtiH5p5ZMkudss519VeVZ6RPo=.4ebb64b0-ec72-482f-8bad-86ae0efd3a0b@github.com>
Message-ID: <TGD7tujLrehk9_YZ_dmOq74Tgxynf_C5OSGz4oHJJw0=.a7982a77-3818-4d53-ba58-09fa9f5f6fd9@github.com>

On Wed, 5 May 2021 18:22:38 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This is rebased pull request of https://github.com/openjdk/shenandoah/pull/32.
> 
> There are bugs in region promotion.
> 
> * Humongous object continuation region remainder object starts are not registered for remembered set scanning purposes.
> * Also, iterating over humongous objects at promotion time to set card table values is inefficient. This is better delayed until card table scanning.
> * Regions that are not either regular or humongous should not get promoted, in particular the cset region state needs to be excluded.
> * We need to prevent regions that contain TLABs or GCLABs from getting promoted with these remaining active. One could retire those LABs once again at final update refs, but that would be inefficient. An alternative defensive and robust fix is to exclude regions that contain any TLAB or GCLAB from promotion. This should not restrict promotions much, because regions tend to fill up as they age and eventually they won't have LABs.
> 
> All of the above is fixed by the commits proposed here.
> This PR is best applied after PR #30, the outcome of which it is rebased to at the moment.

Right. Ok then.

-------------

Marked as reviewed by rkennke (Lead).

PR: https://git.openjdk.java.net/shenandoah/pull/36

From github.com+80801020+wattsun110 at openjdk.java.net  Thu May  6 09:19:51 2021
From: github.com+80801020+wattsun110 at openjdk.java.net (wattsun110)
Date: Thu, 6 May 2021 09:19:51 GMT
Subject: Integrated: 8266018: Shenandoah: fix an incorrect assert
In-Reply-To: <-b7LvBwZVRgeB0LNtWPCcs7lm8Rac6l7tcZSvPTVu54=.87356765-2b10-48bb-bca4-cc103a6ec611@github.com>
References: <-b7LvBwZVRgeB0LNtWPCcs7lm8Rac6l7tcZSvPTVu54=.87356765-2b10-48bb-bca4-cc103a6ec611@github.com>
Message-ID: <C68Skoff4jx-smfviDW_lC3AYnhVm09cWoEY6oAkWy8=.bffa5826-9acd-482a-9a63-0b6db9db0614@github.com>

On Tue, 27 Apr 2021 03:10:52 GMT, wattsun110 <github.com+80801020+wattsun110 at openjdk.org> wrote:

> May I get reviews for this one-line change which fixes an incorrect assert?
> 
> thanks
> watt

This pull request has now been integrated.

Changeset: 2dd56676
Author:    wattsun <wattsun at tencent.com>
Committer: Roman Kennke <rkennke at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/2dd56676fb564072f73f5fd9e591bda7922523ac
Stats:     1 line in 1 file changed: 0 ins; 0 del; 1 mod

8266018: Shenandoah: fix an incorrect assert

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/jdk/pull/3710

From zgu at openjdk.java.net  Thu May  6 14:50:07 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 6 May 2021 14:50:07 GMT
Subject: RFR: 8266440: Shenandoah: TestReferenceShortcutCycle.java test failed
 on AArch64
Message-ID: <ct9Rnr2wDEzWMNhtadFCykyU0vqgkhXHH5cm6oyFUPE=.3ba1acd9-7ee9-4056-9d1f-3d38d58b9208@github.com>

JDK-8263427 missed corresponding changes for aarch64.

LRB needs to be enabled during weak roots/references processing, regardless if heap is stable.

Test:

- [x] hotspot_gc_shenandoah on Macosx/AArch64
- [ ] hotspot_gc_shenandoah on Linux 64/AArch64

-------------

Commit messages:
 - update
 - v1

Changes: https://git.openjdk.java.net/jdk/pull/3902/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3902&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8266440
  Stats: 15 lines in 1 file changed: 14 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3902.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3902/head:pull/3902

PR: https://git.openjdk.java.net/jdk/pull/3902

From wkemper at openjdk.java.net  Thu May  6 15:41:17 2021
From: wkemper at openjdk.java.net (William Kemper)
Date: Thu, 6 May 2021 15:41:17 GMT
Subject: Integrated: Fix Region Promotion
In-Reply-To: <c0ceGxliYEOg3eJ3vqUtiH5p5ZMkudss519VeVZ6RPo=.4ebb64b0-ec72-482f-8bad-86ae0efd3a0b@github.com>
References: <c0ceGxliYEOg3eJ3vqUtiH5p5ZMkudss519VeVZ6RPo=.4ebb64b0-ec72-482f-8bad-86ae0efd3a0b@github.com>
Message-ID: <OWuco9t-GNQNRp9hp8FM1URtC1ym39I5LS8PjzCuIpU=.a205ad0f-4b28-466f-bebe-110994f613c1@github.com>

On Wed, 5 May 2021 18:22:38 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This is rebased pull request of https://github.com/openjdk/shenandoah/pull/32.
> 
> There are bugs in region promotion.
> 
> * Humongous object continuation region remainder object starts are not registered for remembered set scanning purposes.
> * Also, iterating over humongous objects at promotion time to set card table values is inefficient. This is better delayed until card table scanning.
> * Regions that are not either regular or humongous should not get promoted, in particular the cset region state needs to be excluded.
> * We need to prevent regions that contain TLABs or GCLABs from getting promoted with these remaining active. One could retire those LABs once again at final update refs, but that would be inefficient. An alternative defensive and robust fix is to exclude regions that contain any TLAB or GCLAB from promotion. This should not restrict promotions much, because regions tend to fill up as they age and eventually they won't have LABs.
> 
> All of the above is fixed by the commits proposed here.
> This PR is best applied after PR #30, the outcome of which it is rebased to at the moment.

This pull request has now been integrated.

Changeset: 41ed8466
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.java.net/shenandoah/commit/41ed8466724ff576815ac35a1f89ca55f105eca6
Stats:     140 lines in 7 files changed: 73 ins; 58 del; 9 mod

Fix Region Promotion

Co-authored-by: Bernd Mathiske <bmathiske at openjdk.org>
Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/36

From ngasson at openjdk.java.net  Fri May  7 03:22:07 2021
From: ngasson at openjdk.java.net (Nick Gasson)
Date: Fri, 7 May 2021 03:22:07 GMT
Subject: RFR: 8266440: Shenandoah: TestReferenceShortcutCycle.java test
 failed on AArch64
In-Reply-To: <ct9Rnr2wDEzWMNhtadFCykyU0vqgkhXHH5cm6oyFUPE=.3ba1acd9-7ee9-4056-9d1f-3d38d58b9208@github.com>
References: <ct9Rnr2wDEzWMNhtadFCykyU0vqgkhXHH5cm6oyFUPE=.3ba1acd9-7ee9-4056-9d1f-3d38d58b9208@github.com>
Message-ID: <KlAW3tExd6Px1Fhfi6fzPz85VItw35q8amZaizIZvpU=.0f9fde05-22ca-4907-8cad-39587a48739c@github.com>

On Thu, 6 May 2021 14:40:51 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> JDK-8263427 missed corresponding changes for aarch64.
> 
> LRB needs to be enabled during weak roots/references processing, regardless if heap is stable.
> 
> Test:
> 
> - [x] hotspot_gc_shenandoah on Macosx/AArch64
> - [ ] hotspot_gc_shenandoah on Linux 64/AArch64

src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp line 249:

> 247:   } else {
> 248:     if (dst == rscratch1) {
> 249:       __ push(rscratch1, sp);

Could we avoid the push/pop by passing in an extra temporary? We have `tmp1` available in `ShenandoahBarrierSetAssembler::load_at()` which is the only place that calls this.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3902

From zgu at openjdk.java.net  Fri May  7 12:36:53 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Fri, 7 May 2021 12:36:53 GMT
Subject: RFR: 8266440: Shenandoah: TestReferenceShortcutCycle.java test
 failed on AArch64
In-Reply-To: <KlAW3tExd6Px1Fhfi6fzPz85VItw35q8amZaizIZvpU=.0f9fde05-22ca-4907-8cad-39587a48739c@github.com>
References: <ct9Rnr2wDEzWMNhtadFCykyU0vqgkhXHH5cm6oyFUPE=.3ba1acd9-7ee9-4056-9d1f-3d38d58b9208@github.com>
 <KlAW3tExd6Px1Fhfi6fzPz85VItw35q8amZaizIZvpU=.0f9fde05-22ca-4907-8cad-39587a48739c@github.com>
Message-ID: <FWojWvyXJR47qji7jS63POPC9slszN47yH70KB_fTf8=.750f5a6f-a0f8-466d-a0c1-8efabf322b5b@github.com>

On Fri, 7 May 2021 03:19:08 GMT, Nick Gasson <ngasson at openjdk.org> wrote:

>> JDK-8263427 missed corresponding changes for aarch64.
>> 
>> LRB needs to be enabled during weak roots/references processing, regardless if heap is stable.
>> 
>> Test:
>> 
>> - [x] hotspot_gc_shenandoah on Macosx/AArch64
>> - [ ] hotspot_gc_shenandoah on Linux 64/AArch64
>
> src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp line 249:
> 
>> 247:   } else {
>> 248:     if (dst == rscratch1) {
>> 249:       __ push(rscratch1, sp);
> 
> Could we avoid the push/pop by passing in an extra temporary? We have `tmp1` available in `ShenandoahBarrierSetAssembler::load_at()` which is the only place that calls this.

Unfortunately, tmp1 == noreg here.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3902

From aph at openjdk.java.net  Fri May  7 14:34:00 2021
From: aph at openjdk.java.net (Andrew Haley)
Date: Fri, 7 May 2021 14:34:00 GMT
Subject: RFR: 8266440: Shenandoah: TestReferenceShortcutCycle.java test
 failed on AArch64
In-Reply-To: <KlAW3tExd6Px1Fhfi6fzPz85VItw35q8amZaizIZvpU=.0f9fde05-22ca-4907-8cad-39587a48739c@github.com>
References: <ct9Rnr2wDEzWMNhtadFCykyU0vqgkhXHH5cm6oyFUPE=.3ba1acd9-7ee9-4056-9d1f-3d38d58b9208@github.com>
 <KlAW3tExd6Px1Fhfi6fzPz85VItw35q8amZaizIZvpU=.0f9fde05-22ca-4907-8cad-39587a48739c@github.com>
Message-ID: <5l0vBLGhIxGcPTyRnM2o3fVnK03myq0CZoB7PipwMO0=.24f80b40-78c1-4f82-98e2-6d34febc8de9@github.com>

On Fri, 7 May 2021 03:19:08 GMT, Nick Gasson <ngasson at openjdk.org> wrote:

>> JDK-8263427 missed corresponding changes for aarch64.
>> 
>> LRB needs to be enabled during weak roots/references processing, regardless if heap is stable.
>> 
>> Test:
>> 
>> - [x] hotspot_gc_shenandoah on Macosx/AArch64
>> - [ ] hotspot_gc_shenandoah on Linux 64/AArch64
>
> src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp line 249:
> 
>> 247:   } else {
>> 248:     if (dst == rscratch1) {
>> 249:       __ push(rscratch1, sp);
> 
> Could we avoid the push/pop by passing in an extra temporary? We have `tmp1` available in `ShenandoahBarrierSetAssembler::load_at()` which is the only place that calls this.

We don't need a register. It's just a TBNZ and a TBZ, isn't it?

-------------

PR: https://git.openjdk.java.net/jdk/pull/3902

From zgu at openjdk.java.net  Fri May  7 15:09:52 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Fri, 7 May 2021 15:09:52 GMT
Subject: RFR: 8266440: Shenandoah: TestReferenceShortcutCycle.java test
 failed on AArch64
In-Reply-To: <FWojWvyXJR47qji7jS63POPC9slszN47yH70KB_fTf8=.750f5a6f-a0f8-466d-a0c1-8efabf322b5b@github.com>
References: <ct9Rnr2wDEzWMNhtadFCykyU0vqgkhXHH5cm6oyFUPE=.3ba1acd9-7ee9-4056-9d1f-3d38d58b9208@github.com>
 <KlAW3tExd6Px1Fhfi6fzPz85VItw35q8amZaizIZvpU=.0f9fde05-22ca-4907-8cad-39587a48739c@github.com>
 <FWojWvyXJR47qji7jS63POPC9slszN47yH70KB_fTf8=.750f5a6f-a0f8-466d-a0c1-8efabf322b5b@github.com>
Message-ID: <fhRKGqqXcUQ6yVL3Kh623Pe-hwjeaSz-Ol57plriUHs=.566b15a9-d9d6-4f79-9913-3abe5a96d51b@github.com>

On Fri, 7 May 2021 12:33:35 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> We don't need a register. It's just a TBNZ and a TBZ, isn't it?
>
> Unfortunately, tmp1 == noreg here.

so, this is better alternative?
    `Label lrb;`
    `__ tbnz(rscratch2, ShenandoahHeap::HAS_FORWARDED_BITPOS, lrb);`
    `__ tbnz(rscratch2, ShenandoahHeap::WEAK_ROOTS_BITPOS, lrb);`
    `__ b(heap_stable);`
    `__ bind(lrb);`

-------------

PR: https://git.openjdk.java.net/jdk/pull/3902

From zgu at openjdk.java.net  Fri May  7 19:45:19 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Fri, 7 May 2021 19:45:19 GMT
Subject: RFR: 8266740: Shenandoah: Heap region count calculation still wrong
 after JDK-8265239
Message-ID: <x1kon6odNQapKNmmfKPteFBLi6xTWNKyrKW5JyATuOc=.c22a0a6d-9a71-4dec-96b9-29f7295bc3c5@github.com>

JDK-8265239 only addressed heap size that needed to be aligned to region size, it did not address the adjustment needed when use large page.

To avoid all the complication, I purpose to defer region count calculation until heap size adjustment is done.

Test:
- [x] hotspot_gc_shenandoah
- [x] runtime/os/TestTracePageSizes.java now passed

-------------

Commit messages:
 - Merge branch 'master' into JDK-8266740-region-count-2
 - v1

Changes: https://git.openjdk.java.net/jdk/pull/3922/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3922&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8266740
  Stats: 15 lines in 3 files changed: 9 ins; 5 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3922.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3922/head:pull/3922

PR: https://git.openjdk.java.net/jdk/pull/3922

From aph at openjdk.java.net  Sat May  8 08:59:19 2021
From: aph at openjdk.java.net (Andrew Haley)
Date: Sat, 8 May 2021 08:59:19 GMT
Subject: RFR: 8266440: Shenandoah: TestReferenceShortcutCycle.java test
 failed on AArch64
In-Reply-To: <fhRKGqqXcUQ6yVL3Kh623Pe-hwjeaSz-Ol57plriUHs=.566b15a9-d9d6-4f79-9913-3abe5a96d51b@github.com>
References: <ct9Rnr2wDEzWMNhtadFCykyU0vqgkhXHH5cm6oyFUPE=.3ba1acd9-7ee9-4056-9d1f-3d38d58b9208@github.com>
 <KlAW3tExd6Px1Fhfi6fzPz85VItw35q8amZaizIZvpU=.0f9fde05-22ca-4907-8cad-39587a48739c@github.com>
 <FWojWvyXJR47qji7jS63POPC9slszN47yH70KB_fTf8=.750f5a6f-a0f8-466d-a0c1-8efabf322b5b@github.com>
 <fhRKGqqXcUQ6yVL3Kh623Pe-hwjeaSz-Ol57plriUHs=.566b15a9-d9d6-4f79-9913-3abe5a96d51b@github.com>
Message-ID: <0A3WWPkNU0nlPYeUfJ7BJkIVZASDqf6KzJPbrJ92xvI=.c0abd4fd-a002-41e0-bc9b-808f567083ca@github.com>

On Fri, 7 May 2021 15:05:38 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> Unfortunately, tmp1 == noreg here.
>
> so, this is better alternative?
>     `Label lrb;`
>     `__ tbnz(rscratch2, ShenandoahHeap::HAS_FORWARDED_BITPOS, lrb);`
>     `__ tbnz(rscratch2, ShenandoahHeap::WEAK_ROOTS_BITPOS, lrb);`
>     `__ b(heap_stable);`
>     `__ bind(lrb);`

How about

 // Check for heap stability                                                                                                                                                                
  if (is_strong) {
    __ tbz(rscratch2, ShenandoahHeap::HAS_FORWARDED_BITPOS, heap_stable);
  } else {
    label lrb;
    __ tbnz(rscratch2, ShenandoahHeap::WEAK_ROOTS_BITPOS, lrb);
    __ tbz(rscratch2, ShenandoahHeap::HAS_FORWARDED_BITPOS, heap_stable);
    __ bind(lrb);
  }

-------------

PR: https://git.openjdk.java.net/jdk/pull/3902

From zgu at openjdk.java.net  Sat May  8 15:22:30 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Sat, 8 May 2021 15:22:30 GMT
Subject: RFR: 8266440: Shenandoah: TestReferenceShortcutCycle.java test
 failed on AArch64 [v2]
In-Reply-To: <ct9Rnr2wDEzWMNhtadFCykyU0vqgkhXHH5cm6oyFUPE=.3ba1acd9-7ee9-4056-9d1f-3d38d58b9208@github.com>
References: <ct9Rnr2wDEzWMNhtadFCykyU0vqgkhXHH5cm6oyFUPE=.3ba1acd9-7ee9-4056-9d1f-3d38d58b9208@github.com>
Message-ID: <SSeIRCcO-q3OgoUtR2bP-MxryYgi0C9fwUab7zEP9lE=.59e2e5f8-471e-4415-9465-5fd80715c703@github.com>

> JDK-8263427 missed corresponding changes for aarch64.
> 
> LRB needs to be enabled during weak roots/references processing, regardless if heap is stable.
> 
> Test:
> 
> - [x] hotspot_gc_shenandoah on Macosx/AArch64
> - [ ] hotspot_gc_shenandoah on Linux 64/AArch64

Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:

  v2

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/3902/files
  - new: https://git.openjdk.java.net/jdk/pull/3902/files/fea11bc4..4bb453b9

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3902&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3902&range=00-01

  Stats: 11 lines in 1 file changed: 0 ins; 7 del; 4 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3902.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3902/head:pull/3902

PR: https://git.openjdk.java.net/jdk/pull/3902

From zgu at openjdk.java.net  Sat May  8 15:22:32 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Sat, 8 May 2021 15:22:32 GMT
Subject: RFR: 8266440: Shenandoah: TestReferenceShortcutCycle.java test
 failed on AArch64 [v2]
In-Reply-To: <0A3WWPkNU0nlPYeUfJ7BJkIVZASDqf6KzJPbrJ92xvI=.c0abd4fd-a002-41e0-bc9b-808f567083ca@github.com>
References: <ct9Rnr2wDEzWMNhtadFCykyU0vqgkhXHH5cm6oyFUPE=.3ba1acd9-7ee9-4056-9d1f-3d38d58b9208@github.com>
 <KlAW3tExd6Px1Fhfi6fzPz85VItw35q8amZaizIZvpU=.0f9fde05-22ca-4907-8cad-39587a48739c@github.com>
 <FWojWvyXJR47qji7jS63POPC9slszN47yH70KB_fTf8=.750f5a6f-a0f8-466d-a0c1-8efabf322b5b@github.com>
 <fhRKGqqXcUQ6yVL3Kh623Pe-hwjeaSz-Ol57plriUHs=.566b15a9-d9d6-4f79-9913-3abe5a96d51b@github.com>
 <0A3WWPkNU0nlPYeUfJ7BJkIVZASDqf6KzJPbrJ92xvI=.c0abd4fd-a002-41e0-bc9b-808f567083ca@github.com>
Message-ID: <GR7pgoDRdzs6ZCCwiejdwwyFvMWctb16CHxa53njdEM=.d07b6737-06f5-4d53-b6d8-5bdd87f42ae0@github.com>

On Sat, 8 May 2021 08:55:47 GMT, Andrew Haley <aph at openjdk.org> wrote:

>> so, this is better alternative?
>>     `Label lrb;`
>>     `__ tbnz(rscratch2, ShenandoahHeap::HAS_FORWARDED_BITPOS, lrb);`
>>     `__ tbnz(rscratch2, ShenandoahHeap::WEAK_ROOTS_BITPOS, lrb);`
>>     `__ b(heap_stable);`
>>     `__ bind(lrb);`
>
> How about
> 
>  // Check for heap stability                                                                                                                                                                
>   if (is_strong) {
>     __ tbz(rscratch2, ShenandoahHeap::HAS_FORWARDED_BITPOS, heap_stable);
>   } else {
>     label lrb;
>     __ tbnz(rscratch2, ShenandoahHeap::WEAK_ROOTS_BITPOS, lrb);
>     __ tbz(rscratch2, ShenandoahHeap::HAS_FORWARDED_BITPOS, heap_stable);
>     __ bind(lrb);
>   }

Yep, much better. Thanks.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3902

From aph at openjdk.java.net  Mon May 10 09:51:22 2021
From: aph at openjdk.java.net (Andrew Haley)
Date: Mon, 10 May 2021 09:51:22 GMT
Subject: RFR: 8266440: Shenandoah: TestReferenceShortcutCycle.java test
 failed on AArch64 [v2]
In-Reply-To: <SSeIRCcO-q3OgoUtR2bP-MxryYgi0C9fwUab7zEP9lE=.59e2e5f8-471e-4415-9465-5fd80715c703@github.com>
References: <ct9Rnr2wDEzWMNhtadFCykyU0vqgkhXHH5cm6oyFUPE=.3ba1acd9-7ee9-4056-9d1f-3d38d58b9208@github.com>
 <SSeIRCcO-q3OgoUtR2bP-MxryYgi0C9fwUab7zEP9lE=.59e2e5f8-471e-4415-9465-5fd80715c703@github.com>
Message-ID: <uE9Sk60ZRPaVSQIdQ_WpbpMHHHQvlAVTw1FOnp2BT3A=.acca6921-a885-4f5e-b67e-d015f89d44a7@github.com>

On Sat, 8 May 2021 15:22:30 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> JDK-8263427 missed corresponding changes for aarch64.
>> 
>> LRB needs to be enabled during weak roots/references processing, regardless if heap is stable.
>> 
>> Test:
>> 
>> - [x] hotspot_gc_shenandoah on Macosx/AArch64
>> - [ ] hotspot_gc_shenandoah on Linux 64/AArch64
>
> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
> 
>   v2

Marked as reviewed by aph (Reviewer).

-------------

PR: https://git.openjdk.java.net/jdk/pull/3902

From shade at openjdk.java.net  Mon May 10 13:35:40 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 10 May 2021 13:35:40 GMT
Subject: RFR: JDK-8266802: Shenandoah: Round up region size to page size
 unconditionally
Message-ID: <pfANUKmt7PAlHVuzLYcG2VfibLp9AwnNZ6mPkwn_BGw=.b7bacee0-c22d-46b4-8030-3ebf833a2ae9@github.com>

Since JDK-8265239, runtime/os/TestTracePageSizes.java fails with Shenandoah like this:


# Internal Error (/home/buildbot/worker/test-jdkX-linux/build/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp:156), pid=55712, tid=55720
# assert(_num_regions == (max_byte_size / reg_size_bytes)) failed: Must match


This is because Shenandoah region sizes are rounded up to page size only when `ShenandoahUncommit` is true. In the failing test configuration, `ShenandoahUncommit` is false, because Xmx == Xms.

Additional testing:
 - [x] New regression test, now passes
 - [x] `tier1` with Shenandoah

-------------

Commit messages:
 - Minor whitespace
 - Fix and regression tests

Changes: https://git.openjdk.java.net/jdk/pull/3945/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3945&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8266802
  Stats: 65 lines in 3 files changed: 60 ins; 0 del; 5 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3945.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3945/head:pull/3945

PR: https://git.openjdk.java.net/jdk/pull/3945

From zgu at openjdk.java.net  Mon May 10 13:50:53 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 10 May 2021 13:50:53 GMT
Subject: RFR: 8266813: Use shorter instruction sequence for checking if
 marking in progress
Message-ID: <yeombb6w6WduVD-KsuJ-_89WLX42tgN-Fr9lOkyzKXk=.f835637c-eb02-4d58-b56a-1739487f8378@github.com>

ShenandoahBarrierSetAssembler::generate_c1_pre_barrier_runtime_stub() on aarch64, we can replace ld, tst, br instruction sequence with a single tbz.

Test:
- [x] hotspot_gc_shenandoah on Linux aarch64

-------------

Commit messages:
 - v1

Changes: https://git.openjdk.java.net/jdk/pull/3948/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3948&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8266813
  Stats: 3 lines in 1 file changed: 0 ins; 2 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3948.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3948/head:pull/3948

PR: https://git.openjdk.java.net/jdk/pull/3948

From zgu at openjdk.java.net  Mon May 10 14:05:06 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 10 May 2021 14:05:06 GMT
Subject: RFR: JDK-8266802: Shenandoah: Round up region size to page size
 unconditionally
In-Reply-To: <pfANUKmt7PAlHVuzLYcG2VfibLp9AwnNZ6mPkwn_BGw=.b7bacee0-c22d-46b4-8030-3ebf833a2ae9@github.com>
References: <pfANUKmt7PAlHVuzLYcG2VfibLp9AwnNZ6mPkwn_BGw=.b7bacee0-c22d-46b4-8030-3ebf833a2ae9@github.com>
Message-ID: <2VdTzWYz2gIvbF8J5UIVHWiftCuectuAZsuIuy5SIno=.b92dfd02-b678-4a08-a55a-57723cb5b1b8@github.com>

On Mon, 10 May 2021 08:35:06 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> Since JDK-8265239, runtime/os/TestTracePageSizes.java fails with Shenandoah like this:
> 
> 
> # Internal Error (/home/buildbot/worker/test-jdkX-linux/build/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp:156), pid=55712, tid=55720
> # assert(_num_regions == (max_byte_size / reg_size_bytes)) failed: Must match
> 
> 
> This is because Shenandoah region sizes are rounded up to page size only when `ShenandoahUncommit` is true. In the failing test configuration, `ShenandoahUncommit` is false, because Xmx == Xms.
> 
> Additional testing:
>  - [x] New regression test, now passes
>  - [x] `tier1` with Shenandoah

I filed JDK-8266740, that is related. I believe we should defer region count calculation after heap size adjustment to avoid miscalculating the value. Do you think we should consolidate the 2 CRs?

-------------

PR: https://git.openjdk.java.net/jdk/pull/3945

From shade at openjdk.java.net  Mon May 10 15:04:27 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 10 May 2021 15:04:27 GMT
Subject: RFR: JDK-8266802: Shenandoah: Round up region size to page size
 unconditionally [v2]
In-Reply-To: <pfANUKmt7PAlHVuzLYcG2VfibLp9AwnNZ6mPkwn_BGw=.b7bacee0-c22d-46b4-8030-3ebf833a2ae9@github.com>
References: <pfANUKmt7PAlHVuzLYcG2VfibLp9AwnNZ6mPkwn_BGw=.b7bacee0-c22d-46b4-8030-3ebf833a2ae9@github.com>
Message-ID: <Xb3oDCIRPQG-CERoW9ZZ9mCanFP6mu2uxKisBp8ycwo=.c717ed27-ae46-477f-8c48-7ccf76e81303@github.com>

> Since JDK-8265239, runtime/os/TestTracePageSizes.java fails with Shenandoah like this:
> 
> 
> # Internal Error (/home/buildbot/worker/test-jdkX-linux/build/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp:156), pid=55712, tid=55720
> # assert(_num_regions == (max_byte_size / reg_size_bytes)) failed: Must match
> 
> 
> This is because Shenandoah region sizes are rounded up to page size only when `ShenandoahUncommit` is true. In the failing test configuration, `ShenandoahUncommit` is false, because Xmx == Xms.
> 
> Additional testing:
>  - [x] New regression test, now passes
>  - [x] `tier1` with Shenandoah

Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:

 - Merge branch 'master' into JDK-8266802-shenandoah-region-size
 - Minor whitespace
 - Fix and regression tests

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/3945/files
  - new: https://git.openjdk.java.net/jdk/pull/3945/files/94d4f023..96a4778b

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3945&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3945&range=00-01

  Stats: 722 lines in 17 files changed: 390 ins; 203 del; 129 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3945.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3945/head:pull/3945

PR: https://git.openjdk.java.net/jdk/pull/3945

From zgu at openjdk.java.net  Mon May 10 15:18:28 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 10 May 2021 15:18:28 GMT
Subject: RFR: JDK-8266802: Shenandoah: Round up region size to page size
 unconditionally
In-Reply-To: <2VdTzWYz2gIvbF8J5UIVHWiftCuectuAZsuIuy5SIno=.b92dfd02-b678-4a08-a55a-57723cb5b1b8@github.com>
References: <pfANUKmt7PAlHVuzLYcG2VfibLp9AwnNZ6mPkwn_BGw=.b7bacee0-c22d-46b4-8030-3ebf833a2ae9@github.com>
 <2VdTzWYz2gIvbF8J5UIVHWiftCuectuAZsuIuy5SIno=.b92dfd02-b678-4a08-a55a-57723cb5b1b8@github.com>
Message-ID: <Oeoc0mGWDlWsDTsu9QyeobcZXyf_--u6M0xUqI7uUCY=.765cadd0-2637-45dd-88c8-dc5c92b19ad2@github.com>

On Mon, 10 May 2021 14:03:00 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> I filed JDK-8266740, that is related. I believe we should defer region count calculation after heap size adjustment to avoid miscalculating the value. Do you think we should consolidate the 2 CRs?

Never mind. Your change looks good. I will make JDK-8266740 a dup.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3945

From zgu at openjdk.java.net  Mon May 10 15:18:27 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 10 May 2021 15:18:27 GMT
Subject: RFR: JDK-8266802: Shenandoah: Round up region size to page size
 unconditionally [v2]
In-Reply-To: <Xb3oDCIRPQG-CERoW9ZZ9mCanFP6mu2uxKisBp8ycwo=.c717ed27-ae46-477f-8c48-7ccf76e81303@github.com>
References: <pfANUKmt7PAlHVuzLYcG2VfibLp9AwnNZ6mPkwn_BGw=.b7bacee0-c22d-46b4-8030-3ebf833a2ae9@github.com>
 <Xb3oDCIRPQG-CERoW9ZZ9mCanFP6mu2uxKisBp8ycwo=.c717ed27-ae46-477f-8c48-7ccf76e81303@github.com>
Message-ID: <Mo_UMOyoayL_v8kyfdEpCNQQglkweevMdgmFvNvgRfY=.4435bc7d-8e74-4d35-9615-69a554c1d738@github.com>

On Mon, 10 May 2021 15:04:27 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Since JDK-8265239, runtime/os/TestTracePageSizes.java fails with Shenandoah like this:
>> 
>> 
>> # Internal Error (/home/buildbot/worker/test-jdkX-linux/build/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp:156), pid=55712, tid=55720
>> # assert(_num_regions == (max_byte_size / reg_size_bytes)) failed: Must match
>> 
>> 
>> This is because Shenandoah region sizes are rounded up to page size only when `ShenandoahUncommit` is true. In the failing test configuration, `ShenandoahUncommit` is false, because Xmx == Xms.
>> 
>> Additional testing:
>>  - [x] New regression test, now passes
>>  - [x] `tier1` with Shenandoah
>
> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
> 
>  - Merge branch 'master' into JDK-8266802-shenandoah-region-size
>  - Minor whitespace
>  - Fix and regression tests

Looks good

-------------

Marked as reviewed by zgu (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/3945

From zgu at openjdk.java.net  Mon May 10 15:19:57 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 10 May 2021 15:19:57 GMT
Subject: Withdrawn: 8266740: Shenandoah: Heap region count calculation still
 wrong after JDK-8265239
In-Reply-To: <x1kon6odNQapKNmmfKPteFBLi6xTWNKyrKW5JyATuOc=.c22a0a6d-9a71-4dec-96b9-29f7295bc3c5@github.com>
References: <x1kon6odNQapKNmmfKPteFBLi6xTWNKyrKW5JyATuOc=.c22a0a6d-9a71-4dec-96b9-29f7295bc3c5@github.com>
Message-ID: <ExJIG_1l493KWiG4scNHmbHRMl1RcPNE_JiwEhqfUw8=.4ea810c6-e30f-48a2-b51b-28bbed523d07@github.com>

On Fri, 7 May 2021 18:09:50 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> JDK-8265239 only addressed heap size that needed to be aligned to region size, it did not address the adjustment needed when use large page.
> 
> To avoid all the complication, I purpose to defer region count calculation until heap size adjustment is done.
> 
> Test:
> - [x] hotspot_gc_shenandoah
> - [x] runtime/os/TestTracePageSizes.java now passed

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3922

From zgu at openjdk.java.net  Mon May 10 15:19:57 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 10 May 2021 15:19:57 GMT
Subject: RFR: 8266740: Shenandoah: Heap region count calculation still
 wrong after JDK-8265239
In-Reply-To: <x1kon6odNQapKNmmfKPteFBLi6xTWNKyrKW5JyATuOc=.c22a0a6d-9a71-4dec-96b9-29f7295bc3c5@github.com>
References: <x1kon6odNQapKNmmfKPteFBLi6xTWNKyrKW5JyATuOc=.c22a0a6d-9a71-4dec-96b9-29f7295bc3c5@github.com>
Message-ID: <-1Hhh3qcJjq3IV1gyRN2x5rJfPofZHBnNsUXIu1rcyk=.9bf874ae-0485-42b4-bbce-ff3fbe3ab1df@github.com>

On Fri, 7 May 2021 18:09:50 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> JDK-8265239 only addressed heap size that needed to be aligned to region size, it did not address the adjustment needed when use large page.
> 
> To avoid all the complication, I purpose to defer region count calculation until heap size adjustment is done.
> 
> Test:
> - [x] hotspot_gc_shenandoah
> - [x] runtime/os/TestTracePageSizes.java now passed

A duplicate of JDK-8266802

-------------

PR: https://git.openjdk.java.net/jdk/pull/3922

From shade at openjdk.java.net  Mon May 10 16:36:07 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 10 May 2021 16:36:07 GMT
Subject: RFR: JDK-8266802: Shenandoah: Round up region size to page size
 unconditionally
In-Reply-To: <Oeoc0mGWDlWsDTsu9QyeobcZXyf_--u6M0xUqI7uUCY=.765cadd0-2637-45dd-88c8-dc5c92b19ad2@github.com>
References: <pfANUKmt7PAlHVuzLYcG2VfibLp9AwnNZ6mPkwn_BGw=.b7bacee0-c22d-46b4-8030-3ebf833a2ae9@github.com>
 <2VdTzWYz2gIvbF8J5UIVHWiftCuectuAZsuIuy5SIno=.b92dfd02-b678-4a08-a55a-57723cb5b1b8@github.com>
 <Oeoc0mGWDlWsDTsu9QyeobcZXyf_--u6M0xUqI7uUCY=.765cadd0-2637-45dd-88c8-dc5c92b19ad2@github.com>
Message-ID: <KSZZhybDa_n_99YYQm79GXv1dDkTTxvy91MtcsMbJUk=.295b92b2-d00e-4750-80c2-acf0c7e0776b@github.com>

On Mon, 10 May 2021 15:15:04 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> I filed JDK-8266740, that is related. I believe we should defer region count calculation after heap size adjustment to avoid miscalculating the value. Do you think we should consolidate the 2 CRs?

Oh, I have not seen that. Yes, I reckon doing the way this PR approaches it is cleaner. I'll integrate once tests are green.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3945

From shade at openjdk.java.net  Mon May 10 16:38:19 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 10 May 2021 16:38:19 GMT
Subject: RFR: 8266813: Use shorter instruction sequence for checking if
 marking in progress
In-Reply-To: <yeombb6w6WduVD-KsuJ-_89WLX42tgN-Fr9lOkyzKXk=.f835637c-eb02-4d58-b56a-1739487f8378@github.com>
References: <yeombb6w6WduVD-KsuJ-_89WLX42tgN-Fr9lOkyzKXk=.f835637c-eb02-4d58-b56a-1739487f8378@github.com>
Message-ID: <STs8VbOh2ZO62QowWj96gKk-AqmylqNwPQWHhe6lozg=.38861f07-d639-42dd-ae42-487a36a6ece2@github.com>

On Mon, 10 May 2021 13:21:44 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> ShenandoahBarrierSetAssembler::generate_c1_pre_barrier_runtime_stub() on aarch64, we can replace ld, tst, br instruction sequence with a single tbz.
> 
> Test:
> - [x] hotspot_gc_shenandoah on Linux aarch64

This looks fine to me.

-------------

Marked as reviewed by shade (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/3948

From shade at openjdk.java.net  Mon May 10 17:28:50 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 10 May 2021 17:28:50 GMT
Subject: RFR: JDK-8266802: Shenandoah: Round up region size to page size
 unconditionally [v3]
In-Reply-To: <pfANUKmt7PAlHVuzLYcG2VfibLp9AwnNZ6mPkwn_BGw=.b7bacee0-c22d-46b4-8030-3ebf833a2ae9@github.com>
References: <pfANUKmt7PAlHVuzLYcG2VfibLp9AwnNZ6mPkwn_BGw=.b7bacee0-c22d-46b4-8030-3ebf833a2ae9@github.com>
Message-ID: <BNqV3WYJp6DOqXXZ3hkFEjO__tsF9hwJh9i6R2V4HCg=.f63c549c-6775-4f5c-9d1a-12734a9e97dc@github.com>

> Since JDK-8265239, runtime/os/TestTracePageSizes.java fails with Shenandoah like this:
> 
> 
> # Internal Error (/home/buildbot/worker/test-jdkX-linux/build/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp:156), pid=55712, tid=55720
> # assert(_num_regions == (max_byte_size / reg_size_bytes)) failed: Must match
> 
> 
> This is because Shenandoah region sizes are rounded up to page size only when `ShenandoahUncommit` is true. In the failing test configuration, `ShenandoahUncommit` is false, because Xmx == Xms.
> 
> Additional testing:
>  - [x] New regression test, now passes
>  - [x] `tier1` with Shenandoah

Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:

  UseTransparentHugePages is only available on Linux

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/3945/files
  - new: https://git.openjdk.java.net/jdk/pull/3945/files/96a4778b..dd3ba65d

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3945&range=02
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3945&range=01-02

  Stats: 12 lines in 1 file changed: 11 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3945.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3945/head:pull/3945

PR: https://git.openjdk.java.net/jdk/pull/3945

From zgu at openjdk.java.net  Mon May 10 18:02:56 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 10 May 2021 18:02:56 GMT
Subject: Integrated: 8266440: Shenandoah: TestReferenceShortcutCycle.java test
 failed on AArch64
In-Reply-To: <ct9Rnr2wDEzWMNhtadFCykyU0vqgkhXHH5cm6oyFUPE=.3ba1acd9-7ee9-4056-9d1f-3d38d58b9208@github.com>
References: <ct9Rnr2wDEzWMNhtadFCykyU0vqgkhXHH5cm6oyFUPE=.3ba1acd9-7ee9-4056-9d1f-3d38d58b9208@github.com>
Message-ID: <kCC3zyKraJCt0h2ilfoDFfYe39XgcWOtpMm8LCMkOoY=.bb88bb79-a03d-4fac-ac36-cbf3ea3afc54@github.com>

On Thu, 6 May 2021 14:40:51 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> JDK-8263427 missed corresponding changes for aarch64.
> 
> LRB needs to be enabled during weak roots/references processing, regardless if heap is stable.
> 
> Test:
> 
> - [x] hotspot_gc_shenandoah on Macosx/AArch64
> - [ ] hotspot_gc_shenandoah on Linux 64/AArch64

This pull request has now been integrated.

Changeset: f78440ad
Author:    Zhengyu Gu <zgu at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/f78440ad438bca2a8929475fbecfbd3c920d48df
Stats:     8 lines in 1 file changed: 7 ins; 0 del; 1 mod

8266440: Shenandoah: TestReferenceShortcutCycle.java test failed on AArch64

Reviewed-by: aph

-------------

PR: https://git.openjdk.java.net/jdk/pull/3902

From shade at openjdk.java.net  Mon May 10 20:32:16 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 10 May 2021 20:32:16 GMT
Subject: Integrated: JDK-8266802: Shenandoah: Round up region size to page
 size unconditionally
In-Reply-To: <pfANUKmt7PAlHVuzLYcG2VfibLp9AwnNZ6mPkwn_BGw=.b7bacee0-c22d-46b4-8030-3ebf833a2ae9@github.com>
References: <pfANUKmt7PAlHVuzLYcG2VfibLp9AwnNZ6mPkwn_BGw=.b7bacee0-c22d-46b4-8030-3ebf833a2ae9@github.com>
Message-ID: <TEVIT8kG2OGBZtGFhuW9dAx3pT_FfON3NjVBvMGX-cc=.05b2b3e1-84cd-47ce-a08b-669014a6edb6@github.com>

On Mon, 10 May 2021 08:35:06 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> Since JDK-8265239, runtime/os/TestTracePageSizes.java fails with Shenandoah like this:
> 
> 
> # Internal Error (/home/buildbot/worker/test-jdkX-linux/build/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp:156), pid=55712, tid=55720
> # assert(_num_regions == (max_byte_size / reg_size_bytes)) failed: Must match
> 
> 
> This is because Shenandoah region sizes are rounded up to page size only when `ShenandoahUncommit` is true. In the failing test configuration, `ShenandoahUncommit` is false, because Xmx == Xms.
> 
> Additional testing:
>  - [x] New regression test, now passes
>  - [x] `tier1` with Shenandoah

This pull request has now been integrated.

Changeset: e5d3ee39
Author:    Aleksey Shipilev <shade at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/e5d3ee394ae940ee0111489e6e072f327ec29c3b
Stats:     76 lines in 3 files changed: 71 ins; 0 del; 5 mod

8266802: Shenandoah: Round up region size to page size unconditionally

Reviewed-by: zgu

-------------

PR: https://git.openjdk.java.net/jdk/pull/3945

From zgu at openjdk.java.net  Mon May 10 23:29:05 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 10 May 2021 23:29:05 GMT
Subject: RFR: 8266845: Shenandoah: Simplify SBS::load_reference_barrier
 implementation
Message-ID: <aEaMXVGa3wjSWDDJ6vRDeYXMfyqx99RyP4gLKd0-szk=.5bfb0e95-7f42-41a7-933d-82485b7e7f10@github.com>

Early return on null obj simplifies implementation and makes code more readable.

Test:
- [x] hotspot_gc_shenandoah

-------------

Commit messages:
 - update
 - Merge branch 'master' into lrb-null-obj
 - v1

Changes: https://git.openjdk.java.net/jdk/pull/3956/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3956&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8266845
  Stats: 8 lines in 1 file changed: 4 ins; 0 del; 4 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3956.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3956/head:pull/3956

PR: https://git.openjdk.java.net/jdk/pull/3956

From shade at openjdk.java.net  Tue May 11 06:44:55 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 11 May 2021 06:44:55 GMT
Subject: RFR: 8266845: Shenandoah: Simplify SBS::load_reference_barrier
 implementation
In-Reply-To: <aEaMXVGa3wjSWDDJ6vRDeYXMfyqx99RyP4gLKd0-szk=.5bfb0e95-7f42-41a7-933d-82485b7e7f10@github.com>
References: <aEaMXVGa3wjSWDDJ6vRDeYXMfyqx99RyP4gLKd0-szk=.5bfb0e95-7f42-41a7-933d-82485b7e7f10@github.com>
Message-ID: <pxSb3FXUymfwdFlWnu8CfCtcyoaQpvq1hr7PuuS3lJw=.a829e479-ca68-4082-9aae-c98c1bc177ef@github.com>

On Mon, 10 May 2021 20:14:14 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Early return on null obj simplifies implementation and makes code more readable.
> 
> Test:
> - [x] hotspot_gc_shenandoah

Changes requested by shade (Reviewer).

src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 105:

> 103:   if (obj == nullptr) {
> 104:     return nullptr;
> 105:   }

Let's use `NULL` consistently? Yes, it is an old style, but it is consistent with the code in the same file.

src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 131:

> 129:   oop fwd = load_reference_barrier(obj);
> 130:   assert(fwd != nullptr, "Sanity");
> 131:   if (ShenandoahSelfFixing && load_addr != NULL) {

I think dropping `fwd != obj` is not about the null-check. It is about not doing the CAS when evacuation fails. It is a minor performance optimization: do not try to perform obviously no-op update. Let's leave it as is.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3956

From zgu at openjdk.java.net  Tue May 11 12:22:09 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 11 May 2021 12:22:09 GMT
Subject: RFR: 8266845: Shenandoah: Simplify SBS::load_reference_barrier
 implementation [v2]
In-Reply-To: <aEaMXVGa3wjSWDDJ6vRDeYXMfyqx99RyP4gLKd0-szk=.5bfb0e95-7f42-41a7-933d-82485b7e7f10@github.com>
References: <aEaMXVGa3wjSWDDJ6vRDeYXMfyqx99RyP4gLKd0-szk=.5bfb0e95-7f42-41a7-933d-82485b7e7f10@github.com>
Message-ID: <PxzT-wC5hDtk6hqvucGdXNmy9KZ4zHYJjl_bsHtyS5Y=.2b648b4d-f407-47b3-8676-6567b6e6795b@github.com>

> Early return on null obj simplifies implementation and makes code more readable.
> 
> Test:
> - [x] hotspot_gc_shenandoah

Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:

  Aleksey's comments

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/3956/files
  - new: https://git.openjdk.java.net/jdk/pull/3956/files/8ed81d6c..4a70d935

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3956&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3956&range=00-01

  Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3956.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3956/head:pull/3956

PR: https://git.openjdk.java.net/jdk/pull/3956

From zgu at openjdk.java.net  Tue May 11 12:22:10 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 11 May 2021 12:22:10 GMT
Subject: RFR: 8266845: Shenandoah: Simplify SBS::load_reference_barrier
 implementation [v2]
In-Reply-To: <pxSb3FXUymfwdFlWnu8CfCtcyoaQpvq1hr7PuuS3lJw=.a829e479-ca68-4082-9aae-c98c1bc177ef@github.com>
References: <aEaMXVGa3wjSWDDJ6vRDeYXMfyqx99RyP4gLKd0-szk=.5bfb0e95-7f42-41a7-933d-82485b7e7f10@github.com>
 <pxSb3FXUymfwdFlWnu8CfCtcyoaQpvq1hr7PuuS3lJw=.a829e479-ca68-4082-9aae-c98c1bc177ef@github.com>
Message-ID: <bn0ut-eTCjAI7bTrfxRg59v-NCg7R-gfgxKWxx6hs2A=.709999c2-2520-4203-b66b-a80fcefe8e76@github.com>

On Tue, 11 May 2021 06:31:30 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Aleksey's comments
>
> src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 105:
> 
>> 103:   if (obj == nullptr) {
>> 104:     return nullptr;
>> 105:   }
> 
> Let's use `NULL` consistently? Yes, it is an old style, but it is consistent with the code in the same file.

Okay.

> src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 131:
> 
>> 129:   oop fwd = load_reference_barrier(obj);
>> 130:   assert(fwd != nullptr, "Sanity");
>> 131:   if (ShenandoahSelfFixing && load_addr != NULL) {
> 
> I think dropping `fwd != obj` is not about the null-check. It is about not doing the CAS when evacuation fails. It is a minor performance optimization: do not try to perform obviously no-op update. Let's leave it as is.

Right, my mistake.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3956

From shade at openjdk.java.net  Tue May 11 12:24:57 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 11 May 2021 12:24:57 GMT
Subject: RFR: 8266845: Shenandoah: Simplify SBS::load_reference_barrier
 implementation [v2]
In-Reply-To: <PxzT-wC5hDtk6hqvucGdXNmy9KZ4zHYJjl_bsHtyS5Y=.2b648b4d-f407-47b3-8676-6567b6e6795b@github.com>
References: <aEaMXVGa3wjSWDDJ6vRDeYXMfyqx99RyP4gLKd0-szk=.5bfb0e95-7f42-41a7-933d-82485b7e7f10@github.com>
 <PxzT-wC5hDtk6hqvucGdXNmy9KZ4zHYJjl_bsHtyS5Y=.2b648b4d-f407-47b3-8676-6567b6e6795b@github.com>
Message-ID: <_lAfRMaiqO3uuJ1D1bYkU6zVhCJut-3UAukqVDfjUDo=.a13b3baf-d0eb-48f7-b29c-427d7b693690@github.com>

On Tue, 11 May 2021 12:22:09 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> Early return on null obj simplifies implementation and makes code more readable.
>> 
>> Test:
>> - [x] hotspot_gc_shenandoah
>
> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Aleksey's comments

Provisionally approving, modulo the nit.

src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 130:

> 128: 
> 129:   oop fwd = load_reference_barrier(obj);
> 130:   assert(fwd != nullptr, "Sanity");

`nullptr` -> `NULL` here too. Actually, why do we need this assert? I don't mind leaving it, if there is a reason.

-------------

Marked as reviewed by shade (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/3956

From shade at openjdk.java.net  Tue May 11 12:27:59 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 11 May 2021 12:27:59 GMT
Subject: RFR: 8266813: Use shorter instruction sequence for checking if
 marking in progress
In-Reply-To: <yeombb6w6WduVD-KsuJ-_89WLX42tgN-Fr9lOkyzKXk=.f835637c-eb02-4d58-b56a-1739487f8378@github.com>
References: <yeombb6w6WduVD-KsuJ-_89WLX42tgN-Fr9lOkyzKXk=.f835637c-eb02-4d58-b56a-1739487f8378@github.com>
Message-ID: <RSj0D2p3YaW2YenMwB5H9SAXW9OC4pmSixKRvhFp3FA=.dac9113b-1f00-4053-99ca-0050aad556d1@github.com>

On Mon, 10 May 2021 13:21:44 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> ShenandoahBarrierSetAssembler::generate_c1_pre_barrier_runtime_stub() on aarch64, we can replace ld, tst, br instruction sequence with a single tbz.
> 
> Test:
> - [x] hotspot_gc_shenandoah on Linux aarch64

PR title mismatches, BTW.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3948

From zgu at openjdk.java.net  Tue May 11 12:36:44 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 11 May 2021 12:36:44 GMT
Subject: RFR: 8266845: Shenandoah: Simplify SBS::load_reference_barrier
 implementation [v3]
In-Reply-To: <aEaMXVGa3wjSWDDJ6vRDeYXMfyqx99RyP4gLKd0-szk=.5bfb0e95-7f42-41a7-933d-82485b7e7f10@github.com>
References: <aEaMXVGa3wjSWDDJ6vRDeYXMfyqx99RyP4gLKd0-szk=.5bfb0e95-7f42-41a7-933d-82485b7e7f10@github.com>
Message-ID: <3_wFojQs6mjdV9yTLpTUfJQyaSs-2rUVchP9MNQJZf8=.99ac29bb-0a64-47c3-8a94-9c2cdc9b841b@github.com>

> Early return on null obj simplifies implementation and makes code more readable.
> 
> Test:
> - [x] hotspot_gc_shenandoah

Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:

  Removed unnecessary assert

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/3956/files
  - new: https://git.openjdk.java.net/jdk/pull/3956/files/4a70d935..900185fb

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3956&range=02
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3956&range=01-02

  Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3956.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3956/head:pull/3956

PR: https://git.openjdk.java.net/jdk/pull/3956

From zgu at openjdk.java.net  Tue May 11 12:36:46 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 11 May 2021 12:36:46 GMT
Subject: RFR: 8266845: Shenandoah: Simplify SBS::load_reference_barrier
 implementation [v2]
In-Reply-To: <_lAfRMaiqO3uuJ1D1bYkU6zVhCJut-3UAukqVDfjUDo=.a13b3baf-d0eb-48f7-b29c-427d7b693690@github.com>
References: <aEaMXVGa3wjSWDDJ6vRDeYXMfyqx99RyP4gLKd0-szk=.5bfb0e95-7f42-41a7-933d-82485b7e7f10@github.com>
 <PxzT-wC5hDtk6hqvucGdXNmy9KZ4zHYJjl_bsHtyS5Y=.2b648b4d-f407-47b3-8676-6567b6e6795b@github.com>
 <_lAfRMaiqO3uuJ1D1bYkU6zVhCJut-3UAukqVDfjUDo=.a13b3baf-d0eb-48f7-b29c-427d7b693690@github.com>
Message-ID: <H4ySknWlL9lt1fCow8k1r2txKViGtoMKDyIKYUBUYKI=.1a0f5d16-b5cd-4ac9-ac62-1cc8775e2ffc@github.com>

On Tue, 11 May 2021 12:21:56 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Aleksey's comments
>
> src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 130:
> 
>> 128: 
>> 129:   oop fwd = load_reference_barrier(obj);
>> 130:   assert(fwd != nullptr, "Sanity");
> 
> `nullptr` -> `NULL` here too. Actually, why do we need this assert? I don't mind leaving it, if there is a reason.

No good reason. I removed it.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3956

From zgu at openjdk.java.net  Tue May 11 12:53:11 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 11 May 2021 12:53:11 GMT
Subject: Integrated: 8266813: Shenandoah: Use shorter instruction sequence for
 checking if marking in progress
In-Reply-To: <yeombb6w6WduVD-KsuJ-_89WLX42tgN-Fr9lOkyzKXk=.f835637c-eb02-4d58-b56a-1739487f8378@github.com>
References: <yeombb6w6WduVD-KsuJ-_89WLX42tgN-Fr9lOkyzKXk=.f835637c-eb02-4d58-b56a-1739487f8378@github.com>
Message-ID: <Y5xSwi46OEJk5CZaY1E7UoShlyYiTs4YmUbLSXtEBO0=.798acb9e-f9dd-4f5f-8b83-14e3868afce2@github.com>

On Mon, 10 May 2021 13:21:44 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> ShenandoahBarrierSetAssembler::generate_c1_pre_barrier_runtime_stub() on aarch64, we can replace ld, tst, br instruction sequence with a single tbz.
> 
> Test:
> - [x] hotspot_gc_shenandoah on Linux aarch64

This pull request has now been integrated.

Changeset: 9c9c47e4
Author:    Zhengyu Gu <zgu at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/9c9c47e403c6d38afd1ce89f46a1887a01964796
Stats:     3 lines in 1 file changed: 0 ins; 2 del; 1 mod

8266813: Shenandoah: Use shorter instruction sequence for checking if marking in progress

Reviewed-by: shade

-------------

PR: https://git.openjdk.java.net/jdk/pull/3948

From maoliang.ml at alibaba-inc.com  Wed May 12 10:53:48 2021
From: maoliang.ml at alibaba-inc.com (Liang Mao)
Date: Wed, 12 May 2021 18:53:48 +0800
Subject: =?UTF-8?B?U2hlbmFuZG9haCBoYW5ncyBvbiBzcGVjanZtMjAwOCBkdWUgdG8gcmVlbnRyYW50IGhhbmRz?=
 =?UTF-8?B?aGFrZQ==?=
Message-ID: <b095da3e-4a77-4b3f-a63c-1cc05c83553e.maoliang.ml@alibaba-inc.com>

Hi Shenandoah team,

We found this issue while running Shenandoah with jdk/master:
https://bugs.openjdk.java.net/browse/JDK-8266963 in which we had a simple fix
so far.

The hanging comes from https://bugs.openjdk.java.net/browse/JDK-8262443 which
introduced the possibility of reentrant handshake and looks like only shenandoah
would hit the scenario. I have a question specific to shenandoah that why do we
 need the phase of "entry_update_thread_roots"?
I think we already concurrently relocate the oops in thread roots by
"entry_thread_roots".
And "entry_update_thread_roots" doesn't use a general concurrent stack processing
via stack watermark but just a handshake.

Thanks,
Liang


From zgu at redhat.com  Wed May 12 12:25:32 2021
From: zgu at redhat.com (Zhengyu Gu)
Date: Wed, 12 May 2021 08:25:32 -0400
Subject: Shenandoah hangs on specjvm2008 due to reentrant handshake
In-Reply-To: <b095da3e-4a77-4b3f-a63c-1cc05c83553e.maoliang.ml@alibaba-inc.com>
References: <b095da3e-4a77-4b3f-a63c-1cc05c83553e.maoliang.ml@alibaba-inc.com>
Message-ID: <d3a9da7d-0e4f-c3ff-4dc1-8527399dae5e@redhat.com>

Hi Liang,


On 5/12/21 6:53 AM, Liang Mao wrote:
> Hi Shenandoah team,
> 
> We found this issue while running Shenandoah with jdk/master:
> https://bugs.openjdk.java.net/browse/JDK-8266963 in which we had a simple fix
> so far.

Thanks for reporting. Could you please post your fix?

> 
> The hanging comes from https://bugs.openjdk.java.net/browse/JDK-8262443 which
> introduced the possibility of reentrant handshake and looks like only shenandoah
> would hit the scenario. I have a question specific to shenandoah that why do we
>   need the phase of "entry_update_thread_roots"?
> I think we already concurrently relocate the oops in thread roots by
> "entry_thread_roots".

entry_thread_roots fixes on-stack frames, but can not prevent thread 
from loading from-space oops during continuing execution. Ideally, 
Shenandoah LRB should prevent this from happening, if LRB is "atomic" ( 
no safepoints between load and LRB), unfortunately, current LRB is not.

> And "entry_update_thread_roots" doesn't use a general concurrent stack processing
> via stack watermark but just a handshake.

At point we execute entry_update_thread_roots, cset has been completely 
evacuated, therefore, no more from-space oops in heap. There are very 
rare cases that threads can still hold stalled oops, can be flushed out 
very quick, especially, if we can handshake individual Java thread and 
not cause global safepoint.

-Zhengyu

> 
> Thanks,
> Liang
> 


From zgu at openjdk.java.net  Wed May 12 12:38:05 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 12 May 2021 12:38:05 GMT
Subject: Integrated: 8266845: Shenandoah: Simplify SBS::load_reference_barrier
 implementation
In-Reply-To: <aEaMXVGa3wjSWDDJ6vRDeYXMfyqx99RyP4gLKd0-szk=.5bfb0e95-7f42-41a7-933d-82485b7e7f10@github.com>
References: <aEaMXVGa3wjSWDDJ6vRDeYXMfyqx99RyP4gLKd0-szk=.5bfb0e95-7f42-41a7-933d-82485b7e7f10@github.com>
Message-ID: <a9Y1IlPlWraHTkArN_0sfzHSsW9D-_Qbwaa1T8LHz6I=.af557c2e-edb6-4830-99cd-7575a54c580f@github.com>

On Mon, 10 May 2021 20:14:14 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Early return on null obj simplifies implementation and makes code more readable.
> 
> Test:
> - [x] hotspot_gc_shenandoah

This pull request has now been integrated.

Changeset: 69daedfd
Author:    Zhengyu Gu <zgu at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/69daedfd2316815f63d7f3c6ba79d2bb77b427ec
Stats:     6 lines in 1 file changed: 3 ins; 0 del; 3 mod

8266845: Shenandoah: Simplify SBS::load_reference_barrier implementation

Reviewed-by: shade

-------------

PR: https://git.openjdk.java.net/jdk/pull/3956

From zgu at redhat.com  Wed May 12 12:40:20 2021
From: zgu at redhat.com (Zhengyu Gu)
Date: Wed, 12 May 2021 08:40:20 -0400
Subject: Shenandoah hangs on specjvm2008 due to reentrant handshake
In-Reply-To: <d3a9da7d-0e4f-c3ff-4dc1-8527399dae5e@redhat.com>
References: <b095da3e-4a77-4b3f-a63c-1cc05c83553e.maoliang.ml@alibaba-inc.com>
 <d3a9da7d-0e4f-c3ff-4dc1-8527399dae5e@redhat.com>
Message-ID: <1edb20a5-b84b-4ff7-4dbe-6d0538931f7c@redhat.com>


> 
> Thanks for reporting. Could you please post your fix?
> 
Saw your fix in CR.

Thanks,

-Zhengyu

>>
>> The hanging comes from 
>> https://bugs.openjdk.java.net/browse/JDK-8262443 which
>> introduced the possibility of reentrant handshake and looks like only 
>> shenandoah
>> would hit the scenario. I have a question specific to shenandoah that 
>> why do we
>> ? need the phase of "entry_update_thread_roots"?
>> I think we already concurrently relocate the oops in thread roots by
>> "entry_thread_roots".
> 
> entry_thread_roots fixes on-stack frames, but can not prevent thread 
> from loading from-space oops during continuing execution. Ideally, 
> Shenandoah LRB should prevent this from happening, if LRB is "atomic" ( 
> no safepoints between load and LRB), unfortunately, current LRB is not.
> 
>> And "entry_update_thread_roots" doesn't use a general concurrent stack 
>> processing
>> via stack watermark but just a handshake.
> 
> At point we execute entry_update_thread_roots, cset has been completely 
> evacuated, therefore, no more from-space oops in heap. There are very 
> rare cases that threads can still hold stalled oops, can be flushed out 
> very quick, especially, if we can handshake individual Java thread and 
> not cause global safepoint.
> 
> -Zhengyu
> 
>>
>> Thanks,
>> Liang
>>


From maoliang.ml at alibaba-inc.com  Thu May 13 02:17:23 2021
From: maoliang.ml at alibaba-inc.com (Liang Mao)
Date: Thu, 13 May 2021 10:17:23 +0800
Subject: =?UTF-8?B?U2hlbmFuZG9haCBoYW5ncyBvbiBzcGVjanZtMjAwOCBkdWUgdG8gcmVlbnRyYW50IGhhbmRz?=
 =?UTF-8?B?aGFrZQ==?=
Message-ID: <99e2a54a-2a2b-4ac2-9fad-44095da65556.maoliang.ml@alibaba-inc.com>

Hi Zhengyu,

> entry_thread_roots fixes on-stack frames, but can not prevent thread 
> from loading from-space oops during continuing execution. Ideally, 
> Shenandoah LRB should prevent this from happening, if LRB is "atomic" ( 
> no safepoints between load and LRB), unfortunately, current LRB is not.

Is there a chance that threads will load stale oop in from-space  and
hold it after fix-ref is down and then store the stale oop into heap before
entry_update_thread_roots?

Is there a plan to re-use the barrier data to implement LRB to completely
avoid "non-atomic" of loads and LRB like ZGC?

Thanks,
Liang

------------------------------------------------------------------
From:Zhengyu Gu <zgu at redhat.com>
Send Time:2021 May 12 (Wed.) 20:25
To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>; shenandoah-dev <shenandoah-dev at openjdk.java.net>
Subject:Re: Shenandoah hangs on specjvm2008 due to reentrant handshake

Hi Liang,


On 5/12/21 6:53 AM, Liang Mao wrote:
> Hi Shenandoah team,
> 
> We found this issue while running Shenandoah with jdk/master:
> https://bugs.openjdk.java.net/browse/JDK-8266963 in which we had a simple fix
> so far.

Thanks for reporting. Could you please post your fix?

> 
> The hanging comes from https://bugs.openjdk.java.net/browse/JDK-8262443 which
> introduced the possibility of reentrant handshake and looks like only shenandoah
> would hit the scenario. I have a question specific to shenandoah that why do we
>   need the phase of "entry_update_thread_roots"?
> I think we already concurrently relocate the oops in thread roots by
> "entry_thread_roots".

entry_thread_roots fixes on-stack frames, but can not prevent thread 
from loading from-space oops during continuing execution. Ideally, 
Shenandoah LRB should prevent this from happening, if LRB is "atomic" ( 
no safepoints between load and LRB), unfortunately, current LRB is not.

> And "entry_update_thread_roots" doesn't use a general concurrent stack processing
> via stack watermark but just a handshake.

At point we execute entry_update_thread_roots, cset has been completely 
evacuated, therefore, no more from-space oops in heap. There are very 
rare cases that threads can still hold stalled oops, can be flushed out 
very quick, especially, if we can handshake individual Java thread and 
not cause global safepoint.

-Zhengyu

> 
> Thanks,
> Liang
> 


From zgu at redhat.com  Thu May 13 12:19:45 2021
From: zgu at redhat.com (Zhengyu Gu)
Date: Thu, 13 May 2021 08:19:45 -0400
Subject: Shenandoah hangs on specjvm2008 due to reentrant handshake
In-Reply-To: <99e2a54a-2a2b-4ac2-9fad-44095da65556.maoliang.ml@alibaba-inc.com>
References: <99e2a54a-2a2b-4ac2-9fad-44095da65556.maoliang.ml@alibaba-inc.com>
Message-ID: <9a57a8b4-d273-9114-7983-2510b0a69e77@redhat.com>



> 
> Is there a chance that threads will load stale oop in from-space? and
> hold it after fix-ref is down and then store the stale oop into heap before
> entry_update_thread_roots?
> 

No, LRB guarantees no from-space oops can be written to heap when it is 
armed.

> Is there a plan to re-use the barrier data to implement LRB to completely
> avoid "non-atomic" of loads and LRB like ZGC?
>

ZGC uses late barrier expansion to solve the problem, where Shenandoah 
has trouble to do. Roman (cc'd) can explain better.

-Zhengyu

> Thanks,
> Liang
> 
> ------------------------------------------------------------------
> From:Zhengyu Gu <zgu at redhat.com>
> Send Time:2021 May 12 (Wed.) 20:25
> To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>; shenandoah-dev 
> <shenandoah-dev at openjdk.java.net>
> Subject:Re: Shenandoah hangs on specjvm2008 due to reentrant handshake
> 
> Hi?Liang,
> 
> 
> On?5/12/21?6:53?AM,?Liang?Mao?wrote:
>  >?Hi?Shenandoah?team,
>  >
>  >?We?found?this?issue?while?running?Shenandoah?with?jdk/master:
>  > https://bugs.openjdk.java.net/browse/JDK-8266963 
> <https://bugs.openjdk.java.net/browse/JDK-8266963>?in?which?we?had?a?simple?fix
>  >?so?far.
> 
> Thanks?for?reporting.?Could?you?please?post?your?fix?
> 
>  >
>  >?The?hanging?comes?from 
> https://bugs.openjdk.java.net/browse/JDK-8262443 
> <https://bugs.openjdk.java.net/browse/JDK-8262443>?which
>  >?introduced?the?possibility?of?reentrant?handshake?and?looks?like?only?shenandoah
>  >?would?hit?the?scenario.?I?have?a?question?specific?to?shenandoah?that?why?do?we
>  >???need?the?phase?of?"entry_update_thread_roots"?
>  >?I?think?we?already?concurrently?relocate?the?oops?in?thread?roots?by
>  >?"entry_thread_roots".
> 
> entry_thread_roots?fixes?on-stack?frames,?but?can?not?prevent?thread
> from?loading?from-space?oops?during?continuing?execution.?Ideally,
> Shenandoah?LRB?should?prevent?this?from?happening,?if?LRB?is?"atomic"?(
> no?safepoints?between?load?and?LRB),?unfortunately,?current?LRB?is?not.
> 
>  >?And?"entry_update_thread_roots"?doesn't?use?a?general?concurrent?stack?processing
>  >?via?stack?watermark?but?just?a?handshake.
> 
> At?point?we?execute?entry_update_thread_roots,?cset?has?been?completely
> evacuated,?therefore,?no?more?from-space?oops?in?heap.?There?are?very
> rare?cases?that?threads?can?still?hold?stalled?oops,?can?be?flushed?out
> very?quick,?especially,?if?we?can?handshake?individual?Java?thread?and
> not?cause?global?safepoint.
> 
> -Zhengyu
> 
>  >
>  >?Thanks,
>  >?Liang
>  >
> 


From wkemper at openjdk.java.net  Thu May 13 18:36:50 2021
From: wkemper at openjdk.java.net (William Kemper)
Date: Thu, 13 May 2021 18:36:50 GMT
Subject: RFR: Abandon old gen cset candidates upon global collection
Message-ID: <Sb8X3NDRVTOLNlMWWC7HHikgkK0kSTKj1kNFkBACIYo=.31dc41be-8c7f-4e61-a28b-42d061f1881d@github.com>

When a _global_ collection occurs, it will include the old generation. When this happens, it doesn't make sense to continue with mixed collections of the old generation. This changes fixes an assert caused by the same old region being added more than once to the collection set.

-------------

Commit messages:
 - Abandon old gen cset candidates upon global collection

Changes: https://git.openjdk.java.net/shenandoah/pull/38/files
 Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=38&range=00
  Stats: 31 lines in 7 files changed: 30 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/38.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/38/head:pull/38

PR: https://git.openjdk.java.net/shenandoah/pull/38

From wkemper at openjdk.java.net  Fri May 14 00:02:38 2021
From: wkemper at openjdk.java.net (William Kemper)
Date: Fri, 14 May 2021 00:02:38 GMT
Subject: RFR: More promotion fixes
Message-ID: <DDfzX_ueYsbp7DS_NBDeTP1a3Hb2pXjamRZ6lqOl5xY=.92bbc295-9028-48b7-acce-7e509210a3c3@github.com>

This PR addresses a few issues with region promotion:
* Fix typo in code which loops over continuation regions of humongous objects
* Don't rely on mark bits above TAMS when iterating oops
* Due to an as-yet-undiagnosed bug, we need to mark the cards for an entire region dirty when it's promoted

-------------

Commit messages:
 - Use card marking abstraction in heap
 - Revert unnecessary change
 - Use cast_from_oop rather than C-style cast.
 - Dirty entire regions during promotion.
 - Operate on region in loop, rather than 'this' region
 - Don't rely on mark bit map above TAMS when iterating oops in region.

Changes: https://git.openjdk.java.net/shenandoah/pull/39/files
 Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=39&range=00
  Stats: 31 lines in 4 files changed: 16 ins; 2 del; 13 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/39.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/39/head:pull/39

PR: https://git.openjdk.java.net/shenandoah/pull/39

From stuefe at openjdk.java.net  Fri May 14 03:29:37 2021
From: stuefe at openjdk.java.net (Thomas Stuefe)
Date: Fri, 14 May 2021 03:29:37 GMT
Subject: RFR: 8251392: Consolidate Metaspace Statistics
In-Reply-To: <GXWRwtpSZOjwEPKp5b-0G4euKgw1nPYXgbOX8diO-8o=.e07ccfe6-e6c7-4469-b7fe-ac24ae9d70a1@github.com>
References: <GXWRwtpSZOjwEPKp5b-0G4euKgw1nPYXgbOX8diO-8o=.e07ccfe6-e6c7-4469-b7fe-ac24ae9d70a1@github.com>
Message-ID: <nm8JH__rQ9-Gbt0AMGHmsbQKQ4mzXhYTQPJF2JxHnXg=.95bb8bbe-14d5-4044-b0d7-2d60a27c1239@github.com>

On Thu, 29 Apr 2021 04:50:32 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> There is a lot of duplicate coding when it comes to the consumption of Metaspace Statistics.
> 
> Metaspace offers statistical APIs via `MetaspaceUtils::(reserved|committed|used)_(words|bytes)` for either the whole metaspace or non-class/class space separately. But many callers need some sort of combination of these values, which they keep in various data holder structures which all are named somewhat similarly and all do a very similar job. In particular, we have:
> 
> - MetaspaceSizesSnapshot (used for gc logs)
> - MetaspaceSnapshot (used in NMT)
> - MetaspaceSizes, MetaspaceSummary, JFRMetaspaceSummary (JFR)
> - MetaspaceCounters, CompressedClassSpaceCounters, MetaspacePerfCounters (jstat performance counters)
> - CompressedKlassSpacePool and MetaspacePool (used for MXBeans)
> 
> As much as possible coding should be unified.
> 
> In addition to that, all these callers share a common problem, in that retrieving individual statistical values via `MetaspaceUtils::(reserved|committed|used)_(words|bytes)` may yield inconsistent values. For example, "reserved" < "committed", or "committed" < "used". This is the cause of a variety of rare intermittent test errors in different areas, e.g. Performance counters (JDK-8163413, JDK-8153323), the gc log, the MemoryPoolMXBean and NMT (JDK-8237872).
> 
> -----------------------
> 
> This patch introduces two new data holder structures:
> - `MetaspaceStats` holds reserved/committed/used counter
> - `MetaspaceCombinedStats` holds an expression of these counters for class- and non-class metaspace each, as well as total counter.
> 
> Furthermore, the patch introduces two new APIs in MetaspaceUtils:
> - `MetaspaceStats get_statistics(type)`
> - `MetaspaceCombinedStats get_combined_statistics()`
> The former is used to get statistics for either non-class-space or class space; the latter gets statistics for the whole, including total values. Both APIs guarantee consistent values - reserved >= committed >= used - and never lock.
> 
> The patch uses these new data holders and these APIs to consolidate, clean up and simplify a lot of caller code, in addition to making that code resistant against inconsistent statistics:
> 
> - GC Log: class `MetaspaceSizesSnapshot` in *metaspace/metaspaceSizesSnapshot.cpp/hpp* had been used to hold metaspace statistics. Its default constructor sneakishly queried all values. `MetaspaceSizesSnapshot` has neem removed, caller code rewritten for MetaspaceCombinedStats and explicit value querying.
> 
> - NMT had class `MetaspaceSnapshot` to keep metaspace statistics and to print out baseline diffs. 
>    - `MetaspaceSizesSnapshot` was removed and replaced with `MetaspaceCombinedStats`.
>    - `MemSummaryDiffReporter::print_metaspace_diff()` has been modified: fixed a potential div-by-zero, removed the "free" statistics which is meaningless, and massaged the code a bit.
>    - Similar fixes have been done in `MemSummaryReporter::report_metadata()` (which, if backported, should fix JDK-8237872).
> 
> - jstat & co: Metaspace performance counters are realized by `MetaspaceCounters` and `CompressedClassSpaceCounters`, implementation resides in `MetaspacePerfCounters` (metaspace/metaspaceCounters.hpp/cpp). 
> 	- I completely removed `CompressedClassSpaceCounters` since there is no need for two APIs, class space is part of metaspace.
> 	- I simplified the counter coding a lot. I think the complexity was not needed.
> 	- The Metaspace counters are now retrieved in a consistent manner. This should take care of JDK-8163413, JDK-8153323 and similar bugs.
> 
> - `MetaspaceMemoryPool`, `CompressedClassSpaceMemoryPool` used in MxBeans: I changed the implementation to return consistent values. 
> 
> - JFR reports metaspace allocations (which, confusingly, uses a different terminology: "data_space" == non-class portion of metaspace). It used `MetaspaceSummary` to hold the statistics, which were composed of `MetaspaceSizes`.
> 	- I completely removed `MetaspaceSizes`
> 	- I rewrote `MetaspaceSummary` to use `MetaspaceCombinedStats`
> 
> - MetaspaceUtils::print_on() has been used to print metaspace statistics (by GCs). Function has been corrected to print consistent values.
> 
> - Added a simple gtest for the new APIs
> 
> Tests: 
> - manually executed hotspot tier1 tests
> - explicitly tested runtime/Metaspace, gc/metaspace, runtime/NMT
> - SAP nightlies ran on all our platforms

Gentle ping.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3786

From maoliang.ml at alibaba-inc.com  Fri May 14 03:34:42 2021
From: maoliang.ml at alibaba-inc.com (Liang Mao)
Date: Fri, 14 May 2021 11:34:42 +0800
Subject: =?UTF-8?B?U2FmZXBvaW50IGJldHdlZW4gbG9hZCBhbmQgTFJC?=
Message-ID: <a6eaf856-c810-4bb6-92a2-15ab61fc8adb.maoliang.ml@alibaba-inc.com>

Hi Zhengyu and Roman,

I'm still got a little confused by the "non-atomic" loads and LRB.

Say we have the following scenario:

We had an oop_A:

live_range_1:
step 1: load "oop_A_from_space" from heap
step 2: safe_point: oop_A_from_space spill into stack
step 3: LRB: oop_A_from_space updated to oop_A_to_space
step 4: end of live_range_1
     // oop_A_to_space will not be spilled into stack again

live_range_2:
step 1: load oop_A(register) from stack,which is oop_A_from_space

So looks like we still may get from-space oop, right?

Thanks,
Liang





------------------------------------------------------------------
From:Zhengyu Gu <zgu at redhat.com>
Send Time:2021 May 13 (Thu.) 20:19
To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>; shenandoah-dev <shenandoah-dev at openjdk.java.net>; Roman Kennke <rkennke at redhat.com>
Subject:Re: Shenandoah hangs on specjvm2008 due to reentrant handshake



> 
> Is there a chance that threads will load stale oop in from-space  and
> hold it after fix-ref is down and then store the stale oop into heap before
> entry_update_thread_roots?
> 

No, LRB guarantees no from-space oops can be written to heap when it is 
armed.

> Is there a plan to re-use the barrier data to implement LRB to completely
> avoid "non-atomic" of loads and LRB like ZGC?
>

ZGC uses late barrier expansion to solve the problem, where Shenandoah 
has trouble to do. Roman (cc'd) can explain better.

-Zhengyu

> Thanks,
> Liang
> 
> ------------------------------------------------------------------
> From:Zhengyu Gu <zgu at redhat.com>
> Send Time:2021 May 12 (Wed.) 20:25
> To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>; shenandoah-dev 
> <shenandoah-dev at openjdk.java.net>
> Subject:Re: Shenandoah hangs on specjvm2008 due to reentrant handshake
> 
> Hi Liang,
> 
> 
> On 5/12/21 6:53 AM, Liang Mao wrote:
>  > Hi Shenandoah team,
>  >
>  > We found this issue while running Shenandoah with jdk/master:
>  > https://bugs.openjdk.java.net/browse/JDK-8266963 
> <https://bugs.openjdk.java.net/browse/JDK-8266963> in which we had a simple fix
>  > so far.
> 
> Thanks for reporting. Could you please post your fix?
> 
>  >
>  > The hanging comes from 
> https://bugs.openjdk.java.net/browse/JDK-8262443 
> <https://bugs.openjdk.java.net/browse/JDK-8262443> which
>  > introduced the possibility of reentrant handshake and looks like only shenandoah
>  > would hit the scenario. I have a question specific to shenandoah that why do we
>  >   need the phase of "entry_update_thread_roots"?
>  > I think we already concurrently relocate the oops in thread roots by
>  > "entry_thread_roots".
> 
> entry_thread_roots fixes on-stack frames, but can not prevent thread
> from loading from-space oops during continuing execution. Ideally,
> Shenandoah LRB should prevent this from happening, if LRB is "atomic" (
> no safepoints between load and LRB), unfortunately, current LRB is not.
> 
>  > And "entry_update_thread_roots" doesn't use a general concurrent stack processing
>  > via stack watermark but just a handshake.
> 
> At point we execute entry_update_thread_roots, cset has been completely
> evacuated, therefore, no more from-space oops in heap. There are very
> rare cases that threads can still hold stalled oops, can be flushed out
> very quick, especially, if we can handshake individual Java thread and
> not cause global safepoint.
> 
> -Zhengyu
> 
>  >
>  > Thanks,
>  > Liang
>  >
> 


From rkennke at redhat.com  Fri May 14 07:46:43 2021
From: rkennke at redhat.com (Roman Kennke)
Date: Fri, 14 May 2021 09:46:43 +0200
Subject: Safepoint between load and LRB
In-Reply-To: <a6eaf856-c810-4bb6-92a2-15ab61fc8adb.maoliang.ml@alibaba-inc.com>
References: <a6eaf856-c810-4bb6-92a2-15ab61fc8adb.maoliang.ml@alibaba-inc.com>
Message-ID: <54b9d70a-3af1-0b4b-498c-73c7b2d575f5@redhat.com>

I am not totally sure what you mean.

In general, it is not a problem if we get a safepoint between a load and 
the LRB. If we reach a GC safepoint between load and LRB, we would fix 
the from-space-oop in the register/stack before proceeding.

There is one peculiar case where this matters, that is deoptimization. 
If we deopt at the safepoint between load and LRB, then we might see a 
from-space oop. That is why we explicitely placed an LRB in deopt code.

Does that answer your concern?

Roman


> Hi Zhengyu and Roman,
> 
> I'm still got a little confused by the "non-atomic" loads and LRB.
> 
> Say we have the following scenario:
> 
> We had an oop_A:
> 
> live_range_1:
> step 1: load "oop_A_from_space" from heap
> step 2: safe_point: oop_A_from_space spill into stack
> step 3: LRB: oop_A_from_space updated to oop_A_to_space
> step 4: end of live_range_1
>  ? ? ?// oop_A_to_space will not be spilled into stack again
> 
> live_range_2:
> step 1: load oop_A(register) from stack,which is oop_A_from_space
> 
> So looks like we still may get from-space oop, right?
> 
> Thanks,
> Liang
> 
> 
> 
> 
>     ------------------------------------------------------------------
>     From:Zhengyu Gu <zgu at redhat.com>
>     Send Time:2021 May 13 (Thu.) 20:19
>     To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>; shenandoah-dev
>     <shenandoah-dev at openjdk.java.net>; Roman Kennke <rkennke at redhat.com>
>     Subject:Re: Shenandoah hangs on specjvm2008 due to reentrant handshake
> 
> 
> 
>      >
>      >?Is?there?a?chance?that?threads?will?load?stale?oop?in?from-space??and
>      >?hold?it?after?fix-ref?is?down?and?then?store?the?stale?oop?into?heap?before
>      >?entry_update_thread_roots?
>      >
> 
>     No,?LRB?guarantees?no?from-space?oops?can?be?written?to?heap?when?it?is
>     armed.
> 
>      >?Is?there?a?plan?to?re-use?the?barrier?data?to?implement?LRB?to?completely
>      >?avoid?"non-atomic"?of?loads?and?LRB?like?ZGC?
>      >
> 
>     ZGC?uses?late?barrier?expansion?to?solve?the?problem,?where?Shenandoah
>     has?trouble?to?do.?Roman?(cc'd)?can?explain?better.
> 
>     -Zhengyu
> 
>      >?Thanks,
>      >?Liang
>      >
>      >?------------------------------------------------------------------
>      >?From:Zhengyu?Gu?<zgu at redhat.com>
>      >?Send?Time:2021?May?12?(Wed.)?20:25
>      >?To:"MAO,?Liang"?<maoliang.ml at alibaba-inc.com>;?shenandoah-dev
>      >?<shenandoah-dev at openjdk.java.net>
>      >?Subject:Re:?Shenandoah?hangs?on?specjvm2008?due?to?reentrant?handshake
>      >
>      >?Hi?Liang,
>      >
>      >
>      >?On?5/12/21?6:53?AM,?Liang?Mao?wrote:
>      >??>?Hi?Shenandoah?team,
>      >??>
>      >??>?We?found?this?issue?while?running?Shenandoah?with?jdk/master:
>      >??> https://bugs.openjdk.java.net/browse/JDK-8266963
>     <https://bugs.openjdk.java.net/browse/JDK-8266963>
>      >?<https://bugs.openjdk.java.net/browse/JDK-8266963>
>     <https://bugs.openjdk.java.net/browse/JDK-8266963>>?in?which?we?had?a?simple?fix
>      >??>?so?far.
>      >
>      >?Thanks?for?reporting.?Could?you?please?post?your?fix?
>      >
>      >??>
>      >??>?The?hanging?comes?from
>      > https://bugs.openjdk.java.net/browse/JDK-8262443
>     <https://bugs.openjdk.java.net/browse/JDK-8262443>
>      >?<https://bugs.openjdk.java.net/browse/JDK-8262443>
>     <https://bugs.openjdk.java.net/browse/JDK-8262443>>?which
>      >??>?introduced?the?possibility?of?reentrant?handshake?and?looks?like?only?shenandoah
>      >??>?would?hit?the?scenario.?I?have?a?question?specific?to?shenandoah?that?why?do?we
>      >??>???need?the?phase?of?"entry_update_thread_roots"?
>      >??>?I?think?we?already?concurrently?relocate?the?oops?in?thread?roots?by
>      >??>?"entry_thread_roots".
>      >
>      >?entry_thread_roots?fixes?on-stack?frames,?but?can?not?prevent?thread
>      >?from?loading?from-space?oops?during?continuing?execution.?Ideally,
>      >?Shenandoah?LRB?should?prevent?this?from?happening,?if?LRB?is?"atomic"?(
>      >?no?safepoints?between?load?and?LRB),?unfortunately,?current?LRB?is?not.
>      >
>      >??>?And?"entry_update_thread_roots"?doesn't?use?a?general?concurrent?stack?processing
>      >??>?via?stack?watermark?but?just?a?handshake.
>      >
>      >?At?point?we?execute?entry_update_thread_roots,?cset?has?been?completely
>      >?evacuated,?therefore,?no?more?from-space?oops?in?heap.?There?are?very
>      >?rare?cases?that?threads?can?still?hold?stalled?oops,?can?be?flushed?out
>      >?very?quick,?especially,?if?we?can?handshake?individual?Java?thread?and
>      >?not?cause?global?safepoint.
>      >
>      >?-Zhengyu
>      >
>      >??>
>      >??>?Thanks,
>      >??>?Liang
>      >??>
>      > 
> 
> 


From maoliang.ml at alibaba-inc.com  Fri May 14 08:00:15 2021
From: maoliang.ml at alibaba-inc.com (Liang Mao)
Date: Fri, 14 May 2021 16:00:15 +0800
Subject: =?UTF-8?B?U2FmZXBvaW50IGJldHdlZW4gbG9hZCBhbmQgTFJC?=
Message-ID: <aaa49c19-3df1-4148-ac57-635735a33fe2.maoliang.ml@alibaba-inc.com>

Hi Roman,

> In general, it is not a problem if we get a safepoint between a load and 
> the LRB. If we reach a GC safepoint between load and LRB, we would fix 
> the from-space-oop in the register/stack before proceeding.

Yes. The from-space-oop(register) will be fixed by the LRB after safepoint
before proceeding. But the from-space-oop in stack will not fixed at 
the same time and would be fixed until entry_update_thread_roots,
right? So the register may be used by another SSA later and then load the
 from-space-oop again from stack.

Thanks,
Liang


------------------------------------------------------------------
From:Roman Kennke <rkennke at redhat.com>
Send Time:2021 May 14 (Fri.) 15:46
To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>; Zhengyu Gu <zgu at redhat.com>; shenandoah-dev <shenandoah-dev at openjdk.java.net>
Subject:Re: Safepoint between load and LRB

I am not totally sure what you mean.

In general, it is not a problem if we get a safepoint between a load and 
the LRB. If we reach a GC safepoint between load and LRB, we would fix 
the from-space-oop in the register/stack before proceeding.

There is one peculiar case where this matters, that is deoptimization. 
If we deopt at the safepoint between load and LRB, then we might see a 
from-space oop. That is why we explicitely placed an LRB in deopt code.

Does that answer your concern?

Roman


> Hi Zhengyu and Roman,
> 
> I'm still got a little confused by the "non-atomic" loads and LRB.
> 
> Say we have the following scenario:
> 
> We had an oop_A:
> 
> live_range_1:
> step 1: load "oop_A_from_space" from heap
> step 2: safe_point: oop_A_from_space spill into stack
> step 3: LRB: oop_A_from_space updated to oop_A_to_space
> step 4: end of live_range_1
>       // oop_A_to_space will not be spilled into stack again
> 
> live_range_2:
> step 1: load oop_A(register) from stack,which is oop_A_from_space
> 
> So looks like we still may get from-space oop, right?
> 
> Thanks,
> Liang
> 
> 
> 
> 
>     ------------------------------------------------------------------
>     From:Zhengyu Gu <zgu at redhat.com>
>     Send Time:2021 May 13 (Thu.) 20:19
>     To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>; shenandoah-dev
>     <shenandoah-dev at openjdk.java.net>; Roman Kennke <rkennke at redhat.com>
>     Subject:Re: Shenandoah hangs on specjvm2008 due to reentrant handshake
> 
> 
> 
>      >
>      > Is there a chance that threads will load stale oop in from-space  and
>      > hold it after fix-ref is down and then store the stale oop into heap before
>      > entry_update_thread_roots?
>      >
> 
>     No, LRB guarantees no from-space oops can be written to heap when it is
>     armed.
> 
>      > Is there a plan to re-use the barrier data to implement LRB to completely
>      > avoid "non-atomic" of loads and LRB like ZGC?
>      >
> 
>     ZGC uses late barrier expansion to solve the problem, where Shenandoah
>     has trouble to do. Roman (cc'd) can explain better.
> 
>     -Zhengyu
> 
>      > Thanks,
>      > Liang
>      >
>      > ------------------------------------------------------------------
>      > From:Zhengyu Gu <zgu at redhat.com>
>      > Send Time:2021 May 12 (Wed.) 20:25
>      > To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>; shenandoah-dev
>      > <shenandoah-dev at openjdk.java.net>
>      > Subject:Re: Shenandoah hangs on specjvm2008 due to reentrant handshake
>      >
>      > Hi Liang,
>      >
>      >
>      > On 5/12/21 6:53 AM, Liang Mao wrote:
>      >  > Hi Shenandoah team,
>      >  >
>      >  > We found this issue while running Shenandoah with jdk/master:
>      >  > https://bugs.openjdk.java.net/browse/JDK-8266963
>     <https://bugs.openjdk.java.net/browse/JDK-8266963>
>      > <https://bugs.openjdk.java.net/browse/JDK-8266963>
>     <https://bugs.openjdk.java.net/browse/JDK-8266963>> in which we had a simple fix
>      >  > so far.
>      >
>      > Thanks for reporting. Could you please post your fix?
>      >
>      >  >
>      >  > The hanging comes from
>      > https://bugs.openjdk.java.net/browse/JDK-8262443
>     <https://bugs.openjdk.java.net/browse/JDK-8262443>
>      > <https://bugs.openjdk.java.net/browse/JDK-8262443>
>     <https://bugs.openjdk.java.net/browse/JDK-8262443>> which
>      >  > introduced the possibility of reentrant handshake and looks like only shenandoah
>      >  > would hit the scenario. I have a question specific to shenandoah that why do we
>      >  >   need the phase of "entry_update_thread_roots"?
>      >  > I think we already concurrently relocate the oops in thread roots by
>      >  > "entry_thread_roots".
>      >
>      > entry_thread_roots fixes on-stack frames, but can not prevent thread
>      > from loading from-space oops during continuing execution. Ideally,
>      > Shenandoah LRB should prevent this from happening, if LRB is "atomic" (
>      > no safepoints between load and LRB), unfortunately, current LRB is not.
>      >
>      >  > And "entry_update_thread_roots" doesn't use a general concurrent stack processing
>      >  > via stack watermark but just a handshake.
>      >
>      > At point we execute entry_update_thread_roots, cset has been completely
>      > evacuated, therefore, no more from-space oops in heap. There are very
>      > rare cases that threads can still hold stalled oops, can be flushed out
>      > very quick, especially, if we can handshake individual Java thread and
>      > not cause global safepoint.
>      >
>      > -Zhengyu
>      >
>      >  >
>      >  > Thanks,
>      >  > Liang
>      >  >
>      > 
> 
> 


From rkennke at openjdk.java.net  Fri May 14 11:58:02 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Fri, 14 May 2021 11:58:02 GMT
Subject: RFR: Abandon old gen cset candidates upon global collection
In-Reply-To: <Sb8X3NDRVTOLNlMWWC7HHikgkK0kSTKj1kNFkBACIYo=.31dc41be-8c7f-4e61-a28b-42d061f1881d@github.com>
References: <Sb8X3NDRVTOLNlMWWC7HHikgkK0kSTKj1kNFkBACIYo=.31dc41be-8c7f-4e61-a28b-42d061f1881d@github.com>
Message-ID: <9QRK2uaX0Q4O9INu_mEXclRYoI7S8nvUHbrfFo2Zngk=.5eb03c36-2bf4-4211-b330-0f64a845f148@github.com>

On Thu, 13 May 2021 18:31:59 GMT, William Kemper <wkemper at openjdk.org> wrote:

> When a _global_ collection occurs, it will include the old generation. When this happens, it doesn't make sense to continue with mixed collections of the old generation. This changes fixes an assert caused by the same old region being added more than once to the collection set.

Looks good! Thank you!

-------------

Marked as reviewed by rkennke (Lead).

PR: https://git.openjdk.java.net/shenandoah/pull/38

From rkennke at openjdk.java.net  Fri May 14 12:00:17 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Fri, 14 May 2021 12:00:17 GMT
Subject: RFR: More promotion fixes
In-Reply-To: <DDfzX_ueYsbp7DS_NBDeTP1a3Hb2pXjamRZ6lqOl5xY=.92bbc295-9028-48b7-acce-7e509210a3c3@github.com>
References: <DDfzX_ueYsbp7DS_NBDeTP1a3Hb2pXjamRZ6lqOl5xY=.92bbc295-9028-48b7-acce-7e509210a3c3@github.com>
Message-ID: <Hc3yJrf2vMyB5qN5dOU-EpXKg_rER3Cm1uC1XMNV6lc=.d2926d6d-a03c-4b22-be4d-57116d5c3031@github.com>

On Thu, 13 May 2021 23:57:06 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This PR addresses a few issues with region promotion:
> * Fix typo in code which loops over continuation regions of humongous objects
> * Don't rely on mark bits above TAMS when iterating oops
> * Due to an as-yet-undiagnosed bug, we need to mark the cards for an entire region dirty when it's promoted

Looks good!

-------------

Marked as reviewed by rkennke (Lead).

PR: https://git.openjdk.java.net/shenandoah/pull/39

From zgu at redhat.com  Fri May 14 12:20:08 2021
From: zgu at redhat.com (Zhengyu Gu)
Date: Fri, 14 May 2021 08:20:08 -0400
Subject: Safepoint between load and LRB
In-Reply-To: <54b9d70a-3af1-0b4b-498c-73c7b2d575f5@redhat.com>
References: <a6eaf856-c810-4bb6-92a2-15ab61fc8adb.maoliang.ml@alibaba-inc.com>
 <54b9d70a-3af1-0b4b-498c-73c7b2d575f5@redhat.com>
Message-ID: <0241e692-4c6e-43ca-9626-b0b2d859bb5e@redhat.com>



On 5/14/21 3:46 AM, Roman Kennke wrote:
> I am not totally sure what you mean.
> 
> In general, it is not a problem if we get a safepoint between a load and 
> the LRB. If we reach a GC safepoint between load and LRB, we would fix 
> the from-space-oop in the register/stack before proceeding.
> 
> There is one peculiar case where this matters, that is deoptimization. 
> If we deopt at the safepoint between load and LRB, then we might see a 
> from-space oop. That is why we explicitely placed an LRB in deopt code.

Another scenario, is that we disable LRB after the safepoint, e.g. after 
final ref-update safepoint.

-Zhengyu

> 
> Does that answer your concern?
> 
> Roman
> 
> 
>> Hi Zhengyu and Roman,
>>
>> I'm still got a little confused by the "non-atomic" loads and LRB.
>>
>> Say we have the following scenario:
>>
>> We had an oop_A:
>>
>> live_range_1:
>> step 1: load "oop_A_from_space" from heap
>> step 2: safe_point: oop_A_from_space spill into stack
>> step 3: LRB: oop_A_from_space updated to oop_A_to_space
>> step 4: end of live_range_1
>> ?? ? ?// oop_A_to_space will not be spilled into stack again
>>
>> live_range_2:
>> step 1: load oop_A(register) from stack,which is oop_A_from_space
>>
>> So looks like we still may get from-space oop, right?
>>
>> Thanks,
>> Liang
>>
>>
>>
>>
>> ??? ------------------------------------------------------------------
>> ??? From:Zhengyu Gu <zgu at redhat.com>
>> ??? Send Time:2021 May 13 (Thu.) 20:19
>> ??? To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>; shenandoah-dev
>> ??? <shenandoah-dev at openjdk.java.net>; Roman Kennke <rkennke at redhat.com>
>> ??? Subject:Re: Shenandoah hangs on specjvm2008 due to reentrant 
>> handshake
>>
>>
>>
>> ???? >
>>      
>> >?Is?there?a?chance?that?threads?will?load?stale?oop?in?from-space??and
>>      
>> >?hold?it?after?fix-ref?is?down?and?then?store?the?stale?oop?into?heap?before 
>>
>> ???? >?entry_update_thread_roots?
>> ???? >
>>
>>     
>> No,?LRB?guarantees?no?from-space?oops?can?be?written?to?heap?when?it?is
>> ??? armed.
>>
>>      
>> >?Is?there?a?plan?to?re-use?the?barrier?data?to?implement?LRB?to?completely 
>>
>> ???? >?avoid?"non-atomic"?of?loads?and?LRB?like?ZGC?
>> ???? >
>>
>>     
>> ZGC?uses?late?barrier?expansion?to?solve?the?problem,?where?Shenandoah
>> ??? has?trouble?to?do.?Roman?(cc'd)?can?explain?better.
>>
>> ??? -Zhengyu
>>
>> ???? >?Thanks,
>> ???? >?Liang
>> ???? >
>> ???? >?------------------------------------------------------------------
>> ???? >?From:Zhengyu?Gu?<zgu at redhat.com>
>> ???? >?Send?Time:2021?May?12?(Wed.)?20:25
>> ???? >?To:"MAO,?Liang"?<maoliang.ml at alibaba-inc.com>;?shenandoah-dev
>> ???? >?<shenandoah-dev at openjdk.java.net>
>>      
>> >?Subject:Re:?Shenandoah?hangs?on?specjvm2008?due?to?reentrant?handshake
>> ???? >
>> ???? >?Hi?Liang,
>> ???? >
>> ???? >
>> ???? >?On?5/12/21?6:53?AM,?Liang?Mao?wrote:
>> ???? >??>?Hi?Shenandoah?team,
>> ???? >??>
>> ???? >??>?We?found?this?issue?while?running?Shenandoah?with?jdk/master:
>> ???? >??> https://bugs.openjdk.java.net/browse/JDK-8266963
>> ??? <https://bugs.openjdk.java.net/browse/JDK-8266963>
>> ???? >?<https://bugs.openjdk.java.net/browse/JDK-8266963>
>>     
>> <https://bugs.openjdk.java.net/browse/JDK-8266963>>?in?which?we?had?a?simple?fix 
>>
>> ???? >??>?so?far.
>> ???? >
>> ???? >?Thanks?for?reporting.?Could?you?please?post?your?fix?
>> ???? >
>> ???? >??>
>> ???? >??>?The?hanging?comes?from
>> ???? > https://bugs.openjdk.java.net/browse/JDK-8262443
>> ??? <https://bugs.openjdk.java.net/browse/JDK-8262443>
>> ???? >?<https://bugs.openjdk.java.net/browse/JDK-8262443>
>> ??? <https://bugs.openjdk.java.net/browse/JDK-8262443>>?which
>>      
>> >??>?introduced?the?possibility?of?reentrant?handshake?and?looks?like?only?shenandoah 
>>
>>      
>> >??>?would?hit?the?scenario.?I?have?a?question?specific?to?shenandoah?that?why?do?we 
>>
>> ???? >??>???need?the?phase?of?"entry_update_thread_roots"?
>>      
>> >??>?I?think?we?already?concurrently?relocate?the?oops?in?thread?roots?by
>> ???? >??>?"entry_thread_roots".
>> ???? >
>>      
>> >?entry_thread_roots?fixes?on-stack?frames,?but?can?not?prevent?thread
>> ???? >?from?loading?from-space?oops?during?continuing?execution.?Ideally,
>>      
>> >?Shenandoah?LRB?should?prevent?this?from?happening,?if?LRB?is?"atomic"?(
>>      
>> >?no?safepoints?between?load?and?LRB),?unfortunately,?current?LRB?is?not.
>> ???? >
>>      
>> >??>?And?"entry_update_thread_roots"?doesn't?use?a?general?concurrent?stack?processing 
>>
>> ???? >??>?via?stack?watermark?but?just?a?handshake.
>> ???? >
>>      
>> >?At?point?we?execute?entry_update_thread_roots,?cset?has?been?completely
>>      
>> >?evacuated,?therefore,?no?more?from-space?oops?in?heap.?There?are?very
>>      
>> >?rare?cases?that?threads?can?still?hold?stalled?oops,?can?be?flushed?out
>>      
>> >?very?quick,?especially,?if?we?can?handshake?individual?Java?thread?and
>> ???? >?not?cause?global?safepoint.
>> ???? >
>> ???? >?-Zhengyu
>> ???? >
>> ???? >??>
>> ???? >??>?Thanks,
>> ???? >??>?Liang
>> ???? >??>
>> ???? >
>>
> 


From zgu at redhat.com  Fri May 14 12:31:41 2021
From: zgu at redhat.com (Zhengyu Gu)
Date: Fri, 14 May 2021 08:31:41 -0400
Subject: Safepoint between load and LRB
In-Reply-To: <a6eaf856-c810-4bb6-92a2-15ab61fc8adb.maoliang.ml@alibaba-inc.com>
References: <a6eaf856-c810-4bb6-92a2-15ab61fc8adb.maoliang.ml@alibaba-inc.com>
Message-ID: <88c9b11a-25fa-c54f-fe80-20bcecc678c4@redhat.com>



On 5/13/21 11:34 PM, Liang Mao wrote:
> Hi Zhengyu and Roman,
> 
> I'm still got a little confused by the "non-atomic" loads and LRB.
> 
> Say we have the following scenario:
> 
> We had an oop_A:
> 
> live_range_1:
> step 1: load "oop_A_from_space" from heap
> step 2: safe_point: oop_A_from_space spill into stack
> step 3: LRB: oop_A_from_space updated to oop_A_to_space
> step 4: end of live_range_1
>  ? ? ?// oop_A_to_space will not be spilled into stack again
> 
> live_range_2:
> step 1: load oop_A(register) from stack,which is oop_A_from_space
> 
> So looks like we still may get from-space oop, right?

I don't understand what live_range_2 is, and how that can happen. Can 
you elaborate?

-Zhengyu




> 
> Thanks,
> Liang
> 
> 
> 
> 
>     ------------------------------------------------------------------
>     From:Zhengyu Gu <zgu at redhat.com>
>     Send Time:2021 May 13 (Thu.) 20:19
>     To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>; shenandoah-dev
>     <shenandoah-dev at openjdk.java.net>; Roman Kennke <rkennke at redhat.com>
>     Subject:Re: Shenandoah hangs on specjvm2008 due to reentrant handshake
> 
> 
> 
>      >
>      >?Is?there?a?chance?that?threads?will?load?stale?oop?in?from-space??and
>      >?hold?it?after?fix-ref?is?down?and?then?store?the?stale?oop?into?heap?before
>      >?entry_update_thread_roots?
>      >
> 
>     No,?LRB?guarantees?no?from-space?oops?can?be?written?to?heap?when?it?is
>     armed.
> 
>      >?Is?there?a?plan?to?re-use?the?barrier?data?to?implement?LRB?to?completely
>      >?avoid?"non-atomic"?of?loads?and?LRB?like?ZGC?
>      >
> 
>     ZGC?uses?late?barrier?expansion?to?solve?the?problem,?where?Shenandoah
>     has?trouble?to?do.?Roman?(cc'd)?can?explain?better.
> 
>     -Zhengyu
> 
>      >?Thanks,
>      >?Liang
>      >
>      >?------------------------------------------------------------------
>      >?From:Zhengyu?Gu?<zgu at redhat.com>
>      >?Send?Time:2021?May?12?(Wed.)?20:25
>      >?To:"MAO,?Liang"?<maoliang.ml at alibaba-inc.com>;?shenandoah-dev
>      >?<shenandoah-dev at openjdk.java.net>
>      >?Subject:Re:?Shenandoah?hangs?on?specjvm2008?due?to?reentrant?handshake
>      >
>      >?Hi?Liang,
>      >
>      >
>      >?On?5/12/21?6:53?AM,?Liang?Mao?wrote:
>      >??>?Hi?Shenandoah?team,
>      >??>
>      >??>?We?found?this?issue?while?running?Shenandoah?with?jdk/master:
>      >??> https://bugs.openjdk.java.net/browse/JDK-8266963
>     <https://bugs.openjdk.java.net/browse/JDK-8266963>
>      >?<https://bugs.openjdk.java.net/browse/JDK-8266963>
>     <https://bugs.openjdk.java.net/browse/JDK-8266963>>?in?which?we?had?a?simple?fix
>      >??>?so?far.
>      >
>      >?Thanks?for?reporting.?Could?you?please?post?your?fix?
>      >
>      >??>
>      >??>?The?hanging?comes?from
>      > https://bugs.openjdk.java.net/browse/JDK-8262443
>     <https://bugs.openjdk.java.net/browse/JDK-8262443>
>      >?<https://bugs.openjdk.java.net/browse/JDK-8262443>
>     <https://bugs.openjdk.java.net/browse/JDK-8262443>>?which
>      >??>?introduced?the?possibility?of?reentrant?handshake?and?looks?like?only?shenandoah
>      >??>?would?hit?the?scenario.?I?have?a?question?specific?to?shenandoah?that?why?do?we
>      >??>???need?the?phase?of?"entry_update_thread_roots"?
>      >??>?I?think?we?already?concurrently?relocate?the?oops?in?thread?roots?by
>      >??>?"entry_thread_roots".
>      >
>      >?entry_thread_roots?fixes?on-stack?frames,?but?can?not?prevent?thread
>      >?from?loading?from-space?oops?during?continuing?execution.?Ideally,
>      >?Shenandoah?LRB?should?prevent?this?from?happening,?if?LRB?is?"atomic"?(
>      >?no?safepoints?between?load?and?LRB),?unfortunately,?current?LRB?is?not.
>      >
>      >??>?And?"entry_update_thread_roots"?doesn't?use?a?general?concurrent?stack?processing
>      >??>?via?stack?watermark?but?just?a?handshake.
>      >
>      >?At?point?we?execute?entry_update_thread_roots,?cset?has?been?completely
>      >?evacuated,?therefore,?no?more?from-space?oops?in?heap.?There?are?very
>      >?rare?cases?that?threads?can?still?hold?stalled?oops,?can?be?flushed?out
>      >?very?quick,?especially,?if?we?can?handshake?individual?Java?thread?and
>      >?not?cause?global?safepoint.
>      >
>      >?-Zhengyu
>      >
>      >??>
>      >??>?Thanks,
>      >??>?Liang
>      >??>
>      > 
> 
> 


From maoliang.ml at alibaba-inc.com  Fri May 14 12:45:59 2021
From: maoliang.ml at alibaba-inc.com (Liang Mao)
Date: Fri, 14 May 2021 20:45:59 +0800
Subject: =?UTF-8?B?U2FmZXBvaW50IGJldHdlZW4gbG9hZCBhbmQgTFJC?=
Message-ID: <bfa96318-7747-47e7-af71-5c8c3dea201f.maoliang.ml@alibaba-inc.com>

Hi Zhengyu and Roman,

I think I figured out the my puzzle.

The LRB is generated in nodes level so the from_space_oop
and to_space_oop are 2 different SSA and from_space_oop will be never
used later except for storing into stack slot. I treated them as same
register incorrectly. There should be no problem.

Thanks a lot for explanation!

Liang





------------------------------------------------------------------
From:Zhengyu Gu <zgu at redhat.com>
Send Time:2021 May 14 (Fri.) 20:31
To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>; shenandoah-dev <shenandoah-dev at openjdk.java.net>; Roman Kennke <rkennke at redhat.com>
Subject:Re: Safepoint between load and LRB



On 5/13/21 11:34 PM, Liang Mao wrote:
> Hi Zhengyu and Roman,
> 
> I'm still got a little confused by the "non-atomic" loads and LRB.
> 
> Say we have the following scenario:
> 
> We had an oop_A:
> 
> live_range_1:
> step 1: load "oop_A_from_space" from heap
> step 2: safe_point: oop_A_from_space spill into stack
> step 3: LRB: oop_A_from_space updated to oop_A_to_space
> step 4: end of live_range_1
>       // oop_A_to_space will not be spilled into stack again
> 
> live_range_2:
> step 1: load oop_A(register) from stack,which is oop_A_from_space
> 
> So looks like we still may get from-space oop, right?

I don't understand what live_range_2 is, and how that can happen. Can 
you elaborate?

-Zhengyu




> 
> Thanks,
> Liang
> 
> 
> 
> 
>     ------------------------------------------------------------------
>     From:Zhengyu Gu <zgu at redhat.com>
>     Send Time:2021 May 13 (Thu.) 20:19
>     To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>; shenandoah-dev
>     <shenandoah-dev at openjdk.java.net>; Roman Kennke <rkennke at redhat.com>
>     Subject:Re: Shenandoah hangs on specjvm2008 due to reentrant handshake
> 
> 
> 
>      >
>      > Is there a chance that threads will load stale oop in from-space  and
>      > hold it after fix-ref is down and then store the stale oop into heap before
>      > entry_update_thread_roots?
>      >
> 
>     No, LRB guarantees no from-space oops can be written to heap when it is
>     armed.
> 
>      > Is there a plan to re-use the barrier data to implement LRB to completely
>      > avoid "non-atomic" of loads and LRB like ZGC?
>      >
> 
>     ZGC uses late barrier expansion to solve the problem, where Shenandoah
>     has trouble to do. Roman (cc'd) can explain better.
> 
>     -Zhengyu
> 
>      > Thanks,
>      > Liang
>      >
>      > ------------------------------------------------------------------
>      > From:Zhengyu Gu <zgu at redhat.com>
>      > Send Time:2021 May 12 (Wed.) 20:25
>      > To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>; shenandoah-dev
>      > <shenandoah-dev at openjdk.java.net>
>      > Subject:Re: Shenandoah hangs on specjvm2008 due to reentrant handshake
>      >
>      > Hi Liang,
>      >
>      >
>      > On 5/12/21 6:53 AM, Liang Mao wrote:
>      >  > Hi Shenandoah team,
>      >  >
>      >  > We found this issue while running Shenandoah with jdk/master:
>      >  > https://bugs.openjdk.java.net/browse/JDK-8266963
>     <https://bugs.openjdk.java.net/browse/JDK-8266963>
>      > <https://bugs.openjdk.java.net/browse/JDK-8266963>
>     <https://bugs.openjdk.java.net/browse/JDK-8266963>> in which we had a simple fix
>      >  > so far.
>      >
>      > Thanks for reporting. Could you please post your fix?
>      >
>      >  >
>      >  > The hanging comes from
>      > https://bugs.openjdk.java.net/browse/JDK-8262443
>     <https://bugs.openjdk.java.net/browse/JDK-8262443>
>      > <https://bugs.openjdk.java.net/browse/JDK-8262443>
>     <https://bugs.openjdk.java.net/browse/JDK-8262443>> which
>      >  > introduced the possibility of reentrant handshake and looks like only shenandoah
>      >  > would hit the scenario. I have a question specific to shenandoah that why do we
>      >  >   need the phase of "entry_update_thread_roots"?
>      >  > I think we already concurrently relocate the oops in thread roots by
>      >  > "entry_thread_roots".
>      >
>      > entry_thread_roots fixes on-stack frames, but can not prevent thread
>      > from loading from-space oops during continuing execution. Ideally,
>      > Shenandoah LRB should prevent this from happening, if LRB is "atomic" (
>      > no safepoints between load and LRB), unfortunately, current LRB is not.
>      >
>      >  > And "entry_update_thread_roots" doesn't use a general concurrent stack processing
>      >  > via stack watermark but just a handshake.
>      >
>      > At point we execute entry_update_thread_roots, cset has been completely
>      > evacuated, therefore, no more from-space oops in heap. There are very
>      > rare cases that threads can still hold stalled oops, can be flushed out
>      > very quick, especially, if we can handshake individual Java thread and
>      > not cause global safepoint.
>      >
>      > -Zhengyu
>      >
>      >  >
>      >  > Thanks,
>      >  > Liang
>      >  >
>      > 
> 
> 


From wkemper at openjdk.java.net  Fri May 14 16:17:27 2021
From: wkemper at openjdk.java.net (William Kemper)
Date: Fri, 14 May 2021 16:17:27 GMT
Subject: Integrated: More promotion fixes
In-Reply-To: <DDfzX_ueYsbp7DS_NBDeTP1a3Hb2pXjamRZ6lqOl5xY=.92bbc295-9028-48b7-acce-7e509210a3c3@github.com>
References: <DDfzX_ueYsbp7DS_NBDeTP1a3Hb2pXjamRZ6lqOl5xY=.92bbc295-9028-48b7-acce-7e509210a3c3@github.com>
Message-ID: <oASwSa4YtnhbAtp0XgvuKx196cobxNw-pe_CAtwFmSg=.8ed35b10-b384-4411-ad0f-a26acf1faa53@github.com>

On Thu, 13 May 2021 23:57:06 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This PR addresses a few issues with region promotion:
> * Fix typo in code which loops over continuation regions of humongous objects
> * Don't rely on mark bits above TAMS when iterating oops
> * Due to an as-yet-undiagnosed bug, we need to mark the cards for an entire region dirty when it's promoted

This pull request has now been integrated.

Changeset: c6740a8f
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.java.net/shenandoah/commit/c6740a8fa30be0e0977ae69ea255c76fc306491c
Stats:     31 lines in 4 files changed: 16 ins; 2 del; 13 mod

More promotion fixes

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/39

From wkemper at openjdk.java.net  Fri May 14 16:18:18 2021
From: wkemper at openjdk.java.net (William Kemper)
Date: Fri, 14 May 2021 16:18:18 GMT
Subject: Integrated: Abandon old gen cset candidates upon global collection
In-Reply-To: <Sb8X3NDRVTOLNlMWWC7HHikgkK0kSTKj1kNFkBACIYo=.31dc41be-8c7f-4e61-a28b-42d061f1881d@github.com>
References: <Sb8X3NDRVTOLNlMWWC7HHikgkK0kSTKj1kNFkBACIYo=.31dc41be-8c7f-4e61-a28b-42d061f1881d@github.com>
Message-ID: <LtjAeostdn11OcuL8JehsDKCYbIfti6bkvWFPSieYRU=.2eaa75aa-f3be-4971-8465-a39424427c00@github.com>

On Thu, 13 May 2021 18:31:59 GMT, William Kemper <wkemper at openjdk.org> wrote:

> When a _global_ collection occurs, it will include the old generation. When this happens, it doesn't make sense to continue with mixed collections of the old generation. This changes fixes an assert caused by the same old region being added more than once to the collection set.

This pull request has now been integrated.

Changeset: 207da43e
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.java.net/shenandoah/commit/207da43eeaf7dc7817fae622a8c36c83fe3e1d1e
Stats:     31 lines in 7 files changed: 30 ins; 0 del; 1 mod

Abandon old gen cset candidates upon global collection

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/38

From coleenp at openjdk.java.net  Fri May 14 17:12:37 2021
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Fri, 14 May 2021 17:12:37 GMT
Subject: RFR: 8251392: Consolidate Metaspace Statistics
In-Reply-To: <GXWRwtpSZOjwEPKp5b-0G4euKgw1nPYXgbOX8diO-8o=.e07ccfe6-e6c7-4469-b7fe-ac24ae9d70a1@github.com>
References: <GXWRwtpSZOjwEPKp5b-0G4euKgw1nPYXgbOX8diO-8o=.e07ccfe6-e6c7-4469-b7fe-ac24ae9d70a1@github.com>
Message-ID: <dVnlfzQvSEFNZMK0W6hmgp-G7jT56G8E9PqQVsGJ0iU=.df491fd7-33f3-47eb-8685-f5005f6498f5@github.com>

On Thu, 29 Apr 2021 04:50:32 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> There is a lot of duplicate coding when it comes to the consumption of Metaspace Statistics.
> 
> Metaspace offers statistical APIs via `MetaspaceUtils::(reserved|committed|used)_(words|bytes)` for either the whole metaspace or non-class/class space separately. But many callers need some sort of combination of these values, which they keep in data holder structures which all are named "Metaspace" something and all do a very similar job. In particular, we have:
> 
> - MetaspaceSizesSnapshot (used for gc logs)
> - MetaspaceSnapshot (used in NMT)
> - MetaspaceSizes, MetaspaceSummary, JFRMetaspaceSummary (JFR)
> - MetaspaceCounters, CompressedClassSpaceCounters, MetaspacePerfCounters (jstat performance counters)
> - CompressedKlassSpacePool and MetaspacePool (used for MXBeans)
> 
> As much as possible coding should be unified.
> 
> In addition to that, all these callers share a common problem, in that retrieving individual statistical values via `MetaspaceUtils::(reserved|committed|used)_(words|bytes)` may yield inconsistent values. For example, "reserved" < "committed", or "committed" < "used". This is the cause of a variety of rare intermittent test errors in different areas, e.g. Performance counters (JDK-8163413, JDK-8153323), the gc log, the MemoryPoolMXBean and NMT (JDK-8237872).
> 
> -----------------------
> 
> This patch introduces two new data holder structures:
> - `MetaspaceStats` holds reserved/committed/used counter
> - `MetaspaceCombinedStats` holds an expression of these counters for class- and non-class metaspace each, as well as total counter.
> 
> Furthermore, the patch introduces two new APIs in MetaspaceUtils:
> - `MetaspaceStats get_statistics(type)`
> - `MetaspaceCombinedStats get_combined_statistics()`
> The former is used to get statistics for either non-class-space or class space; the latter gets statistics for the whole, including total values. Both APIs guarantee consistent values - reserved >= committed >= used - and never lock.
> 
> The patch uses these new data holders and these APIs to consolidate, clean up and simplify a lot of caller code, in addition to making that code resistant against inconsistent statistics:
> 
> - GC Log: class `MetaspaceSizesSnapshot` in *metaspace/metaspaceSizesSnapshot.cpp/hpp* had been used to hold metaspace statistics. Its default constructor sneakishly queried all values. `MetaspaceSizesSnapshot` has neem removed, caller code rewritten for MetaspaceCombinedStats and explicit value querying.
> 
> - NMT had class `MetaspaceSnapshot` to keep metaspace statistics and to print out baseline diffs. 
>    - `MetaspaceSizesSnapshot` was removed and replaced with `MetaspaceCombinedStats`.
>    - `MemSummaryDiffReporter::print_metaspace_diff()` has been modified: fixed a potential div-by-zero, removed the "free" statistics which is meaningless, and massaged the code a bit.
>    - Similar fixes have been done in `MemSummaryReporter::report_metadata()` (which, if backported, should fix JDK-8237872).
> 
> - jstat & co: Metaspace performance counters are realized by `MetaspaceCounters` and `CompressedClassSpaceCounters`, implementation resides in `MetaspacePerfCounters` (metaspace/metaspaceCounters.hpp/cpp). 
> 	- I completely removed `CompressedClassSpaceCounters` since there is no need for two APIs, class space is part of metaspace.
> 	- I simplified the counter coding a lot. I think the complexity was not needed.
> 	- The Metaspace counters are now retrieved in a consistent manner. This should take care of JDK-8163413, JDK-8153323 and similar bugs.
> 
> - `MetaspaceMemoryPool`, `CompressedClassSpaceMemoryPool` used in MxBeans: I changed the implementation to return consistent values. 
> 
> - JFR reports metaspace allocations (which, confusingly, uses a different terminology: "data_space" == non-class portion of metaspace). It used `MetaspaceSummary` to hold the statistics, which were composed of `MetaspaceSizes`.
> 	- I completely removed `MetaspaceSizes`
> 	- I rewrote `MetaspaceSummary` to use `MetaspaceCombinedStats`
> 
> - MetaspaceUtils::print_on() has been used to print metaspace statistics (by GCs). Function has been corrected to print consistent values.
> 
> - Added a simple gtest for the new APIs
> 
> Tests: 
> - manually executed hotspot tier1 tests
> - explicitly tested runtime/Metaspace, gc/metaspace, runtime/NMT
> - SAP tests ran for several weeks on all our platforms

This looks good to me.  Nice cleanup removing duplicated metaspace counting code!

-------------

Marked as reviewed by coleenp (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/3786

From stuefe at openjdk.java.net  Fri May 14 18:10:39 2021
From: stuefe at openjdk.java.net (Thomas Stuefe)
Date: Fri, 14 May 2021 18:10:39 GMT
Subject: RFR: 8251392: Consolidate Metaspace Statistics
In-Reply-To: <dVnlfzQvSEFNZMK0W6hmgp-G7jT56G8E9PqQVsGJ0iU=.df491fd7-33f3-47eb-8685-f5005f6498f5@github.com>
References: <GXWRwtpSZOjwEPKp5b-0G4euKgw1nPYXgbOX8diO-8o=.e07ccfe6-e6c7-4469-b7fe-ac24ae9d70a1@github.com>
 <dVnlfzQvSEFNZMK0W6hmgp-G7jT56G8E9PqQVsGJ0iU=.df491fd7-33f3-47eb-8685-f5005f6498f5@github.com>
Message-ID: <EyXQlWRjgsEG7AfBopvUrLDTdGALYczy8Rr2jMVo8C4=.4b053093-3d04-46ed-89e5-bf44a1389bfb@github.com>

On Fri, 14 May 2021 17:09:41 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

> This looks good to me. Nice cleanup removing duplicated metaspace counting code!

Thank you Coleen!

-------------

PR: https://git.openjdk.java.net/jdk/pull/3786

From rkennke at redhat.com  Fri May 14 20:55:31 2021
From: rkennke at redhat.com (Roman Kennke)
Date: Fri, 14 May 2021 22:55:31 +0200
Subject: Safepoint between load and LRB
In-Reply-To: <bfa96318-7747-47e7-af71-5c8c3dea201f.maoliang.ml@alibaba-inc.com>
References: <bfa96318-7747-47e7-af71-5c8c3dea201f.maoliang.ml@alibaba-inc.com>
Message-ID: <75a57e23-e7b3-5d28-3718-6185c5cae324@redhat.com>

Ah yes, indeed! It seemed obvious to me, but it is in-fact far from 
obvious (as is anything to do with C2).

There is one little detail that is worth mentioning in this context: 
there is one use of the from-space-oop, that is in the context of 
NULL-checks: if we find that an obj is *only* used in comparison with 
NULL, then we allow the LRB to be elided. And it's obviously ok: it 
doesn't matter if we compare the from-space-oop with NULL, or the 
to-space-oop, the result is the same. This is frequent enough to make a 
measurable difference in performance.

Thanks and regards,
Roman


> Hi Zhengyu and Roman,
> 
> I think I figured out the my puzzle.
> 
> The LRB is generated in nodes level so the from_space_oop
> and to_space_oop are 2 different SSA and from_space_oop will be never
> used later except for storing into stack slot. I treated them as same
> register incorrectly. There should be no problem.
> 
> Thanks a lot for explanation!
> 
> Liang
> 
> 
> 
> 
>     ------------------------------------------------------------------
>     From:Zhengyu Gu <zgu at redhat.com>
>     Send Time:2021 May 14 (Fri.) 20:31
>     To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>; shenandoah-dev
>     <shenandoah-dev at openjdk.java.net>; Roman Kennke <rkennke at redhat.com>
>     Subject:Re: Safepoint between load and LRB
> 
> 
> 
>     On?5/13/21?11:34?PM,?Liang?Mao?wrote:
>      >?Hi?Zhengyu?and?Roman,
>      >
>      >?I'm?still?got?a?little?confused?by?the?"non-atomic"?loads?and?LRB.
>      >
>      >?Say?we?have?the?following?scenario:
>      >
>      >?We?had?an?oop_A:
>      >
>      >?live_range_1:
>      >?step?1:?load?"oop_A_from_space"?from?heap
>      >?step?2:?safe_point:?oop_A_from_space?spill?into?stack
>      >?step?3:?LRB:?oop_A_from_space?updated?to?oop_A_to_space
>      >?step?4:?end?of?live_range_1
>      >???????//?oop_A_to_space?will?not?be?spilled?into?stack?again
>      >
>      >?live_range_2:
>      >?step?1:?load?oop_A(register)?from?stack,which?is?oop_A_from_space
>      >
>      >?So?looks?like?we?still?may?get?from-space?oop,?right?
> 
>     I?don't?understand?what?live_range_2?is,?and?how?that?can?happen.?Can
>     you?elaborate?
> 
>     -Zhengyu
> 
> 
> 
> 
>      >
>      >?Thanks,
>      >?Liang
>      >
>      >
>      >
>      >
>      >?????------------------------------------------------------------------
>      >?????From:Zhengyu?Gu?<zgu at redhat.com>
>      >?????Send?Time:2021?May?13?(Thu.)?20:19
>      >?????To:"MAO,?Liang"?<maoliang.ml at alibaba-inc.com>;?shenandoah-dev
>      >?????<shenandoah-dev at openjdk.java.net>;?Roman?Kennke?<rkennke at redhat.com>
>      >?????Subject:Re:?Shenandoah?hangs?on?specjvm2008?due?to?reentrant?handshake
>      >
>      >
>      >
>      >??????>
>      >??????>?Is?there?a?chance?that?threads?will?load?stale?oop?in?from-space??and
>      >??????>?hold?it?after?fix-ref?is?down?and?then?store?the?stale?oop?into?heap?before
>      >??????>?entry_update_thread_roots?
>      >??????>
>      >
>      >?????No,?LRB?guarantees?no?from-space?oops?can?be?written?to?heap?when?it?is
>      >?????armed.
>      >
>      >??????>?Is?there?a?plan?to?re-use?the?barrier?data?to?implement?LRB?to?completely
>      >??????>?avoid?"non-atomic"?of?loads?and?LRB?like?ZGC?
>      >??????>
>      >
>      >?????ZGC?uses?late?barrier?expansion?to?solve?the?problem,?where?Shenandoah
>      >?????has?trouble?to?do.?Roman?(cc'd)?can?explain?better.
>      >
>      >?????-Zhengyu
>      >
>      >??????>?Thanks,
>      >??????>?Liang
>      >??????>
>      >??????>?------------------------------------------------------------------
>      >??????>?From:Zhengyu?Gu?<zgu at redhat.com>
>      >??????>?Send?Time:2021?May?12?(Wed.)?20:25
>      >??????>?To:"MAO,?Liang"?<maoliang.ml at alibaba-inc.com>;?shenandoah-dev
>      >??????>?<shenandoah-dev at openjdk.java.net>
>      >??????>?Subject:Re:?Shenandoah?hangs?on?specjvm2008?due?to?reentrant?handshake
>      >??????>
>      >??????>?Hi?Liang,
>      >??????>
>      >??????>
>      >??????>?On?5/12/21?6:53?AM,?Liang?Mao?wrote:
>      >??????>??>?Hi?Shenandoah?team,
>      >??????>??>
>      >??????>??>?We?found?this?issue?while?running?Shenandoah?with?jdk/master:
>      >??????>??> https://bugs.openjdk.java.net/browse/JDK-8266963
>     <https://bugs.openjdk.java.net/browse/JDK-8266963>
>      >?????<https://bugs.openjdk.java.net/browse/JDK-8266963>
>     <https://bugs.openjdk.java.net/browse/JDK-8266963>>
>      >??????>?<https://bugs.openjdk.java.net/browse/JDK-8266963>
>     <https://bugs.openjdk.java.net/browse/JDK-8266963>>
>      >?????<https://bugs.openjdk.java.net/browse/JDK-8266963>>
>     <https://bugs.openjdk.java.net/browse/JDK-8266963>>>?in?which?we?had?a?simple?fix
>      >??????>??>?so?far.
>      >??????>
>      >??????>?Thanks?for?reporting.?Could?you?please?post?your?fix?
>      >??????>
>      >??????>??>
>      >??????>??>?The?hanging?comes?from
>      >??????> https://bugs.openjdk.java.net/browse/JDK-8262443
>     <https://bugs.openjdk.java.net/browse/JDK-8262443>
>      >?????<https://bugs.openjdk.java.net/browse/JDK-8262443>
>     <https://bugs.openjdk.java.net/browse/JDK-8262443>>
>      >??????>?<https://bugs.openjdk.java.net/browse/JDK-8262443>
>     <https://bugs.openjdk.java.net/browse/JDK-8262443>>
>      >?????<https://bugs.openjdk.java.net/browse/JDK-8262443>>
>     <https://bugs.openjdk.java.net/browse/JDK-8262443>>>?which
>      >??????>??>?introduced?the?possibility?of?reentrant?handshake?and?looks?like?only?shenandoah
>      >??????>??>?would?hit?the?scenario.?I?have?a?question?specific?to?shenandoah?that?why?do?we
>      >??????>??>???need?the?phase?of?"entry_update_thread_roots"?
>      >??????>??>?I?think?we?already?concurrently?relocate?the?oops?in?thread?roots?by
>      >??????>??>?"entry_thread_roots".
>      >??????>
>      >??????>?entry_thread_roots?fixes?on-stack?frames,?but?can?not?prevent?thread
>      >??????>?from?loading?from-space?oops?during?continuing?execution.?Ideally,
>      >??????>?Shenandoah?LRB?should?prevent?this?from?happening,?if?LRB?is?"atomic"?(
>      >??????>?no?safepoints?between?load?and?LRB),?unfortunately,?current?LRB?is?not.
>      >??????>
>      >??????>??>?And?"entry_update_thread_roots"?doesn't?use?a?general?concurrent?stack?processing
>      >??????>??>?via?stack?watermark?but?just?a?handshake.
>      >??????>
>      >??????>?At?point?we?execute?entry_update_thread_roots,?cset?has?been?completely
>      >??????>?evacuated,?therefore,?no?more?from-space?oops?in?heap.?There?are?very
>      >??????>?rare?cases?that?threads?can?still?hold?stalled?oops,?can?be?flushed?out
>      >??????>?very?quick,?especially,?if?we?can?handshake?individual?Java?thread?and
>      >??????>?not?cause?global?safepoint.
>      >??????>
>      >??????>?-Zhengyu
>      >??????>
>      >??????>??>
>      >??????>??>?Thanks,
>      >??????>??>?Liang
>      >??????>??>
>      >??????>
>      >
>      > 
> 
> 


From zgu at openjdk.java.net  Mon May 17 12:28:45 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 17 May 2021 12:28:45 GMT
Subject: RFR: 8251392: Consolidate Metaspace Statistics
In-Reply-To: <GXWRwtpSZOjwEPKp5b-0G4euKgw1nPYXgbOX8diO-8o=.e07ccfe6-e6c7-4469-b7fe-ac24ae9d70a1@github.com>
References: <GXWRwtpSZOjwEPKp5b-0G4euKgw1nPYXgbOX8diO-8o=.e07ccfe6-e6c7-4469-b7fe-ac24ae9d70a1@github.com>
Message-ID: <3bDQ9FhybxaLW2WO4K2iuxiCCFM9Bfqlqext3gBbx-w=.f0cba338-246b-4022-a64e-700929c863d7@github.com>

On Thu, 29 Apr 2021 04:50:32 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> There is a lot of duplicate coding when it comes to the consumption of Metaspace Statistics.
> 
> Metaspace offers statistical APIs via `MetaspaceUtils::(reserved|committed|used)_(words|bytes)` for either the whole metaspace or non-class/class space separately. But many callers need some sort of combination of these values, which they keep in data holder structures which all are named "Metaspace" something and all do a very similar job. In particular, we have:
> 
> - MetaspaceSizesSnapshot (used for gc logs)
> - MetaspaceSnapshot (used in NMT)
> - MetaspaceSizes, MetaspaceSummary, JFRMetaspaceSummary (JFR)
> - MetaspaceCounters, CompressedClassSpaceCounters, MetaspacePerfCounters (jstat performance counters)
> - CompressedKlassSpacePool and MetaspacePool (used for MXBeans)
> 
> As much as possible coding should be unified.
> 
> In addition to that, all these callers share a common problem, in that retrieving individual statistical values via `MetaspaceUtils::(reserved|committed|used)_(words|bytes)` may yield inconsistent values. For example, "reserved" < "committed", or "committed" < "used". This is the cause of a variety of rare intermittent test errors in different areas, e.g. Performance counters (JDK-8163413, JDK-8153323), the gc log, the MemoryPoolMXBean and NMT (JDK-8237872).
> 
> -----------------------
> 
> This patch introduces two new data holder structures:
> - `MetaspaceStats` holds reserved/committed/used counter
> - `MetaspaceCombinedStats` holds an expression of these counters for class- and non-class metaspace each, as well as total counter.
> 
> Furthermore, the patch introduces two new APIs in MetaspaceUtils:
> - `MetaspaceStats get_statistics(type)`
> - `MetaspaceCombinedStats get_combined_statistics()`
> The former is used to get statistics for either non-class-space or class space; the latter gets statistics for the whole, including total values. Both APIs guarantee consistent values - reserved >= committed >= used - and never lock.
> 
> The patch uses these new data holders and these APIs to consolidate, clean up and simplify a lot of caller code, in addition to making that code resistant against inconsistent statistics:
> 
> - GC Log: class `MetaspaceSizesSnapshot` in *metaspace/metaspaceSizesSnapshot.cpp/hpp* had been used to hold metaspace statistics. Its default constructor sneakishly queried all values. `MetaspaceSizesSnapshot` has neem removed, caller code rewritten for MetaspaceCombinedStats and explicit value querying.
> 
> - NMT had class `MetaspaceSnapshot` to keep metaspace statistics and to print out baseline diffs. 
>    - `MetaspaceSizesSnapshot` was removed and replaced with `MetaspaceCombinedStats`.
>    - `MemSummaryDiffReporter::print_metaspace_diff()` has been modified: fixed a potential div-by-zero, removed the "free" statistics which is meaningless, and massaged the code a bit.
>    - Similar fixes have been done in `MemSummaryReporter::report_metadata()` (which, if backported, should fix JDK-8237872).
> 
> - jstat & co: Metaspace performance counters are realized by `MetaspaceCounters` and `CompressedClassSpaceCounters`, implementation resides in `MetaspacePerfCounters` (metaspace/metaspaceCounters.hpp/cpp). 
> 	- I completely removed `CompressedClassSpaceCounters` since there is no need for two APIs, class space is part of metaspace.
> 	- I simplified the counter coding a lot. I think the complexity was not needed.
> 	- The Metaspace counters are now retrieved in a consistent manner. This should take care of JDK-8163413, JDK-8153323 and similar bugs.
> 
> - `MetaspaceMemoryPool`, `CompressedClassSpaceMemoryPool` used in MxBeans: I changed the implementation to return consistent values. 
> 
> - JFR reports metaspace allocations (which, confusingly, uses a different terminology: "data_space" == non-class portion of metaspace). It used `MetaspaceSummary` to hold the statistics, which were composed of `MetaspaceSizes`.
> 	- I completely removed `MetaspaceSizes`
> 	- I rewrote `MetaspaceSummary` to use `MetaspaceCombinedStats`
> 
> - MetaspaceUtils::print_on() has been used to print metaspace statistics (by GCs). Function has been corrected to print consistent values.
> 
> - Added a simple gtest for the new APIs
> 
> Tests: 
> - manually executed hotspot tier1 tests
> - explicitly tested runtime/Metaspace, gc/metaspace, runtime/NMT
> - SAP tests ran for several weeks on all our platforms

Looks good.

-------------

Marked as reviewed by zgu (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/3786

From stuefe at openjdk.java.net  Mon May 17 12:49:46 2021
From: stuefe at openjdk.java.net (Thomas Stuefe)
Date: Mon, 17 May 2021 12:49:46 GMT
Subject: RFR: 8251392: Consolidate Metaspace Statistics
In-Reply-To: <3bDQ9FhybxaLW2WO4K2iuxiCCFM9Bfqlqext3gBbx-w=.f0cba338-246b-4022-a64e-700929c863d7@github.com>
References: <GXWRwtpSZOjwEPKp5b-0G4euKgw1nPYXgbOX8diO-8o=.e07ccfe6-e6c7-4469-b7fe-ac24ae9d70a1@github.com>
 <3bDQ9FhybxaLW2WO4K2iuxiCCFM9Bfqlqext3gBbx-w=.f0cba338-246b-4022-a64e-700929c863d7@github.com>
Message-ID: <L2pBlWkwG9NXd3uAz-FcOBPXaC51ujVI8ucEsLPxpYY=.cc3e2874-877a-482d-949b-d7bee4fc6bf4@github.com>

On Mon, 17 May 2021 12:26:06 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Looks good.

Thanks Zhengyu!

-------------

PR: https://git.openjdk.java.net/jdk/pull/3786

From stuefe at openjdk.java.net  Tue May 18 06:38:50 2021
From: stuefe at openjdk.java.net (Thomas Stuefe)
Date: Tue, 18 May 2021 06:38:50 GMT
Subject: RFR: 8251392: Consolidate Metaspace Statistics
In-Reply-To: <GXWRwtpSZOjwEPKp5b-0G4euKgw1nPYXgbOX8diO-8o=.e07ccfe6-e6c7-4469-b7fe-ac24ae9d70a1@github.com>
References: <GXWRwtpSZOjwEPKp5b-0G4euKgw1nPYXgbOX8diO-8o=.e07ccfe6-e6c7-4469-b7fe-ac24ae9d70a1@github.com>
Message-ID: <-bOCwk4gzEM_D1lsnZJQL_YGVeG7pY65oNylcC07qdY=.441a40c4-2b78-49a4-871a-90fd0f0d70b9@github.com>

On Thu, 29 Apr 2021 04:50:32 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> There is a lot of duplicate coding when it comes to the consumption of Metaspace Statistics.
> 
> Metaspace offers statistical APIs via `MetaspaceUtils::(reserved|committed|used)_(words|bytes)` for either the whole metaspace or non-class/class space separately. But many callers need some sort of combination of these values, which they keep in data holder structures which all are named "Metaspace" something and all do a very similar job. In particular, we have:
> 
> - MetaspaceSizesSnapshot (used for gc logs)
> - MetaspaceSnapshot (used in NMT)
> - MetaspaceSizes, MetaspaceSummary, JFRMetaspaceSummary (JFR)
> - MetaspaceCounters, CompressedClassSpaceCounters, MetaspacePerfCounters (jstat performance counters)
> - CompressedKlassSpacePool and MetaspacePool (used for MXBeans)
> 
> As much as possible coding should be unified.
> 
> In addition to that, all these callers share a common problem, in that retrieving individual statistical values via `MetaspaceUtils::(reserved|committed|used)_(words|bytes)` may yield inconsistent values. For example, "reserved" < "committed", or "committed" < "used". This is the cause of a variety of rare intermittent test errors in different areas, e.g. Performance counters (JDK-8163413, JDK-8153323), the gc log, the MemoryPoolMXBean and NMT (JDK-8237872).
> 
> -----------------------
> 
> This patch introduces two new data holder structures:
> - `MetaspaceStats` holds reserved/committed/used counter
> - `MetaspaceCombinedStats` holds an expression of these counters for class- and non-class metaspace each, as well as total counter.
> 
> Furthermore, the patch introduces two new APIs in MetaspaceUtils:
> - `MetaspaceStats get_statistics(type)`
> - `MetaspaceCombinedStats get_combined_statistics()`
> The former is used to get statistics for either non-class-space or class space; the latter gets statistics for the whole, including total values. Both APIs guarantee consistent values - reserved >= committed >= used - and never lock.
> 
> The patch uses these new data holders and these APIs to consolidate, clean up and simplify a lot of caller code, in addition to making that code resistant against inconsistent statistics:
> 
> - GC Log: class `MetaspaceSizesSnapshot` in *metaspace/metaspaceSizesSnapshot.cpp/hpp* had been used to hold metaspace statistics. Its default constructor sneakishly queried all values. `MetaspaceSizesSnapshot` has neem removed, caller code rewritten for MetaspaceCombinedStats and explicit value querying.
> 
> - NMT had class `MetaspaceSnapshot` to keep metaspace statistics and to print out baseline diffs. 
>    - `MetaspaceSizesSnapshot` was removed and replaced with `MetaspaceCombinedStats`.
>    - `MemSummaryDiffReporter::print_metaspace_diff()` has been modified: fixed a potential div-by-zero, removed the "free" statistics which is meaningless, and massaged the code a bit.
>    - Similar fixes have been done in `MemSummaryReporter::report_metadata()` (which, if backported, should fix JDK-8237872).
> 
> - jstat & co: Metaspace performance counters are realized by `MetaspaceCounters` and `CompressedClassSpaceCounters`, implementation resides in `MetaspacePerfCounters` (metaspace/metaspaceCounters.hpp/cpp). 
> 	- I completely removed `CompressedClassSpaceCounters` since there is no need for two APIs, class space is part of metaspace.
> 	- I simplified the counter coding a lot. I think the complexity was not needed.
> 	- The Metaspace counters are now retrieved in a consistent manner. This should take care of JDK-8163413, JDK-8153323 and similar bugs.
> 
> - `MetaspaceMemoryPool`, `CompressedClassSpaceMemoryPool` used in MxBeans: I changed the implementation to return consistent values. 
> 
> - JFR reports metaspace allocations (which, confusingly, uses a different terminology: "data_space" == non-class portion of metaspace). It used `MetaspaceSummary` to hold the statistics, which were composed of `MetaspaceSizes`.
> 	- I completely removed `MetaspaceSizes`
> 	- I rewrote `MetaspaceSummary` to use `MetaspaceCombinedStats`
> 
> - MetaspaceUtils::print_on() has been used to print metaspace statistics (by GCs). Function has been corrected to print consistent values.
> 
> - Added a simple gtest for the new APIs
> 
> Tests: 
> - manually executed hotspot tier1 tests
> - explicitly tested runtime/Metaspace, gc/metaspace, runtime/NMT
> - SAP tests ran for several weeks on all our platforms

Okay, I waited but no more reviews are forthcoming. Our nightlies show no errors attributable to this change.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3786

From stuefe at openjdk.java.net  Tue May 18 06:38:52 2021
From: stuefe at openjdk.java.net (Thomas Stuefe)
Date: Tue, 18 May 2021 06:38:52 GMT
Subject: Integrated: 8251392: Consolidate Metaspace Statistics
In-Reply-To: <GXWRwtpSZOjwEPKp5b-0G4euKgw1nPYXgbOX8diO-8o=.e07ccfe6-e6c7-4469-b7fe-ac24ae9d70a1@github.com>
References: <GXWRwtpSZOjwEPKp5b-0G4euKgw1nPYXgbOX8diO-8o=.e07ccfe6-e6c7-4469-b7fe-ac24ae9d70a1@github.com>
Message-ID: <YARcYjHgWASfX_cn4Y-NwEXg8hB4Y_MIBRIEVbHzW5U=.3a4c336f-cb88-4799-995d-22b827451143@github.com>

On Thu, 29 Apr 2021 04:50:32 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> There is a lot of duplicate coding when it comes to the consumption of Metaspace Statistics.
> 
> Metaspace offers statistical APIs via `MetaspaceUtils::(reserved|committed|used)_(words|bytes)` for either the whole metaspace or non-class/class space separately. But many callers need some sort of combination of these values, which they keep in data holder structures which all are named "Metaspace" something and all do a very similar job. In particular, we have:
> 
> - MetaspaceSizesSnapshot (used for gc logs)
> - MetaspaceSnapshot (used in NMT)
> - MetaspaceSizes, MetaspaceSummary, JFRMetaspaceSummary (JFR)
> - MetaspaceCounters, CompressedClassSpaceCounters, MetaspacePerfCounters (jstat performance counters)
> - CompressedKlassSpacePool and MetaspacePool (used for MXBeans)
> 
> As much as possible coding should be unified.
> 
> In addition to that, all these callers share a common problem, in that retrieving individual statistical values via `MetaspaceUtils::(reserved|committed|used)_(words|bytes)` may yield inconsistent values. For example, "reserved" < "committed", or "committed" < "used". This is the cause of a variety of rare intermittent test errors in different areas, e.g. Performance counters (JDK-8163413, JDK-8153323), the gc log, the MemoryPoolMXBean and NMT (JDK-8237872).
> 
> -----------------------
> 
> This patch introduces two new data holder structures:
> - `MetaspaceStats` holds reserved/committed/used counter
> - `MetaspaceCombinedStats` holds an expression of these counters for class- and non-class metaspace each, as well as total counter.
> 
> Furthermore, the patch introduces two new APIs in MetaspaceUtils:
> - `MetaspaceStats get_statistics(type)`
> - `MetaspaceCombinedStats get_combined_statistics()`
> The former is used to get statistics for either non-class-space or class space; the latter gets statistics for the whole, including total values. Both APIs guarantee consistent values - reserved >= committed >= used - and never lock.
> 
> The patch uses these new data holders and these APIs to consolidate, clean up and simplify a lot of caller code, in addition to making that code resistant against inconsistent statistics:
> 
> - GC Log: class `MetaspaceSizesSnapshot` in *metaspace/metaspaceSizesSnapshot.cpp/hpp* had been used to hold metaspace statistics. Its default constructor sneakishly queried all values. `MetaspaceSizesSnapshot` has neem removed, caller code rewritten for MetaspaceCombinedStats and explicit value querying.
> 
> - NMT had class `MetaspaceSnapshot` to keep metaspace statistics and to print out baseline diffs. 
>    - `MetaspaceSizesSnapshot` was removed and replaced with `MetaspaceCombinedStats`.
>    - `MemSummaryDiffReporter::print_metaspace_diff()` has been modified: fixed a potential div-by-zero, removed the "free" statistics which is meaningless, and massaged the code a bit.
>    - Similar fixes have been done in `MemSummaryReporter::report_metadata()` (which, if backported, should fix JDK-8237872).
> 
> - jstat & co: Metaspace performance counters are realized by `MetaspaceCounters` and `CompressedClassSpaceCounters`, implementation resides in `MetaspacePerfCounters` (metaspace/metaspaceCounters.hpp/cpp). 
> 	- I completely removed `CompressedClassSpaceCounters` since there is no need for two APIs, class space is part of metaspace.
> 	- I simplified the counter coding a lot. I think the complexity was not needed.
> 	- The Metaspace counters are now retrieved in a consistent manner. This should take care of JDK-8163413, JDK-8153323 and similar bugs.
> 
> - `MetaspaceMemoryPool`, `CompressedClassSpaceMemoryPool` used in MxBeans: I changed the implementation to return consistent values. 
> 
> - JFR reports metaspace allocations (which, confusingly, uses a different terminology: "data_space" == non-class portion of metaspace). It used `MetaspaceSummary` to hold the statistics, which were composed of `MetaspaceSizes`.
> 	- I completely removed `MetaspaceSizes`
> 	- I rewrote `MetaspaceSummary` to use `MetaspaceCombinedStats`
> 
> - MetaspaceUtils::print_on() has been used to print metaspace statistics (by GCs). Function has been corrected to print consistent values.
> 
> - Added a simple gtest for the new APIs
> 
> Tests: 
> - manually executed hotspot tier1 tests
> - explicitly tested runtime/Metaspace, gc/metaspace, runtime/NMT
> - SAP tests ran for several weeks on all our platforms

This pull request has now been integrated.

Changeset: 554caf33
Author:    Thomas Stuefe <stuefe at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/554caf33a01ac9ca2e3e9170557e8348750f3971
Stats:     593 lines in 33 files changed: 193 ins; 304 del; 96 mod

8251392: Consolidate Metaspace Statistics

Reviewed-by: coleenp, zgu

-------------

PR: https://git.openjdk.java.net/jdk/pull/3786

From zgu at openjdk.java.net  Tue May 18 15:23:58 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 18 May 2021 15:23:58 GMT
Subject: RFR: 8267257: Shenandoah: Always deduplicate strings when it is
 enabled during full gc
Message-ID: <kIuFm2BBLer5fVXMH-sLttijUxqOU1WHzSVlwsHPM7U=.6c25fc4e-81db-4e68-a9cb-ff519c6b0073@github.com>

Full GC should recover as much memory as possible, so it should deduplicate all string candidates regardless their ages, when it is enabled. Also enhanced TestStringDedup.java test to ensure all candidates are deduplicated.

Note: Full GC does not preserve age bits, so current implementation will not deduplicate any strings even with back-to-back full GCs.

Test:
- [x] hotspot_gc_shenandoah

-------------

Commit messages:
 - Update test
 - Merge
 - Merge
 - v1
 - update
 - init

Changes: https://git.openjdk.java.net/jdk/pull/4096/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4096&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8267257
  Stats: 140 lines in 9 files changed: 58 ins; 45 del; 37 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4096.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4096/head:pull/4096

PR: https://git.openjdk.java.net/jdk/pull/4096

From rkennke at openjdk.java.net  Wed May 19 10:49:51 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 19 May 2021 10:49:51 GMT
Subject: RFR: 8267257: Shenandoah: Always deduplicate strings when it is
 enabled during full gc
In-Reply-To: <kIuFm2BBLer5fVXMH-sLttijUxqOU1WHzSVlwsHPM7U=.6c25fc4e-81db-4e68-a9cb-ff519c6b0073@github.com>
References: <kIuFm2BBLer5fVXMH-sLttijUxqOU1WHzSVlwsHPM7U=.6c25fc4e-81db-4e68-a9cb-ff519c6b0073@github.com>
Message-ID: <xRDPhFU8DW_IbZWnYskVbRKnWwc4ing4xbbWYKB9rNM=.1d92fe4b-9b71-454d-afec-f0a57fc84df0@github.com>

On Tue, 18 May 2021 15:18:32 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Full GC should recover as much memory as possible, so it should deduplicate all string candidates regardless their ages, when it is enabled. Also enhanced TestStringDedup.java test to ensure all candidates are deduplicated.
> 
> Note: Full GC does not preserve age bits, so current implementation will not deduplicate any strings even with back-to-back full GCs.
> 
> Test:
> - [x] hotspot_gc_shenandoah

Looks good to me. Thank you!

-------------

Marked as reviewed by rkennke (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/4096

From zgu at openjdk.java.net  Wed May 19 13:16:44 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 19 May 2021 13:16:44 GMT
Subject: Integrated: 8267257: Shenandoah: Always deduplicate strings when it
 is enabled during full gc
In-Reply-To: <kIuFm2BBLer5fVXMH-sLttijUxqOU1WHzSVlwsHPM7U=.6c25fc4e-81db-4e68-a9cb-ff519c6b0073@github.com>
References: <kIuFm2BBLer5fVXMH-sLttijUxqOU1WHzSVlwsHPM7U=.6c25fc4e-81db-4e68-a9cb-ff519c6b0073@github.com>
Message-ID: <2qHDgOE4d9VKYweVydphsMxe99e4aV23eOCoAfBN-9A=.112f430c-2d41-4a37-9ecf-41a2ec89dda3@github.com>

On Tue, 18 May 2021 15:18:32 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Full GC should recover as much memory as possible, so it should deduplicate all string candidates regardless their ages, when it is enabled. Also enhanced TestStringDedup.java test to ensure all candidates are deduplicated.
> 
> Note: Full GC does not preserve age bits, so current implementation will not deduplicate any strings even with back-to-back full GCs.
> 
> Test:
> - [x] hotspot_gc_shenandoah

This pull request has now been integrated.

Changeset: 0b49f5ae
Author:    Zhengyu Gu <zgu at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/0b49f5ae19b51d7ff7548b8492ea0f58dfe53180
Stats:     140 lines in 9 files changed: 58 ins; 45 del; 37 mod

8267257: Shenandoah: Always deduplicate strings when it is enabled during full gc

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/jdk/pull/4096

From shade at openjdk.java.net  Wed May 19 16:00:51 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Wed, 19 May 2021 16:00:51 GMT
Subject: RFR: 8261492: Shenandoah: reconsider forwardee accesses memory
 ordering [v3]
In-Reply-To: <OeK5Ff0cZaQ7qV2HFv8IsffVTbME9-ozEXSSu-gWsO8=.72593252-988a-478d-9360-51282530b8a6@github.com>
References: <eP3bMKbgFS1MJ1MO2DUcTom0eNJT8zszs4A-PJXIgsI=.3cead947-d093-466a-9cfd-0c312eec592a@github.com>
 <OeK5Ff0cZaQ7qV2HFv8IsffVTbME9-ozEXSSu-gWsO8=.72593252-988a-478d-9360-51282530b8a6@github.com>
Message-ID: <ueyc5n-jOVWrJN5hY7hqoSPpHpiObOONrbUUzv1Gu3o=.4ef20859-6a28-4be1-a670-040739ae3941@github.com>

On Tue, 16 Feb 2021 10:26:06 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Shenandoah carries forwardee information in object's mark word. Installing the new mark word is effectively "releasing" the object copy, and reading from the new mark word is "acquiring" that object copy.
>> 
>> For the forwardee update side, Hotspot's default for atomic operations is memory_order_conservative, which emits two-way memory fences around the CASes at least on AArch64 and PPC64. This seems to be excessive for Shenandoah forwardee updates, and "release" is enough.
>> 
>> For the forwardee load side, we need to guarantee "acquire". We do not do it now, reading the markword without memory semantics. It does not seem to pose a practical problem today, because GC does not access the object contents in the new copy, and mutators get this from the JRT-called stub that separates the fwdptr access and object contents access by a lot. It still should be cleaner to "acquire" the mark on load to avoid surprises.
>> 
>> Additional testing:
>>  - [x] Linux x86_64 `hotspot_gc_shenandoah`
>>  - [x] Linux AArch64 `hotspot_gc_shenandoah`
>>  - [x] Linux AArch64 `tier1` with Shenandoah
>
> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
> 
>  - A few minor touchups
>  - Add a blurb to x86 code as well
>  - Use implicit "consume" in AArch64, add more notes.
>  - Merge branch 'master' into JDK-8261492-shenandoah-forwardee-memord
>  - Make sure to access fwdptr with acquire semantics in assembler code
>  - 8261492: Shenandoah: reconsider forwardee accesses memory ordering

Not yet, bot. Still waiting.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2496

From shade at openjdk.java.net  Wed May 19 16:01:50 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Wed, 19 May 2021 16:01:50 GMT
Subject: RFR: 8261495: Shenandoah: reconsider update references memory
 ordering [v4]
In-Reply-To: <mmDtPov974Ez620ICbTzeSGQ6DJpv9CMAzfRW0Ai3ro=.a3b6d2c2-7b2e-4902-a5dc-c7241b7e6ada@github.com>
References: <4RLKvcdaWu0Cu6owC3yGoVY1KVEsYjBZEFJhfdwnhWg=.65fbeae1-58f6-48d3-a2ed-981858ef7da9@github.com>
 <mmDtPov974Ez620ICbTzeSGQ6DJpv9CMAzfRW0Ai3ro=.a3b6d2c2-7b2e-4902-a5dc-c7241b7e6ada@github.com>
Message-ID: <n2TOurA-3-ZaqmPzVz49qhQ8X9k7bZHud--8_HmxfdE=.3a9d826d-54db-473d-aec2-544079cf16a0@github.com>

On Tue, 16 Feb 2021 13:21:00 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Shenandoah update heap references code uses default Atomic::cmpxchg to avoid races with mutator updates. Hotspot's default for atomic operations is memory_order_conservative, which emits two-way memory fences around the CASes at least on AArch64 and PPC64.
>> 
>> This seems to be excessive for Shenandoah update references code, and "release" is enough. We do not seem to piggyback on update-references memory effects anywhere (in fact, if not for mutator, we would not even need a CAS). But, there is an interplay with concurrent evacuation and updates from self-healing.
>> 
>> Average time goes down, the number of GC cycles go up, since the cycles are shorter.
>> 
>> Additional testing:
>>  - [x] Linux x86_64 hotspot_gc_shenandoah
>>  - [x] Linux AArch64 hotspot_gc_shenandoah
>>  - [x] Linux AArch64 tier1 with Shenandoah
>
> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 10 additional commits since the last revision:
> 
>  - Comment touchup
>  - Specialize out witness-checking methods, drop acquire again
>  - Even more explanation
>  - Move the comment
>  - Also handle clearing the oops
>  - Minor touchups to the comment
>  - Merge branch 'master' into JDK-8261495-shenandoah-updaterefs-memord
>  - Use release only
>  - Do acq_rel instead
>  - 8261495: Shenandoah: reconsider update references memory ordering

Not yet, bot. Still waiting.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2498

From wkemper at openjdk.java.net  Wed May 19 22:07:40 2021
From: wkemper at openjdk.java.net (William Kemper)
Date: Wed, 19 May 2021 22:07:40 GMT
Subject: RFR: Fix verifier handling of weak references when scanning the
 remembered set
Message-ID: <9Dyr73qOVVO1hOrbc_jxYIrGfPn68pHMRg6Nzb4BFIU=.9ae39f8d-251a-49bc-835a-5788c8d5f623@github.com>

In some use cases, `ShenandoahVerifier` is required to _ignore_ weak references (because they will be processed concurrently). Because of the way the remembered set scan handles oops, it must use an `OopIterateClosure` (not a bare `OopClosure`). To make use of the `OopClosure` passed to the remembered set scanner, it was wrapped in an "adapter" that sheared off the `ShenandoahIgnoreReferenceDiscoverer` configured on the passed in closure. This change removes the need to use such an adapter during the verifier's root scan.

-------------

Commit messages:
 - Use more descriptive method names
 - Fix verifier handling of weak references when scanning the remembered set

Changes: https://git.openjdk.java.net/shenandoah/pull/41/files
 Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=41&range=00
  Stats: 13 lines in 4 files changed: 5 ins; 0 del; 8 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/41.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/41/head:pull/41

PR: https://git.openjdk.java.net/shenandoah/pull/41

From zgu at openjdk.java.net  Wed May 19 23:31:12 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 19 May 2021 23:31:12 GMT
Subject: RFR: Fix verifier handling of weak references when scanning the
 remembered set
In-Reply-To: <9Dyr73qOVVO1hOrbc_jxYIrGfPn68pHMRg6Nzb4BFIU=.9ae39f8d-251a-49bc-835a-5788c8d5f623@github.com>
References: <9Dyr73qOVVO1hOrbc_jxYIrGfPn68pHMRg6Nzb4BFIU=.9ae39f8d-251a-49bc-835a-5788c8d5f623@github.com>
Message-ID: <0BXk2o6JpBZVJz6foXfjtimpKm7ZyiZk1Bl5lEMRe8o=.7fa8b90f-5a6e-43f5-ab30-464f8dbeebc4@github.com>

On Wed, 19 May 2021 22:02:11 GMT, William Kemper <wkemper at openjdk.org> wrote:

> In some use cases, `ShenandoahVerifier` is required to _ignore_ weak references (because they will be processed concurrently). Because of the way the remembered set scan handles oops, it must use an `OopIterateClosure` (not a bare `OopClosure`). To make use of the `OopClosure` passed to the remembered set scanner, it was wrapped in an "adapter" that sheared off the `ShenandoahIgnoreReferenceDiscoverer` configured on the passed in closure. This change removes the need to use such an adapter during the verifier's root scan.

Looks good.

-------------

Marked as reviewed by zgu (Committer).

PR: https://git.openjdk.java.net/shenandoah/pull/41

From yude.lyd at alibaba-inc.com  Thu May 20 05:17:04 2021
From: yude.lyd at alibaba-inc.com (=?UTF-8?B?5p6X6IKy5b63KOW+oeW+tyk=?=)
Date: Thu, 20 May 2021 13:17:04 +0800
Subject: =?UTF-8?B?SGVhcCBzaXplIHNsb3dseSBncm93cyBpbiBqZGsxMXUgU2hlbmFuZG9haCBwYXNzaXZlIG1v?=
 =?UTF-8?B?ZGU=?=
In-Reply-To: <94a73cf4-0497-4bd2-a791-7dda292a6c06.>
References: <a41a2051-67b9-47b3-96e8-d2ce9d9574ca.>,
 <94a73cf4-0497-4bd2-a791-7dda292a6c06.>
Message-ID: <4e6701bd-c5b2-495c-b01b-09d5565da50d.yude.lyd@alibaba-inc.com>



Hi,

I was running jdk11u Shenandoah on specjvm2008, and found the passive mode has a leak-like behaviour:

$java -Xmx24g -Xms24g -XX:ParallelGCThreads=1 -XX:+UseShenandoahGC -XX:+UnlockDiagnosticVMOptions -XX:ShenandoahGCMode=passive -Xlog:gc*=debug:file=512sp.log -jar SPECjvm2008.jar -ict -coe -i 1 derby

$grep 'Pause .*ms' 512sp.log
[63.407s][info ][gc             ] GC(0) Pause Degenerated GC (Outside of Cycle) 23344M->15824M(24576M) 6994.533ms
[71.029s][info ][gc             ] GC(1) Pause Degenerated GC (Outside of Cycle) 23343M->15520M(24576M) 7325.687ms
[80.319s][info ][gc             ] GC(2) Pause Degenerated GC (Outside of Cycle) 23343M->11720M(24576M) 8721.156ms
[88.396s][info ][gc             ] GC(3) Pause Degenerated GC (Outside of Cycle) 23343M->4176M(24576M) 7549.126ms
[94.300s][info ][gc             ] GC(4) Pause Degenerated GC (Outside of Cycle) 23343M->4192M(24576M) 5216.467ms
[99.941s][info ][gc             ] GC(5) Pause Degenerated GC (Outside of Cycle) 23343M->4200M(24576M) 4984.741ms
[105.506s][info ][gc             ] GC(6) Pause Degenerated GC (Outside of Cycle) 23343M->4208M(24576M) 4901.316ms
[111.339s][info ][gc             ] GC(7) Pause Degenerated GC (Outside of Cycle) 23343M->4216M(24576M) 5175.875ms
[117.154s][info ][gc             ] GC(8) Pause Degenerated GC (Outside of Cycle) 23343M->4224M(24576M) 5134.298ms
[123.078s][info ][gc             ] GC(9) Pause Degenerated GC (Outside of Cycle) 23343M->4232M(24576M) 5255.877ms
[129.249s][info ][gc             ] GC(10) Pause Degenerated GC (Outside of Cycle) 23343M->4232M(24576M) 5505.492ms
[135.258s][info ][gc             ] GC(11) Pause Degenerated GC (Outside of Cycle) 23343M->4240M(24576M) 5355.191ms
[141.378s][info ][gc             ] GC(12) Pause Degenerated GC (Outside of Cycle) 23343M->4248M(24576M) 5487.325ms
[147.217s][info ][gc             ] GC(13) Pause Degenerated GC (Outside of Cycle) 23343M->4256M(24576M) 5214.325ms
[152.963s][info ][gc             ] GC(14) Pause Degenerated GC (Outside of Cycle) 23343M->4264M(24576M) 5083.129ms
[158.798s][info ][gc             ] GC(15) Pause Degenerated GC (Outside of Cycle) 23343M->4272M(24576M) 5162.959ms
[164.950s][info ][gc             ] GC(16) Pause Degenerated GC (Outside of Cycle) 23343M->4280M(24576M) 5475.542ms
[170.856s][info ][gc             ] GC(17) Pause Degenerated GC (Outside of Cycle) 23343M->4288M(24576M) 5222.228ms
[177.007s][info ][gc             ] GC(18) Pause Degenerated GC (Outside of Cycle) 23343M->4304M(24576M) 5300.388ms
[182.809s][info ][gc             ] GC(19) Pause Degenerated GC (Outside of Cycle) 23343M->4312M(24576M) 4833.105ms
[189.589s][info ][gc             ] GC(20) Pause Degenerated GC (Outside of Cycle) 23344M->4312M(24576M) 5441.911ms
[195.528s][info ][gc             ] GC(21) Pause Degenerated GC (Outside of Cycle) 23343M->4320M(24576M) 5262.478ms
[201.654s][info ][gc             ] GC(22) Pause Degenerated GC (Outside of Cycle) 23343M->4328M(24576M) 5491.649ms
[208.601s][info ][gc             ] GC(23) Pause Degenerated GC (Outside of Cycle) 23343M->4336M(24576M) 6293.675ms
[214.483s][info ][gc             ] GC(24) Pause Degenerated GC (Outside of Cycle) 23343M->4344M(24576M) 5259.813ms
[220.657s][info ][gc             ] GC(25) Pause Degenerated GC (Outside of Cycle) 23343M->4352M(24576M) 5533.361ms
[227.005s][info ][gc             ] GC(26) Pause Degenerated GC (Outside of Cycle) 23343M->4352M(24576M) 5712.247ms
[233.210s][info ][gc             ] GC(27) Pause Degenerated GC (Outside of Cycle) 23343M->4368M(24576M) 5580.975ms
[239.202s][info ][gc             ] GC(28) Pause Degenerated GC (Outside of Cycle) 23343M->4368M(24576M) 5379.766ms
[245.565s][info ][gc             ] GC(29) Pause Degenerated GC (Outside of Cycle) 23343M->4376M(24576M) 5714.096ms
[251.450s][info ][gc             ] GC(30) Pause Degenerated GC (Outside of Cycle) 23343M->4384M(24576M) 5250.005ms
[257.447s][info ][gc             ] GC(31) Pause Degenerated GC (Outside of Cycle) 23343M->4392M(24576M) 5354.806ms
[263.840s][info ][gc             ] GC(32) Pause Degenerated GC (Outside of Cycle) 23343M->4400M(24576M) 5716.993ms
[270.792s][info ][gc             ] GC(33) Pause Degenerated GC (Outside of Cycle) 23343M->4408M(24576M) 6264.228ms
[277.201s][info ][gc             ] GC(34) Pause Degenerated GC (Outside of Cycle) 23343M->4408M(24576M) 5743.778ms
[283.828s][info ][gc             ] GC(35) Pause Degenerated GC (Outside of Cycle) 23343M->4416M(24576M) 5973.385ms
[291.227s][info ][gc             ] GC(36) Pause Degenerated GC (Outside of Cycle) 23343M->4424M(24576M) 6587.729ms
[298.153s][info ][gc             ] GC(37) Pause Degenerated GC (Outside of Cycle) 23343M->4432M(24576M) 6122.285ms
[304.722s][info ][gc             ] GC(38) Pause Degenerated GC (Outside of Cycle) 23343M->4440M(24576M) 5938.363ms
[311.097s][info ][gc             ] GC(39) Pause Degenerated GC (Outside of Cycle) 23343M->4448M(24576M) 5737.377ms
[317.710s][info ][gc             ] GC(40) Pause Degenerated GC (Outside of Cycle) 23343M->4448M(24576M) 5994.975ms
[323.969s][info ][gc             ] GC(41) Pause Degenerated GC (Outside of Cycle) 23343M->4456M(24576M) 5624.046ms
[330.910s][info ][gc             ] GC(42) Pause Degenerated GC (Outside of Cycle) 23343M->4464M(24576M) 6298.006ms
[337.761s][info ][gc             ] GC(43) Pause Degenerated GC (Outside of Cycle) 23343M->4472M(24576M) 6248.352ms
[344.369s][info ][gc             ] GC(44) Pause Degenerated GC (Outside of Cycle) 23343M->4480M(24576M) 5947.428ms
[351.096s][info ][gc             ] GC(45) Pause Degenerated GC (Outside of Cycle) 23343M->4488M(24576M) 6094.857ms
[357.776s][info ][gc             ] GC(46) Pause Degenerated GC (Outside of Cycle) 23343M->4496M(24576M) 6048.758ms
[364.830s][info ][gc             ] GC(47) Pause Degenerated GC (Outside of Cycle) 23343M->4504M(24576M) 6418.273ms
[370.804s][info ][gc             ] GC(48) Pause Degenerated GC (Outside of Cycle) 23343M->4512M(24576M) 5348.577ms
[376.787s][info ][gc             ] GC(49) Pause Degenerated GC (Outside of Cycle) 23344M->4520M(24576M) 5339.928ms
[383.407s][info ][gc             ] GC(50) Pause Degenerated GC (Outside of Cycle) 23343M->4528M(24576M) 5942.465ms
[389.858s][info ][gc             ] GC(51) Pause Degenerated GC (Outside of Cycle) 23343M->4536M(24576M) 5801.355ms
[396.714s][info ][gc             ] GC(52) Pause Degenerated GC (Outside of Cycle) 23343M->4544M(24576M) 6190.560ms
[403.752s][info ][gc             ] GC(53) Pause Degenerated GC (Outside of Cycle) 23343M->4560M(24576M) 6226.817ms
[410.816s][info ][gc             ] GC(54) Pause Degenerated GC (Outside of Cycle) 23343M->4560M(24576M) 6400.091ms
[417.651s][info ][gc             ] GC(55) Pause Degenerated GC (Outside of Cycle) 23343M->4568M(24576M) 6206.848ms
[424.040s][info ][gc             ] GC(56) Pause Degenerated GC (Outside of Cycle) 23343M->4576M(24576M) 5774.497ms
[431.115s][info ][gc             ] GC(57) Pause Degenerated GC (Outside of Cycle) 23343M->4584M(24576M) 6433.645ms
[437.740s][info ][gc             ] GC(58) Pause Degenerated GC (Outside of Cycle) 23343M->4592M(24576M) 5980.660ms



The heap seems to grow slowly, and the pause time tends to grow too.
The jdk16 or 17 version appears okay (with constant after-gc heap size).

I wonder if there is a known reason?

Thanks!

Yude Lin


From stefank at openjdk.java.net  Thu May 20 12:03:00 2021
From: stefank at openjdk.java.net (Stefan Karlsson)
Date: Thu, 20 May 2021 12:03:00 GMT
Subject: RFR: 8267464: Circular-dependency resiliant inline headers
Message-ID: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>

I'd like to propose a small adjustment to how we write .inline.hpp files, in the hope to reduce include problems because of circular dependencies between inline headers.

This is a small, contrived example to show the problem:

// a.hpp
#pragma once

void a1();
void a2();


// a.inline.hpp
#pragma once

#include "a.hpp"
#include "b.inline.hpp"

inline void a1() {
  b1();
}

inline void a2() {
}


// b.hpp
#pragma once

void b1();
void b2();


// b.inline.hpp
#pragma once

#include "a.inline.hpp"
#include "b.hpp"

inline void b1() {
}

inline void b2() {
  a1();
}

The following code compiles fine:

// a.cpp
#include "a.inline.hpp"

int main() {
  a1();

  return 0;
}

But the following:

// b.cpp
#include "b.inline.hpp"

int main() {
  b1();

  return 0;
}


fails with the this error message:

In file included from b.inline.hpp:3,
                 from b.cpp:1:
a.inline.hpp: In function ?void a1()?:
a.inline.hpp:8:3: error: ?b1? was not declared in this scope; did you mean ?a1??

We can see the problem with g++ -E:

# 1 "a.inline.hpp" 1
# 1 "a.hpp" 1

void a1();
void a2();

# 4 "a.inline.hpp" 2

inline void a1() {
  b1();
}

inline void a2() {
}

# 4 "b.inline.hpp" 2
# 1 "b.hpp" 1

void b1();
void b2();

# 5 "b.inline.hpp" 2

inline void b1() {
}

inline void b2() {
  a1();
}

# 2 "b.cpp" 2

int main() {
  b1();

  return 0;
}


b1() is called before it has been declared. b.inline.hpp inlined a.inline.hpp, which uses functions declared in b.hpp. However, at that point we've not included enough of b.inline.hpp to have declared b1().

This is a small and easy example. In the JVM the circular dependencies usually involves more than two files, and often from different sub-systems of the JVM. We have so-far solved this by restructuring the code, making functions out-of-line, creating proxy objects, etc. However, the more we use templates, the more this is going to become a reoccurring nuisance. And in experiments with lambdas, which requires templates, this very quickly showed up as a problem.

I propose that we solve most (all?) of these issues by adding a rule that states that .inline.hpp files should start by including the corresponding .hpp, and that the .hpp should contain the declarations of all externally used parts of the .inline.hpp file. This should guarantee that the declarations are always present before any subsequent include can create a circular include dependency back to the original file.

In the example above, b.inline.hpp would become:

// b.inline.hpp
#pragma once

#include "b.hpp"
#include "a.inline.hpp"

inline void b1() {
}

inline void b2() {
  a1();
}

and now both a.cpp and b.cpp compiles. The generated output now looks like this:

# 1 "b.inline.hpp" 1
# 1 "b.hpp" 1

void b1();
void b2();

# 4 "b.inline.hpp" 2
# 1 "a.inline.hpp" 1

# 1 "a.hpp" 1

void a1();
void a2();

# 4 "a.inline.hpp" 2

inline void a1() {
  b1();
}

inline void a2() {
}

# 5 "b.inline.hpp" 2

inline void b1() {
}

inline void b2() {
  a1();
}

# 2 "b.cpp" 2

int main() {
  b1();

  return 0;
}

The declarations come first, and the compiler is happy. 

An alternative to this, that has been proposed by other HotSpot GC devs have been to always include all .hpp files first, and then have a section of .inline.hpp includes. I've experimented with that as well, but I think it requires more maintenance and vigilance of  *users* .inline.hpp files (add .hpp include to the first section, add .inline.hpp section to second). The proposed structures only requires extra care from *writers* of .inline.hpp files. All others can include .hpp and .inline.hpp as we've been doing for a long time now.

Some notes about this patch:
1) Some externally-used declarations have been placed in .inline.hpp files, and not in the .hpp. Those should be moved. I have not done that.
2) Some .inline.hpp files don't have a corresponding .hpp file. I could either try to fix that in this patch, or we could take care of that as separate patches later.
3) The style I chose was to add the .hpp include and a extra blank line, separating it from the rest of the includes. I think I like that style, because it makes it easy for those writing .inline.hpp to recognize this pattern. And it removes the confusion why this include isn't sorted into the other includes.
4) We have a few *special* platform dependent header files. Both those that simply are included into platform independent files, and those that inject code *into* the platform independent classes. Extra care, as always, need to be taken around those files.
5) Mostly tested locally, but I'll test on more platforms if the idea is accepted.

What do you think?

-------------

Commit messages:
 - 8267464: Circular-dependency resiliant inline headers

Changes: https://git.openjdk.java.net/jdk/pull/4127/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4127&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8267464
  Stats: 459 lines in 224 files changed: 350 ins; 104 del; 5 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4127.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4127/head:pull/4127

PR: https://git.openjdk.java.net/jdk/pull/4127

From zgu at redhat.com  Thu May 20 15:30:44 2021
From: zgu at redhat.com (Zhengyu Gu)
Date: Thu, 20 May 2021 11:30:44 -0400
Subject: Heap size slowly grows in jdk11u Shenandoah passive mode
In-Reply-To: <4e6701bd-c5b2-495c-b01b-09d5565da50d.yude.lyd@alibaba-inc.com>
References: <a41a2051-67b9-47b3-96e8-d2ce9d9574ca.>
 <94a73cf4-0497-4bd2-a791-7dda292a6c06.>
 <4e6701bd-c5b2-495c-b01b-09d5565da50d.yude.lyd@alibaba-inc.com>
Message-ID: <841bbfaf-72a4-5510-febf-c3eb111a20b5@redhat.com>

Hi,

This may be due to class unloading. Shenandoah/jdk11u, by default, 
unloads classes every 100 cycles, while jdk16/17 unloads classes 
(concurrently) every cycle.

You may want to try: -XX:ShenandoahUnloadClassesFrequency=1

-Zhengyu

On 5/20/21 1:17 AM, ???(??) wrote:
> 
> 
> Hi,
> 
> I was running jdk11u Shenandoah on specjvm2008, and found the passive mode has a leak-like behaviour:
> 
> $java -Xmx24g -Xms24g -XX:ParallelGCThreads=1 -XX:+UseShenandoahGC -XX:+UnlockDiagnosticVMOptions -XX:ShenandoahGCMode=passive -Xlog:gc*=debug:file=512sp.log -jar SPECjvm2008.jar -ict -coe -i 1 derby
> 
> $grep 'Pause .*ms' 512sp.log
> [63.407s][info ][gc             ] GC(0) Pause Degenerated GC (Outside of Cycle) 23344M->15824M(24576M) 6994.533ms
> [71.029s][info ][gc             ] GC(1) Pause Degenerated GC (Outside of Cycle) 23343M->15520M(24576M) 7325.687ms
> [80.319s][info ][gc             ] GC(2) Pause Degenerated GC (Outside of Cycle) 23343M->11720M(24576M) 8721.156ms
> [88.396s][info ][gc             ] GC(3) Pause Degenerated GC (Outside of Cycle) 23343M->4176M(24576M) 7549.126ms
> [94.300s][info ][gc             ] GC(4) Pause Degenerated GC (Outside of Cycle) 23343M->4192M(24576M) 5216.467ms
> [99.941s][info ][gc             ] GC(5) Pause Degenerated GC (Outside of Cycle) 23343M->4200M(24576M) 4984.741ms
> [105.506s][info ][gc             ] GC(6) Pause Degenerated GC (Outside of Cycle) 23343M->4208M(24576M) 4901.316ms
> [111.339s][info ][gc             ] GC(7) Pause Degenerated GC (Outside of Cycle) 23343M->4216M(24576M) 5175.875ms
> [117.154s][info ][gc             ] GC(8) Pause Degenerated GC (Outside of Cycle) 23343M->4224M(24576M) 5134.298ms
> [123.078s][info ][gc             ] GC(9) Pause Degenerated GC (Outside of Cycle) 23343M->4232M(24576M) 5255.877ms
> [129.249s][info ][gc             ] GC(10) Pause Degenerated GC (Outside of Cycle) 23343M->4232M(24576M) 5505.492ms
> [135.258s][info ][gc             ] GC(11) Pause Degenerated GC (Outside of Cycle) 23343M->4240M(24576M) 5355.191ms
> [141.378s][info ][gc             ] GC(12) Pause Degenerated GC (Outside of Cycle) 23343M->4248M(24576M) 5487.325ms
> [147.217s][info ][gc             ] GC(13) Pause Degenerated GC (Outside of Cycle) 23343M->4256M(24576M) 5214.325ms
> [152.963s][info ][gc             ] GC(14) Pause Degenerated GC (Outside of Cycle) 23343M->4264M(24576M) 5083.129ms
> [158.798s][info ][gc             ] GC(15) Pause Degenerated GC (Outside of Cycle) 23343M->4272M(24576M) 5162.959ms
> [164.950s][info ][gc             ] GC(16) Pause Degenerated GC (Outside of Cycle) 23343M->4280M(24576M) 5475.542ms
> [170.856s][info ][gc             ] GC(17) Pause Degenerated GC (Outside of Cycle) 23343M->4288M(24576M) 5222.228ms
> [177.007s][info ][gc             ] GC(18) Pause Degenerated GC (Outside of Cycle) 23343M->4304M(24576M) 5300.388ms
> [182.809s][info ][gc             ] GC(19) Pause Degenerated GC (Outside of Cycle) 23343M->4312M(24576M) 4833.105ms
> [189.589s][info ][gc             ] GC(20) Pause Degenerated GC (Outside of Cycle) 23344M->4312M(24576M) 5441.911ms
> [195.528s][info ][gc             ] GC(21) Pause Degenerated GC (Outside of Cycle) 23343M->4320M(24576M) 5262.478ms
> [201.654s][info ][gc             ] GC(22) Pause Degenerated GC (Outside of Cycle) 23343M->4328M(24576M) 5491.649ms
> [208.601s][info ][gc             ] GC(23) Pause Degenerated GC (Outside of Cycle) 23343M->4336M(24576M) 6293.675ms
> [214.483s][info ][gc             ] GC(24) Pause Degenerated GC (Outside of Cycle) 23343M->4344M(24576M) 5259.813ms
> [220.657s][info ][gc             ] GC(25) Pause Degenerated GC (Outside of Cycle) 23343M->4352M(24576M) 5533.361ms
> [227.005s][info ][gc             ] GC(26) Pause Degenerated GC (Outside of Cycle) 23343M->4352M(24576M) 5712.247ms
> [233.210s][info ][gc             ] GC(27) Pause Degenerated GC (Outside of Cycle) 23343M->4368M(24576M) 5580.975ms
> [239.202s][info ][gc             ] GC(28) Pause Degenerated GC (Outside of Cycle) 23343M->4368M(24576M) 5379.766ms
> [245.565s][info ][gc             ] GC(29) Pause Degenerated GC (Outside of Cycle) 23343M->4376M(24576M) 5714.096ms
> [251.450s][info ][gc             ] GC(30) Pause Degenerated GC (Outside of Cycle) 23343M->4384M(24576M) 5250.005ms
> [257.447s][info ][gc             ] GC(31) Pause Degenerated GC (Outside of Cycle) 23343M->4392M(24576M) 5354.806ms
> [263.840s][info ][gc             ] GC(32) Pause Degenerated GC (Outside of Cycle) 23343M->4400M(24576M) 5716.993ms
> [270.792s][info ][gc             ] GC(33) Pause Degenerated GC (Outside of Cycle) 23343M->4408M(24576M) 6264.228ms
> [277.201s][info ][gc             ] GC(34) Pause Degenerated GC (Outside of Cycle) 23343M->4408M(24576M) 5743.778ms
> [283.828s][info ][gc             ] GC(35) Pause Degenerated GC (Outside of Cycle) 23343M->4416M(24576M) 5973.385ms
> [291.227s][info ][gc             ] GC(36) Pause Degenerated GC (Outside of Cycle) 23343M->4424M(24576M) 6587.729ms
> [298.153s][info ][gc             ] GC(37) Pause Degenerated GC (Outside of Cycle) 23343M->4432M(24576M) 6122.285ms
> [304.722s][info ][gc             ] GC(38) Pause Degenerated GC (Outside of Cycle) 23343M->4440M(24576M) 5938.363ms
> [311.097s][info ][gc             ] GC(39) Pause Degenerated GC (Outside of Cycle) 23343M->4448M(24576M) 5737.377ms
> [317.710s][info ][gc             ] GC(40) Pause Degenerated GC (Outside of Cycle) 23343M->4448M(24576M) 5994.975ms
> [323.969s][info ][gc             ] GC(41) Pause Degenerated GC (Outside of Cycle) 23343M->4456M(24576M) 5624.046ms
> [330.910s][info ][gc             ] GC(42) Pause Degenerated GC (Outside of Cycle) 23343M->4464M(24576M) 6298.006ms
> [337.761s][info ][gc             ] GC(43) Pause Degenerated GC (Outside of Cycle) 23343M->4472M(24576M) 6248.352ms
> [344.369s][info ][gc             ] GC(44) Pause Degenerated GC (Outside of Cycle) 23343M->4480M(24576M) 5947.428ms
> [351.096s][info ][gc             ] GC(45) Pause Degenerated GC (Outside of Cycle) 23343M->4488M(24576M) 6094.857ms
> [357.776s][info ][gc             ] GC(46) Pause Degenerated GC (Outside of Cycle) 23343M->4496M(24576M) 6048.758ms
> [364.830s][info ][gc             ] GC(47) Pause Degenerated GC (Outside of Cycle) 23343M->4504M(24576M) 6418.273ms
> [370.804s][info ][gc             ] GC(48) Pause Degenerated GC (Outside of Cycle) 23343M->4512M(24576M) 5348.577ms
> [376.787s][info ][gc             ] GC(49) Pause Degenerated GC (Outside of Cycle) 23344M->4520M(24576M) 5339.928ms
> [383.407s][info ][gc             ] GC(50) Pause Degenerated GC (Outside of Cycle) 23343M->4528M(24576M) 5942.465ms
> [389.858s][info ][gc             ] GC(51) Pause Degenerated GC (Outside of Cycle) 23343M->4536M(24576M) 5801.355ms
> [396.714s][info ][gc             ] GC(52) Pause Degenerated GC (Outside of Cycle) 23343M->4544M(24576M) 6190.560ms
> [403.752s][info ][gc             ] GC(53) Pause Degenerated GC (Outside of Cycle) 23343M->4560M(24576M) 6226.817ms
> [410.816s][info ][gc             ] GC(54) Pause Degenerated GC (Outside of Cycle) 23343M->4560M(24576M) 6400.091ms
> [417.651s][info ][gc             ] GC(55) Pause Degenerated GC (Outside of Cycle) 23343M->4568M(24576M) 6206.848ms
> [424.040s][info ][gc             ] GC(56) Pause Degenerated GC (Outside of Cycle) 23343M->4576M(24576M) 5774.497ms
> [431.115s][info ][gc             ] GC(57) Pause Degenerated GC (Outside of Cycle) 23343M->4584M(24576M) 6433.645ms
> [437.740s][info ][gc             ] GC(58) Pause Degenerated GC (Outside of Cycle) 23343M->4592M(24576M) 5980.660ms
> 
> 
> 
> The heap seems to grow slowly, and the pause time tends to grow too.
> The jdk16 or 17 version appears okay (with constant after-gc heap size).
> 
> I wonder if there is a known reason?
> 
> Thanks!
> 
> Yude Lin
> 


From yude.lyd at alibaba-inc.com  Fri May 21 04:42:10 2021
From: yude.lyd at alibaba-inc.com (Yude Lin)
Date: Fri, 21 May 2021 12:42:10 +0800
Subject: =?UTF-8?B?UmU6IEhlYXAgc2l6ZSBzbG93bHkgZ3Jvd3MgaW4gamRrMTF1IFNoZW5hbmRvYWggcGFzc2l2?=
 =?UTF-8?B?ZSBtb2Rl?=
In-Reply-To: <841bbfaf-72a4-5510-febf-c3eb111a20b5@redhat.com>
References: <a41a2051-67b9-47b3-96e8-d2ce9d9574ca.>
 <94a73cf4-0497-4bd2-a791-7dda292a6c06.>
 <4e6701bd-c5b2-495c-b01b-09d5565da50d.yude.lyd@alibaba-inc.com>,
 <841bbfaf-72a4-5510-febf-c3eb111a20b5@redhat.com>
Message-ID: <2c7be7af-dce2-42e3-81b2-bfe689a79c84.yude.lyd@alibaba-inc.com>

Hi Zhengyu,

I tried -XX:ShenandoahUnloadClassesFrequency=1 but I still get the same symptom.
I checked the log and it has "Unload Classes" every cycle, so unloading is on. Any ideas?
Thanks!

Yude
------------------------------------------------------------------
From:Zhengyu Gu <zgu at redhat.com>
Send Time:2021?5?20?(???) 23:30
To:???(??) <yude.lyd at alibaba-inc.com>; shenandoah-dev <shenandoah-dev at openjdk.java.net>
Subject:Re: Heap size slowly grows in jdk11u Shenandoah passive mode

Hi,

This may be due to class unloading. Shenandoah/jdk11u, by default, 
unloads classes every 100 cycles, while jdk16/17 unloads classes 
(concurrently) every cycle.

You may want to try: -XX:ShenandoahUnloadClassesFrequency=1

-Zhengyu

On 5/20/21 1:17 AM, ???(??) wrote:
> 
> 
> Hi,
> 
> I was running jdk11u Shenandoah on specjvm2008, and found the passive mode has a leak-like behaviour:
> 
> $java -Xmx24g -Xms24g -XX:ParallelGCThreads=1 -XX:+UseShenandoahGC -XX:+UnlockDiagnosticVMOptions -XX:ShenandoahGCMode=passive -Xlog:gc*=debug:file=512sp.log -jar SPECjvm2008.jar -ict -coe -i 1 derby
> 
> $grep 'Pause .*ms' 512sp.log
> [63.407s][info ][gc             ] GC(0) Pause Degenerated GC (Outside of Cycle) 23344M->15824M(24576M) 6994.533ms
> [71.029s][info ][gc             ] GC(1) Pause Degenerated GC (Outside of Cycle) 23343M->15520M(24576M) 7325.687ms
> [80.319s][info ][gc             ] GC(2) Pause Degenerated GC (Outside of Cycle) 23343M->11720M(24576M) 8721.156ms
> [88.396s][info ][gc             ] GC(3) Pause Degenerated GC (Outside of Cycle) 23343M->4176M(24576M) 7549.126ms
> [94.300s][info ][gc             ] GC(4) Pause Degenerated GC (Outside of Cycle) 23343M->4192M(24576M) 5216.467ms
> [99.941s][info ][gc             ] GC(5) Pause Degenerated GC (Outside of Cycle) 23343M->4200M(24576M) 4984.741ms
> [105.506s][info ][gc             ] GC(6) Pause Degenerated GC (Outside of Cycle) 23343M->4208M(24576M) 4901.316ms
> [111.339s][info ][gc             ] GC(7) Pause Degenerated GC (Outside of Cycle) 23343M->4216M(24576M) 5175.875ms
> [117.154s][info ][gc             ] GC(8) Pause Degenerated GC (Outside of Cycle) 23343M->4224M(24576M) 5134.298ms
> [123.078s][info ][gc             ] GC(9) Pause Degenerated GC (Outside of Cycle) 23343M->4232M(24576M) 5255.877ms
> [129.249s][info ][gc             ] GC(10) Pause Degenerated GC (Outside of Cycle) 23343M->4232M(24576M) 5505.492ms
> [135.258s][info ][gc             ] GC(11) Pause Degenerated GC (Outside of Cycle) 23343M->4240M(24576M) 5355.191ms
> [141.378s][info ][gc             ] GC(12) Pause Degenerated GC (Outside of Cycle) 23343M->4248M(24576M) 5487.325ms
> [147.217s][info ][gc             ] GC(13) Pause Degenerated GC (Outside of Cycle) 23343M->4256M(24576M) 5214.325ms
> [152.963s][info ][gc             ] GC(14) Pause Degenerated GC (Outside of Cycle) 23343M->4264M(24576M) 5083.129ms
> [158.798s][info ][gc             ] GC(15) Pause Degenerated GC (Outside of Cycle) 23343M->4272M(24576M) 5162.959ms
> [164.950s][info ][gc             ] GC(16) Pause Degenerated GC (Outside of Cycle) 23343M->4280M(24576M) 5475.542ms
> [170.856s][info ][gc             ] GC(17) Pause Degenerated GC (Outside of Cycle) 23343M->4288M(24576M) 5222.228ms
> [177.007s][info ][gc             ] GC(18) Pause Degenerated GC (Outside of Cycle) 23343M->4304M(24576M) 5300.388ms
> [182.809s][info ][gc             ] GC(19) Pause Degenerated GC (Outside of Cycle) 23343M->4312M(24576M) 4833.105ms
> [189.589s][info ][gc             ] GC(20) Pause Degenerated GC (Outside of Cycle) 23344M->4312M(24576M) 5441.911ms
> [195.528s][info ][gc             ] GC(21) Pause Degenerated GC (Outside of Cycle) 23343M->4320M(24576M) 5262.478ms
> [201.654s][info ][gc             ] GC(22) Pause Degenerated GC (Outside of Cycle) 23343M->4328M(24576M) 5491.649ms
> [208.601s][info ][gc             ] GC(23) Pause Degenerated GC (Outside of Cycle) 23343M->4336M(24576M) 6293.675ms
> [214.483s][info ][gc             ] GC(24) Pause Degenerated GC (Outside of Cycle) 23343M->4344M(24576M) 5259.813ms
> [220.657s][info ][gc             ] GC(25) Pause Degenerated GC (Outside of Cycle) 23343M->4352M(24576M) 5533.361ms
> [227.005s][info ][gc             ] GC(26) Pause Degenerated GC (Outside of Cycle) 23343M->4352M(24576M) 5712.247ms
> [233.210s][info ][gc             ] GC(27) Pause Degenerated GC (Outside of Cycle) 23343M->4368M(24576M) 5580.975ms
> [239.202s][info ][gc             ] GC(28) Pause Degenerated GC (Outside of Cycle) 23343M->4368M(24576M) 5379.766ms
> [245.565s][info ][gc             ] GC(29) Pause Degenerated GC (Outside of Cycle) 23343M->4376M(24576M) 5714.096ms
> [251.450s][info ][gc             ] GC(30) Pause Degenerated GC (Outside of Cycle) 23343M->4384M(24576M) 5250.005ms
> [257.447s][info ][gc             ] GC(31) Pause Degenerated GC (Outside of Cycle) 23343M->4392M(24576M) 5354.806ms
> [263.840s][info ][gc             ] GC(32) Pause Degenerated GC (Outside of Cycle) 23343M->4400M(24576M) 5716.993ms
> [270.792s][info ][gc             ] GC(33) Pause Degenerated GC (Outside of Cycle) 23343M->4408M(24576M) 6264.228ms
> [277.201s][info ][gc             ] GC(34) Pause Degenerated GC (Outside of Cycle) 23343M->4408M(24576M) 5743.778ms
> [283.828s][info ][gc             ] GC(35) Pause Degenerated GC (Outside of Cycle) 23343M->4416M(24576M) 5973.385ms
> [291.227s][info ][gc             ] GC(36) Pause Degenerated GC (Outside of Cycle) 23343M->4424M(24576M) 6587.729ms
> [298.153s][info ][gc             ] GC(37) Pause Degenerated GC (Outside of Cycle) 23343M->4432M(24576M) 6122.285ms
> [304.722s][info ][gc             ] GC(38) Pause Degenerated GC (Outside of Cycle) 23343M->4440M(24576M) 5938.363ms
> [311.097s][info ][gc             ] GC(39) Pause Degenerated GC (Outside of Cycle) 23343M->4448M(24576M) 5737.377ms
> [317.710s][info ][gc             ] GC(40) Pause Degenerated GC (Outside of Cycle) 23343M->4448M(24576M) 5994.975ms
> [323.969s][info ][gc             ] GC(41) Pause Degenerated GC (Outside of Cycle) 23343M->4456M(24576M) 5624.046ms
> [330.910s][info ][gc             ] GC(42) Pause Degenerated GC (Outside of Cycle) 23343M->4464M(24576M) 6298.006ms
> [337.761s][info ][gc             ] GC(43) Pause Degenerated GC (Outside of Cycle) 23343M->4472M(24576M) 6248.352ms
> [344.369s][info ][gc             ] GC(44) Pause Degenerated GC (Outside of Cycle) 23343M->4480M(24576M) 5947.428ms
> [351.096s][info ][gc             ] GC(45) Pause Degenerated GC (Outside of Cycle) 23343M->4488M(24576M) 6094.857ms
> [357.776s][info ][gc             ] GC(46) Pause Degenerated GC (Outside of Cycle) 23343M->4496M(24576M) 6048.758ms
> [364.830s][info ][gc             ] GC(47) Pause Degenerated GC (Outside of Cycle) 23343M->4504M(24576M) 6418.273ms
> [370.804s][info ][gc             ] GC(48) Pause Degenerated GC (Outside of Cycle) 23343M->4512M(24576M) 5348.577ms
> [376.787s][info ][gc             ] GC(49) Pause Degenerated GC (Outside of Cycle) 23344M->4520M(24576M) 5339.928ms
> [383.407s][info ][gc             ] GC(50) Pause Degenerated GC (Outside of Cycle) 23343M->4528M(24576M) 5942.465ms
> [389.858s][info ][gc             ] GC(51) Pause Degenerated GC (Outside of Cycle) 23343M->4536M(24576M) 5801.355ms
> [396.714s][info ][gc             ] GC(52) Pause Degenerated GC (Outside of Cycle) 23343M->4544M(24576M) 6190.560ms
> [403.752s][info ][gc             ] GC(53) Pause Degenerated GC (Outside of Cycle) 23343M->4560M(24576M) 6226.817ms
> [410.816s][info ][gc             ] GC(54) Pause Degenerated GC (Outside of Cycle) 23343M->4560M(24576M) 6400.091ms
> [417.651s][info ][gc             ] GC(55) Pause Degenerated GC (Outside of Cycle) 23343M->4568M(24576M) 6206.848ms
> [424.040s][info ][gc             ] GC(56) Pause Degenerated GC (Outside of Cycle) 23343M->4576M(24576M) 5774.497ms
> [431.115s][info ][gc             ] GC(57) Pause Degenerated GC (Outside of Cycle) 23343M->4584M(24576M) 6433.645ms
> [437.740s][info ][gc             ] GC(58) Pause Degenerated GC (Outside of Cycle) 23343M->4592M(24576M) 5980.660ms
> 
> 
> 
> The heap seems to grow slowly, and the pause time tends to grow too.
> The jdk16 or 17 version appears okay (with constant after-gc heap size).
> 
> I wonder if there is a known reason?
> 
> Thanks!
> 
> Yude Lin
> 

From stefank at openjdk.java.net  Fri May 21 09:11:09 2021
From: stefank at openjdk.java.net (Stefan Karlsson)
Date: Fri, 21 May 2021 09:11:09 GMT
Subject: RFR: 8267464: Circular-dependency resilient inline headers [v2]
In-Reply-To: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
Message-ID: <2umZkQrx2-G55b24P-HV35dWJ84VMaumFaNizjpNw_k=.27e4e0b0-1aef-4867-a49b-903f9b860bec@github.com>

> I'd like to propose a small adjustment to how we write .inline.hpp files, in the hope to reduce include problems because of circular dependencies between inline headers.
> 
> This is a small, contrived example to show the problem:
> 
> // a.hpp
> #pragma once
> 
> void a1();
> void a2();
> 
> 
> // a.inline.hpp
> #pragma once
> 
> #include "a.hpp"
> #include "b.inline.hpp"
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> 
> // b.hpp
> #pragma once
> 
> void b1();
> void b2();
> 
> 
> // b.inline.hpp
> #pragma once
> 
> #include "a.inline.hpp"
> #include "b.hpp"
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> The following code compiles fine:
> 
> // a.cpp
> #include "a.inline.hpp"
> 
> int main() {
>   a1();
> 
>   return 0;
> }
> 
> But the following:
> 
> // b.cpp
> #include "b.inline.hpp"
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> 
> fails with the this error message:
> 
> In file included from b.inline.hpp:3,
>                  from b.cpp:1:
> a.inline.hpp: In function ?void a1()?:
> a.inline.hpp:8:3: error: ?b1? was not declared in this scope; did you mean ?a1??
> 
> We can see the problem with g++ -E:
> 
> # 1 "a.inline.hpp" 1
> # 1 "a.hpp" 1
> 
> void a1();
> void a2();
> 
> # 4 "a.inline.hpp" 2
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> # 4 "b.inline.hpp" 2
> # 1 "b.hpp" 1
> 
> void b1();
> void b2();
> 
> # 5 "b.inline.hpp" 2
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> # 2 "b.cpp" 2
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> 
> b1() is called before it has been declared. b.inline.hpp inlined a.inline.hpp, which uses functions declared in b.hpp. However, at that point we've not included enough of b.inline.hpp to have declared b1().
> 
> This is a small and easy example. In the JVM the circular dependencies usually involves more than two files, and often from different sub-systems of the JVM. We have so-far solved this by restructuring the code, making functions out-of-line, creating proxy objects, etc. However, the more we use templates, the more this is going to become a reoccurring nuisance. And in experiments with lambdas, which requires templates, this very quickly showed up as a problem.
> 
> I propose that we solve most (all?) of these issues by adding a rule that states that .inline.hpp files should start by including the corresponding .hpp, and that the .hpp should contain the declarations of all externally used parts of the .inline.hpp file. This should guarantee that the declarations are always present before any subsequent include can create a circular include dependency back to the original file.
> 
> In the example above, b.inline.hpp would become:
> 
> // b.inline.hpp
> #pragma once
> 
> #include "b.hpp"
> #include "a.inline.hpp"
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> and now both a.cpp and b.cpp compiles. The generated output now looks like this:
> 
> # 1 "b.inline.hpp" 1
> # 1 "b.hpp" 1
> 
> void b1();
> void b2();
> 
> # 4 "b.inline.hpp" 2
> # 1 "a.inline.hpp" 1
> 
> # 1 "a.hpp" 1
> 
> void a1();
> void a2();
> 
> # 4 "a.inline.hpp" 2
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> # 5 "b.inline.hpp" 2
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> # 2 "b.cpp" 2
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> The declarations come first, and the compiler is happy. 
> 
> An alternative to this, that has been proposed by other HotSpot GC devs have been to always include all .hpp files first, and then have a section of .inline.hpp includes. I've experimented with that as well, but I think it requires more maintenance and vigilance of  *users* .inline.hpp files (add .hpp include to the first section, add .inline.hpp section to second). The proposed structures only requires extra care from *writers* of .inline.hpp files. All others can include .hpp and .inline.hpp as we've been doing for a long time now.
> 
> Some notes about this patch:
> 1) Some externally-used declarations have been placed in .inline.hpp files, and not in the .hpp. Those should be moved. I have not done that.
> 2) Some .inline.hpp files don't have a corresponding .hpp file. I could either try to fix that in this patch, or we could take care of that as separate patches later.
> 3) The style I chose was to add the .hpp include and a extra blank line, separating it from the rest of the includes. I think I like that style, because it makes it easy for those writing .inline.hpp to recognize this pattern. And it removes the confusion why this include isn't sorted into the other includes.
> 4) We have a few *special* platform dependent header files. Both those that simply are included into platform independent files, and those that inject code *into* the platform independent classes. Extra care, as always, need to be taken around those files.
> 5) Mostly tested locally, but I'll test on more platforms if the idea is accepted.
> 
> What do you think?

Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision:

  Clean up assembler_<cpu>.inline.hpp

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4127/files
  - new: https://git.openjdk.java.net/jdk/pull/4127/files/6f3fe6af..260c1115

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4127&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4127&range=00-01

  Stats: 45 lines in 28 files changed: 3 ins; 24 del; 18 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4127.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4127/head:pull/4127

PR: https://git.openjdk.java.net/jdk/pull/4127

From wkemper at openjdk.java.net  Fri May 21 15:10:50 2021
From: wkemper at openjdk.java.net (William Kemper)
Date: Fri, 21 May 2021 15:10:50 GMT
Subject: RFR: Mixed evacuation fixes
Message-ID: <NLpPuw0RDh4g7o6TWjhCl-141wzVTLYcjAixqeJRUXw=.e4831d22-25f6-4a5c-8006-359641444f94@github.com>

This PR has a few bug fixes, the most important of which corrects an error that prevented references between old regions from being updated. To handle this case we expose the notion of _mixed_ collections to the update reference loop. This lets us continue using the optimization of only updating references in the remembered set during a _pure_ young collection. The other fixes are comparatively minor and described in their commit message.

-------------

Commit messages:
 - Formatting fixes
 - Do not include cset regions when scanning the remembered set for the verifier
 - Only look at old regions when purging old SATB queue.
 - Only register successfully evacuated object with remembered set
 - Update references in old regions during a mixed gc cycle

Changes: https://git.openjdk.java.net/shenandoah/pull/43/files
 Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=43&range=00
  Stats: 59 lines in 6 files changed: 28 ins; 18 del; 13 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/43.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/43/head:pull/43

PR: https://git.openjdk.java.net/shenandoah/pull/43

From wkemper at openjdk.java.net  Fri May 21 21:16:58 2021
From: wkemper at openjdk.java.net (William Kemper)
Date: Fri, 21 May 2021 21:16:58 GMT
Subject: RFR: Collect old humongous regions immediately at end of concurrent
 old mark
Message-ID: <9M98PgtZ16qRHGhxPXP4RTdq4WIZ44imMcXLUwbtVsc=.16885e78-d54d-4f11-8630-face7e472539@github.com>

Recycle dead humongous objects at the end of concurrent old mark, rather than add them to the cset. This behavior is similar to young gen cset selection. It also increases the amount of free memory in old gen to support evacuations. Shenandoah has a specific code path for recycling humongous objects and (debug builds) will assert when an old region is added to the cset.

-------------

Commit messages:
 - Log the number of regions trashed when humongous object is reclaimed
 - Recycle old humongous objects immediately at the end of old marking

Changes: https://git.openjdk.java.net/shenandoah/pull/44/files
 Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=44&range=00
  Stats: 57 lines in 3 files changed: 3 ins; 34 del; 20 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/44.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/44/head:pull/44

PR: https://git.openjdk.java.net/shenandoah/pull/44

From zgu at redhat.com  Fri May 21 21:33:23 2021
From: zgu at redhat.com (Zhengyu Gu)
Date: Fri, 21 May 2021 17:33:23 -0400
Subject: Heap size slowly grows in jdk11u Shenandoah passive mode
In-Reply-To: <2c7be7af-dce2-42e3-81b2-bfe689a79c84.yude.lyd@alibaba-inc.com>
References: <a41a2051-67b9-47b3-96e8-d2ce9d9574ca.>
 <94a73cf4-0497-4bd2-a791-7dda292a6c06.>
 <4e6701bd-c5b2-495c-b01b-09d5565da50d.yude.lyd@alibaba-inc.com>
 <841bbfaf-72a4-5510-febf-c3eb111a20b5@redhat.com>
 <2c7be7af-dce2-42e3-81b2-bfe689a79c84.yude.lyd@alibaba-inc.com>
Message-ID: <b66a8402-333a-f1cc-0173-45c841f5c008@redhat.com>

Hi Yude,

Thanks for reporting and confirming. I found the root cause [1], it only 
affect 11u, as 16u and 17 we switched to concurrent reference processing.


-Zhengyu


[1] https://bugs.openjdk.java.net/browse/JDK-8267561


On 5/21/21 12:42 AM, Yude Lin wrote:
> Hi Zhengyu,
> 
> I tried -XX:ShenandoahUnloadClassesFrequency=1 but I still get the same 
> symptom.
> I checked the log and it has "Unload Classes" every cycle, so unloading 
> is on. Any ideas?
> Thanks!
> 
> Yude
> 
>     ------------------------------------------------------------------
>     From:Zhengyu Gu <zgu at redhat.com>
>     Send Time:2021?5?20?(???) 23:30
>     To:???(??) <yude.lyd at alibaba-inc.com>; shenandoah-dev
>     <shenandoah-dev at openjdk.java.net>
>     Subject:Re: Heap size slowly grows in jdk11u Shenandoah passive mode
> 
>     Hi,
> 
>     This?may?be?due?to?class?unloading.?Shenandoah/jdk11u,?by?default,
>     unloads?classes?every?100?cycles,?while?jdk16/17?unloads?classes
>     (concurrently)?every?cycle.
> 
>     You?may?want?to?try:?-XX:ShenandoahUnloadClassesFrequency=1
> 
>     -Zhengyu
> 
>     On?5/20/21?1:17?AM,????(??)?wrote:
>      >
>      >
>      >?Hi,
>      >
>      >?I?was?running?jdk11u?Shenandoah?on?specjvm2008,?and?found?the?passive?mode?has?a?leak-like?behaviour:
>      >
>      >?$java?-Xmx24g?-Xms24g?-XX:ParallelGCThreads=1?-XX:+UseShenandoahGC?-XX:+UnlockDiagnosticVMOptions?-XX:ShenandoahGCMode=passive?-Xlog:gc*=debug:file=512sp.log?-jar?SPECjvm2008.jar?-ict?-coe?-i?1?derby
>      >
>      >?$grep?'Pause?.*ms'?512sp.log
>      >?[63.407s][info?][gc?????????????]?GC(0)?Pause?Degenerated?GC?(Outside?of?Cycle)?23344M->15824M(24576M)?6994.533ms
>      >?[71.029s][info?][gc?????????????]?GC(1)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->15520M(24576M)?7325.687ms
>      >?[80.319s][info?][gc?????????????]?GC(2)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->11720M(24576M)?8721.156ms
>      >?[88.396s][info?][gc?????????????]?GC(3)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4176M(24576M)?7549.126ms
>      >?[94.300s][info?][gc?????????????]?GC(4)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4192M(24576M)?5216.467ms
>      >?[99.941s][info?][gc?????????????]?GC(5)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4200M(24576M)?4984.741ms
>      >?[105.506s][info?][gc?????????????]?GC(6)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4208M(24576M)?4901.316ms
>      >?[111.339s][info?][gc?????????????]?GC(7)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4216M(24576M)?5175.875ms
>      >?[117.154s][info?][gc?????????????]?GC(8)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4224M(24576M)?5134.298ms
>      >?[123.078s][info?][gc?????????????]?GC(9)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4232M(24576M)?5255.877ms
>      >?[129.249s][info?][gc?????????????]?GC(10)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4232M(24576M)?5505.492ms
>      >?[135.258s][info?][gc?????????????]?GC(11)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4240M(24576M)?5355.191ms
>      >?[141.378s][info?][gc?????????????]?GC(12)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4248M(24576M)?5487.325ms
>      >?[147.217s][info?][gc?????????????]?GC(13)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4256M(24576M)?5214.325ms
>      >?[152.963s][info?][gc?????????????]?GC(14)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4264M(24576M)?5083.129ms
>      >?[158.798s][info?][gc?????????????]?GC(15)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4272M(24576M)?5162.959ms
>      >?[164.950s][info?][gc?????????????]?GC(16)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4280M(24576M)?5475.542ms
>      >?[170.856s][info?][gc?????????????]?GC(17)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4288M(24576M)?5222.228ms
>      >?[177.007s][info?][gc?????????????]?GC(18)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4304M(24576M)?5300.388ms
>      >?[182.809s][info?][gc?????????????]?GC(19)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4312M(24576M)?4833.105ms
>      >?[189.589s][info?][gc?????????????]?GC(20)?Pause?Degenerated?GC?(Outside?of?Cycle)?23344M->4312M(24576M)?5441.911ms
>      >?[195.528s][info?][gc?????????????]?GC(21)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4320M(24576M)?5262.478ms
>      >?[201.654s][info?][gc?????????????]?GC(22)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4328M(24576M)?5491.649ms
>      >?[208.601s][info?][gc?????????????]?GC(23)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4336M(24576M)?6293.675ms
>      >?[214.483s][info?][gc?????????????]?GC(24)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4344M(24576M)?5259.813ms
>      >?[220.657s][info?][gc?????????????]?GC(25)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4352M(24576M)?5533.361ms
>      >?[227.005s][info?][gc?????????????]?GC(26)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4352M(24576M)?5712.247ms
>      >?[233.210s][info?][gc?????????????]?GC(27)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4368M(24576M)?5580.975ms
>      >?[239.202s][info?][gc?????????????]?GC(28)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4368M(24576M)?5379.766ms
>      >?[245.565s][info?][gc?????????????]?GC(29)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4376M(24576M)?5714.096ms
>      >?[251.450s][info?][gc?????????????]?GC(30)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4384M(24576M)?5250.005ms
>      >?[257.447s][info?][gc?????????????]?GC(31)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4392M(24576M)?5354.806ms
>      >?[263.840s][info?][gc?????????????]?GC(32)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4400M(24576M)?5716.993ms
>      >?[270.792s][info?][gc?????????????]?GC(33)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4408M(24576M)?6264.228ms
>      >?[277.201s][info?][gc?????????????]?GC(34)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4408M(24576M)?5743.778ms
>      >?[283.828s][info?][gc?????????????]?GC(35)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4416M(24576M)?5973.385ms
>      >?[291.227s][info?][gc?????????????]?GC(36)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4424M(24576M)?6587.729ms
>      >?[298.153s][info?][gc?????????????]?GC(37)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4432M(24576M)?6122.285ms
>      >?[304.722s][info?][gc?????????????]?GC(38)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4440M(24576M)?5938.363ms
>      >?[311.097s][info?][gc?????????????]?GC(39)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4448M(24576M)?5737.377ms
>      >?[317.710s][info?][gc?????????????]?GC(40)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4448M(24576M)?5994.975ms
>      >?[323.969s][info?][gc?????????????]?GC(41)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4456M(24576M)?5624.046ms
>      >?[330.910s][info?][gc?????????????]?GC(42)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4464M(24576M)?6298.006ms
>      >?[337.761s][info?][gc?????????????]?GC(43)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4472M(24576M)?6248.352ms
>      >?[344.369s][info?][gc?????????????]?GC(44)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4480M(24576M)?5947.428ms
>      >?[351.096s][info?][gc?????????????]?GC(45)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4488M(24576M)?6094.857ms
>      >?[357.776s][info?][gc?????????????]?GC(46)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4496M(24576M)?6048.758ms
>      >?[364.830s][info?][gc?????????????]?GC(47)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4504M(24576M)?6418.273ms
>      >?[370.804s][info?][gc?????????????]?GC(48)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4512M(24576M)?5348.577ms
>      >?[376.787s][info?][gc?????????????]?GC(49)?Pause?Degenerated?GC?(Outside?of?Cycle)?23344M->4520M(24576M)?5339.928ms
>      >?[383.407s][info?][gc?????????????]?GC(50)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4528M(24576M)?5942.465ms
>      >?[389.858s][info?][gc?????????????]?GC(51)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4536M(24576M)?5801.355ms
>      >?[396.714s][info?][gc?????????????]?GC(52)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4544M(24576M)?6190.560ms
>      >?[403.752s][info?][gc?????????????]?GC(53)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4560M(24576M)?6226.817ms
>      >?[410.816s][info?][gc?????????????]?GC(54)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4560M(24576M)?6400.091ms
>      >?[417.651s][info?][gc?????????????]?GC(55)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4568M(24576M)?6206.848ms
>      >?[424.040s][info?][gc?????????????]?GC(56)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4576M(24576M)?5774.497ms
>      >?[431.115s][info?][gc?????????????]?GC(57)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4584M(24576M)?6433.645ms
>      >?[437.740s][info?][gc?????????????]?GC(58)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4592M(24576M)?5980.660ms
>      >
>      >
>      >
>      >?The?heap?seems?to?grow?slowly,?and?the?pause?time?tends?to?grow?too.
>      >?The?jdk16?or?17?version?appears?okay?(with?constant?after-gc?heap?size).
>      >
>      >?I?wonder?if?there?is?a?known?reason?
>      >
>      >?Thanks!
>      >
>      >?Yude?Lin
>      > 
> 


From zgu at redhat.com  Sat May 22 23:20:54 2021
From: zgu at redhat.com (Zhengyu Gu)
Date: Sat, 22 May 2021 19:20:54 -0400
Subject: [11u] RFR 8267561: Shenandoah: Reference processing not properly
 setup for outside of cycle degenerated GC
Message-ID: <7ef26b6b-fd30-9e48-6543-3c3fb1c472e9@redhat.com>

Alibaba reported that Shenandoah outside-of-cycle degenerated GC slowly 
leak, please see CR for details.

The cause is that reference processing not setup properly during 
outside-of-cycle degenerated GC.

Webrev: http://cr.openjdk.java.net/~zgu/JDK-8267561/webrev.00/

Test:
   hotspot_gc_shenandoah
   specJVM Debery with configuration reported in CR.

Thanks,

-Zhengyu


From kbarrett at openjdk.java.net  Mon May 24 00:17:01 2021
From: kbarrett at openjdk.java.net (Kim Barrett)
Date: Mon, 24 May 2021 00:17:01 GMT
Subject: RFR: 8267464: Circular-dependency resilient inline headers [v2]
In-Reply-To: <2umZkQrx2-G55b24P-HV35dWJ84VMaumFaNizjpNw_k=.27e4e0b0-1aef-4867-a49b-903f9b860bec@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
 <2umZkQrx2-G55b24P-HV35dWJ84VMaumFaNizjpNw_k=.27e4e0b0-1aef-4867-a49b-903f9b860bec@github.com>
Message-ID: <6K69ZSv9jA-M9nRoeCAkw-PygKL2GVUg30NqfI0wbwo=.b082724d-986d-4bbb-8bd0-175200f3031f@github.com>

On Fri, 21 May 2021 09:11:09 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> I'd like to propose a small adjustment to how we write .inline.hpp files, in the hope to reduce include problems because of circular dependencies between inline headers.
>> 
>> This is a small, contrived example to show the problem:
>> 
>> // a.hpp
>> #pragma once
>> 
>> void a1();
>> void a2();
>> 
>> 
>> // a.inline.hpp
>> #pragma once
>> 
>> #include "a.hpp"
>> #include "b.inline.hpp"
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> 
>> // b.hpp
>> #pragma once
>> 
>> void b1();
>> void b2();
>> 
>> 
>> // b.inline.hpp
>> #pragma once
>> 
>> #include "a.inline.hpp"
>> #include "b.hpp"
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> The following code compiles fine:
>> 
>> // a.cpp
>> #include "a.inline.hpp"
>> 
>> int main() {
>>   a1();
>> 
>>   return 0;
>> }
>> 
>> But the following:
>> 
>> // b.cpp
>> #include "b.inline.hpp"
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> 
>> fails with the this error message:
>> 
>> In file included from b.inline.hpp:3,
>>                  from b.cpp:1:
>> a.inline.hpp: In function ?void a1()?:
>> a.inline.hpp:8:3: error: ?b1? was not declared in this scope; did you mean ?a1??
>> 
>> We can see the problem with g++ -E:
>> 
>> # 1 "a.inline.hpp" 1
>> # 1 "a.hpp" 1
>> 
>> void a1();
>> void a2();
>> 
>> # 4 "a.inline.hpp" 2
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> # 4 "b.inline.hpp" 2
>> # 1 "b.hpp" 1
>> 
>> void b1();
>> void b2();
>> 
>> # 5 "b.inline.hpp" 2
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> # 2 "b.cpp" 2
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> 
>> b1() is called before it has been declared. b.inline.hpp inlined a.inline.hpp, which uses functions declared in b.hpp. However, at that point we've not included enough of b.inline.hpp to have declared b1().
>> 
>> This is a small and easy example. In the JVM the circular dependencies usually involves more than two files, and often from different sub-systems of the JVM. We have so-far solved this by restructuring the code, making functions out-of-line, creating proxy objects, etc. However, the more we use templates, the more this is going to become a reoccurring nuisance. And in experiments with lambdas, which requires templates, this very quickly showed up as a problem.
>> 
>> I propose that we solve most (all?) of these issues by adding a rule that states that .inline.hpp files should start by including the corresponding .hpp, and that the .hpp should contain the declarations of all externally used parts of the .inline.hpp file. This should guarantee that the declarations are always present before any subsequent include can create a circular include dependency back to the original file.
>> 
>> In the example above, b.inline.hpp would become:
>> 
>> // b.inline.hpp
>> #pragma once
>> 
>> #include "b.hpp"
>> #include "a.inline.hpp"
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> and now both a.cpp and b.cpp compiles. The generated output now looks like this:
>> 
>> # 1 "b.inline.hpp" 1
>> # 1 "b.hpp" 1
>> 
>> void b1();
>> void b2();
>> 
>> # 4 "b.inline.hpp" 2
>> # 1 "a.inline.hpp" 1
>> 
>> # 1 "a.hpp" 1
>> 
>> void a1();
>> void a2();
>> 
>> # 4 "a.inline.hpp" 2
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> # 5 "b.inline.hpp" 2
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> # 2 "b.cpp" 2
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> The declarations come first, and the compiler is happy. 
>> 
>> An alternative to this, that has been proposed by other HotSpot GC devs have been to always include all .hpp files first, and then have a section of .inline.hpp includes. I've experimented with that as well, but I think it requires more maintenance and vigilance of  *users* .inline.hpp files (add .hpp include to the first section, add .inline.hpp section to second). The proposed structures only requires extra care from *writers* of .inline.hpp files. All others can include .hpp and .inline.hpp as we've been doing for a long time now.
>> 
>> Some notes about this patch:
>> 1) Some externally-used declarations have been placed in .inline.hpp files, and not in the .hpp. Those should be moved. I have not done that.
>> 2) Some .inline.hpp files don't have a corresponding .hpp file. I could either try to fix that in this patch, or we could take care of that as separate patches later.
>> 3) The style I chose was to add the .hpp include and a extra blank line, separating it from the rest of the includes. I think I like that style, because it makes it easy for those writing .inline.hpp to recognize this pattern. And it removes the confusion why this include isn't sorted into the other includes.
>> 4) We have a few *special* platform dependent header files. Both those that simply are included into platform independent files, and those that inject code *into* the platform independent classes. Extra care, as always, need to be taken around those files.
>> 5) Mostly tested locally, but I'll test on more platforms if the idea is accepted.
>> 
>> What do you think?
>
> Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Clean up assembler_<cpu>.inline.hpp

Code changes look fine.  This needs an update to the style guide too.

-------------

Changes requested by kbarrett (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/4127

From eosterlund at openjdk.java.net  Mon May 24 08:37:11 2021
From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=)
Date: Mon, 24 May 2021 08:37:11 GMT
Subject: RFR: 8267464: Circular-dependency resilient inline headers [v2]
In-Reply-To: <2umZkQrx2-G55b24P-HV35dWJ84VMaumFaNizjpNw_k=.27e4e0b0-1aef-4867-a49b-903f9b860bec@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
 <2umZkQrx2-G55b24P-HV35dWJ84VMaumFaNizjpNw_k=.27e4e0b0-1aef-4867-a49b-903f9b860bec@github.com>
Message-ID: <uUCAaBd3TtMTQuUvhGwTvKH3xtZC_je0Fg9oFUX1qmg=.0f12157d-a0fc-44c9-95b7-8b0a55a80c73@github.com>

On Fri, 21 May 2021 09:11:09 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> I'd like to propose a small adjustment to how we write .inline.hpp files, in the hope to reduce include problems because of circular dependencies between inline headers.
>> 
>> This is a small, contrived example to show the problem:
>> 
>> // a.hpp
>> #pragma once
>> 
>> void a1();
>> void a2();
>> 
>> 
>> // a.inline.hpp
>> #pragma once
>> 
>> #include "a.hpp"
>> #include "b.inline.hpp"
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> 
>> // b.hpp
>> #pragma once
>> 
>> void b1();
>> void b2();
>> 
>> 
>> // b.inline.hpp
>> #pragma once
>> 
>> #include "a.inline.hpp"
>> #include "b.hpp"
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> The following code compiles fine:
>> 
>> // a.cpp
>> #include "a.inline.hpp"
>> 
>> int main() {
>>   a1();
>> 
>>   return 0;
>> }
>> 
>> But the following:
>> 
>> // b.cpp
>> #include "b.inline.hpp"
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> 
>> fails with the this error message:
>> 
>> In file included from b.inline.hpp:3,
>>                  from b.cpp:1:
>> a.inline.hpp: In function ?void a1()?:
>> a.inline.hpp:8:3: error: ?b1? was not declared in this scope; did you mean ?a1??
>> 
>> We can see the problem with g++ -E:
>> 
>> # 1 "a.inline.hpp" 1
>> # 1 "a.hpp" 1
>> 
>> void a1();
>> void a2();
>> 
>> # 4 "a.inline.hpp" 2
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> # 4 "b.inline.hpp" 2
>> # 1 "b.hpp" 1
>> 
>> void b1();
>> void b2();
>> 
>> # 5 "b.inline.hpp" 2
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> # 2 "b.cpp" 2
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> 
>> b1() is called before it has been declared. b.inline.hpp inlined a.inline.hpp, which uses functions declared in b.hpp. However, at that point we've not included enough of b.inline.hpp to have declared b1().
>> 
>> This is a small and easy example. In the JVM the circular dependencies usually involves more than two files, and often from different sub-systems of the JVM. We have so-far solved this by restructuring the code, making functions out-of-line, creating proxy objects, etc. However, the more we use templates, the more this is going to become a reoccurring nuisance. And in experiments with lambdas, which requires templates, this very quickly showed up as a problem.
>> 
>> I propose that we solve most (all?) of these issues by adding a rule that states that .inline.hpp files should start by including the corresponding .hpp, and that the .hpp should contain the declarations of all externally used parts of the .inline.hpp file. This should guarantee that the declarations are always present before any subsequent include can create a circular include dependency back to the original file.
>> 
>> In the example above, b.inline.hpp would become:
>> 
>> // b.inline.hpp
>> #pragma once
>> 
>> #include "b.hpp"
>> #include "a.inline.hpp"
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> and now both a.cpp and b.cpp compiles. The generated output now looks like this:
>> 
>> # 1 "b.inline.hpp" 1
>> # 1 "b.hpp" 1
>> 
>> void b1();
>> void b2();
>> 
>> # 4 "b.inline.hpp" 2
>> # 1 "a.inline.hpp" 1
>> 
>> # 1 "a.hpp" 1
>> 
>> void a1();
>> void a2();
>> 
>> # 4 "a.inline.hpp" 2
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> # 5 "b.inline.hpp" 2
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> # 2 "b.cpp" 2
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> The declarations come first, and the compiler is happy. 
>> 
>> An alternative to this, that has been proposed by other HotSpot GC devs have been to always include all .hpp files first, and then have a section of .inline.hpp includes. I've experimented with that as well, but I think it requires more maintenance and vigilance of  *users* .inline.hpp files (add .hpp include to the first section, add .inline.hpp section to second). The proposed structures only requires extra care from *writers* of .inline.hpp files. All others can include .hpp and .inline.hpp as we've been doing for a long time now.
>> 
>> Some notes about this patch:
>> 1) Some externally-used declarations have been placed in .inline.hpp files, and not in the .hpp. Those should be moved. I have not done that.
>> 2) Some .inline.hpp files don't have a corresponding .hpp file. I could either try to fix that in this patch, or we could take care of that as separate patches later.
>> 3) The style I chose was to add the .hpp include and a extra blank line, separating it from the rest of the includes. I think I like that style, because it makes it easy for those writing .inline.hpp to recognize this pattern. And it removes the confusion why this include isn't sorted into the other includes.
>> 4) We have a few *special* platform dependent header files. Both those that simply are included into platform independent files, and those that inject code *into* the platform independent classes. Extra care, as always, need to be taken around those files.
>> 5) Mostly tested locally, but I'll test on more platforms if the idea is accepted.
>> 
>> What do you think?
>
> Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Clean up assembler_<cpu>.inline.hpp

Looks great!

-------------

Marked as reviewed by eosterlund (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/4127

From stefank at openjdk.java.net  Mon May 24 12:14:10 2021
From: stefank at openjdk.java.net (Stefan Karlsson)
Date: Mon, 24 May 2021 12:14:10 GMT
Subject: RFR: 8267464: Circular-dependency resilient inline headers [v3]
In-Reply-To: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
Message-ID: <aybWpryDEkLIPzdH-5zU4yu2nMcknZyxlyKX2oVHutE=.6232d6e0-13f4-4e3a-b202-491ff54e41ab@github.com>

> I'd like to propose a small adjustment to how we write .inline.hpp files, in the hope to reduce include problems because of circular dependencies between inline headers.
> 
> This is a small, contrived example to show the problem:
> 
> // a.hpp
> #pragma once
> 
> void a1();
> void a2();
> 
> 
> // a.inline.hpp
> #pragma once
> 
> #include "a.hpp"
> #include "b.inline.hpp"
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> 
> // b.hpp
> #pragma once
> 
> void b1();
> void b2();
> 
> 
> // b.inline.hpp
> #pragma once
> 
> #include "a.inline.hpp"
> #include "b.hpp"
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> The following code compiles fine:
> 
> // a.cpp
> #include "a.inline.hpp"
> 
> int main() {
>   a1();
> 
>   return 0;
> }
> 
> But the following:
> 
> // b.cpp
> #include "b.inline.hpp"
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> 
> fails with the this error message:
> 
> In file included from b.inline.hpp:3,
>                  from b.cpp:1:
> a.inline.hpp: In function ?void a1()?:
> a.inline.hpp:8:3: error: ?b1? was not declared in this scope; did you mean ?a1??
> 
> We can see the problem with g++ -E:
> 
> # 1 "a.inline.hpp" 1
> # 1 "a.hpp" 1
> 
> void a1();
> void a2();
> 
> # 4 "a.inline.hpp" 2
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> # 4 "b.inline.hpp" 2
> # 1 "b.hpp" 1
> 
> void b1();
> void b2();
> 
> # 5 "b.inline.hpp" 2
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> # 2 "b.cpp" 2
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> 
> b1() is called before it has been declared. b.inline.hpp inlined a.inline.hpp, which uses functions declared in b.hpp. However, at that point we've not included enough of b.inline.hpp to have declared b1().
> 
> This is a small and easy example. In the JVM the circular dependencies usually involves more than two files, and often from different sub-systems of the JVM. We have so-far solved this by restructuring the code, making functions out-of-line, creating proxy objects, etc. However, the more we use templates, the more this is going to become a reoccurring nuisance. And in experiments with lambdas, which requires templates, this very quickly showed up as a problem.
> 
> I propose that we solve most (all?) of these issues by adding a rule that states that .inline.hpp files should start by including the corresponding .hpp, and that the .hpp should contain the declarations of all externally used parts of the .inline.hpp file. This should guarantee that the declarations are always present before any subsequent include can create a circular include dependency back to the original file.
> 
> In the example above, b.inline.hpp would become:
> 
> // b.inline.hpp
> #pragma once
> 
> #include "b.hpp"
> #include "a.inline.hpp"
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> and now both a.cpp and b.cpp compiles. The generated output now looks like this:
> 
> # 1 "b.inline.hpp" 1
> # 1 "b.hpp" 1
> 
> void b1();
> void b2();
> 
> # 4 "b.inline.hpp" 2
> # 1 "a.inline.hpp" 1
> 
> # 1 "a.hpp" 1
> 
> void a1();
> void a2();
> 
> # 4 "a.inline.hpp" 2
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> # 5 "b.inline.hpp" 2
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> # 2 "b.cpp" 2
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> The declarations come first, and the compiler is happy. 
> 
> An alternative to this, that has been proposed by other HotSpot GC devs have been to always include all .hpp files first, and then have a section of .inline.hpp includes. I've experimented with that as well, but I think it requires more maintenance and vigilance of  *users* .inline.hpp files (add .hpp include to the first section, add .inline.hpp section to second). The proposed structures only requires extra care from *writers* of .inline.hpp files. All others can include .hpp and .inline.hpp as we've been doing for a long time now.
> 
> Some notes about this patch:
> 1) Some externally-used declarations have been placed in .inline.hpp files, and not in the .hpp. Those should be moved. I have not done that.
> 2) Some .inline.hpp files don't have a corresponding .hpp file. I could either try to fix that in this patch, or we could take care of that as separate patches later.
> 3) The style I chose was to add the .hpp include and a extra blank line, separating it from the rest of the includes. I think I like that style, because it makes it easy for those writing .inline.hpp to recognize this pattern. And it removes the confusion why this include isn't sorted into the other includes.
> 4) We have a few *special* platform dependent header files. Both those that simply are included into platform independent files, and those that inject code *into* the platform independent classes. Extra care, as always, need to be taken around those files.
> 5) Mostly tested locally, but I'll test on more platforms if the idea is accepted.
> 
> What do you think?

Stefan Karlsson has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:

 - Merge remote-tracking branch 'origin/master' into 8267464_circular-dependency_resilient_inline_headers
 - Clean up assembler_<cpu>.inline.hpp
 - 8267464: Circular-dependency resiliant inline headers

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4127/files
  - new: https://git.openjdk.java.net/jdk/pull/4127/files/260c1115..4bcd4348

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4127&range=02
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4127&range=01-02

  Stats: 5145 lines in 153 files changed: 2603 ins; 1960 del; 582 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4127.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4127/head:pull/4127

PR: https://git.openjdk.java.net/jdk/pull/4127

From stefank at openjdk.java.net  Mon May 24 12:14:10 2021
From: stefank at openjdk.java.net (Stefan Karlsson)
Date: Mon, 24 May 2021 12:14:10 GMT
Subject: RFR: 8267464: Circular-dependency resilient inline headers [v2]
In-Reply-To: <6K69ZSv9jA-M9nRoeCAkw-PygKL2GVUg30NqfI0wbwo=.b082724d-986d-4bbb-8bd0-175200f3031f@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
 <2umZkQrx2-G55b24P-HV35dWJ84VMaumFaNizjpNw_k=.27e4e0b0-1aef-4867-a49b-903f9b860bec@github.com>
 <6K69ZSv9jA-M9nRoeCAkw-PygKL2GVUg30NqfI0wbwo=.b082724d-986d-4bbb-8bd0-175200f3031f@github.com>
Message-ID: <NEjQTx8fQoTPVaeuCiXBa5u-sSrvgLj_TV3jyuNndo0=.2dee63f3-535d-45af-a566-09ee1120e05d@github.com>

On Mon, 24 May 2021 00:14:06 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:

>> Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Clean up assembler_<cpu>.inline.hpp
>
> Code changes look fine.  This needs an update to the style guide too.

Thanks @kimbarrett and @fisk for reviewing!

-------------

PR: https://git.openjdk.java.net/jdk/pull/4127

From zgu at redhat.com  Mon May 24 12:47:02 2021
From: zgu at redhat.com (Zhengyu Gu)
Date: Mon, 24 May 2021 08:47:02 -0400
Subject: Heap size slowly grows in jdk11u Shenandoah passive mode
In-Reply-To: <b66a8402-333a-f1cc-0173-45c841f5c008@redhat.com>
References: <a41a2051-67b9-47b3-96e8-d2ce9d9574ca.>
 <94a73cf4-0497-4bd2-a791-7dda292a6c06.>
 <4e6701bd-c5b2-495c-b01b-09d5565da50d.yude.lyd@alibaba-inc.com>
 <841bbfaf-72a4-5510-febf-c3eb111a20b5@redhat.com>
 <2c7be7af-dce2-42e3-81b2-bfe689a79c84.yude.lyd@alibaba-inc.com>
 <b66a8402-333a-f1cc-0173-45c841f5c008@redhat.com>
Message-ID: <131c378e-08dc-bd9f-d220-f2afe16d5c11@redhat.com>

Hi Yude,

Here is the fix: http://cr.openjdk.java.net/~zgu/JDK-8267561/webrev.00/

Could you verify it?

Thanks,

-Zhengyu

On 5/21/21 5:33 PM, Zhengyu Gu wrote:
> Hi Yude,
> 
> Thanks for reporting and confirming. I found the root cause [1], it only 
> affect 11u, as 16u and 17 we switched to concurrent reference processing.
> 
> 
> -Zhengyu
> 
> 
> [1] https://bugs.openjdk.java.net/browse/JDK-8267561
> 
> 
> On 5/21/21 12:42 AM, Yude Lin wrote:
>> Hi Zhengyu,
>>
>> I tried -XX:ShenandoahUnloadClassesFrequency=1 but I still get the 
>> same symptom.
>> I checked the log and it has "Unload Classes" every cycle, so 
>> unloading is on. Any ideas?
>> Thanks!
>>
>> Yude
>>
>> ??? ------------------------------------------------------------------
>> ??? From:Zhengyu Gu <zgu at redhat.com>
>> ??? Send Time:2021?5?20?(???) 23:30
>> ??? To:???(??) <yude.lyd at alibaba-inc.com>; shenandoah-dev
>> ??? <shenandoah-dev at openjdk.java.net>
>> ??? Subject:Re: Heap size slowly grows in jdk11u Shenandoah passive mode
>>
>> ??? Hi,
>>
>> ??? This?may?be?due?to?class?unloading.?Shenandoah/jdk11u,?by?default,
>> ??? unloads?classes?every?100?cycles,?while?jdk16/17?unloads?classes
>> ??? (concurrently)?every?cycle.
>>
>> ??? You?may?want?to?try:?-XX:ShenandoahUnloadClassesFrequency=1
>>
>> ??? -Zhengyu
>>
>> ??? On?5/20/21?1:17?AM,????(??)?wrote:
>> ???? >
>> ???? >
>> ???? >?Hi,
>> ???? >
>>      
>> >?I?was?running?jdk11u?Shenandoah?on?specjvm2008,?and?found?the?passive?mode?has?a?leak-like?behaviour: 
>>
>> ???? >
>>      
>> >?$java?-Xmx24g?-Xms24g?-XX:ParallelGCThreads=1?-XX:+UseShenandoahGC?-XX:+UnlockDiagnosticVMOptions?-XX:ShenandoahGCMode=passive?-Xlog:gc*=debug:file=512sp.log?-jar?SPECjvm2008.jar?-ict?-coe?-i?1?derby 
>>
>> ???? >
>> ???? >?$grep?'Pause?.*ms'?512sp.log
>>      
>> >?[63.407s][info?][gc?????????????]?GC(0)?Pause?Degenerated?GC?(Outside?of?Cycle)?23344M->15824M(24576M)?6994.533ms 
>>
>>      
>> >?[71.029s][info?][gc?????????????]?GC(1)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->15520M(24576M)?7325.687ms 
>>
>>      
>> >?[80.319s][info?][gc?????????????]?GC(2)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->11720M(24576M)?8721.156ms 
>>
>>      
>> >?[88.396s][info?][gc?????????????]?GC(3)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4176M(24576M)?7549.126ms 
>>
>>      
>> >?[94.300s][info?][gc?????????????]?GC(4)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4192M(24576M)?5216.467ms 
>>
>>      
>> >?[99.941s][info?][gc?????????????]?GC(5)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4200M(24576M)?4984.741ms 
>>
>>      
>> >?[105.506s][info?][gc?????????????]?GC(6)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4208M(24576M)?4901.316ms 
>>
>>      
>> >?[111.339s][info?][gc?????????????]?GC(7)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4216M(24576M)?5175.875ms 
>>
>>      
>> >?[117.154s][info?][gc?????????????]?GC(8)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4224M(24576M)?5134.298ms 
>>
>>      
>> >?[123.078s][info?][gc?????????????]?GC(9)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4232M(24576M)?5255.877ms 
>>
>>      
>> >?[129.249s][info?][gc?????????????]?GC(10)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4232M(24576M)?5505.492ms 
>>
>>      
>> >?[135.258s][info?][gc?????????????]?GC(11)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4240M(24576M)?5355.191ms 
>>
>>      
>> >?[141.378s][info?][gc?????????????]?GC(12)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4248M(24576M)?5487.325ms 
>>
>>      
>> >?[147.217s][info?][gc?????????????]?GC(13)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4256M(24576M)?5214.325ms 
>>
>>      
>> >?[152.963s][info?][gc?????????????]?GC(14)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4264M(24576M)?5083.129ms 
>>
>>      
>> >?[158.798s][info?][gc?????????????]?GC(15)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4272M(24576M)?5162.959ms 
>>
>>      
>> >?[164.950s][info?][gc?????????????]?GC(16)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4280M(24576M)?5475.542ms 
>>
>>      
>> >?[170.856s][info?][gc?????????????]?GC(17)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4288M(24576M)?5222.228ms 
>>
>>      
>> >?[177.007s][info?][gc?????????????]?GC(18)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4304M(24576M)?5300.388ms 
>>
>>      
>> >?[182.809s][info?][gc?????????????]?GC(19)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4312M(24576M)?4833.105ms 
>>
>>      
>> >?[189.589s][info?][gc?????????????]?GC(20)?Pause?Degenerated?GC?(Outside?of?Cycle)?23344M->4312M(24576M)?5441.911ms 
>>
>>      
>> >?[195.528s][info?][gc?????????????]?GC(21)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4320M(24576M)?5262.478ms 
>>
>>      
>> >?[201.654s][info?][gc?????????????]?GC(22)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4328M(24576M)?5491.649ms 
>>
>>      
>> >?[208.601s][info?][gc?????????????]?GC(23)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4336M(24576M)?6293.675ms 
>>
>>      
>> >?[214.483s][info?][gc?????????????]?GC(24)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4344M(24576M)?5259.813ms 
>>
>>      
>> >?[220.657s][info?][gc?????????????]?GC(25)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4352M(24576M)?5533.361ms 
>>
>>      
>> >?[227.005s][info?][gc?????????????]?GC(26)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4352M(24576M)?5712.247ms 
>>
>>      
>> >?[233.210s][info?][gc?????????????]?GC(27)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4368M(24576M)?5580.975ms 
>>
>>      
>> >?[239.202s][info?][gc?????????????]?GC(28)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4368M(24576M)?5379.766ms 
>>
>>      
>> >?[245.565s][info?][gc?????????????]?GC(29)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4376M(24576M)?5714.096ms 
>>
>>      
>> >?[251.450s][info?][gc?????????????]?GC(30)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4384M(24576M)?5250.005ms 
>>
>>      
>> >?[257.447s][info?][gc?????????????]?GC(31)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4392M(24576M)?5354.806ms 
>>
>>      
>> >?[263.840s][info?][gc?????????????]?GC(32)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4400M(24576M)?5716.993ms 
>>
>>      
>> >?[270.792s][info?][gc?????????????]?GC(33)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4408M(24576M)?6264.228ms 
>>
>>      
>> >?[277.201s][info?][gc?????????????]?GC(34)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4408M(24576M)?5743.778ms 
>>
>>      
>> >?[283.828s][info?][gc?????????????]?GC(35)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4416M(24576M)?5973.385ms 
>>
>>      
>> >?[291.227s][info?][gc?????????????]?GC(36)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4424M(24576M)?6587.729ms 
>>
>>      
>> >?[298.153s][info?][gc?????????????]?GC(37)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4432M(24576M)?6122.285ms 
>>
>>      
>> >?[304.722s][info?][gc?????????????]?GC(38)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4440M(24576M)?5938.363ms 
>>
>>      
>> >?[311.097s][info?][gc?????????????]?GC(39)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4448M(24576M)?5737.377ms 
>>
>>      
>> >?[317.710s][info?][gc?????????????]?GC(40)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4448M(24576M)?5994.975ms 
>>
>>      
>> >?[323.969s][info?][gc?????????????]?GC(41)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4456M(24576M)?5624.046ms 
>>
>>      
>> >?[330.910s][info?][gc?????????????]?GC(42)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4464M(24576M)?6298.006ms 
>>
>>      
>> >?[337.761s][info?][gc?????????????]?GC(43)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4472M(24576M)?6248.352ms 
>>
>>      
>> >?[344.369s][info?][gc?????????????]?GC(44)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4480M(24576M)?5947.428ms 
>>
>>      
>> >?[351.096s][info?][gc?????????????]?GC(45)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4488M(24576M)?6094.857ms 
>>
>>      
>> >?[357.776s][info?][gc?????????????]?GC(46)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4496M(24576M)?6048.758ms 
>>
>>      
>> >?[364.830s][info?][gc?????????????]?GC(47)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4504M(24576M)?6418.273ms 
>>
>>      
>> >?[370.804s][info?][gc?????????????]?GC(48)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4512M(24576M)?5348.577ms 
>>
>>      
>> >?[376.787s][info?][gc?????????????]?GC(49)?Pause?Degenerated?GC?(Outside?of?Cycle)?23344M->4520M(24576M)?5339.928ms 
>>
>>      
>> >?[383.407s][info?][gc?????????????]?GC(50)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4528M(24576M)?5942.465ms 
>>
>>      
>> >?[389.858s][info?][gc?????????????]?GC(51)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4536M(24576M)?5801.355ms 
>>
>>      
>> >?[396.714s][info?][gc?????????????]?GC(52)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4544M(24576M)?6190.560ms 
>>
>>      
>> >?[403.752s][info?][gc?????????????]?GC(53)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4560M(24576M)?6226.817ms 
>>
>>      
>> >?[410.816s][info?][gc?????????????]?GC(54)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4560M(24576M)?6400.091ms 
>>
>>      
>> >?[417.651s][info?][gc?????????????]?GC(55)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4568M(24576M)?6206.848ms 
>>
>>      
>> >?[424.040s][info?][gc?????????????]?GC(56)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4576M(24576M)?5774.497ms 
>>
>>      
>> >?[431.115s][info?][gc?????????????]?GC(57)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4584M(24576M)?6433.645ms 
>>
>>      
>> >?[437.740s][info?][gc?????????????]?GC(58)?Pause?Degenerated?GC?(Outside?of?Cycle)?23343M->4592M(24576M)?5980.660ms 
>>
>> ???? >
>> ???? >
>> ???? >
>>      
>> >?The?heap?seems?to?grow?slowly,?and?the?pause?time?tends?to?grow?too.
>>      
>> >?The?jdk16?or?17?version?appears?okay?(with?constant?after-gc?heap?size). 
>>
>> ???? >
>> ???? >?I?wonder?if?there?is?a?known?reason?
>> ???? >
>> ???? >?Thanks!
>> ???? >
>> ???? >?Yude?Lin
>> ???? >


From stefank at openjdk.java.net  Mon May 24 12:58:45 2021
From: stefank at openjdk.java.net (Stefan Karlsson)
Date: Mon, 24 May 2021 12:58:45 GMT
Subject: RFR: 8267464: Circular-dependency resilient inline headers [v4]
In-Reply-To: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
Message-ID: <TU_1CODSKvFOzDmPEOShcp92o2pleQDQQXZFMfQ0j-Q=.0a220517-a6e1-4b10-871e-2c1ffeb58baf@github.com>

> I'd like to propose a small adjustment to how we write .inline.hpp files, in the hope to reduce include problems because of circular dependencies between inline headers.
> 
> This is a small, contrived example to show the problem:
> 
> // a.hpp
> #pragma once
> 
> void a1();
> void a2();
> 
> 
> // a.inline.hpp
> #pragma once
> 
> #include "a.hpp"
> #include "b.inline.hpp"
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> 
> // b.hpp
> #pragma once
> 
> void b1();
> void b2();
> 
> 
> // b.inline.hpp
> #pragma once
> 
> #include "a.inline.hpp"
> #include "b.hpp"
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> The following code compiles fine:
> 
> // a.cpp
> #include "a.inline.hpp"
> 
> int main() {
>   a1();
> 
>   return 0;
> }
> 
> But the following:
> 
> // b.cpp
> #include "b.inline.hpp"
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> 
> fails with the this error message:
> 
> In file included from b.inline.hpp:3,
>                  from b.cpp:1:
> a.inline.hpp: In function ?void a1()?:
> a.inline.hpp:8:3: error: ?b1? was not declared in this scope; did you mean ?a1??
> 
> We can see the problem with g++ -E:
> 
> # 1 "a.inline.hpp" 1
> # 1 "a.hpp" 1
> 
> void a1();
> void a2();
> 
> # 4 "a.inline.hpp" 2
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> # 4 "b.inline.hpp" 2
> # 1 "b.hpp" 1
> 
> void b1();
> void b2();
> 
> # 5 "b.inline.hpp" 2
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> # 2 "b.cpp" 2
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> 
> b1() is called before it has been declared. b.inline.hpp inlined a.inline.hpp, which uses functions declared in b.hpp. However, at that point we've not included enough of b.inline.hpp to have declared b1().
> 
> This is a small and easy example. In the JVM the circular dependencies usually involves more than two files, and often from different sub-systems of the JVM. We have so-far solved this by restructuring the code, making functions out-of-line, creating proxy objects, etc. However, the more we use templates, the more this is going to become a reoccurring nuisance. And in experiments with lambdas, which requires templates, this very quickly showed up as a problem.
> 
> I propose that we solve most (all?) of these issues by adding a rule that states that .inline.hpp files should start by including the corresponding .hpp, and that the .hpp should contain the declarations of all externally used parts of the .inline.hpp file. This should guarantee that the declarations are always present before any subsequent include can create a circular include dependency back to the original file.
> 
> In the example above, b.inline.hpp would become:
> 
> // b.inline.hpp
> #pragma once
> 
> #include "b.hpp"
> #include "a.inline.hpp"
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> and now both a.cpp and b.cpp compiles. The generated output now looks like this:
> 
> # 1 "b.inline.hpp" 1
> # 1 "b.hpp" 1
> 
> void b1();
> void b2();
> 
> # 4 "b.inline.hpp" 2
> # 1 "a.inline.hpp" 1
> 
> # 1 "a.hpp" 1
> 
> void a1();
> void a2();
> 
> # 4 "a.inline.hpp" 2
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> # 5 "b.inline.hpp" 2
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> # 2 "b.cpp" 2
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> The declarations come first, and the compiler is happy. 
> 
> An alternative to this, that has been proposed by other HotSpot GC devs have been to always include all .hpp files first, and then have a section of .inline.hpp includes. I've experimented with that as well, but I think it requires more maintenance and vigilance of  *users* .inline.hpp files (add .hpp include to the first section, add .inline.hpp section to second). The proposed structures only requires extra care from *writers* of .inline.hpp files. All others can include .hpp and .inline.hpp as we've been doing for a long time now.
> 
> Some notes about this patch:
> 1) Some externally-used declarations have been placed in .inline.hpp files, and not in the .hpp. Those should be moved. I have not done that.
> 2) Some .inline.hpp files don't have a corresponding .hpp file. I could either try to fix that in this patch, or we could take care of that as separate patches later.
> 3) The style I chose was to add the .hpp include and a extra blank line, separating it from the rest of the includes. I think I like that style, because it makes it easy for those writing .inline.hpp to recognize this pattern. And it removes the confusion why this include isn't sorted into the other includes.
> 4) We have a few *special* platform dependent header files. Both those that simply are included into platform independent files, and those that inject code *into* the platform independent classes. Extra care, as always, need to be taken around those files.
> 5) Mostly tested locally, but I'll test on more platforms if the idea is accepted.
> 
> What do you think?

Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision:

  Update HotSpot style guide documents

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4127/files
  - new: https://git.openjdk.java.net/jdk/pull/4127/files/4bcd4348..ba9ad1b1

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4127&range=03
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4127&range=02-03

  Stats: 6 lines in 2 files changed: 6 ins; 0 del; 0 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4127.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4127/head:pull/4127

PR: https://git.openjdk.java.net/jdk/pull/4127

From yude.lyd at alibaba-inc.com  Tue May 25 05:19:49 2021
From: yude.lyd at alibaba-inc.com (Yude Lin)
Date: Tue, 25 May 2021 13:19:49 +0800
Subject: =?UTF-8?B?UmU6IEhlYXAgc2l6ZSBzbG93bHkgZ3Jvd3MgaW4gamRrMTF1IFNoZW5hbmRvYWggcGFzc2l2?=
 =?UTF-8?B?ZSBtb2Rl?=
In-Reply-To: <131c378e-08dc-bd9f-d220-f2afe16d5c11@redhat.com>
References: <a41a2051-67b9-47b3-96e8-d2ce9d9574ca.>
 <94a73cf4-0497-4bd2-a791-7dda292a6c06.>
 <4e6701bd-c5b2-495c-b01b-09d5565da50d.yude.lyd@alibaba-inc.com>
 <841bbfaf-72a4-5510-febf-c3eb111a20b5@redhat.com>
 <2c7be7af-dce2-42e3-81b2-bfe689a79c84.yude.lyd@alibaba-inc.com>
 <b66a8402-333a-f1cc-0173-45c841f5c008@redhat.com>,
 <131c378e-08dc-bd9f-d220-f2afe16d5c11@redhat.com>
Message-ID: <375483d3-32ce-43df-884a-7719e8ea1622.yude.lyd@alibaba-inc.com>

Hi Zhengyu,

Yes, I confirm this solves the issue. The fix looks good to me. Thanks!

Yude



------------------------------------------------------------------
From:Zhengyu Gu <zgu at redhat.com>
Send Time:2021?5?24?(???) 20:47
To:???(??) <yude.lyd at alibaba-inc.com>; shenandoah-dev <shenandoah-dev at openjdk.java.net>
Subject:Re: Heap size slowly grows in jdk11u Shenandoah passive mode

Hi Yude,

Here is the fix: http://cr.openjdk.java.net/~zgu/JDK-8267561/webrev.00/

Could you verify it?

Thanks,

-Zhengyu

On 5/21/21 5:33 PM, Zhengyu Gu wrote:
> Hi Yude,
> 
> Thanks for reporting and confirming. I found the root cause [1], it only 
> affect 11u, as 16u and 17 we switched to concurrent reference processing.
> 
> 
> -Zhengyu
> 
> 
> [1] https://bugs.openjdk.java.net/browse/JDK-8267561
> 
> 
> On 5/21/21 12:42 AM, Yude Lin wrote:
>> Hi Zhengyu,
>>
>> I tried -XX:ShenandoahUnloadClassesFrequency=1 but I still get the 
>> same symptom.
>> I checked the log and it has "Unload Classes" every cycle, so 
>> unloading is on. Any ideas?
>> Thanks!
>>
>> Yude
>>
>>     ------------------------------------------------------------------
>>     From:Zhengyu Gu <zgu at redhat.com>
>>     Send Time:2021?5?20?(???) 23:30
>>     To:???(??) <yude.lyd at alibaba-inc.com>; shenandoah-dev
>>     <shenandoah-dev at openjdk.java.net>
>>     Subject:Re: Heap size slowly grows in jdk11u Shenandoah passive mode
>>
>>     Hi,
>>
>>     This may be due to class unloading. Shenandoah/jdk11u, by default,
>>     unloads classes every 100 cycles, while jdk16/17 unloads classes
>>     (concurrently) every cycle.
>>
>>     You may want to try: -XX:ShenandoahUnloadClassesFrequency=1
>>
>>     -Zhengyu
>>
>>     On 5/20/21 1:17 AM, ???(??) wrote:
>>      >
>>      >
>>      > Hi,
>>      >
>>      
>> > I was running jdk11u Shenandoah on specjvm2008, and found the passive mode has a leak-like behaviour: 
>>
>>      >
>>      
>> > $java -Xmx24g -Xms24g -XX:ParallelGCThreads=1 -XX:+UseShenandoahGC -XX:+UnlockDiagnosticVMOptions -XX:ShenandoahGCMode=passive -Xlog:gc*=debug:file=512sp.log -jar SPECjvm2008.jar -ict -coe -i 1 derby 
>>
>>      >
>>      > $grep 'Pause .*ms' 512sp.log
>>      
>> > [63.407s][info ][gc             ] GC(0) Pause Degenerated GC (Outside of Cycle) 23344M->15824M(24576M) 6994.533ms 
>>
>>      
>> > [71.029s][info ][gc             ] GC(1) Pause Degenerated GC (Outside of Cycle) 23343M->15520M(24576M) 7325.687ms 
>>
>>      
>> > [80.319s][info ][gc             ] GC(2) Pause Degenerated GC (Outside of Cycle) 23343M->11720M(24576M) 8721.156ms 
>>
>>      
>> > [88.396s][info ][gc             ] GC(3) Pause Degenerated GC (Outside of Cycle) 23343M->4176M(24576M) 7549.126ms 
>>
>>      
>> > [94.300s][info ][gc             ] GC(4) Pause Degenerated GC (Outside of Cycle) 23343M->4192M(24576M) 5216.467ms 
>>
>>      
>> > [99.941s][info ][gc             ] GC(5) Pause Degenerated GC (Outside of Cycle) 23343M->4200M(24576M) 4984.741ms 
>>
>>      
>> > [105.506s][info ][gc             ] GC(6) Pause Degenerated GC (Outside of Cycle) 23343M->4208M(24576M) 4901.316ms 
>>
>>      
>> > [111.339s][info ][gc             ] GC(7) Pause Degenerated GC (Outside of Cycle) 23343M->4216M(24576M) 5175.875ms 
>>
>>      
>> > [117.154s][info ][gc             ] GC(8) Pause Degenerated GC (Outside of Cycle) 23343M->4224M(24576M) 5134.298ms 
>>
>>      
>> > [123.078s][info ][gc             ] GC(9) Pause Degenerated GC (Outside of Cycle) 23343M->4232M(24576M) 5255.877ms 
>>
>>      
>> > [129.249s][info ][gc             ] GC(10) Pause Degenerated GC (Outside of Cycle) 23343M->4232M(24576M) 5505.492ms 
>>
>>      
>> > [135.258s][info ][gc             ] GC(11) Pause Degenerated GC (Outside of Cycle) 23343M->4240M(24576M) 5355.191ms 
>>
>>      
>> > [141.378s][info ][gc             ] GC(12) Pause Degenerated GC (Outside of Cycle) 23343M->4248M(24576M) 5487.325ms 
>>
>>      
>> > [147.217s][info ][gc             ] GC(13) Pause Degenerated GC (Outside of Cycle) 23343M->4256M(24576M) 5214.325ms 
>>
>>      
>> > [152.963s][info ][gc             ] GC(14) Pause Degenerated GC (Outside of Cycle) 23343M->4264M(24576M) 5083.129ms 
>>
>>      
>> > [158.798s][info ][gc             ] GC(15) Pause Degenerated GC (Outside of Cycle) 23343M->4272M(24576M) 5162.959ms 
>>
>>      
>> > [164.950s][info ][gc             ] GC(16) Pause Degenerated GC (Outside of Cycle) 23343M->4280M(24576M) 5475.542ms 
>>
>>      
>> > [170.856s][info ][gc             ] GC(17) Pause Degenerated GC (Outside of Cycle) 23343M->4288M(24576M) 5222.228ms 
>>
>>      
>> > [177.007s][info ][gc             ] GC(18) Pause Degenerated GC (Outside of Cycle) 23343M->4304M(24576M) 5300.388ms 
>>
>>      
>> > [182.809s][info ][gc             ] GC(19) Pause Degenerated GC (Outside of Cycle) 23343M->4312M(24576M) 4833.105ms 
>>
>>      
>> > [189.589s][info ][gc             ] GC(20) Pause Degenerated GC (Outside of Cycle) 23344M->4312M(24576M) 5441.911ms 
>>
>>      
>> > [195.528s][info ][gc             ] GC(21) Pause Degenerated GC (Outside of Cycle) 23343M->4320M(24576M) 5262.478ms 
>>
>>      
>> > [201.654s][info ][gc             ] GC(22) Pause Degenerated GC (Outside of Cycle) 23343M->4328M(24576M) 5491.649ms 
>>
>>      
>> > [208.601s][info ][gc             ] GC(23) Pause Degenerated GC (Outside of Cycle) 23343M->4336M(24576M) 6293.675ms 
>>
>>      
>> > [214.483s][info ][gc             ] GC(24) Pause Degenerated GC (Outside of Cycle) 23343M->4344M(24576M) 5259.813ms 
>>
>>      
>> > [220.657s][info ][gc             ] GC(25) Pause Degenerated GC (Outside of Cycle) 23343M->4352M(24576M) 5533.361ms 
>>
>>      
>> > [227.005s][info ][gc             ] GC(26) Pause Degenerated GC (Outside of Cycle) 23343M->4352M(24576M) 5712.247ms 
>>
>>      
>> > [233.210s][info ][gc             ] GC(27) Pause Degenerated GC (Outside of Cycle) 23343M->4368M(24576M) 5580.975ms 
>>
>>      
>> > [239.202s][info ][gc             ] GC(28) Pause Degenerated GC (Outside of Cycle) 23343M->4368M(24576M) 5379.766ms 
>>
>>      
>> > [245.565s][info ][gc             ] GC(29) Pause Degenerated GC (Outside of Cycle) 23343M->4376M(24576M) 5714.096ms 
>>
>>      
>> > [251.450s][info ][gc             ] GC(30) Pause Degenerated GC (Outside of Cycle) 23343M->4384M(24576M) 5250.005ms 
>>
>>      
>> > [257.447s][info ][gc             ] GC(31) Pause Degenerated GC (Outside of Cycle) 23343M->4392M(24576M) 5354.806ms 
>>
>>      
>> > [263.840s][info ][gc             ] GC(32) Pause Degenerated GC (Outside of Cycle) 23343M->4400M(24576M) 5716.993ms 
>>
>>      
>> > [270.792s][info ][gc             ] GC(33) Pause Degenerated GC (Outside of Cycle) 23343M->4408M(24576M) 6264.228ms 
>>
>>      
>> > [277.201s][info ][gc             ] GC(34) Pause Degenerated GC (Outside of Cycle) 23343M->4408M(24576M) 5743.778ms 
>>
>>      
>> > [283.828s][info ][gc             ] GC(35) Pause Degenerated GC (Outside of Cycle) 23343M->4416M(24576M) 5973.385ms 
>>
>>      
>> > [291.227s][info ][gc             ] GC(36) Pause Degenerated GC (Outside of Cycle) 23343M->4424M(24576M) 6587.729ms 
>>
>>      
>> > [298.153s][info ][gc             ] GC(37) Pause Degenerated GC (Outside of Cycle) 23343M->4432M(24576M) 6122.285ms 
>>
>>      
>> > [304.722s][info ][gc             ] GC(38) Pause Degenerated GC (Outside of Cycle) 23343M->4440M(24576M) 5938.363ms 
>>
>>      
>> > [311.097s][info ][gc             ] GC(39) Pause Degenerated GC (Outside of Cycle) 23343M->4448M(24576M) 5737.377ms 
>>
>>      
>> > [317.710s][info ][gc             ] GC(40) Pause Degenerated GC (Outside of Cycle) 23343M->4448M(24576M) 5994.975ms 
>>
>>      
>> > [323.969s][info ][gc             ] GC(41) Pause Degenerated GC (Outside of Cycle) 23343M->4456M(24576M) 5624.046ms 
>>
>>      
>> > [330.910s][info ][gc             ] GC(42) Pause Degenerated GC (Outside of Cycle) 23343M->4464M(24576M) 6298.006ms 
>>
>>      
>> > [337.761s][info ][gc             ] GC(43) Pause Degenerated GC (Outside of Cycle) 23343M->4472M(24576M) 6248.352ms 
>>
>>      
>> > [344.369s][info ][gc             ] GC(44) Pause Degenerated GC (Outside of Cycle) 23343M->4480M(24576M) 5947.428ms 
>>
>>      
>> > [351.096s][info ][gc             ] GC(45) Pause Degenerated GC (Outside of Cycle) 23343M->4488M(24576M) 6094.857ms 
>>
>>      
>> > [357.776s][info ][gc             ] GC(46) Pause Degenerated GC (Outside of Cycle) 23343M->4496M(24576M) 6048.758ms 
>>
>>      
>> > [364.830s][info ][gc             ] GC(47) Pause Degenerated GC (Outside of Cycle) 23343M->4504M(24576M) 6418.273ms 
>>
>>      
>> > [370.804s][info ][gc             ] GC(48) Pause Degenerated GC (Outside of Cycle) 23343M->4512M(24576M) 5348.577ms 
>>
>>      
>> > [376.787s][info ][gc             ] GC(49) Pause Degenerated GC (Outside of Cycle) 23344M->4520M(24576M) 5339.928ms 
>>
>>      
>> > [383.407s][info ][gc             ] GC(50) Pause Degenerated GC (Outside of Cycle) 23343M->4528M(24576M) 5942.465ms 
>>
>>      
>> > [389.858s][info ][gc             ] GC(51) Pause Degenerated GC (Outside of Cycle) 23343M->4536M(24576M) 5801.355ms 
>>
>>      
>> > [396.714s][info ][gc             ] GC(52) Pause Degenerated GC (Outside of Cycle) 23343M->4544M(24576M) 6190.560ms 
>>
>>      
>> > [403.752s][info ][gc             ] GC(53) Pause Degenerated GC (Outside of Cycle) 23343M->4560M(24576M) 6226.817ms 
>>
>>      
>> > [410.816s][info ][gc             ] GC(54) Pause Degenerated GC (Outside of Cycle) 23343M->4560M(24576M) 6400.091ms 
>>
>>      
>> > [417.651s][info ][gc             ] GC(55) Pause Degenerated GC (Outside of Cycle) 23343M->4568M(24576M) 6206.848ms 
>>
>>      
>> > [424.040s][info ][gc             ] GC(56) Pause Degenerated GC (Outside of Cycle) 23343M->4576M(24576M) 5774.497ms 
>>
>>      
>> > [431.115s][info ][gc             ] GC(57) Pause Degenerated GC (Outside of Cycle) 23343M->4584M(24576M) 6433.645ms 
>>
>>      
>> > [437.740s][info ][gc             ] GC(58) Pause Degenerated GC (Outside of Cycle) 23343M->4592M(24576M) 5980.660ms 
>>
>>      >
>>      >
>>      >
>>      
>> > The heap seems to grow slowly, and the pause time tends to grow too.
>>      
>> > The jdk16 or 17 version appears okay (with constant after-gc heap size). 
>>
>>      >
>>      > I wonder if there is a known reason?
>>      >
>>      > Thanks!
>>      >
>>      > Yude Lin
>>      >


From rkennke at redhat.com  Tue May 25 10:46:11 2021
From: rkennke at redhat.com (Roman Kennke)
Date: Tue, 25 May 2021 12:46:11 +0200
Subject: [11u] RFR 8267561: Shenandoah: Reference processing not properly
 setup for outside of cycle degenerated GC
In-Reply-To: <7ef26b6b-fd30-9e48-6543-3c3fb1c472e9@redhat.com>
References: <7ef26b6b-fd30-9e48-6543-3c3fb1c472e9@redhat.com>
Message-ID: <d41bb71c-e641-6cc3-58e7-13a0a5388bf2@redhat.com>

Hi Zhengyu,

The fix looks good.

Is sh/jdk8 affected by this? If so, please mark the bug accordingly (and 
handle the backport if you can).

Thanks,
Roman

> Alibaba reported that Shenandoah outside-of-cycle degenerated GC slowly 
> leak, please see CR for details.
> 
> The cause is that reference processing not setup properly during 
> outside-of-cycle degenerated GC.
> 
> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8267561/webrev.00/
> 
> Test:
>  ?hotspot_gc_shenandoah
>  ?specJVM Debery with configuration reported in CR.
> 
> Thanks,
> 
> -Zhengyu
> 


From rkennke at openjdk.java.net  Tue May 25 11:48:33 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 25 May 2021 11:48:33 GMT
Subject: RFR: Fix verifier handling of weak references when scanning the
 remembered set
In-Reply-To: <9Dyr73qOVVO1hOrbc_jxYIrGfPn68pHMRg6Nzb4BFIU=.9ae39f8d-251a-49bc-835a-5788c8d5f623@github.com>
References: <9Dyr73qOVVO1hOrbc_jxYIrGfPn68pHMRg6Nzb4BFIU=.9ae39f8d-251a-49bc-835a-5788c8d5f623@github.com>
Message-ID: <H1aCmFU3gshN4lHleYtt_tJYy0TgicXpBF5x2XAUF1Q=.f221c334-b341-451d-9c3c-0533a466e625@github.com>

On Wed, 19 May 2021 22:02:11 GMT, William Kemper <wkemper at openjdk.org> wrote:

> In some use cases, `ShenandoahVerifier` is required to _ignore_ weak references (because they will be processed concurrently). Because of the way the remembered set scan handles oops, it must use an `OopIterateClosure` (not a bare `OopClosure`). To make use of the `OopClosure` passed to the remembered set scanner, it was wrapped in an "adapter" that sheared off the `ShenandoahIgnoreReferenceDiscoverer` configured on the passed in closure. This change removes the need to use such an adapter during the verifier's root scan.

Marked as reviewed by rkennke (Lead).

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/41

From rkennke at openjdk.java.net  Tue May 25 11:48:35 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 25 May 2021 11:48:35 GMT
Subject: RFR: Collect old humongous regions immediately at end of
 concurrent old mark
In-Reply-To: <9M98PgtZ16qRHGhxPXP4RTdq4WIZ44imMcXLUwbtVsc=.16885e78-d54d-4f11-8630-face7e472539@github.com>
References: <9M98PgtZ16qRHGhxPXP4RTdq4WIZ44imMcXLUwbtVsc=.16885e78-d54d-4f11-8630-face7e472539@github.com>
Message-ID: <XS8b-H9Ob1qX-hwF-ajhqfpTWYyKXZoDEfoh6IUdAuo=.df1d346f-f338-4a8a-98c5-2e8b9d0a88f9@github.com>

On Fri, 21 May 2021 21:10:13 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Recycle dead humongous objects at the end of concurrent old mark, rather than add them to the cset. This behavior is similar to young gen cset selection. It also increases the amount of free memory in old gen to support evacuations. Shenandoah has a specific code path for recycling humongous objects and (debug builds) will assert when an old region is added to the cset.

Looks good to me, thanks!

-------------

Marked as reviewed by rkennke (Lead).

PR: https://git.openjdk.java.net/shenandoah/pull/44

From rkennke at openjdk.java.net  Tue May 25 11:48:34 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 25 May 2021 11:48:34 GMT
Subject: RFR: Mixed evacuation fixes
In-Reply-To: <NLpPuw0RDh4g7o6TWjhCl-141wzVTLYcjAixqeJRUXw=.e4831d22-25f6-4a5c-8006-359641444f94@github.com>
References: <NLpPuw0RDh4g7o6TWjhCl-141wzVTLYcjAixqeJRUXw=.e4831d22-25f6-4a5c-8006-359641444f94@github.com>
Message-ID: <MwbFJsU_TJbmE2drM4DzuMFmtOEm3g8tvIBXUqANqzU=.99c6a98a-c4b4-4dd0-84f1-a81ec524a551@github.com>

On Fri, 21 May 2021 15:06:42 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This PR has a few bug fixes, the most important of which corrects an error that prevented references between old regions from being updated. To handle this case we expose the notion of _mixed_ collections to the update reference loop. This lets us continue using the optimization of only updating references in the remembered set during a _pure_ young collection. The other fixes are comparatively minor and described in their commit message.

Looks good to me! Thanks!

-------------

Marked as reviewed by rkennke (Lead).

PR: https://git.openjdk.java.net/shenandoah/pull/43

From rkennke at openjdk.java.net  Tue May 25 12:07:07 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 25 May 2021 12:07:07 GMT
Subject: RFR: Allow committers to review Shenandoah changes
Message-ID: <heMmOURLGnj7ewuUBwjwHLrVEvBIN0-4MVfgBaXZoaY=.4e6c8a38-7c41-49a1-8038-8cef9158a010@github.com>

We shouldn't need formal distinction between reviewers and commiters in Shenandoah. I propose to allow all committers to also review changes.

-------------

Commit messages:
 - Allow committers to review Shenandoah changes

Changes: https://git.openjdk.java.net/shenandoah/pull/45/files
 Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=45&range=00
  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/45.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/45/head:pull/45

PR: https://git.openjdk.java.net/shenandoah/pull/45

From shade at openjdk.java.net  Tue May 25 12:07:07 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 25 May 2021 12:07:07 GMT
Subject: RFR: Allow committers to review Shenandoah changes
In-Reply-To: <heMmOURLGnj7ewuUBwjwHLrVEvBIN0-4MVfgBaXZoaY=.4e6c8a38-7c41-49a1-8038-8cef9158a010@github.com>
References: <heMmOURLGnj7ewuUBwjwHLrVEvBIN0-4MVfgBaXZoaY=.4e6c8a38-7c41-49a1-8038-8cef9158a010@github.com>
Message-ID: <GJEpotMD7DtKfzzq4uuimM9A9v9c6z9PL8bJkl2IIJE=.e025b437-2b48-4ff4-917a-df583a43c67a@github.com>

On Tue, 25 May 2021 10:57:09 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> We shouldn't need formal distinction between reviewers and commiters in Shenandoah. I propose to allow all committers to also review changes.

That's fine.

-------------

Marked as reviewed by shade (Committer).

PR: https://git.openjdk.java.net/shenandoah/pull/45

From zgu at openjdk.java.net  Tue May 25 12:23:35 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 25 May 2021 12:23:35 GMT
Subject: RFR: Allow committers to review Shenandoah changes
In-Reply-To: <heMmOURLGnj7ewuUBwjwHLrVEvBIN0-4MVfgBaXZoaY=.4e6c8a38-7c41-49a1-8038-8cef9158a010@github.com>
References: <heMmOURLGnj7ewuUBwjwHLrVEvBIN0-4MVfgBaXZoaY=.4e6c8a38-7c41-49a1-8038-8cef9158a010@github.com>
Message-ID: <-AvZ33VXg3aruwQueNBhMZXIU9ej2d6rKxZ7Xjzv7wA=.0340ef1f-3edb-4438-888b-e23d74f23b63@github.com>

On Tue, 25 May 2021 10:57:09 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> We shouldn't need formal distinction between reviewers and commiters in Shenandoah. I propose to allow all committers to also review changes.

Marked as reviewed by zgu (Committer).

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/45

From duke at openjdk.java.net  Tue May 25 12:55:58 2021
From: duke at openjdk.java.net (duke)
Date: Tue, 25 May 2021 12:55:58 GMT
Subject: git: openjdk/shenandoah: master: Allow committers to review
 Shenandoah changes
Message-ID: <c9c6a6a9-3510-4ff0-bac9-c26ebc8108d2@openjdk.java.net>

Changeset: 627ca100
Author:    Roman Kennke <rkennke at redhat.com>
Date:      2021-05-25 12:56:08 +0000
URL:       https://git.openjdk.java.net/shenandoah/commit/627ca1006a7239a87c327dee7a4ea342236ec725

Allow committers to review Shenandoah changes

! .jcheck/conf


From rkennke at openjdk.java.net  Tue May 25 12:58:18 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 25 May 2021 12:58:18 GMT
Subject: Withdrawn: Allow committers to review Shenandoah changes
In-Reply-To: <heMmOURLGnj7ewuUBwjwHLrVEvBIN0-4MVfgBaXZoaY=.4e6c8a38-7c41-49a1-8038-8cef9158a010@github.com>
References: <heMmOURLGnj7ewuUBwjwHLrVEvBIN0-4MVfgBaXZoaY=.4e6c8a38-7c41-49a1-8038-8cef9158a010@github.com>
Message-ID: <Ah3LRgwxrbTEG8ke8iw_jSoS4SHq5dO6D9JXi-zEeQY=.ea0298a6-862d-44ef-ace9-f481568e0f48@github.com>

On Tue, 25 May 2021 10:57:09 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> We shouldn't need formal distinction between reviewers and commiters in Shenandoah. I propose to allow all committers to also review changes.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/45

From coleenp at openjdk.java.net  Tue May 25 13:27:16 2021
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Tue, 25 May 2021 13:27:16 GMT
Subject: RFR: 8267653: Remove Mutex::_safepoint_check_sometimes
Message-ID: <OpIHO8GcqBWsONZ6MNwlQzaQ6Z4dfekskdRrmUhT8M8=.853bdb38-9966-42fc-8d38-4147836d93e6@github.com>

Since SR_lock is removed, this state of declaring locks as sometimes safepoint checking and not is no longer used.  JavaThreads either always or never check for safepoint, depending on the lock.  The Heap_lock was always a bit special because it's taken by a JavaThread after it exits, but the code in mutex::lock_contended already deals with this (ie doesn't safepoint check while exiting).
Tested with tier 2-3 and tier1 in progress.
Thanks to Zhengu for testing and confirming shenandoah (I built and ran shenandoah tests too).

-------------

Commit messages:
 - 8267653: Remove Mutex::_safepoint_check_sometimes

Changes: https://git.openjdk.java.net/jdk/pull/4184/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4184&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8267653
  Stats: 22 lines in 5 files changed: 0 ins; 19 del; 3 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4184.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4184/head:pull/4184

PR: https://git.openjdk.java.net/jdk/pull/4184

From zgu at redhat.com  Tue May 25 17:48:38 2021
From: zgu at redhat.com (Zhengyu Gu)
Date: Tue, 25 May 2021 13:48:38 -0400
Subject: [sh/8u] RFR 8267561: Shenandoah: Reference processing not properly
 setup for outside of cycle degenerated GC
Message-ID: <d940e070-e999-d239-ae0c-56a99b176f6c@redhat.com>

Shenandoah 8u suffers the same problem as 11u, that could slowly leak 
heap memory due to not process weak references.

11u patch applies cleanly to Shenandoah 8u, however, it does not 
compile, as APIs are slightly different between 11u and 8u, fixed as follow:

diff -r 0a4911d2aac7 
src/share/vm/gc_implementation/shenandoah/shenandoahHeap.cpp
--- a/src/share/vm/gc_implementation/shenandoah/shenandoahHeap.cpp 
Tue May 25 13:04:23 2021 -0400
+++ b/src/share/vm/gc_implementation/shenandoah/shenandoahHeap.cpp 
Tue May 25 13:27:02 2021 -0400
@@ -1647,8 +1647,8 @@
          rp->set_active_mt_degree(_heap->workers()->active_workers());

          // enable ("weak") refs discovery
-        rp->enable_discovery(true /*verify_no_refs*/);
- 
rp->setup_policy(_heap->soft_ref_policy()->should_clear_all_soft_refs());
+        rp->enable_discovery(true /*verify_no_refs*/, true);
+ 
rp->setup_policy(_heap->collector_policy()->should_clear_all_soft_refs());
        }


Shenandoah 8u webrev: 
http://cr.openjdk.java.net/~zgu/JDK-8267561-8u/webrev.00/

Test:
   hotspot_gc_shenandoah

Thanks,

-Zhengyu


From rkennke at redhat.com  Tue May 25 20:20:55 2021
From: rkennke at redhat.com (Roman Kennke)
Date: Tue, 25 May 2021 22:20:55 +0200
Subject: [sh/8u] RFR 8267561: Shenandoah: Reference processing not
 properly setup for outside of cycle degenerated GC
In-Reply-To: <d940e070-e999-d239-ae0c-56a99b176f6c@redhat.com>
References: <d940e070-e999-d239-ae0c-56a99b176f6c@redhat.com>
Message-ID: <fa219e49-1091-ca4d-5f43-c361591dd95b@redhat.com>

Looks good!

Thanks, Roman


> Shenandoah 8u suffers the same problem as 11u, that could slowly leak 
> heap memory due to not process weak references.
> 
> 11u patch applies cleanly to Shenandoah 8u, however, it does not 
> compile, as APIs are slightly different between 11u and 8u, fixed as 
> follow:
> 
> diff -r 0a4911d2aac7 
> src/share/vm/gc_implementation/shenandoah/shenandoahHeap.cpp
> --- a/src/share/vm/gc_implementation/shenandoah/shenandoahHeap.cpp Tue 
> May 25 13:04:23 2021 -0400
> +++ b/src/share/vm/gc_implementation/shenandoah/shenandoahHeap.cpp Tue 
> May 25 13:27:02 2021 -0400
> @@ -1647,8 +1647,8 @@
>  ??????? rp->set_active_mt_degree(_heap->workers()->active_workers());
> 
>  ??????? // enable ("weak") refs discovery
> -??????? rp->enable_discovery(true /*verify_no_refs*/);
> - rp->setup_policy(_heap->soft_ref_policy()->should_clear_all_soft_refs());
> +??????? rp->enable_discovery(true /*verify_no_refs*/, true);
> + 
> rp->setup_policy(_heap->collector_policy()->should_clear_all_soft_refs());
>  ????? }
> 
> 
> Shenandoah 8u webrev: 
> http://cr.openjdk.java.net/~zgu/JDK-8267561-8u/webrev.00/
> 
> Test:
>  ?hotspot_gc_shenandoah
> 
> Thanks,
> 
> -Zhengyu
> 


From zgu at redhat.com  Tue May 25 20:36:29 2021
From: zgu at redhat.com (zgu at redhat.com)
Date: Tue, 25 May 2021 20:36:29 +0000
Subject: hg: shenandoah/jdk8/hotspot: [backport] 8267561: Shenandoah:
 Reference processing not properly setup for outside of cycle
 degenerated GC
Message-ID: <202105252036.14PKaTPr005763@aojmv0008.oracle.com>

Changeset: 4b77b88eb115
Author:    zgu
Date:      2021-05-25 13:04 -0400
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/4b77b88eb115

[backport] 8267561: Shenandoah: Reference processing not properly setup for outside of cycle degenerated GC
Reviewed-by: rkennke

! src/share/vm/gc_implementation/shenandoah/shenandoahHeap.cpp


From wkemper at openjdk.java.net  Tue May 25 21:12:09 2021
From: wkemper at openjdk.java.net (William Kemper)
Date: Tue, 25 May 2021 21:12:09 GMT
Subject: RFR: Encode phase for each generation separately
Message-ID: <eoVIZ0mxm99dZFLuMJTG-lHGTaBQjpmpXe5GH6PqbYA=.89271678-76fe-4c8f-a9bb-290b869bfe68@github.com>

This change encodes the state/activity of each generation separately for the overall collector status. There are complementary changes in [my fork of the visualizer](https://github.com/earthling-amzn/shenandoah-visualizer/tree/genshen-visualizer). There is no explicit versioning of the protocol between the encoding in the collector and the decoding in the visualizer. The change in this PR is compatible with the `genshen-visualizer` branch in my fork, but is _not_ compatible with the [upstream visualizer](https://github.com/openjdk/shenandoah-visualizer). Should I open a PR in the visualizer? Should we add a versioning scheme to the prototcol and try to maintain compatibility with older versions?

-------------

Commit messages:
 - Encode phase for each generation separately

Changes: https://git.openjdk.java.net/shenandoah/pull/46/files
 Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=46&range=00
  Stats: 80 lines in 2 files changed: 65 ins; 11 del; 4 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/46.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/46/head:pull/46

PR: https://git.openjdk.java.net/shenandoah/pull/46

From wkemper at openjdk.java.net  Tue May 25 21:58:25 2021
From: wkemper at openjdk.java.net (William Kemper)
Date: Tue, 25 May 2021 21:58:25 GMT
Subject: Integrated: Mixed evacuation fixes
In-Reply-To: <NLpPuw0RDh4g7o6TWjhCl-141wzVTLYcjAixqeJRUXw=.e4831d22-25f6-4a5c-8006-359641444f94@github.com>
References: <NLpPuw0RDh4g7o6TWjhCl-141wzVTLYcjAixqeJRUXw=.e4831d22-25f6-4a5c-8006-359641444f94@github.com>
Message-ID: <4P8pM0UG6J9Qv0F-NdTwsK1EZJ5ThQ_rfvFLoBRfQkk=.2ee59b10-cb8c-494a-8373-7f7023438360@github.com>

On Fri, 21 May 2021 15:06:42 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This PR has a few bug fixes, the most important of which corrects an error that prevented references between old regions from being updated. To handle this case we expose the notion of _mixed_ collections to the update reference loop. This lets us continue using the optimization of only updating references in the remembered set during a _pure_ young collection. The other fixes are comparatively minor and described in their commit message.

This pull request has now been integrated.

Changeset: a21de226
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.java.net/shenandoah/commit/a21de22639315fe7d5eb06ef9707e594046bd38b
Stats:     59 lines in 6 files changed: 28 ins; 18 del; 13 mod

Mixed evacuation fixes

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/43

From wkemper at openjdk.java.net  Tue May 25 21:58:51 2021
From: wkemper at openjdk.java.net (William Kemper)
Date: Tue, 25 May 2021 21:58:51 GMT
Subject: Integrated: Collect old humongous regions immediately at end of
 concurrent old mark
In-Reply-To: <9M98PgtZ16qRHGhxPXP4RTdq4WIZ44imMcXLUwbtVsc=.16885e78-d54d-4f11-8630-face7e472539@github.com>
References: <9M98PgtZ16qRHGhxPXP4RTdq4WIZ44imMcXLUwbtVsc=.16885e78-d54d-4f11-8630-face7e472539@github.com>
Message-ID: <AtOrCr4uvKTsMkxz9rR1NnmhL9Kk9Ihcpf9ITIh9yzA=.4f3eaae8-11f4-442c-864e-5c7c2197f520@github.com>

On Fri, 21 May 2021 21:10:13 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Recycle dead humongous objects at the end of concurrent old mark, rather than add them to the cset. This behavior is similar to young gen cset selection. It also increases the amount of free memory in old gen to support evacuations. Shenandoah has a specific code path for recycling humongous objects and (debug builds) will assert when an old region is added to the cset.

This pull request has now been integrated.

Changeset: 27732e58
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.java.net/shenandoah/commit/27732e58e555860ea08eefc6980b780b4628be89
Stats:     57 lines in 3 files changed: 3 ins; 34 del; 20 mod

Collect old humongous regions immediately at end of concurrent old mark

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/44

From wkemper at openjdk.java.net  Tue May 25 21:58:54 2021
From: wkemper at openjdk.java.net (William Kemper)
Date: Tue, 25 May 2021 21:58:54 GMT
Subject: Integrated: Fix verifier handling of weak references when scanning
 the remembered set
In-Reply-To: <9Dyr73qOVVO1hOrbc_jxYIrGfPn68pHMRg6Nzb4BFIU=.9ae39f8d-251a-49bc-835a-5788c8d5f623@github.com>
References: <9Dyr73qOVVO1hOrbc_jxYIrGfPn68pHMRg6Nzb4BFIU=.9ae39f8d-251a-49bc-835a-5788c8d5f623@github.com>
Message-ID: <RehbkD_5HWnvMc4Jdy3RI0MzuBeyS7mj3vWQdOszC5k=.b7e92fc5-b71c-4ddc-98a6-cc64ce307890@github.com>

On Wed, 19 May 2021 22:02:11 GMT, William Kemper <wkemper at openjdk.org> wrote:

> In some use cases, `ShenandoahVerifier` is required to _ignore_ weak references (because they will be processed concurrently). Because of the way the remembered set scan handles oops, it must use an `OopIterateClosure` (not a bare `OopClosure`). To make use of the `OopClosure` passed to the remembered set scanner, it was wrapped in an "adapter" that sheared off the `ShenandoahIgnoreReferenceDiscoverer` configured on the passed in closure. This change removes the need to use such an adapter during the verifier's root scan.

This pull request has now been integrated.

Changeset: fff87a60
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.java.net/shenandoah/commit/fff87a60f33d60690a9335982738d968f3b9e1ab
Stats:     13 lines in 4 files changed: 5 ins; 0 del; 8 mod

Fix verifier handling of weak references when scanning the remembered set

Reviewed-by: zgu, rkennke

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/41

From rkennke at openjdk.java.net  Tue May 25 22:44:45 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 25 May 2021 22:44:45 GMT
Subject: RFR: Encode phase for each generation separately
In-Reply-To: <eoVIZ0mxm99dZFLuMJTG-lHGTaBQjpmpXe5GH6PqbYA=.89271678-76fe-4c8f-a9bb-290b869bfe68@github.com>
References: <eoVIZ0mxm99dZFLuMJTG-lHGTaBQjpmpXe5GH6PqbYA=.89271678-76fe-4c8f-a9bb-290b869bfe68@github.com>
Message-ID: <rESsZzQWCC-r1m7cAz2XPMHhmPM39YsE5-fU0S1kVo0=.7d1159c0-f96f-49ed-aa7a-2402ace00ca2@github.com>

On Tue, 25 May 2021 21:05:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This change encodes the state/activity of each generation separately for the overall collector status. There are complementary changes in [my fork of the visualizer](https://github.com/earthling-amzn/shenandoah-visualizer/tree/genshen-visualizer). There is no explicit versioning of the protocol between the encoding in the collector and the decoding in the visualizer. The change in this PR is compatible with the `genshen-visualizer` branch in my fork, but is _not_ compatible with the [upstream visualizer](https://github.com/openjdk/shenandoah-visualizer). Should I open a PR in the visualizer? Should we add a versioning scheme to the prototcol and try to maintain compatibility with older versions?

Looks good. Thanks!

If the genshen-visualizer is still compatible with upstream's Shenandoah, then please file a PR against upstream visualizer. I don't think we need versioning. It would be nice if we can maintain backwards compatibility if it's not too much trouble.

-------------

Marked as reviewed by rkennke (Lead).

PR: https://git.openjdk.java.net/shenandoah/pull/46

From dholmes at openjdk.java.net  Tue May 25 23:06:14 2021
From: dholmes at openjdk.java.net (David Holmes)
Date: Tue, 25 May 2021 23:06:14 GMT
Subject: RFR: 8267464: Circular-dependency resilient inline headers [v4]
In-Reply-To: <TU_1CODSKvFOzDmPEOShcp92o2pleQDQQXZFMfQ0j-Q=.0a220517-a6e1-4b10-871e-2c1ffeb58baf@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
 <TU_1CODSKvFOzDmPEOShcp92o2pleQDQQXZFMfQ0j-Q=.0a220517-a6e1-4b10-871e-2c1ffeb58baf@github.com>
Message-ID: <JJ2sP-JmHk_FqKvNN5nNupat_4C05iDLnnhJ6ftzj3s=.51e6bcba-1a1e-44e0-9788-68889cca4506@github.com>

On Mon, 24 May 2021 12:58:45 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> I'd like to propose a small adjustment to how we write .inline.hpp files, in the hope to reduce include problems because of circular dependencies between inline headers.
>> 
>> This is a small, contrived example to show the problem:
>> 
>> // a.hpp
>> #pragma once
>> 
>> void a1();
>> void a2();
>> 
>> 
>> // a.inline.hpp
>> #pragma once
>> 
>> #include "a.hpp"
>> #include "b.inline.hpp"
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> 
>> // b.hpp
>> #pragma once
>> 
>> void b1();
>> void b2();
>> 
>> 
>> // b.inline.hpp
>> #pragma once
>> 
>> #include "a.inline.hpp"
>> #include "b.hpp"
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> The following code compiles fine:
>> 
>> // a.cpp
>> #include "a.inline.hpp"
>> 
>> int main() {
>>   a1();
>> 
>>   return 0;
>> }
>> 
>> But the following:
>> 
>> // b.cpp
>> #include "b.inline.hpp"
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> 
>> fails with the this error message:
>> 
>> In file included from b.inline.hpp:3,
>>                  from b.cpp:1:
>> a.inline.hpp: In function ?void a1()?:
>> a.inline.hpp:8:3: error: ?b1? was not declared in this scope; did you mean ?a1??
>> 
>> We can see the problem with g++ -E:
>> 
>> # 1 "a.inline.hpp" 1
>> # 1 "a.hpp" 1
>> 
>> void a1();
>> void a2();
>> 
>> # 4 "a.inline.hpp" 2
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> # 4 "b.inline.hpp" 2
>> # 1 "b.hpp" 1
>> 
>> void b1();
>> void b2();
>> 
>> # 5 "b.inline.hpp" 2
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> # 2 "b.cpp" 2
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> 
>> b1() is called before it has been declared. b.inline.hpp inlined a.inline.hpp, which uses functions declared in b.hpp. However, at that point we've not included enough of b.inline.hpp to have declared b1().
>> 
>> This is a small and easy example. In the JVM the circular dependencies usually involves more than two files, and often from different sub-systems of the JVM. We have so-far solved this by restructuring the code, making functions out-of-line, creating proxy objects, etc. However, the more we use templates, the more this is going to become a reoccurring nuisance. And in experiments with lambdas, which requires templates, this very quickly showed up as a problem.
>> 
>> I propose that we solve most (all?) of these issues by adding a rule that states that .inline.hpp files should start by including the corresponding .hpp, and that the .hpp should contain the declarations of all externally used parts of the .inline.hpp file. This should guarantee that the declarations are always present before any subsequent include can create a circular include dependency back to the original file.
>> 
>> In the example above, b.inline.hpp would become:
>> 
>> // b.inline.hpp
>> #pragma once
>> 
>> #include "b.hpp"
>> #include "a.inline.hpp"
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> and now both a.cpp and b.cpp compiles. The generated output now looks like this:
>> 
>> # 1 "b.inline.hpp" 1
>> # 1 "b.hpp" 1
>> 
>> void b1();
>> void b2();
>> 
>> # 4 "b.inline.hpp" 2
>> # 1 "a.inline.hpp" 1
>> 
>> # 1 "a.hpp" 1
>> 
>> void a1();
>> void a2();
>> 
>> # 4 "a.inline.hpp" 2
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> # 5 "b.inline.hpp" 2
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> # 2 "b.cpp" 2
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> The declarations come first, and the compiler is happy. 
>> 
>> An alternative to this, that has been proposed by other HotSpot GC devs have been to always include all .hpp files first, and then have a section of .inline.hpp includes. I've experimented with that as well, but I think it requires more maintenance and vigilance of  *users* .inline.hpp files (add .hpp include to the first section, add .inline.hpp section to second). The proposed structures only requires extra care from *writers* of .inline.hpp files. All others can include .hpp and .inline.hpp as we've been doing for a long time now.
>> 
>> Some notes about this patch:
>> 1) Some externally-used declarations have been placed in .inline.hpp files, and not in the .hpp. Those should be moved. I have not done that.
>> 2) Some .inline.hpp files don't have a corresponding .hpp file. I could either try to fix that in this patch, or we could take care of that as separate patches later.
>> 3) The style I chose was to add the .hpp include and a extra blank line, separating it from the rest of the includes. I think I like that style, because it makes it easy for those writing .inline.hpp to recognize this pattern. And it removes the confusion why this include isn't sorted into the other includes.
>> 4) We have a few *special* platform dependent header files. Both those that simply are included into platform independent files, and those that inject code *into* the platform independent classes. Extra care, as always, need to be taken around those files.
>> 5) Mostly tested locally, but I'll test on more platforms if the idea is accepted.
>> 
>> What do you think?
>
> Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Update HotSpot style guide documents

doc/hotspot-style.html line 97:

> 95: <li><p>Do not put non-trivial function implementations in .hpp files. If the implementation depends on other .hpp files, put it in a .cpp or a .inline.hpp file.</p></li>
> 96: <li><p>.inline.hpp files should only be included in .cpp or .inline.hpp files.</p></li>
> 97: <li><p>All .inline.hpp files should include their corresponding .hpp file as the first include line. Externally used declarations should be put in the .hpp file, and not in the the .inline.hpp file. This rule exists to resolve problems with circular dependencies between.inline.hpp files.</p></li>

typo: the the
Need space after between on last line

-------------

PR: https://git.openjdk.java.net/jdk/pull/4127

From dholmes at openjdk.java.net  Wed May 26 00:52:13 2021
From: dholmes at openjdk.java.net (David Holmes)
Date: Wed, 26 May 2021 00:52:13 GMT
Subject: RFR: 8267653: Remove Mutex::_safepoint_check_sometimes
In-Reply-To: <OpIHO8GcqBWsONZ6MNwlQzaQ6Z4dfekskdRrmUhT8M8=.853bdb38-9966-42fc-8d38-4147836d93e6@github.com>
References: <OpIHO8GcqBWsONZ6MNwlQzaQ6Z4dfekskdRrmUhT8M8=.853bdb38-9966-42fc-8d38-4147836d93e6@github.com>
Message-ID: <aF7Vsfq_1jsJpPVOpDffrBm4FSux_PRvJqJfC_iiFyI=.58fad6e7-f200-4f95-90ec-f6dcec8eb5de@github.com>

On Tue, 25 May 2021 13:20:54 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

> Since SR_lock is removed, this state of declaring locks as sometimes safepoint checking and not is no longer used.  JavaThreads either always or never check for safepoint, depending on the lock.  The Heap_lock was always a bit special because it's taken by a JavaThread after it exits, but the code in mutex::lock_contended already deals with this (ie doesn't safepoint check while exiting).
> Tested with tier 2-3 and tier1 in progress.
> Thanks to Zhengu for testing and confirming shenandoah (I built and ran shenandoah tests too).

Hi Coleen,

This seems fine. Working through the thread and VM termination logic is always tricky. Just to be clear the condition for the mutex not checking safepoints is that the thread has terminated, not just that it is exiting, so I've suggested an additional assert before taking the Heap-lock.

Thanks,
David

src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp line 543:

> 541:   } else {
> 542:     // Heap_lock protects external pending list
> 543:     MonitorLocker ml(Heap_lock);

For performance you could leave the _no_safepoint_check flag.

src/hotspot/share/runtime/thread.cpp line 3420:

> 3418:     // we'll never emerge out of the safepoint before the VM exits.
> 3419: 
> 3420:     MutexLocker ml(Heap_lock);

Can you add an assert before this that the thread is_terminated() as that is the key to the mutex acquisition not doing a safepoint check.

-------------

Marked as reviewed by dholmes (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/4184

From coleenp at openjdk.java.net  Wed May 26 01:53:33 2021
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Wed, 26 May 2021 01:53:33 GMT
Subject: RFR: 8267653: Remove Mutex::_safepoint_check_sometimes [v2]
In-Reply-To: <OpIHO8GcqBWsONZ6MNwlQzaQ6Z4dfekskdRrmUhT8M8=.853bdb38-9966-42fc-8d38-4147836d93e6@github.com>
References: <OpIHO8GcqBWsONZ6MNwlQzaQ6Z4dfekskdRrmUhT8M8=.853bdb38-9966-42fc-8d38-4147836d93e6@github.com>
Message-ID: <Vlg6EpsZj7VNzjxfLXoA-mwseSNfctrdUPfqFoX98Vg=.6b8be221-b1dd-476b-8596-45c0d4d5fcc6@github.com>

> Since SR_lock is removed, this state of declaring locks as sometimes safepoint checking and not is no longer used.  JavaThreads either always or never check for safepoint, depending on the lock.  The Heap_lock was always a bit special because it's taken by a JavaThread after it exits, but the code in mutex::lock_contended already deals with this (ie doesn't safepoint check while exiting).
> Tested with tier 2-3 and tier1 in progress.
> Thanks to Zhengu for testing and confirming shenandoah (I built and ran shenandoah tests too).

Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:

  Add assert that the thread is terminated before taking Heap_lock

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4184/files
  - new: https://git.openjdk.java.net/jdk/pull/4184/files/d7f55efe..3dc9b3b9

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4184&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4184&range=00-01

  Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4184.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4184/head:pull/4184

PR: https://git.openjdk.java.net/jdk/pull/4184

From coleenp at openjdk.java.net  Wed May 26 01:53:34 2021
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Wed, 26 May 2021 01:53:34 GMT
Subject: RFR: 8267653: Remove Mutex::_safepoint_check_sometimes [v2]
In-Reply-To: <aF7Vsfq_1jsJpPVOpDffrBm4FSux_PRvJqJfC_iiFyI=.58fad6e7-f200-4f95-90ec-f6dcec8eb5de@github.com>
References: <OpIHO8GcqBWsONZ6MNwlQzaQ6Z4dfekskdRrmUhT8M8=.853bdb38-9966-42fc-8d38-4147836d93e6@github.com>
 <aF7Vsfq_1jsJpPVOpDffrBm4FSux_PRvJqJfC_iiFyI=.58fad6e7-f200-4f95-90ec-f6dcec8eb5de@github.com>
Message-ID: <IxSp6HN-X-jvxdg6k5oNY9U3edaagjutslwW-pbn7l8=.e98b7711-1f28-4a50-850b-91ad888fbb19@github.com>

On Wed, 26 May 2021 00:38:00 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Add assert that the thread is terminated before taking Heap_lock
>
> src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp line 543:
> 
>> 541:   } else {
>> 542:     // Heap_lock protects external pending list
>> 543:     MonitorLocker ml(Heap_lock);
> 
> For performance you could leave the _no_safepoint_check flag.

I cannot leave it because there's an assert to verify that we don't try to take this lock without a safepoint check, since it was declared with safepoint_check_always.

> src/hotspot/share/runtime/thread.cpp line 3420:
> 
>> 3418:     // we'll never emerge out of the safepoint before the VM exits.
>> 3419: 
>> 3420:     MutexLocker ml(Heap_lock);
> 
> Can you add an assert before this that the thread is_terminated() as that is the key to the mutex acquisition not doing a safepoint check.

Ok, I can add:
+    assert(thread->is_terminated(), "must be terminated before acquiring Heap_lock");
and rerun tests.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4184

From david.holmes at oracle.com  Wed May 26 02:00:08 2021
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 26 May 2021 12:00:08 +1000
Subject: RFR: 8267653: Remove Mutex::_safepoint_check_sometimes [v2]
In-Reply-To: <IxSp6HN-X-jvxdg6k5oNY9U3edaagjutslwW-pbn7l8=.e98b7711-1f28-4a50-850b-91ad888fbb19@github.com>
References: <OpIHO8GcqBWsONZ6MNwlQzaQ6Z4dfekskdRrmUhT8M8=.853bdb38-9966-42fc-8d38-4147836d93e6@github.com>
 <aF7Vsfq_1jsJpPVOpDffrBm4FSux_PRvJqJfC_iiFyI=.58fad6e7-f200-4f95-90ec-f6dcec8eb5de@github.com>
 <IxSp6HN-X-jvxdg6k5oNY9U3edaagjutslwW-pbn7l8=.e98b7711-1f28-4a50-850b-91ad888fbb19@github.com>
Message-ID: <781224c1-0118-19b4-6cc8-3c8621af4d43@oracle.com>

On 26/05/2021 11:53 am, Coleen Phillimore wrote:
> On Wed, 26 May 2021 00:38:00 GMT, David Holmes <dholmes at openjdk.org> wrote:
> 
>>> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
>>>
>>>    Add assert that the thread is terminated before taking Heap_lock
>>
>> src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp line 543:
>>
>>> 541:   } else {
>>> 542:     // Heap_lock protects external pending list
>>> 543:     MonitorLocker ml(Heap_lock);
>>
>> For performance you could leave the _no_safepoint_check flag.
> 
> I cannot leave it because there's an assert to verify that we don't try to take this lock without a safepoint check, since it was declared with safepoint_check_always.

But that assert only applies to JavaThreads and this is being taken by a 
GC thread isn't it?

>> src/hotspot/share/runtime/thread.cpp line 3420:
>>
>>> 3418:     // we'll never emerge out of the safepoint before the VM exits.
>>> 3419:
>>> 3420:     MutexLocker ml(Heap_lock);
>>
>> Can you add an assert before this that the thread is_terminated() as that is the key to the mutex acquisition not doing a safepoint check.
> 
> Ok, I can add:
> +    assert(thread->is_terminated(), "must be terminated before acquiring Heap_lock");
> and rerun tests.

Thanks,
David

> -------------
> 
> PR: https://git.openjdk.java.net/jdk/pull/4184
> 

From dholmes at openjdk.java.net  Wed May 26 02:21:11 2021
From: dholmes at openjdk.java.net (David Holmes)
Date: Wed, 26 May 2021 02:21:11 GMT
Subject: RFR: 8267464: Circular-dependency resilient inline headers [v4]
In-Reply-To: <TU_1CODSKvFOzDmPEOShcp92o2pleQDQQXZFMfQ0j-Q=.0a220517-a6e1-4b10-871e-2c1ffeb58baf@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
 <TU_1CODSKvFOzDmPEOShcp92o2pleQDQQXZFMfQ0j-Q=.0a220517-a6e1-4b10-871e-2c1ffeb58baf@github.com>
Message-ID: <OZ5FPnyyi36m1M6vwvoHhLIyPaohJWRnLV8BQaoBleo=.552f9b74-912c-44a2-b229-d143ee857302@github.com>

On Mon, 24 May 2021 12:58:45 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> I'd like to propose a small adjustment to how we write .inline.hpp files, in the hope to reduce include problems because of circular dependencies between inline headers.
>> 
>> This is a small, contrived example to show the problem:
>> 
>> // a.hpp
>> #pragma once
>> 
>> void a1();
>> void a2();
>> 
>> 
>> // a.inline.hpp
>> #pragma once
>> 
>> #include "a.hpp"
>> #include "b.inline.hpp"
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> 
>> // b.hpp
>> #pragma once
>> 
>> void b1();
>> void b2();
>> 
>> 
>> // b.inline.hpp
>> #pragma once
>> 
>> #include "a.inline.hpp"
>> #include "b.hpp"
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> The following code compiles fine:
>> 
>> // a.cpp
>> #include "a.inline.hpp"
>> 
>> int main() {
>>   a1();
>> 
>>   return 0;
>> }
>> 
>> But the following:
>> 
>> // b.cpp
>> #include "b.inline.hpp"
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> 
>> fails with the this error message:
>> 
>> In file included from b.inline.hpp:3,
>>                  from b.cpp:1:
>> a.inline.hpp: In function ?void a1()?:
>> a.inline.hpp:8:3: error: ?b1? was not declared in this scope; did you mean ?a1??
>> 
>> We can see the problem with g++ -E:
>> 
>> # 1 "a.inline.hpp" 1
>> # 1 "a.hpp" 1
>> 
>> void a1();
>> void a2();
>> 
>> # 4 "a.inline.hpp" 2
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> # 4 "b.inline.hpp" 2
>> # 1 "b.hpp" 1
>> 
>> void b1();
>> void b2();
>> 
>> # 5 "b.inline.hpp" 2
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> # 2 "b.cpp" 2
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> 
>> b1() is called before it has been declared. b.inline.hpp inlined a.inline.hpp, which uses functions declared in b.hpp. However, at that point we've not included enough of b.inline.hpp to have declared b1().
>> 
>> This is a small and easy example. In the JVM the circular dependencies usually involves more than two files, and often from different sub-systems of the JVM. We have so-far solved this by restructuring the code, making functions out-of-line, creating proxy objects, etc. However, the more we use templates, the more this is going to become a reoccurring nuisance. And in experiments with lambdas, which requires templates, this very quickly showed up as a problem.
>> 
>> I propose that we solve most (all?) of these issues by adding a rule that states that .inline.hpp files should start by including the corresponding .hpp, and that the .hpp should contain the declarations of all externally used parts of the .inline.hpp file. This should guarantee that the declarations are always present before any subsequent include can create a circular include dependency back to the original file.
>> 
>> In the example above, b.inline.hpp would become:
>> 
>> // b.inline.hpp
>> #pragma once
>> 
>> #include "b.hpp"
>> #include "a.inline.hpp"
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> and now both a.cpp and b.cpp compiles. The generated output now looks like this:
>> 
>> # 1 "b.inline.hpp" 1
>> # 1 "b.hpp" 1
>> 
>> void b1();
>> void b2();
>> 
>> # 4 "b.inline.hpp" 2
>> # 1 "a.inline.hpp" 1
>> 
>> # 1 "a.hpp" 1
>> 
>> void a1();
>> void a2();
>> 
>> # 4 "a.inline.hpp" 2
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> # 5 "b.inline.hpp" 2
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> # 2 "b.cpp" 2
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> The declarations come first, and the compiler is happy. 
>> 
>> An alternative to this, that has been proposed by other HotSpot GC devs have been to always include all .hpp files first, and then have a section of .inline.hpp includes. I've experimented with that as well, but I think it requires more maintenance and vigilance of  *users* .inline.hpp files (add .hpp include to the first section, add .inline.hpp section to second). The proposed structures only requires extra care from *writers* of .inline.hpp files. All others can include .hpp and .inline.hpp as we've been doing for a long time now.
>> 
>> Some notes about this patch:
>> 1) Some externally-used declarations have been placed in .inline.hpp files, and not in the .hpp. Those should be moved. I have not done that.
>> 2) Some .inline.hpp files don't have a corresponding .hpp file. I could either try to fix that in this patch, or we could take care of that as separate patches later.
>> 3) The style I chose was to add the .hpp include and a extra blank line, separating it from the rest of the includes. I think I like that style, because it makes it easy for those writing .inline.hpp to recognize this pattern. And it removes the confusion why this include isn't sorted into the other includes.
>> 4) We have a few *special* platform dependent header files. Both those that simply are included into platform independent files, and those that inject code *into* the platform independent classes. Extra care, as always, need to be taken around those files.
>> 5) Mostly tested locally, but I'll test on more platforms if the idea is accepted.
>> 
>> What do you think?
>
> Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Update HotSpot style guide documents

Hi Stefan,

I think this policy makes a lot of sense!

I only scanned through the changes to get a sense of them but it all seemed fine.

Thanks,
David

doc/hotspot-style.md line 142:

> 140: 
> 141: * All .inline.hpp files should include their corresponding .hpp file
> 142: as the first include line. Externally used declarations should be put

Do we need to spell out it should be first unless precompiled.hpp is included?

-------------

Marked as reviewed by dholmes (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/4127

From gnu.andrew at redhat.com  Wed May 26 05:40:39 2021
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Wed, 26 May 2021 06:40:39 +0100
Subject: [RFR] [8u] 8u302-b01 Upstream Sync
Message-ID: <20210526054039.GA1745963@rincewind>

Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/

Merge changesets:
http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/corba/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/jaxp/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/jaxws/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/jdk/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/hotspot/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/langtools/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/nashorn/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/root/merge.changeset

Changes in aarch64-shenandoah-jdk8u302-b01:
  - JDK-6878250: (so) IllegalBlockingModeException thrown when reading from a closed SocketChannel's InputStream
  - JDK-7059970: Test case: javax/imageio/plugins/png/ITXtTest.java is not closing a file
  - JDK-8030123: java/beans/Introspector/Test8027648.java fails
  - JDK-8033289: clang: clean up unused function warning
  - JDK-8036095: RMI tests using testlibrary.RMID and testlibrary.JavaVM do not pass through vmoptions
  - JDK-8042891: Format issues embedded in macros for two g1 source files
  - JDK-8055754: filemap.cpp does not compile with clang
  - JDK-8064909: FragmentMetaspace.java got OutOfMemoryError
  - JDK-8066508: JTReg tests timeout on slow devices when run using JPRT
  - JDK-8066807: langtools/test/Makefile should use -agentvm not -samevm
  - JDK-8071374: -XX:+PrintAssembly -XX:+PrintSignatureHandlers crash fastdebug VM with assert(limit == __null || limit <= nm->code_end()) in RelocIterator::initialize
  - JDK-8073446: TimeZone getOffset API does not  return a dst offset between years 2038-2137
  - JDK-8075071: [TEST_BUG] TimSortStackSize2.java: OOME: Java heap space: MaxHeap shrinked by MaxRAMFraction
  - JDK-8077364: "if( !this )" construct prevents build on Xcode 6.3
  - JDK-8130308: Too low memory usage in TestPromotionFromSurvivorToTenuredAfterMinorGC.java
  - JDK-8132148: G1 hs_err region dump legend out of sync with region values
  - JDK-8132709: [TESTBUG] gc/g1/TestHumongousShrinkHeap.java might fail on embedded
  - JDK-8134672: [TEST_BUG] Some tests should check isDisplayChangeSupported
  - JDK-8134883: C1 hard crash in range check elimination in Nashorn test262parallel
  - JDK-8136592: [TEST_BUG] Fix 2 platform-specific closed regtests for jigsaw
  - JDK-8151786: [TESTBUG] java/beans/XMLEncoder/Test4625418.java timed out intermittently
  - JDK-8159898: Negative array size in java/beans/Introspector/Test8027905.java
  - JDK-8166046: [TESTBUG] compiler/stringopts/TestStringObjectInitialization.java fails with OOME
  - JDK-8166724: gc/g1/TestHumongousShrinkHeap.java fails with OOME
  - JDK-8177809: File.lastModified() is losing milliseconds (always ends in 000)
  - JDK-8178403: DirectAudio in JavaSound may hang and leak
  - JDK-8180478: tools/launcher/MultipleJRE.sh fails on Windows because of extra-''
  - JDK-8183910: gc/arguments/TestAggressiveHeap.java fails intermittently
  - JDK-8190332: PngReader throws NegativeArraySizeException/OOM error when IHDR width is very large
  - JDK-8190679: java/util/Arrays/TimSortStackSize2.java fails with "Initial heap size set to a larger value than the maximum heap size"
  - JDK-8191955: AArch64: incorrect prefetch distance causes an internal error
  - JDK-8199265: java/util/Arrays/TimSortStackSize2.java fails with OOM
  - JDK-8200550: Xcode 9.3 produce warning -Wexpansion-to-defined
  - JDK-8203196: C1 emits incorrect code due to integer overflow in _tableswitch keys
  - JDK-8205014: com/sun/jndi/ldap/DeadSSLLdapTimeoutTest.java failed with "Read timed out"
  - JDK-8209996: [PPC64] Fix JFR profiling
  - JDK-8214345: infinite recursion while checking super class
  - JDK-8217230: assert(t == t_no_spec) failure in NodeHash::check_no_speculative_types()
  - JDK-8217348: assert(thread->is_Java_thread()) failed: just checking
  - JDK-8225081: Remove Telia Company CA certificate expiring in April 2021
  - JDK-8225116: Test OwnedWindowsLeak.java intermittently fails
  - JDK-8230428: Cleanup dead CastIP node code in formssel.cpp
  - JDK-8231631: sun/net/ftp/FtpURLConnectionLeak.java fails intermittently with NPE
  - JDK-8231841: AArch64: debug.cpp help() is missing an AArch64 line for pns
  - JDK-8231949: [PPC64, s390]: Make async profiling more reliable
  - JDK-8234011: (zipfs) Memory leak in ZipFileSystem.releaseDeflater()
  - JDK-8241649: Optimize Character.toString
  - JDK-8243559: Remove root certificates with 1024-bit keys
  - JDK-8247350: [aarch64] assert(false) failed: wrong size of mach node
  - JDK-8249278: Revert JDK-8226253 which breaks the spec of AccessibleState.SHOWING for JList
  - JDK-8255086: Update the root locale display names
  - JDK-8255734: VM should ignore SIGXFSZ on ppc64, s390 too
  - JDK-8256421: Add 2 HARICA roots to cacerts truststore
  - JDK-8257999: Parallel GC crash in gc/parallel/TestDynShrinkHeap.java: new region is not in covered_region
  - JDK-8258419: RSA cipher buffer cleanup
  - JDK-8258669: fastdebug jvm crashes when do event based tracing for monitor inflation
  - JDK-8258753: StartTlsResponse.close() hangs due to synchronization issues
  - JDK-8259271: gc/parallel/TestDynShrinkHeap.java still fails "assert(covered_region.contains(new_memregion)) failed: new region is not in covered_region"
  - JDK-8259619: C1: 3-arg StubAssembler::call_RT stack-use condition is incorrect
  - JDK-8259886: Improve SSL session cache performance and scalability
  - JDK-8260029: aarch64: fix typo in verify_oop_array
  - JDK-8260236: better init AnnotationCollector _contended_group
  - JDK-8260255: C1: LoopInvariantCodeMotion constructor can leave some fields uninitialized
  - JDK-8260484: CheckExamples.java / NoJavaLangTest.java fail with jtreg 4.2
  - JDK-8260704: ParallelGC: oldgen expansion needs release-store for _end
  - JDK-8261355: No data buffering in SunPKCS11 Cipher encryption when the underlying mechanism has no padding
  - JDK-8261867: Backport relevant test changes & additions from JDK-8130125
  - JDK-8262110: DST starts from incorrect time in 2038
  - JDK-8262726: AArch64: C1 StubAssembler::call_RT can corrupt stack
  - JDK-8262730: Enable jdk8u MacOS external debug symbols
  - JDK-8262864: No debug symbols in image for Windows --with-native-debug-symbols=external
  - JDK-8263061: copy wrong unpack200 debuginfo to bin directory after 8252395
  - JDK-8263504: Some OutputMachOpcodes fields are uninitialized
  - JDK-8263600: change rmidRunning to a simple lookup
  - JDK-8264509: jdk8u MacOS zipped debug symbols won't build
  - JDK-8264562: assert(verify_field_bit(1)) failed: Attempting to write an uninitialized event field: type
  - JDK-8264816: Weak handles leak causes GC to take longer
  - JDK-8265832: runtime/StackGap/testme.sh fails to compile in 8u
  - JDK-8265988: Fix sun/text/IntHashtable/Bug4170614 for JDK 8u
  - JDK-8266191: Missing aarch64 parts of JDK-8181872 (C1: possible overflow when strength reducing integer multiply by constant)

Main issues of note:
None, clean merge.

diffstat for root
 b/.hgtags                           |    3 
 b/make/common/NativeCompilation.gmk |  131 ++++++++++++++++++++----------------
 2 files changed, 77 insertions(+), 57 deletions(-)

diffstat for corba
 b/.hgtags |    3 +++
 1 file changed, 3 insertions(+)

diffstat for jaxp
 b/.hgtags |    3 +++
 1 file changed, 3 insertions(+)

diffstat for jaxws
 b/.hgtags |    3 +++
 1 file changed, 3 insertions(+)

diffstat for langtools
 b/.hgtags                                                   |    3 +++
 b/src/share/classes/com/sun/tools/javac/comp/Check.java     |    5 ++++-
 b/test/Makefile                                             |   10 ++++------
 b/test/tools/javac/diags/examples/NoJavaLang.java           |    2 +-
 b/test/tools/javac/fatalErrors/NoJavaLangTest.java          |    2 +-
 b/test/tools/javac/generics/ClassBoundCheckingOverflow.java |   12 ++++++++++++
 b/test/tools/javac/generics/ClassBoundCheckingOverflow.out  |    3 +++
 7 files changed, 28 insertions(+), 9 deletions(-)

diffstat for nashorn
 b/.hgtags |    3 +++
 1 file changed, 3 insertions(+)

diffstat for jdk
 a/make/data/cacerts/soneraclass2ca                                                         |   26 
 a/make/data/cacerts/thawtepremiumserverca                                                  |   27 
 a/make/data/cacerts/verisignclass2g2ca                                                     |   26 
 a/make/data/cacerts/verisignclass3ca                                                       |   21 
 a/make/data/cacerts/verisignclass3g2ca                                                     |   26 
 a/make/data/cacerts/verisigntsaca                                                          |   24 
 b/.hgtags                                                                                  |    3 
 b/make/CompileLaunchers.gmk                                                                |    6 
 b/make/Images.gmk                                                                          |   17 
 b/src/share/classes/com/sun/crypto/provider/RSACipher.java                                 |   80 -
 b/src/share/classes/com/sun/imageio/plugins/png/PNGImageReader.java                        |   25 
 b/src/share/classes/com/sun/media/sound/DirectAudioDevice.java                             |   10 
 b/src/share/classes/java/beans/IndexedPropertyDescriptor.java                              |   56 
 b/src/share/classes/java/beans/PropertyDescriptor.java                                     |    2 
 b/src/share/classes/java/lang/Character.java                                               |    7 
 b/src/share/classes/javax/swing/JList.java                                                 |    8 
 b/src/share/classes/sun/nio/ch/SocketAdaptor.java                                          |    5 
 b/src/share/classes/sun/security/pkcs11/P11Cipher.java                                     |  196 +-
 b/src/share/classes/sun/security/rsa/RSACore.java                                          |    4 
 b/src/share/classes/sun/security/rsa/RSAPadding.java                                       |   39 
 b/src/share/classes/sun/security/ssl/SSLSocketImpl.java                                    |   24 
 b/src/share/classes/sun/security/util/Cache.java                                           |   21 
 b/src/share/classes/sun/util/calendar/ZoneInfo.java                                        |    4 
 b/src/share/classes/sun/util/resources/LocaleNames.properties                              |  208 +--
 b/src/share/demo/nio/zipfs/src/com/sun/nio/zipfs/ZipFileSystem.java                        |    2 
 b/src/solaris/native/java/io/UnixFileSystem_md.c                                           |   19 
 b/src/windows/classes/com/sun/java/accessibility/AccessBridge.java                         |   88 -
 b/test/ProblemList.txt                                                                     |    3 
 b/test/com/sun/jndi/ldap/DeadSSLLdapTimeoutTest.java                                       |    3 
 b/test/demo/zipfs/ReleaseDeflater.java                                                     |  100 +
 b/test/java/awt/EmbeddedFrame/GraphicsConfigTest/GraphicsConfigTest.java                   |  157 ++
 b/test/java/awt/EventQueue/6980209/bug6980209.java                                         |   13 
 b/test/java/awt/Frame/NonEDT_GUI_DeadlockTest/NonEDT_GUI_Deadlock.html                     |   44 
 b/test/java/awt/Frame/NonEDT_GUI_DeadlockTest/NonEDT_GUI_Deadlock.java                     |  323 ++++
 b/test/java/awt/GraphicsDevice/IncorrectDisplayModeExitFullscreen.java                     |    3 
 b/test/java/awt/List/FocusEmptyListTest/FocusEmptyListTest.html                            |   44 
 b/test/java/awt/List/FocusEmptyListTest/FocusEmptyListTest.java                            |   96 +
 b/test/java/awt/Mixing/AWT_Mixing/JSplitPaneOverlapping.java                               |    3 
 b/test/java/awt/Window/OwnedWindowsLeak/OwnedWindowsLeak.java                              |   50 
 b/test/java/awt/font/GlyphVector/TestLayoutFlags.java                                      |  209 +++
 b/test/java/awt/font/Underline/UnderlineTest.java                                          |  160 ++
 b/test/java/awt/xembed/server/TestXEmbedServer.java                                        |    7 
 b/test/java/beans/Introspector/Test8027905.java                                            |    1 
 b/test/java/beans/XMLEncoder/Test4625418.java                                              |  658 +++++-----
 b/test/java/io/File/SetLastModified.java                                                   |    4 
 b/test/java/nio/channels/SocketChannel/AdaptSocketReadAfterClose.java                      |   63 
 b/test/java/rmi/activation/Activatable/checkActivateRef/security.policy                    |    3 
 b/test/java/rmi/activation/Activatable/checkAnnotations/security.policy                    |    3 
 b/test/java/rmi/activation/Activatable/checkImplClassLoader/security.policy                |    3 
 b/test/java/rmi/activation/Activatable/checkRegisterInLog/security.policy                  |    3 
 b/test/java/rmi/activation/Activatable/createPrivateActivable/security.policy              |    3 
 b/test/java/rmi/activation/Activatable/downloadParameterClass/security.policy              |    3 
 b/test/java/rmi/activation/Activatable/elucidateNoSuchMethod/security.policy               |    3 
 b/test/java/rmi/activation/Activatable/extLoadedImpl/security.policy                       |    3 
 b/test/java/rmi/activation/Activatable/forceLogSnapshot/security.policy                    |    3 
 b/test/java/rmi/activation/Activatable/inactiveGroup/security.policy                       |    3 
 b/test/java/rmi/activation/Activatable/nestedActivate/security.policy                      |    3 
 b/test/java/rmi/activation/Activatable/nonExistentActivatable/security.policy              |    3 
 b/test/java/rmi/activation/Activatable/restartCrashedService/security.policy               |    3 
 b/test/java/rmi/activation/Activatable/restartLatecomer/security.policy                    |    3 
 b/test/java/rmi/activation/Activatable/restartService/security.policy                      |    3 
 b/test/java/rmi/activation/Activatable/shutdownGracefully/security.policy                  |    3 
 b/test/java/rmi/activation/Activatable/unregisterInactive/security.policy                  |    3 
 b/test/java/rmi/activation/ActivateFailedException/activateFails/security.policy           |    3 
 b/test/java/rmi/activation/ActivationSystem/activeGroup/security.policy                    |    3 
 b/test/java/rmi/activation/ActivationSystem/modifyDescriptor/security.policy               |    3 
 b/test/java/rmi/activation/ActivationSystem/stubClassesPermitted/security.policy           |    3 
 b/test/java/rmi/activation/ActivationSystem/unregisterGroup/security.policy                |    3 
 b/test/java/rmi/activation/CommandEnvironment/security.policy                              |    3 
 b/test/java/rmi/dgc/VMID/security.policy                                                   |    3 
 b/test/java/rmi/registry/classPathCodebase/security.policy                                 |    3 
 b/test/java/rmi/server/RMIClassLoader/delegateToContextLoader/security.policy              |    3 
 b/test/java/rmi/server/RMIClassLoader/downloadArrayClass/security.policy                   |    3 
 b/test/java/rmi/server/RMIClassLoader/getClassLoader/security.policy                       |    3 
 b/test/java/rmi/server/RMIClassLoader/loadProxyClasses/security.policy                     |    3 
 b/test/java/rmi/server/RMIClassLoader/spi/security.policy                                  |    3 
 b/test/java/rmi/server/RMIClassLoader/useCodebaseOnly/security.policy                      |    3 
 b/test/java/rmi/server/RMIClassLoader/useGetURLs/security.policy                           |    3 
 b/test/java/rmi/server/RMISocketFactory/useSocketFactory/activatable/security.policy       |    3 
 b/test/java/rmi/server/RMISocketFactory/useSocketFactory/registry/security.policy          |    3 
 b/test/java/rmi/server/RMISocketFactory/useSocketFactory/unicast/security.policy           |    3 
 b/test/java/rmi/server/useCustomRef/security.policy                                        |    3 
 b/test/java/rmi/testlibrary/ActivationLibrary.java                                         |   68 -
 b/test/java/rmi/testlibrary/JavaVM.java                                                    |    8 
 b/test/java/rmi/testlibrary/RMID.java                                                      |   61 
 b/test/java/rmi/testlibrary/TestLibrary.java                                               |   24 
 b/test/java/rmi/testlibrary/TestParams.java                                                |   64 
 b/test/java/rmi/transport/checkLeaseInfoLeak/security.policy                               |    3 
 b/test/java/rmi/transport/dgcDeadLock/security.policy                                      |    3 
 b/test/java/util/Arrays/TimSortStackSize2.java                                             |   56 
 b/test/java/util/Locale/LocaleEnhanceTest.java                                             |   17 
 b/test/java/util/Locale/LocaleTest.java                                                    |    6 
 b/test/javax/imageio/plugins/png/ITXtTest.java                                             |   39 
 b/test/javax/imageio/plugins/png/PngLargeIHDRDimensionTest.java                            |   86 +
 b/test/javax/sound/sampled/Clip/ClipCloseLoss.java                                         |   20 
 b/test/jdk/security/infra/java/security/cert/CertPathValidator/certification/HaricaCA.java |  320 ++++
 b/test/sun/net/www/ftptest/FtpCommandHandler.java                                          |    4 
 b/test/sun/rmi/server/MarshalOutputStream/marshalForeignStub/security.policy               |    3 
 b/test/sun/security/lib/cacerts/VerifyCACerts.java                                         |   24 
 b/test/sun/security/pkcs11/Cipher/EncryptionPadding.java                                   |  104 +
 b/test/sun/text/IntHashtable/Bug4170614Test.sh                                             |    3 
 b/test/sun/text/resources/LocaleData                                                       |  123 +
 b/test/sun/text/resources/LocaleDataTest.java                                              |    2 
 b/test/sun/util/calendar/zi/Beyond2037.java                                                |   73 +
 b/test/tools/launcher/MultipleJRE.sh                                                       |    2 
 105 files changed, 2918 insertions(+), 1218 deletions(-)

diffstat for hotspot
 b/.hgtags                                                                       |    3 
 b/src/cpu/aarch64/vm/c1_LIRGenerator_aarch64.cpp                                |   10 -
 b/src/cpu/aarch64/vm/c1_Runtime1_aarch64.cpp                                    |    6 
 b/src/cpu/aarch64/vm/macroAssembler_aarch64.cpp                                 |    8 
 b/src/cpu/aarch64/vm/stubGenerator_aarch64.cpp                                  |    4 
 b/src/cpu/aarch64/vm/vm_version_aarch64.cpp                                     |   11 +
 b/src/cpu/ppc/vm/frame_ppc.cpp                                                  |   11 -
 b/src/cpu/x86/vm/c1_Runtime1_x86.cpp                                            |    4 
 b/src/cpu/x86/vm/templateTable_x86_64.cpp                                       |    4 
 b/src/os_cpu/aix_ppc/vm/os_aix_ppc.cpp                                          |    2 
 b/src/os_cpu/linux_ppc/vm/os_linux_ppc.cpp                                      |    2 
 b/src/os_cpu/linux_ppc/vm/thread_linux_ppc.cpp                                  |   29 +--
 b/src/share/tools/hsdis/hsdis.c                                                 |   19 +-
 b/src/share/vm/adlc/formssel.cpp                                                |   37 ----
 b/src/share/vm/adlc/output_h.cpp                                                |    4 
 b/src/share/vm/c1/c1_IR.cpp                                                     |    7 
 b/src/share/vm/c1/c1_Instruction.hpp                                            |    4 
 b/src/share/vm/c1/c1_LIRGenerator.cpp                                           |    2 
 b/src/share/vm/c1/c1_Runtime1.cpp                                               |    1 
 b/src/share/vm/c1/c1_ValueMap.cpp                                               |    2 
 b/src/share/vm/classfile/classFileParser.hpp                                    |    4 
 b/src/share/vm/code/codeBlob.cpp                                                |    5 
 b/src/share/vm/code/codeBlob.hpp                                                |    1 
 b/src/share/vm/code/nmethod.cpp                                                 |    8 
 b/src/share/vm/code/nmethod.hpp                                                 |    1 
 b/src/share/vm/compiler/disassembler.cpp                                        |    3 
 b/src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp                         |   77 ++++----
 b/src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp                         |   15 -
 b/src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp                       |   12 -
 b/src/share/vm/gc_implementation/g1/g1ErgoVerbose.hpp                           |   65 +++----
 b/src/share/vm/gc_implementation/g1/g1HeapRegionEventSender.cpp                 |    2 
 b/src/share/vm/gc_implementation/g1/heapRegionSet.hpp                           |   16 -
 b/src/share/vm/gc_implementation/parallelScavenge/psOldGen.cpp                  |    6 
 b/src/share/vm/gc_implementation/shared/mutableSpace.cpp                        |   15 +
 b/src/share/vm/interpreter/interpreterRuntime.cpp                               |    1 
 b/src/share/vm/jfr/recorder/checkpoint/types/jfrType.cpp                        |   13 -
 b/src/share/vm/libadt/set.cpp                                                   |    2 
 b/src/share/vm/memory/filemap.cpp                                               |   14 -
 b/src/share/vm/memory/filemap.hpp                                               |    4 
 b/src/share/vm/opto/chaitin.cpp                                                 |    2 
 b/src/share/vm/opto/compile.cpp                                                 |    7 
 b/src/share/vm/opto/compile.hpp                                                 |    2 
 b/src/share/vm/opto/output.cpp                                                  |   12 +
 b/src/share/vm/prims/jvmtiEnvBase.cpp                                           |    2 
 b/src/share/vm/prims/jvmtiTagMap.cpp                                            |    2 
 b/src/share/vm/prims/methodHandles.cpp                                          |    7 
 b/src/share/vm/runtime/mutex.cpp                                                |   10 -
 b/src/share/vm/runtime/sharedRuntime.cpp                                        |    1 
 b/src/share/vm/runtime/synchronizer.cpp                                         |   72 ++++++--
 b/src/share/vm/runtime/synchronizer.hpp                                         |   15 +
 b/src/share/vm/runtime/vframe.hpp                                               |    1 
 b/src/share/vm/utilities/debug.cpp                                              |   14 -
 b/test/Makefile                                                                 |    3 
 b/test/compiler/rangechecks/TestRangeCheckExceptionHandlerLoop.jasm             |   89 ++++++++++
 b/test/compiler/rangechecks/TestRangeCheckExceptionHandlerLoopMain.java         |   41 ++++
 b/test/compiler/stringopts/TestStringObjectInitialization.java                  |    8 
 b/test/gc/arguments/TestAggressiveHeap.java                                     |    7 
 b/test/gc/g1/TestHumongousShrinkHeap.java                                       |   30 ++-
 b/test/gc/survivorAlignment/TestPromotionFromSurvivorToTenuredAfterMinorGC.java |   11 +
 b/test/runtime/Metaspace/FragmentMetaspace.java                                 |   23 +-
 b/test/runtime/StackGap/testme.sh                                               |    3 
 61 files changed, 506 insertions(+), 290 deletions(-)

Successfully built on x86, x86_64, s390 (Zero), s390x (Zero), ppc (Zero),
ppc64, ppc64le & aarch64.

Ok to push?

Thanks,
-- 
Andrew :)

Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222

From stefank at openjdk.java.net  Wed May 26 06:38:47 2021
From: stefank at openjdk.java.net (Stefan Karlsson)
Date: Wed, 26 May 2021 06:38:47 GMT
Subject: RFR: 8267464: Circular-dependency resilient inline headers [v5]
In-Reply-To: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
Message-ID: <c-OaKR1PFHIGfLRVAIG-RHC5EnozbxTHzTMq_5TRl_g=.140285a5-9230-4931-9679-8945670749d8@github.com>

> I'd like to propose a small adjustment to how we write .inline.hpp files, in the hope to reduce include problems because of circular dependencies between inline headers.
> 
> This is a small, contrived example to show the problem:
> 
> // a.hpp
> #pragma once
> 
> void a1();
> void a2();
> 
> 
> // a.inline.hpp
> #pragma once
> 
> #include "a.hpp"
> #include "b.inline.hpp"
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> 
> // b.hpp
> #pragma once
> 
> void b1();
> void b2();
> 
> 
> // b.inline.hpp
> #pragma once
> 
> #include "a.inline.hpp"
> #include "b.hpp"
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> The following code compiles fine:
> 
> // a.cpp
> #include "a.inline.hpp"
> 
> int main() {
>   a1();
> 
>   return 0;
> }
> 
> But the following:
> 
> // b.cpp
> #include "b.inline.hpp"
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> 
> fails with the this error message:
> 
> In file included from b.inline.hpp:3,
>                  from b.cpp:1:
> a.inline.hpp: In function ?void a1()?:
> a.inline.hpp:8:3: error: ?b1? was not declared in this scope; did you mean ?a1??
> 
> We can see the problem with g++ -E:
> 
> # 1 "a.inline.hpp" 1
> # 1 "a.hpp" 1
> 
> void a1();
> void a2();
> 
> # 4 "a.inline.hpp" 2
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> # 4 "b.inline.hpp" 2
> # 1 "b.hpp" 1
> 
> void b1();
> void b2();
> 
> # 5 "b.inline.hpp" 2
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> # 2 "b.cpp" 2
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> 
> b1() is called before it has been declared. b.inline.hpp inlined a.inline.hpp, which uses functions declared in b.hpp. However, at that point we've not included enough of b.inline.hpp to have declared b1().
> 
> This is a small and easy example. In the JVM the circular dependencies usually involves more than two files, and often from different sub-systems of the JVM. We have so-far solved this by restructuring the code, making functions out-of-line, creating proxy objects, etc. However, the more we use templates, the more this is going to become a reoccurring nuisance. And in experiments with lambdas, which requires templates, this very quickly showed up as a problem.
> 
> I propose that we solve most (all?) of these issues by adding a rule that states that .inline.hpp files should start by including the corresponding .hpp, and that the .hpp should contain the declarations of all externally used parts of the .inline.hpp file. This should guarantee that the declarations are always present before any subsequent include can create a circular include dependency back to the original file.
> 
> In the example above, b.inline.hpp would become:
> 
> // b.inline.hpp
> #pragma once
> 
> #include "b.hpp"
> #include "a.inline.hpp"
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> and now both a.cpp and b.cpp compiles. The generated output now looks like this:
> 
> # 1 "b.inline.hpp" 1
> # 1 "b.hpp" 1
> 
> void b1();
> void b2();
> 
> # 4 "b.inline.hpp" 2
> # 1 "a.inline.hpp" 1
> 
> # 1 "a.hpp" 1
> 
> void a1();
> void a2();
> 
> # 4 "a.inline.hpp" 2
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> # 5 "b.inline.hpp" 2
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> # 2 "b.cpp" 2
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> The declarations come first, and the compiler is happy. 
> 
> An alternative to this, that has been proposed by other HotSpot GC devs have been to always include all .hpp files first, and then have a section of .inline.hpp includes. I've experimented with that as well, but I think it requires more maintenance and vigilance of  *users* .inline.hpp files (add .hpp include to the first section, add .inline.hpp section to second). The proposed structures only requires extra care from *writers* of .inline.hpp files. All others can include .hpp and .inline.hpp as we've been doing for a long time now.
> 
> Some notes about this patch:
> 1) Some externally-used declarations have been placed in .inline.hpp files, and not in the .hpp. Those should be moved. I have not done that.
> 2) Some .inline.hpp files don't have a corresponding .hpp file. I could either try to fix that in this patch, or we could take care of that as separate patches later.
> 3) The style I chose was to add the .hpp include and a extra blank line, separating it from the rest of the includes. I think I like that style, because it makes it easy for those writing .inline.hpp to recognize this pattern. And it removes the confusion why this include isn't sorted into the other includes.
> 4) We have a few *special* platform dependent header files. Both those that simply are included into platform independent files, and those that inject code *into* the platform independent classes. Extra care, as always, need to be taken around those files.
> 5) Mostly tested locally, but I'll test on more platforms if the idea is accepted.
> 
> What do you think?

Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision:

  Review dholmes

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4127/files
  - new: https://git.openjdk.java.net/jdk/pull/4127/files/ba9ad1b1..0cf6257d

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4127&range=04
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4127&range=03-04

  Stats: 3 lines in 2 files changed: 0 ins; 0 del; 3 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4127.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4127/head:pull/4127

PR: https://git.openjdk.java.net/jdk/pull/4127

From stefank at openjdk.java.net  Wed May 26 06:38:53 2021
From: stefank at openjdk.java.net (Stefan Karlsson)
Date: Wed, 26 May 2021 06:38:53 GMT
Subject: RFR: 8267464: Circular-dependency resilient inline headers [v4]
In-Reply-To: <OZ5FPnyyi36m1M6vwvoHhLIyPaohJWRnLV8BQaoBleo=.552f9b74-912c-44a2-b229-d143ee857302@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
 <TU_1CODSKvFOzDmPEOShcp92o2pleQDQQXZFMfQ0j-Q=.0a220517-a6e1-4b10-871e-2c1ffeb58baf@github.com>
 <OZ5FPnyyi36m1M6vwvoHhLIyPaohJWRnLV8BQaoBleo=.552f9b74-912c-44a2-b229-d143ee857302@github.com>
Message-ID: <O9teAMehggUiNNgTYGybK7Y7096B-qRUJy9GCCbUFhU=.4ab3e9e9-7a4f-437f-96f1-09125677e709@github.com>

On Wed, 26 May 2021 02:18:06 GMT, David Holmes <dholmes at openjdk.org> wrote:

> Hi Stefan,
> 
> I think this policy makes a lot of sense!
> 
> I only scanned through the changes to get a sense of them but it all seemed fine.
> 
> Thanks,
> David

Thanks a lot for reviewing!

> doc/hotspot-style.html line 97:
> 
>> 95: <li><p>Do not put non-trivial function implementations in .hpp files. If the implementation depends on other .hpp files, put it in a .cpp or a .inline.hpp file.</p></li>
>> 96: <li><p>.inline.hpp files should only be included in .cpp or .inline.hpp files.</p></li>
>> 97: <li><p>All .inline.hpp files should include their corresponding .hpp file as the first include line. Externally used declarations should be put in the .hpp file, and not in the the .inline.hpp file. This rule exists to resolve problems with circular dependencies between.inline.hpp files.</p></li>
> 
> typo: the the
> Need space after between on last line

Fixed.

> doc/hotspot-style.md line 142:
> 
>> 140: 
>> 141: * All .inline.hpp files should include their corresponding .hpp file
>> 142: as the first include line. Externally used declarations should be put
> 
> Do we need to spell out it should be first unless precompiled.hpp is included?

We should never include precompiled.hpp from an .inline.hpp file, so I don't think we need to state that.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4127

From dholmes at openjdk.java.net  Wed May 26 07:56:13 2021
From: dholmes at openjdk.java.net (David Holmes)
Date: Wed, 26 May 2021 07:56:13 GMT
Subject: RFR: 8267464: Circular-dependency resilient inline headers [v4]
In-Reply-To: <O9teAMehggUiNNgTYGybK7Y7096B-qRUJy9GCCbUFhU=.4ab3e9e9-7a4f-437f-96f1-09125677e709@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
 <TU_1CODSKvFOzDmPEOShcp92o2pleQDQQXZFMfQ0j-Q=.0a220517-a6e1-4b10-871e-2c1ffeb58baf@github.com>
 <OZ5FPnyyi36m1M6vwvoHhLIyPaohJWRnLV8BQaoBleo=.552f9b74-912c-44a2-b229-d143ee857302@github.com>
 <O9teAMehggUiNNgTYGybK7Y7096B-qRUJy9GCCbUFhU=.4ab3e9e9-7a4f-437f-96f1-09125677e709@github.com>
Message-ID: <zblvBz4I_k_n6eghRSSvY3Y4RqM6QUS7BqqBiT3JYlA=.6cd16585-e5c4-4a8c-ba73-6fadce910632@github.com>

On Wed, 26 May 2021 06:35:29 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> doc/hotspot-style.md line 142:
>> 
>>> 140: 
>>> 141: * All .inline.hpp files should include their corresponding .hpp file
>>> 142: as the first include line. Externally used declarations should be put
>> 
>> Do we need to spell out it should be first unless precompiled.hpp is included?
>
> We should never include precompiled.hpp from an .inline.hpp file, so I don't think we need to state that.

Ah. Right. :)

-------------

PR: https://git.openjdk.java.net/jdk/pull/4127

From coleen.phillimore at oracle.com  Wed May 26 03:05:52 2021
From: coleen.phillimore at oracle.com (Coleen Phillimore)
Date: Tue, 25 May 2021 23:05:52 -0400
Subject: RFR: 8267653: Remove Mutex::_safepoint_check_sometimes [v2]
In-Reply-To: <781224c1-0118-19b4-6cc8-3c8621af4d43@oracle.com>
References: <OpIHO8GcqBWsONZ6MNwlQzaQ6Z4dfekskdRrmUhT8M8=.853bdb38-9966-42fc-8d38-4147836d93e6@github.com>
 <aF7Vsfq_1jsJpPVOpDffrBm4FSux_PRvJqJfC_iiFyI=.58fad6e7-f200-4f95-90ec-f6dcec8eb5de@github.com>
 <IxSp6HN-X-jvxdg6k5oNY9U3edaagjutslwW-pbn7l8=.e98b7711-1f28-4a50-850b-91ad888fbb19@github.com>
 <781224c1-0118-19b4-6cc8-3c8621af4d43@oracle.com>
Message-ID: <82a3fbab-ea30-1bad-6600-4d52b874712f@oracle.com>



On 5/25/21 10:00 PM, David Holmes wrote:
> On 26/05/2021 11:53 am, Coleen Phillimore wrote:
>> On Wed, 26 May 2021 00:38:00 GMT, David Holmes <dholmes at openjdk.org> 
>> wrote:
>>
>>>> Coleen Phillimore has updated the pull request incrementally with 
>>>> one additional commit since the last revision:
>>>>
>>>> ?? Add assert that the thread is terminated before taking Heap_lock
>>>
>>> src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp 
>>> line 543:
>>>
>>>> 541:?? } else {
>>>> 542:???? // Heap_lock protects external pending list
>>>> 543:???? MonitorLocker ml(Heap_lock);
>>>
>>> For performance you could leave the _no_safepoint_check flag.
>>
>> I cannot leave it because there's an assert to verify that we don't 
>> try to take this lock without a safepoint check, since it was 
>> declared with safepoint_check_always.
>
> But that assert only applies to JavaThreads and this is being taken by 
> a GC thread isn't it?

It's true that if this is a non-Java thread, it wouldn't fire the assert 
that checks that the states are consistent.? I had to go study the code 
a bit to see that, which I wouldn't want to do over again.? But it would 
be very inconsistent in the code since all other Heap_lock acquisitions 
are not made with _no_safepoint_check, we would have to look again to 
see why this is special and other GC threads take Heap_lock without 
no_safepoint_check.

And it wouldn't be a meaningful optimization.

thanks,
Coleen

>
>>> src/hotspot/share/runtime/thread.cpp line 3420:
>>>
>>>> 3418:???? // we'll never emerge out of the safepoint before the VM 
>>>> exits.
>>>> 3419:
>>>> 3420:???? MutexLocker ml(Heap_lock);
>>>
>>> Can you add an assert before this that the thread is_terminated() as 
>>> that is the key to the mutex acquisition not doing a safepoint check.
>>
>> Ok, I can add:
>> +??? assert(thread->is_terminated(), "must be terminated before 
>> acquiring Heap_lock");
>> and rerun tests.
>
> Thanks,
> David
>
>> -------------
>>
>> PR: https://git.openjdk.java.net/jdk/pull/4184
>>


From shade at redhat.com  Wed May 26 11:22:51 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Wed, 26 May 2021 13:22:51 +0200
Subject: [RFR] [8u] 8u302-b01 Upstream Sync
In-Reply-To: <20210526054039.GA1745963@rincewind>
References: <20210526054039.GA1745963@rincewind>
Message-ID: <8be9efc9-f09f-1794-7cf4-8efaf2581247@redhat.com>

On 5/26/21 7:40 AM, Andrew Hughes wrote:
> Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/
> 
> Merge changesets:
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/corba/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/jaxp/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/jaxws/merge.changeset

Look trivially fine.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/jdk/merge.changeset

Looks fine.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/hotspot/merge.changeset

Looks fine.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/langtools/merge.changeset

Looks fine.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/nashorn/merge.changeset

Looks trivially fine.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/root/merge.changeset

Looks fine.

> Ok to push?

Yes.

-- 
Thanks,
-Aleksey


From kbarrett at openjdk.java.net  Wed May 26 14:38:15 2021
From: kbarrett at openjdk.java.net (Kim Barrett)
Date: Wed, 26 May 2021 14:38:15 GMT
Subject: RFR: 8267464: Circular-dependency resilient inline headers [v5]
In-Reply-To: <c-OaKR1PFHIGfLRVAIG-RHC5EnozbxTHzTMq_5TRl_g=.140285a5-9230-4931-9679-8945670749d8@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
 <c-OaKR1PFHIGfLRVAIG-RHC5EnozbxTHzTMq_5TRl_g=.140285a5-9230-4931-9679-8945670749d8@github.com>
Message-ID: <GljqMekBQKJKTwU-8ELrwL-oKyRgC6y3S_vI62IlP1U=.8af96aac-9fbc-42ca-b091-78280473188b@github.com>

On Wed, 26 May 2021 06:38:47 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> I'd like to propose a small adjustment to how we write .inline.hpp files, in the hope to reduce include problems because of circular dependencies between inline headers.
>> 
>> This is a small, contrived example to show the problem:
>> 
>> // a.hpp
>> #pragma once
>> 
>> void a1();
>> void a2();
>> 
>> 
>> // a.inline.hpp
>> #pragma once
>> 
>> #include "a.hpp"
>> #include "b.inline.hpp"
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> 
>> // b.hpp
>> #pragma once
>> 
>> void b1();
>> void b2();
>> 
>> 
>> // b.inline.hpp
>> #pragma once
>> 
>> #include "a.inline.hpp"
>> #include "b.hpp"
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> The following code compiles fine:
>> 
>> // a.cpp
>> #include "a.inline.hpp"
>> 
>> int main() {
>>   a1();
>> 
>>   return 0;
>> }
>> 
>> But the following:
>> 
>> // b.cpp
>> #include "b.inline.hpp"
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> 
>> fails with the this error message:
>> 
>> In file included from b.inline.hpp:3,
>>                  from b.cpp:1:
>> a.inline.hpp: In function ?void a1()?:
>> a.inline.hpp:8:3: error: ?b1? was not declared in this scope; did you mean ?a1??
>> 
>> We can see the problem with g++ -E:
>> 
>> # 1 "a.inline.hpp" 1
>> # 1 "a.hpp" 1
>> 
>> void a1();
>> void a2();
>> 
>> # 4 "a.inline.hpp" 2
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> # 4 "b.inline.hpp" 2
>> # 1 "b.hpp" 1
>> 
>> void b1();
>> void b2();
>> 
>> # 5 "b.inline.hpp" 2
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> # 2 "b.cpp" 2
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> 
>> b1() is called before it has been declared. b.inline.hpp inlined a.inline.hpp, which uses functions declared in b.hpp. However, at that point we've not included enough of b.inline.hpp to have declared b1().
>> 
>> This is a small and easy example. In the JVM the circular dependencies usually involves more than two files, and often from different sub-systems of the JVM. We have so-far solved this by restructuring the code, making functions out-of-line, creating proxy objects, etc. However, the more we use templates, the more this is going to become a reoccurring nuisance. And in experiments with lambdas, which requires templates, this very quickly showed up as a problem.
>> 
>> I propose that we solve most (all?) of these issues by adding a rule that states that .inline.hpp files should start by including the corresponding .hpp, and that the .hpp should contain the declarations of all externally used parts of the .inline.hpp file. This should guarantee that the declarations are always present before any subsequent include can create a circular include dependency back to the original file.
>> 
>> In the example above, b.inline.hpp would become:
>> 
>> // b.inline.hpp
>> #pragma once
>> 
>> #include "b.hpp"
>> #include "a.inline.hpp"
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> and now both a.cpp and b.cpp compiles. The generated output now looks like this:
>> 
>> # 1 "b.inline.hpp" 1
>> # 1 "b.hpp" 1
>> 
>> void b1();
>> void b2();
>> 
>> # 4 "b.inline.hpp" 2
>> # 1 "a.inline.hpp" 1
>> 
>> # 1 "a.hpp" 1
>> 
>> void a1();
>> void a2();
>> 
>> # 4 "a.inline.hpp" 2
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> # 5 "b.inline.hpp" 2
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> # 2 "b.cpp" 2
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> The declarations come first, and the compiler is happy. 
>> 
>> An alternative to this, that has been proposed by other HotSpot GC devs have been to always include all .hpp files first, and then have a section of .inline.hpp includes. I've experimented with that as well, but I think it requires more maintenance and vigilance of  *users* .inline.hpp files (add .hpp include to the first section, add .inline.hpp section to second). The proposed structures only requires extra care from *writers* of .inline.hpp files. All others can include .hpp and .inline.hpp as we've been doing for a long time now.
>> 
>> Some notes about this patch:
>> 1) Some externally-used declarations have been placed in .inline.hpp files, and not in the .hpp. Those should be moved. I have not done that.
>> 2) Some .inline.hpp files don't have a corresponding .hpp file. I could either try to fix that in this patch, or we could take care of that as separate patches later.
>> 3) The style I chose was to add the .hpp include and a extra blank line, separating it from the rest of the includes. I think I like that style, because it makes it easy for those writing .inline.hpp to recognize this pattern. And it removes the confusion why this include isn't sorted into the other includes.
>> 4) We have a few *special* platform dependent header files. Both those that simply are included into platform independent files, and those that inject code *into* the platform independent classes. Extra care, as always, need to be taken around those files.
>> 5) Mostly tested locally, but I'll test on more platforms if the idea is accepted.
>> 
>> What do you think?
>
> Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Review dholmes

doc/hotspot-style.md line 142:

> 140: 
> 141: * All .inline.hpp files should include their corresponding .hpp file
> 142: as the first include line. Externally used declarations should be put

I think by "Externally used declarations" you mean "declarations needed by other headers"?  Maybe say it that way, or some other wording?

-------------

PR: https://git.openjdk.java.net/jdk/pull/4127

From wkemper at openjdk.java.net  Wed May 26 17:08:36 2021
From: wkemper at openjdk.java.net (William Kemper)
Date: Wed, 26 May 2021 17:08:36 GMT
Subject: Integrated: Encode phase for each generation separately
In-Reply-To: <eoVIZ0mxm99dZFLuMJTG-lHGTaBQjpmpXe5GH6PqbYA=.89271678-76fe-4c8f-a9bb-290b869bfe68@github.com>
References: <eoVIZ0mxm99dZFLuMJTG-lHGTaBQjpmpXe5GH6PqbYA=.89271678-76fe-4c8f-a9bb-290b869bfe68@github.com>
Message-ID: <jL8j27Bt_NVDnf1Bx8qJ7Okzq3YUuW6bNR_PdDUkz_s=.86245ce5-2f2c-4ec4-bd98-bad73c3ccf50@github.com>

On Tue, 25 May 2021 21:05:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

> This change encodes the state/activity of each generation separately for the overall collector status. There are complementary changes in [my fork of the visualizer](https://github.com/earthling-amzn/shenandoah-visualizer/tree/genshen-visualizer). There is no explicit versioning of the protocol between the encoding in the collector and the decoding in the visualizer. The change in this PR is compatible with the `genshen-visualizer` branch in my fork, but is _not_ compatible with the [upstream visualizer](https://github.com/openjdk/shenandoah-visualizer). Should I open a PR in the visualizer? Should we add a versioning scheme to the prototcol and try to maintain compatibility with older versions?

This pull request has now been integrated.

Changeset: 6f1f29a1
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.java.net/shenandoah/commit/6f1f29a12d6b91ccb610403e9ea87fe719d0a8c8
Stats:     80 lines in 2 files changed: 65 ins; 11 del; 4 mod

Encode phase for each generation separately

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/46

From stefank at openjdk.java.net  Wed May 26 18:00:54 2021
From: stefank at openjdk.java.net (Stefan Karlsson)
Date: Wed, 26 May 2021 18:00:54 GMT
Subject: RFR: 8267464: Circular-dependency resilient inline headers [v6]
In-Reply-To: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
Message-ID: <-N-g2LtXiLzyl-1Z_61StqnhLz01OMEaH1rrUFgHv5c=.10a88ce2-4a54-4fcd-a1a4-6b8083e8df1f@github.com>

> I'd like to propose a small adjustment to how we write .inline.hpp files, in the hope to reduce include problems because of circular dependencies between inline headers.
> 
> This is a small, contrived example to show the problem:
> 
> // a.hpp
> #pragma once
> 
> void a1();
> void a2();
> 
> 
> // a.inline.hpp
> #pragma once
> 
> #include "a.hpp"
> #include "b.inline.hpp"
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> 
> // b.hpp
> #pragma once
> 
> void b1();
> void b2();
> 
> 
> // b.inline.hpp
> #pragma once
> 
> #include "a.inline.hpp"
> #include "b.hpp"
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> The following code compiles fine:
> 
> // a.cpp
> #include "a.inline.hpp"
> 
> int main() {
>   a1();
> 
>   return 0;
> }
> 
> But the following:
> 
> // b.cpp
> #include "b.inline.hpp"
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> 
> fails with the this error message:
> 
> In file included from b.inline.hpp:3,
>                  from b.cpp:1:
> a.inline.hpp: In function ?void a1()?:
> a.inline.hpp:8:3: error: ?b1? was not declared in this scope; did you mean ?a1??
> 
> We can see the problem with g++ -E:
> 
> # 1 "a.inline.hpp" 1
> # 1 "a.hpp" 1
> 
> void a1();
> void a2();
> 
> # 4 "a.inline.hpp" 2
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> # 4 "b.inline.hpp" 2
> # 1 "b.hpp" 1
> 
> void b1();
> void b2();
> 
> # 5 "b.inline.hpp" 2
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> # 2 "b.cpp" 2
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> 
> b1() is called before it has been declared. b.inline.hpp inlined a.inline.hpp, which uses functions declared in b.hpp. However, at that point we've not included enough of b.inline.hpp to have declared b1().
> 
> This is a small and easy example. In the JVM the circular dependencies usually involves more than two files, and often from different sub-systems of the JVM. We have so-far solved this by restructuring the code, making functions out-of-line, creating proxy objects, etc. However, the more we use templates, the more this is going to become a reoccurring nuisance. And in experiments with lambdas, which requires templates, this very quickly showed up as a problem.
> 
> I propose that we solve most (all?) of these issues by adding a rule that states that .inline.hpp files should start by including the corresponding .hpp, and that the .hpp should contain the declarations of all externally used parts of the .inline.hpp file. This should guarantee that the declarations are always present before any subsequent include can create a circular include dependency back to the original file.
> 
> In the example above, b.inline.hpp would become:
> 
> // b.inline.hpp
> #pragma once
> 
> #include "b.hpp"
> #include "a.inline.hpp"
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> and now both a.cpp and b.cpp compiles. The generated output now looks like this:
> 
> # 1 "b.inline.hpp" 1
> # 1 "b.hpp" 1
> 
> void b1();
> void b2();
> 
> # 4 "b.inline.hpp" 2
> # 1 "a.inline.hpp" 1
> 
> # 1 "a.hpp" 1
> 
> void a1();
> void a2();
> 
> # 4 "a.inline.hpp" 2
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> # 5 "b.inline.hpp" 2
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> # 2 "b.cpp" 2
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> The declarations come first, and the compiler is happy. 
> 
> An alternative to this, that has been proposed by other HotSpot GC devs have been to always include all .hpp files first, and then have a section of .inline.hpp includes. I've experimented with that as well, but I think it requires more maintenance and vigilance of  *users* .inline.hpp files (add .hpp include to the first section, add .inline.hpp section to second). The proposed structures only requires extra care from *writers* of .inline.hpp files. All others can include .hpp and .inline.hpp as we've been doing for a long time now.
> 
> Some notes about this patch:
> 1) Some externally-used declarations have been placed in .inline.hpp files, and not in the .hpp. Those should be moved. I have not done that.
> 2) Some .inline.hpp files don't have a corresponding .hpp file. I could either try to fix that in this patch, or we could take care of that as separate patches later.
> 3) The style I chose was to add the .hpp include and a extra blank line, separating it from the rest of the includes. I think I like that style, because it makes it easy for those writing .inline.hpp to recognize this pattern. And it removes the confusion why this include isn't sorted into the other includes.
> 4) We have a few *special* platform dependent header files. Both those that simply are included into platform independent files, and those that inject code *into* the platform independent classes. Extra care, as always, need to be taken around those files.
> 5) Mostly tested locally, but I'll test on more platforms if the idea is accepted.
> 
> What do you think?

Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision:

  Review kbarrett

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4127/files
  - new: https://git.openjdk.java.net/jdk/pull/4127/files/0cf6257d..340bcf49

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4127&range=05
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4127&range=04-05

  Stats: 3 lines in 2 files changed: 0 ins; 0 del; 3 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4127.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4127/head:pull/4127

PR: https://git.openjdk.java.net/jdk/pull/4127

From pchilanomate at openjdk.java.net  Wed May 26 18:07:16 2021
From: pchilanomate at openjdk.java.net (Patricio Chilano Mateo)
Date: Wed, 26 May 2021 18:07:16 GMT
Subject: RFR: 8267653: Remove Mutex::_safepoint_check_sometimes [v2]
In-Reply-To: <Vlg6EpsZj7VNzjxfLXoA-mwseSNfctrdUPfqFoX98Vg=.6b8be221-b1dd-476b-8596-45c0d4d5fcc6@github.com>
References: <OpIHO8GcqBWsONZ6MNwlQzaQ6Z4dfekskdRrmUhT8M8=.853bdb38-9966-42fc-8d38-4147836d93e6@github.com>
 <Vlg6EpsZj7VNzjxfLXoA-mwseSNfctrdUPfqFoX98Vg=.6b8be221-b1dd-476b-8596-45c0d4d5fcc6@github.com>
Message-ID: <XLE9o1X0Q5Jom6URih8n3RrX9mqPbNLK3wyYWbNK1B8=.502d62ef-fba3-4d72-a4db-a8f6de4c8973@github.com>

On Wed, 26 May 2021 01:53:33 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Since SR_lock is removed, this state of declaring locks as sometimes safepoint checking and not is no longer used.  JavaThreads either always or never check for safepoint, depending on the lock.  The Heap_lock was always a bit special because it's taken by a JavaThread after it exits, but the code in mutex::lock_contended already deals with this (ie doesn't safepoint check while exiting).
>> Tested with tier 2-3 and tier1 in progress.
>> Thanks to Zhengu for testing and confirming shenandoah (I built and ran shenandoah tests too).
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Add assert that the thread is terminated before taking Heap_lock

Looks good to me!

Thanks,
Patricio

-------------

Marked as reviewed by pchilanomate (Committer).

PR: https://git.openjdk.java.net/jdk/pull/4184

From coleenp at openjdk.java.net  Wed May 26 19:11:14 2021
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Wed, 26 May 2021 19:11:14 GMT
Subject: RFR: 8267653: Remove Mutex::_safepoint_check_sometimes [v2]
In-Reply-To: <Vlg6EpsZj7VNzjxfLXoA-mwseSNfctrdUPfqFoX98Vg=.6b8be221-b1dd-476b-8596-45c0d4d5fcc6@github.com>
References: <OpIHO8GcqBWsONZ6MNwlQzaQ6Z4dfekskdRrmUhT8M8=.853bdb38-9966-42fc-8d38-4147836d93e6@github.com>
 <Vlg6EpsZj7VNzjxfLXoA-mwseSNfctrdUPfqFoX98Vg=.6b8be221-b1dd-476b-8596-45c0d4d5fcc6@github.com>
Message-ID: <_CnhaU9EWmjSpv7rDdmykhpe41VDnYxot4hY0wABOBA=.c8f5e256-22f7-4b71-a961-b42a200e797f@github.com>

On Wed, 26 May 2021 01:53:33 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Since SR_lock is removed, this state of declaring locks as sometimes safepoint checking and not is no longer used.  JavaThreads either always or never check for safepoint, depending on the lock.  The Heap_lock was always a bit special because it's taken by a JavaThread after it exits, but the code in mutex::lock_contended already deals with this (ie doesn't safepoint check while exiting).
>> Tested with tier 2-3 and tier1 in progress.
>> Thanks to Zhengu for testing and confirming shenandoah (I built and ran shenandoah tests too).
>
> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Add assert that the thread is terminated before taking Heap_lock

Thanks Patricio and David!

-------------

PR: https://git.openjdk.java.net/jdk/pull/4184

From coleenp at openjdk.java.net  Wed May 26 19:11:15 2021
From: coleenp at openjdk.java.net (Coleen Phillimore)
Date: Wed, 26 May 2021 19:11:15 GMT
Subject: Integrated: 8267653: Remove Mutex::_safepoint_check_sometimes
In-Reply-To: <OpIHO8GcqBWsONZ6MNwlQzaQ6Z4dfekskdRrmUhT8M8=.853bdb38-9966-42fc-8d38-4147836d93e6@github.com>
References: <OpIHO8GcqBWsONZ6MNwlQzaQ6Z4dfekskdRrmUhT8M8=.853bdb38-9966-42fc-8d38-4147836d93e6@github.com>
Message-ID: <GXpw72eJxw8mbJt4ULQ3b_ZfRkwftACZ2R-OJp7Rq14=.823cb645-61b4-4a73-aca0-1de780cf2c96@github.com>

On Tue, 25 May 2021 13:20:54 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

> Since SR_lock is removed, this state of declaring locks as sometimes safepoint checking and not is no longer used.  JavaThreads either always or never check for safepoint, depending on the lock.  The Heap_lock was always a bit special because it's taken by a JavaThread after it exits, but the code in mutex::lock_contended already deals with this (ie doesn't safepoint check while exiting).
> Tested with tier 2-3 and tier1 in progress.
> Thanks to Zhengu for testing and confirming shenandoah (I built and ran shenandoah tests too).

This pull request has now been integrated.

Changeset: c59484e7
Author:    Coleen Phillimore <coleenp at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/c59484e71517ea9fea7df7b0344b633495f8fd81
Stats:     26 lines in 5 files changed: 4 ins; 19 del; 3 mod

8267653: Remove Mutex::_safepoint_check_sometimes

Reviewed-by: dholmes, pchilanomate

-------------

PR: https://git.openjdk.java.net/jdk/pull/4184

From wkemper at openjdk.java.net  Wed May 26 20:48:38 2021
From: wkemper at openjdk.java.net (William Kemper)
Date: Wed, 26 May 2021 20:48:38 GMT
Subject: RFR: Check explicitly for affiliation when updating references
Message-ID: <nrW9UiaxTXMC5hFN_6j8NaRR-hVmul6YzKJsBObxS2k=.7ad7dca5-41a7-4c6c-bf68-a917234c74d8@github.com>

It's possible for a mutator allocation to 'activate' a region before setting the affiliation, which would trip the assert here (or worse, miss the assert and then go off the rails).

-------------

Commit messages:
 - Check explicitly for affiliation when updating references

Changes: https://git.openjdk.java.net/shenandoah/pull/47/files
 Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=47&range=00
  Stats: 2 lines in 1 file changed: 0 ins; 1 del; 1 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/47.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/47/head:pull/47

PR: https://git.openjdk.java.net/shenandoah/pull/47

From kbarrett at openjdk.java.net  Wed May 26 23:06:05 2021
From: kbarrett at openjdk.java.net (Kim Barrett)
Date: Wed, 26 May 2021 23:06:05 GMT
Subject: RFR: 8267464: Circular-dependency resilient inline headers [v6]
In-Reply-To: <-N-g2LtXiLzyl-1Z_61StqnhLz01OMEaH1rrUFgHv5c=.10a88ce2-4a54-4fcd-a1a4-6b8083e8df1f@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
 <-N-g2LtXiLzyl-1Z_61StqnhLz01OMEaH1rrUFgHv5c=.10a88ce2-4a54-4fcd-a1a4-6b8083e8df1f@github.com>
Message-ID: <OPTS0x5KcDzroluKpqAFeuFg0km3_z0BrJuEPgDSPyg=.e2006f2c-07e0-42fc-a685-79dea3d0f5ac@github.com>

On Wed, 26 May 2021 18:00:54 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> I'd like to propose a small adjustment to how we write .inline.hpp files, in the hope to reduce include problems because of circular dependencies between inline headers.
>> 
>> This is a small, contrived example to show the problem:
>> 
>> // a.hpp
>> #pragma once
>> 
>> void a1();
>> void a2();
>> 
>> 
>> // a.inline.hpp
>> #pragma once
>> 
>> #include "a.hpp"
>> #include "b.inline.hpp"
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> 
>> // b.hpp
>> #pragma once
>> 
>> void b1();
>> void b2();
>> 
>> 
>> // b.inline.hpp
>> #pragma once
>> 
>> #include "a.inline.hpp"
>> #include "b.hpp"
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> The following code compiles fine:
>> 
>> // a.cpp
>> #include "a.inline.hpp"
>> 
>> int main() {
>>   a1();
>> 
>>   return 0;
>> }
>> 
>> But the following:
>> 
>> // b.cpp
>> #include "b.inline.hpp"
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> 
>> fails with the this error message:
>> 
>> In file included from b.inline.hpp:3,
>>                  from b.cpp:1:
>> a.inline.hpp: In function ?void a1()?:
>> a.inline.hpp:8:3: error: ?b1? was not declared in this scope; did you mean ?a1??
>> 
>> We can see the problem with g++ -E:
>> 
>> # 1 "a.inline.hpp" 1
>> # 1 "a.hpp" 1
>> 
>> void a1();
>> void a2();
>> 
>> # 4 "a.inline.hpp" 2
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> # 4 "b.inline.hpp" 2
>> # 1 "b.hpp" 1
>> 
>> void b1();
>> void b2();
>> 
>> # 5 "b.inline.hpp" 2
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> # 2 "b.cpp" 2
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> 
>> b1() is called before it has been declared. b.inline.hpp inlined a.inline.hpp, which uses functions declared in b.hpp. However, at that point we've not included enough of b.inline.hpp to have declared b1().
>> 
>> This is a small and easy example. In the JVM the circular dependencies usually involves more than two files, and often from different sub-systems of the JVM. We have so-far solved this by restructuring the code, making functions out-of-line, creating proxy objects, etc. However, the more we use templates, the more this is going to become a reoccurring nuisance. And in experiments with lambdas, which requires templates, this very quickly showed up as a problem.
>> 
>> I propose that we solve most (all?) of these issues by adding a rule that states that .inline.hpp files should start by including the corresponding .hpp, and that the .hpp should contain the declarations of all externally used parts of the .inline.hpp file. This should guarantee that the declarations are always present before any subsequent include can create a circular include dependency back to the original file.
>> 
>> In the example above, b.inline.hpp would become:
>> 
>> // b.inline.hpp
>> #pragma once
>> 
>> #include "b.hpp"
>> #include "a.inline.hpp"
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> and now both a.cpp and b.cpp compiles. The generated output now looks like this:
>> 
>> # 1 "b.inline.hpp" 1
>> # 1 "b.hpp" 1
>> 
>> void b1();
>> void b2();
>> 
>> # 4 "b.inline.hpp" 2
>> # 1 "a.inline.hpp" 1
>> 
>> # 1 "a.hpp" 1
>> 
>> void a1();
>> void a2();
>> 
>> # 4 "a.inline.hpp" 2
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> # 5 "b.inline.hpp" 2
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> # 2 "b.cpp" 2
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> The declarations come first, and the compiler is happy. 
>> 
>> An alternative to this, that has been proposed by other HotSpot GC devs have been to always include all .hpp files first, and then have a section of .inline.hpp includes. I've experimented with that as well, but I think it requires more maintenance and vigilance of  *users* .inline.hpp files (add .hpp include to the first section, add .inline.hpp section to second). The proposed structures only requires extra care from *writers* of .inline.hpp files. All others can include .hpp and .inline.hpp as we've been doing for a long time now.
>> 
>> Some notes about this patch:
>> 1) Some externally-used declarations have been placed in .inline.hpp files, and not in the .hpp. Those should be moved. I have not done that.
>> 2) Some .inline.hpp files don't have a corresponding .hpp file. I could either try to fix that in this patch, or we could take care of that as separate patches later.
>> 3) The style I chose was to add the .hpp include and a extra blank line, separating it from the rest of the includes. I think I like that style, because it makes it easy for those writing .inline.hpp to recognize this pattern. And it removes the confusion why this include isn't sorted into the other includes.
>> 4) We have a few *special* platform dependent header files. Both those that simply are included into platform independent files, and those that inject code *into* the platform independent classes. Extra care, as always, need to be taken around those files.
>> 5) Mostly tested locally, but I'll test on more platforms if the idea is accepted.
>> 
>> What do you think?
>
> Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Review kbarrett

Looks good.

-------------

Marked as reviewed by kbarrett (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/4127

From zgu at openjdk.java.net  Thu May 27 00:12:22 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 27 May 2021 00:12:22 GMT
Subject: RFR: Check explicitly for affiliation when updating references
In-Reply-To: <nrW9UiaxTXMC5hFN_6j8NaRR-hVmul6YzKJsBObxS2k=.7ad7dca5-41a7-4c6c-bf68-a917234c74d8@github.com>
References: <nrW9UiaxTXMC5hFN_6j8NaRR-hVmul6YzKJsBObxS2k=.7ad7dca5-41a7-4c6c-bf68-a917234c74d8@github.com>
Message-ID: <_ptWRXDwK8Hra-eFcmSA-Tp_5rat6pdbhcezsOX5zdI=.34ffa11c-3395-4af5-8d1e-72dbc9b10e22@github.com>

On Wed, 26 May 2021 20:37:21 GMT, William Kemper <wkemper at openjdk.org> wrote:

> It's possible for a mutator allocation to 'activate' a region before setting the affiliation, which would trip the assert here (or worse, miss the assert and then go off the rails).

Marked as reviewed by zgu (Committer).

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/47

From stefank at openjdk.java.net  Thu May 27 07:15:06 2021
From: stefank at openjdk.java.net (Stefan Karlsson)
Date: Thu, 27 May 2021 07:15:06 GMT
Subject: RFR: 8267464: Circular-dependency resilient inline headers [v6]
In-Reply-To: <-N-g2LtXiLzyl-1Z_61StqnhLz01OMEaH1rrUFgHv5c=.10a88ce2-4a54-4fcd-a1a4-6b8083e8df1f@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
 <-N-g2LtXiLzyl-1Z_61StqnhLz01OMEaH1rrUFgHv5c=.10a88ce2-4a54-4fcd-a1a4-6b8083e8df1f@github.com>
Message-ID: <vppezzVF1Dqm4_tYrVa-8ssZ5RFULMteG0dQIRoRhJ8=.cd8da840-13a6-436e-b1c6-07ffbe9eaff7@github.com>

On Wed, 26 May 2021 18:00:54 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> I'd like to propose a small adjustment to how we write .inline.hpp files, in the hope to reduce include problems because of circular dependencies between inline headers.
>> 
>> This is a small, contrived example to show the problem:
>> 
>> // a.hpp
>> #pragma once
>> 
>> void a1();
>> void a2();
>> 
>> 
>> // a.inline.hpp
>> #pragma once
>> 
>> #include "a.hpp"
>> #include "b.inline.hpp"
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> 
>> // b.hpp
>> #pragma once
>> 
>> void b1();
>> void b2();
>> 
>> 
>> // b.inline.hpp
>> #pragma once
>> 
>> #include "a.inline.hpp"
>> #include "b.hpp"
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> The following code compiles fine:
>> 
>> // a.cpp
>> #include "a.inline.hpp"
>> 
>> int main() {
>>   a1();
>> 
>>   return 0;
>> }
>> 
>> But the following:
>> 
>> // b.cpp
>> #include "b.inline.hpp"
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> 
>> fails with the this error message:
>> 
>> In file included from b.inline.hpp:3,
>>                  from b.cpp:1:
>> a.inline.hpp: In function ?void a1()?:
>> a.inline.hpp:8:3: error: ?b1? was not declared in this scope; did you mean ?a1??
>> 
>> We can see the problem with g++ -E:
>> 
>> # 1 "a.inline.hpp" 1
>> # 1 "a.hpp" 1
>> 
>> void a1();
>> void a2();
>> 
>> # 4 "a.inline.hpp" 2
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> # 4 "b.inline.hpp" 2
>> # 1 "b.hpp" 1
>> 
>> void b1();
>> void b2();
>> 
>> # 5 "b.inline.hpp" 2
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> # 2 "b.cpp" 2
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> 
>> b1() is called before it has been declared. b.inline.hpp inlined a.inline.hpp, which uses functions declared in b.hpp. However, at that point we've not included enough of b.inline.hpp to have declared b1().
>> 
>> This is a small and easy example. In the JVM the circular dependencies usually involves more than two files, and often from different sub-systems of the JVM. We have so-far solved this by restructuring the code, making functions out-of-line, creating proxy objects, etc. However, the more we use templates, the more this is going to become a reoccurring nuisance. And in experiments with lambdas, which requires templates, this very quickly showed up as a problem.
>> 
>> I propose that we solve most (all?) of these issues by adding a rule that states that .inline.hpp files should start by including the corresponding .hpp, and that the .hpp should contain the declarations of all externally used parts of the .inline.hpp file. This should guarantee that the declarations are always present before any subsequent include can create a circular include dependency back to the original file.
>> 
>> In the example above, b.inline.hpp would become:
>> 
>> // b.inline.hpp
>> #pragma once
>> 
>> #include "b.hpp"
>> #include "a.inline.hpp"
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> and now both a.cpp and b.cpp compiles. The generated output now looks like this:
>> 
>> # 1 "b.inline.hpp" 1
>> # 1 "b.hpp" 1
>> 
>> void b1();
>> void b2();
>> 
>> # 4 "b.inline.hpp" 2
>> # 1 "a.inline.hpp" 1
>> 
>> # 1 "a.hpp" 1
>> 
>> void a1();
>> void a2();
>> 
>> # 4 "a.inline.hpp" 2
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> # 5 "b.inline.hpp" 2
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> # 2 "b.cpp" 2
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> The declarations come first, and the compiler is happy. 
>> 
>> An alternative to this, that has been proposed by other HotSpot GC devs have been to always include all .hpp files first, and then have a section of .inline.hpp includes. I've experimented with that as well, but I think it requires more maintenance and vigilance of  *users* .inline.hpp files (add .hpp include to the first section, add .inline.hpp section to second). The proposed structures only requires extra care from *writers* of .inline.hpp files. All others can include .hpp and .inline.hpp as we've been doing for a long time now.
>> 
>> Some notes about this patch:
>> 1) Some externally-used declarations have been placed in .inline.hpp files, and not in the .hpp. Those should be moved. I have not done that.
>> 2) Some .inline.hpp files don't have a corresponding .hpp file. I could either try to fix that in this patch, or we could take care of that as separate patches later.
>> 3) The style I chose was to add the .hpp include and a extra blank line, separating it from the rest of the includes. I think I like that style, because it makes it easy for those writing .inline.hpp to recognize this pattern. And it removes the confusion why this include isn't sorted into the other includes.
>> 4) We have a few *special* platform dependent header files. Both those that simply are included into platform independent files, and those that inject code *into* the platform independent classes. Extra care, as always, need to be taken around those files.
>> 5) Mostly tested locally, but I'll test on more platforms if the idea is accepted.
>> 
>> What do you think?
>
> Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Review kbarrett

@vnkozlov Can we considered the proposed changes to HotSpot Coding Style sufficiently reviewed?

-------------

PR: https://git.openjdk.java.net/jdk/pull/4127

From zgu at redhat.com  Thu May 27 14:10:59 2021
From: zgu at redhat.com (Zhengyu Gu)
Date: Thu, 27 May 2021 10:10:59 -0400
Subject: [sh/jdk8u] Backport region count/region size fixes
Message-ID: <883e1358-b3be-3bf0-1f94-974c74fdc280@redhat.com>

I would like to backport following two CRs that fix region count/region 
size calculations.

8265239: Shenandoah: Shenandoah heap region count could be off by 1
8266802: Shenandoah: Round up region size to page size unconditionally

Both patch do not apply cleanly.

8265239:
   - Extra include "utilities/align.hpp"
   - Manually resolved minor merge conflict.

8266802:
   - Fixed new assertion. Cannot format assertion message in 8u.
   - Removed "@requires vm.gc.Shenandoah" annotation in new test, 8u does
     not recognize the annotation.


Test:
   hotspot_gc_shenandoah, including new test.

Thanks,

-Zhengyu


From gnu.andrew at redhat.com  Thu May 27 15:38:59 2021
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Thu, 27 May 2021 16:38:59 +0100
Subject: [RFR] [8u] 8u302-b01 Upstream Sync
In-Reply-To: <8be9efc9-f09f-1794-7cf4-8efaf2581247@redhat.com>
References: <20210526054039.GA1745963@rincewind>
 <8be9efc9-f09f-1794-7cf4-8efaf2581247@redhat.com>
Message-ID: <20210527153859.GA1788663@rincewind>

On 13:22 Wed 26 May     , Aleksey Shipilev wrote:
> On 5/26/21 7:40 AM, Andrew Hughes wrote:
> > Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/
> > 
> > Merge changesets:
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/corba/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/jaxp/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/jaxws/merge.changeset
> 
> Look trivially fine.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/jdk/merge.changeset
> 
> Looks fine.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/hotspot/merge.changeset
> 
> Looks fine.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/langtools/merge.changeset
> 
> Looks fine.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/nashorn/merge.changeset
> 
> Looks trivially fine.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b01/root/merge.changeset
> 
> Looks fine.
> 
> > Ok to push?
> 
> Yes.
> 
> -- 
> Thanks,
> -Aleksey
> 

Thanks. Pushed.
-- 
Andrew :)

Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222

From gnu.andrew at redhat.com  Thu May 27 16:02:05 2021
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Thu, 27 May 2021 17:02:05 +0100
Subject: [RFR] [8u] 8u302-b02 Upstream Sync
Message-ID: <20210527160205.GB1788663@rincewind>

Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b02/

Merge changesets:
http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b02/corba/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b02/jaxp/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b02/jaxws/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b02/jdk/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b02/hotspot/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b02/langtools/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b02/nashorn/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b02/root/merge.changeset

Changes in aarch64-shenandoah-jdk8u302-b02:
  - JDK-8034856: gcc warnings compiling src/solaris/native/sun/security/pkcs11
  - JDK-8034857: gcc warnings compiling src/solaris/native/sun/management
  - JDK-8035054: JarFacade.c should not include ctype.h
  - JDK-8129511: PlatformMidi.c:83 uses malloc without malloc header

Main issues of note:
None, clean merge (no HotSpot changes).

diffstat for root
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for corba
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jaxp
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jaxws
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for langtools
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for nashorn
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jdk
 b/.hgtags                                                      |    1 
 b/src/share/instrument/JarFacade.c                             |   12 +---------
 b/src/share/native/com/sun/media/sound/PlatformMidi.c          |    4 ++-
 b/src/share/native/sun/security/pkcs11/wrapper/pkcs11wrapper.h |    5 ++++
 b/src/solaris/native/sun/management/LinuxOperatingSystem.c     |    3 +-
 b/src/solaris/native/sun/security/pkcs11/j2secmod_md.c         |    1 
 b/src/solaris/native/sun/security/pkcs11/wrapper/p11_md.h      |    5 ++++
 7 files changed, 19 insertions(+), 12 deletions(-)

diffstat for hotspot
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

Successfully built on x86, x86_64, s390 (Zero), s390x (Zero), ppc (Zero),
ppc64, ppc64le & aarch64.

Ok to push?

Thanks,
-- 
Andrew :)

Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222

From wkemper at openjdk.java.net  Thu May 27 16:02:31 2021
From: wkemper at openjdk.java.net (William Kemper)
Date: Thu, 27 May 2021 16:02:31 GMT
Subject: Integrated: Check explicitly for affiliation when updating references
In-Reply-To: <nrW9UiaxTXMC5hFN_6j8NaRR-hVmul6YzKJsBObxS2k=.7ad7dca5-41a7-4c6c-bf68-a917234c74d8@github.com>
References: <nrW9UiaxTXMC5hFN_6j8NaRR-hVmul6YzKJsBObxS2k=.7ad7dca5-41a7-4c6c-bf68-a917234c74d8@github.com>
Message-ID: <aQa_lbO-2zMdB46DiMhRPoEYTE0Y2hoABJuIiAErfhc=.37590553-6757-4115-ad89-507d0751ae62@github.com>

On Wed, 26 May 2021 20:37:21 GMT, William Kemper <wkemper at openjdk.org> wrote:

> It's possible for a mutator allocation to 'activate' a region before setting the affiliation, which would trip the assert here (or worse, miss the assert and then go off the rails).

This pull request has now been integrated.

Changeset: 9011a4b6
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.java.net/shenandoah/commit/9011a4b60e68d7990a5309059d3dc165c4c1e774
Stats:     2 lines in 1 file changed: 0 ins; 1 del; 1 mod

Check explicitly for affiliation when updating references

Reviewed-by: zgu

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/47

From shade at redhat.com  Thu May 27 16:48:38 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Thu, 27 May 2021 18:48:38 +0200
Subject: [RFR] [8u] 8u302-b02 Upstream Sync
In-Reply-To: <20210527160205.GB1788663@rincewind>
References: <20210527160205.GB1788663@rincewind>
Message-ID: <4b71ccc8-a0f3-a040-e742-8a3351246a11@redhat.com>

On 5/27/21 6:02 PM, Andrew Hughes wrote:
> Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b02/
> 
> Merge changesets:
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b02/corba/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b02/jaxp/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b02/jaxws/merge.changeset

Look trivially fine.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b02/jdk/merge.changeset

Looks fine.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b02/hotspot/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b02/langtools/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b02/nashorn/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b02/root/merge.changeset

Look trivially fine.

> Ok to push?

Yes.

-- 
Thanks,
-Aleksey


From shade at redhat.com  Thu May 27 16:48:57 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Thu, 27 May 2021 18:48:57 +0200
Subject: [sh/jdk8u] Backport region count/region size fixes
In-Reply-To: <883e1358-b3be-3bf0-1f94-974c74fdc280@redhat.com>
References: <883e1358-b3be-3bf0-1f94-974c74fdc280@redhat.com>
Message-ID: <ccaa4f0b-b0c1-7996-a6ed-c029e7a975cd@redhat.com>

On 5/27/21 4:10 PM, Zhengyu Gu wrote:
> I would like to backport following two CRs that fix region count/region
> size calculations.
> 
> 8265239: Shenandoah: Shenandoah heap region count could be off by 1
> 8266802: Shenandoah: Round up region size to page size unconditionally
> 
> Both patch do not apply cleanly.

OK, please show the webrevs then?

-- 
Thanks,
-Aleksey


From zgu at redhat.com  Thu May 27 16:52:27 2021
From: zgu at redhat.com (Zhengyu Gu)
Date: Thu, 27 May 2021 12:52:27 -0400
Subject: [sh/jdk8u] Backport region count/region size fixes
In-Reply-To: <ccaa4f0b-b0c1-7996-a6ed-c029e7a975cd@redhat.com>
References: <883e1358-b3be-3bf0-1f94-974c74fdc280@redhat.com>
 <ccaa4f0b-b0c1-7996-a6ed-c029e7a975cd@redhat.com>
Message-ID: <d238f405-b60d-bac6-3d02-b8586081bb0e@redhat.com>

Ah, forgot it.

http://cr.openjdk.java.net/~zgu/shenandoah/sh-region-count/webrev.00/

Thanks,

-Zhengyu

On 5/27/21 12:48 PM, Aleksey Shipilev wrote:
> On 5/27/21 4:10 PM, Zhengyu Gu wrote:
>> I would like to backport following two CRs that fix region count/region
>> size calculations.
>>
>> 8265239: Shenandoah: Shenandoah heap region count could be off by 1
>> 8266802: Shenandoah: Round up region size to page size unconditionally
>>
>> Both patch do not apply cleanly.
> 
> OK, please show the webrevs then?
> 


From shade at redhat.com  Thu May 27 17:04:01 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Thu, 27 May 2021 19:04:01 +0200
Subject: [sh/jdk8u] Backport region count/region size fixes
In-Reply-To: <d238f405-b60d-bac6-3d02-b8586081bb0e@redhat.com>
References: <883e1358-b3be-3bf0-1f94-974c74fdc280@redhat.com>
 <ccaa4f0b-b0c1-7996-a6ed-c029e7a975cd@redhat.com>
 <d238f405-b60d-bac6-3d02-b8586081bb0e@redhat.com>
Message-ID: <dd3ae024-7d12-8c39-374e-1020de4c244a@redhat.com>

On 5/27/21 6:52 PM, Zhengyu Gu wrote:
> Ah, forgot it.
> 
> http://cr.openjdk.java.net/~zgu/shenandoah/sh-region-count/webrev.00/

Looks fine. You can restore the assert message with err_msg.


-- 
Thanks,
-Aleksey


From shade at redhat.com  Thu May 27 17:03:45 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Thu, 27 May 2021 19:03:45 +0200
Subject: [sh/jdk8u] Backport region count/region size fixes
In-Reply-To: <883e1358-b3be-3bf0-1f94-974c74fdc280@redhat.com>
References: <883e1358-b3be-3bf0-1f94-974c74fdc280@redhat.com>
Message-ID: <55cd859f-3695-99f9-e173-accacd5be3db@redhat.com>

On 5/27/21 4:10 PM, Zhengyu Gu wrote:
> 8266802:
>     - Fixed new assertion. Cannot format assertion message in 8u.

Oh, yes you can. Use err_msg, like:

  assert(..., err_msg("%s...", ))


-- 
Thanks,
-Aleksey


From shade at redhat.com  Thu May 27 17:19:43 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Thu, 27 May 2021 19:19:43 +0200
Subject: [sh/jdk8u] Backport region count/region size fixes
In-Reply-To: <d76ee1d9-1021-98cf-5a0d-528d5b0828f5@redhat.com>
References: <883e1358-b3be-3bf0-1f94-974c74fdc280@redhat.com>
 <ccaa4f0b-b0c1-7996-a6ed-c029e7a975cd@redhat.com>
 <d238f405-b60d-bac6-3d02-b8586081bb0e@redhat.com>
 <dd3ae024-7d12-8c39-374e-1020de4c244a@redhat.com>
 <d76ee1d9-1021-98cf-5a0d-528d5b0828f5@redhat.com>
Message-ID: <211f04e4-9346-511d-eb00-6990dfe77063@redhat.com>

On 5/27/21 7:18 PM, Zhengyu Gu wrote:
> Updated:
> http://cr.openjdk.java.net/~zgu/shenandoah/sh-region-count/webrev.01/

OK, cool, looks good!

-- 
Thanks,
-Aleksey


From zgu at redhat.com  Thu May 27 17:24:05 2021
From: zgu at redhat.com (zgu at redhat.com)
Date: Thu, 27 May 2021 17:24:05 +0000
Subject: hg: shenandoah/jdk8/hotspot: [backport] 8265239: Shenandoah:
 Shenandoah heap region count could be off by 1
Message-ID: <202105271724.14RHO51L008923@aojmv0008.oracle.com>

Changeset: 52756aa9369d
Author:    zgu
Date:      2021-04-16 12:27 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/52756aa9369d

[backport] 8265239: Shenandoah: Shenandoah heap region count could be off by 1
Reviewed-by: shade

! src/share/vm/gc_implementation/shenandoah/shenandoahHeap.cpp
! src/share/vm/gc_implementation/shenandoah/shenandoahHeapRegion.cpp


From zgu at redhat.com  Thu May 27 17:24:22 2021
From: zgu at redhat.com (zgu at redhat.com)
Date: Thu, 27 May 2021 17:24:22 +0000
Subject: hg: shenandoah/jdk8/hotspot: [backport] 8266802: Shenandoah: Round up
 region size to page size unconditionally
Message-ID: <202105271724.14RHOM5B009739@aojmv0008.oracle.com>

Changeset: fd89a265e9d3
Author:    shade
Date:      2021-05-10 20:28 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/fd89a265e9d3

[backport] 8266802: Shenandoah: Round up region size to page size unconditionally
Reviewed-by: zgu

! src/share/vm/gc_implementation/shenandoah/shenandoahHeap.cpp
! src/share/vm/gc_implementation/shenandoah/shenandoahHeapRegion.cpp
+ test/gc/shenandoah/options/TestLargePages.java


From zgu at openjdk.java.net  Thu May 27 18:36:15 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 27 May 2021 18:36:15 GMT
Subject: RFR: 8267875: Shenandoah: Duplicated code in
 ShenandoahBarrierSetC2::ideal_node()
Message-ID: <dQJS9gVpm55H7B0E6JhsNoWU8MuxbDngHQA9--xEZfE=.91ffd03f-9bdc-4e56-8dda-c3a1843f7f5f@github.com>

Node::set_req_X definition as:
void Node::set_req_X(uint i, Node *n, PhaseGVN *gvn) {
  PhaseIterGVN* igvn = gvn->is_IterGVN();
  if (igvn == NULL) {
    set_req(i, n);
    return;
  }
  set_req_X(i, n, igvn);
}

ShenandoahBarrierSetC2::ideal_node() duplicates above logic.


Test:
  hotspot_gc_shenandoah

-------------

Commit messages:
 - Merge branch 'master' into sh-c2-ideal-simplify
 - v1

Changes: https://git.openjdk.java.net/jdk/pull/4229/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4229&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8267875
  Stats: 11 lines in 1 file changed: 0 ins; 9 del; 2 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4229.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4229/head:pull/4229

PR: https://git.openjdk.java.net/jdk/pull/4229

From zgu at redhat.com  Thu May 27 17:18:36 2021
From: zgu at redhat.com (Zhengyu Gu)
Date: Thu, 27 May 2021 13:18:36 -0400
Subject: [sh/jdk8u] Backport region count/region size fixes
In-Reply-To: <dd3ae024-7d12-8c39-374e-1020de4c244a@redhat.com>
References: <883e1358-b3be-3bf0-1f94-974c74fdc280@redhat.com>
 <ccaa4f0b-b0c1-7996-a6ed-c029e7a975cd@redhat.com>
 <d238f405-b60d-bac6-3d02-b8586081bb0e@redhat.com>
 <dd3ae024-7d12-8c39-374e-1020de4c244a@redhat.com>
Message-ID: <d76ee1d9-1021-98cf-5a0d-528d5b0828f5@redhat.com>

Updated: 
http://cr.openjdk.java.net/~zgu/shenandoah/sh-region-count/webrev.01/

Thanks,

-Zhengyu

On 5/27/21 1:04 PM, Aleksey Shipilev wrote:
> On 5/27/21 6:52 PM, Zhengyu Gu wrote:
>> Ah, forgot it.
>>
>> http://cr.openjdk.java.net/~zgu/shenandoah/sh-region-count/webrev.00/
> 
> Looks fine. You can restore the assert message with err_msg.
> 
> 


From stefank at openjdk.java.net  Fri May 28 07:24:37 2021
From: stefank at openjdk.java.net (Stefan Karlsson)
Date: Fri, 28 May 2021 07:24:37 GMT
Subject: RFR: 8267464: Circular-dependency resilient inline headers [v7]
In-Reply-To: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
Message-ID: <TWlRdyfYQGw3V7VvZj79_r7HZBH_xpdIas1WlgIR2Gw=.52b9787a-4d85-4109-96ab-b18ab9c61146@github.com>

> I'd like to propose a small adjustment to how we write .inline.hpp files, in the hope to reduce include problems because of circular dependencies between inline headers.
> 
> This is a small, contrived example to show the problem:
> 
> // a.hpp
> #pragma once
> 
> void a1();
> void a2();
> 
> 
> // a.inline.hpp
> #pragma once
> 
> #include "a.hpp"
> #include "b.inline.hpp"
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> 
> // b.hpp
> #pragma once
> 
> void b1();
> void b2();
> 
> 
> // b.inline.hpp
> #pragma once
> 
> #include "a.inline.hpp"
> #include "b.hpp"
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> The following code compiles fine:
> 
> // a.cpp
> #include "a.inline.hpp"
> 
> int main() {
>   a1();
> 
>   return 0;
> }
> 
> But the following:
> 
> // b.cpp
> #include "b.inline.hpp"
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> 
> fails with the this error message:
> 
> In file included from b.inline.hpp:3,
>                  from b.cpp:1:
> a.inline.hpp: In function ?void a1()?:
> a.inline.hpp:8:3: error: ?b1? was not declared in this scope; did you mean ?a1??
> 
> We can see the problem with g++ -E:
> 
> # 1 "a.inline.hpp" 1
> # 1 "a.hpp" 1
> 
> void a1();
> void a2();
> 
> # 4 "a.inline.hpp" 2
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> # 4 "b.inline.hpp" 2
> # 1 "b.hpp" 1
> 
> void b1();
> void b2();
> 
> # 5 "b.inline.hpp" 2
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> # 2 "b.cpp" 2
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> 
> b1() is called before it has been declared. b.inline.hpp inlined a.inline.hpp, which uses functions declared in b.hpp. However, at that point we've not included enough of b.inline.hpp to have declared b1().
> 
> This is a small and easy example. In the JVM the circular dependencies usually involves more than two files, and often from different sub-systems of the JVM. We have so-far solved this by restructuring the code, making functions out-of-line, creating proxy objects, etc. However, the more we use templates, the more this is going to become a reoccurring nuisance. And in experiments with lambdas, which requires templates, this very quickly showed up as a problem.
> 
> I propose that we solve most (all?) of these issues by adding a rule that states that .inline.hpp files should start by including the corresponding .hpp, and that the .hpp should contain the declarations of all externally used parts of the .inline.hpp file. This should guarantee that the declarations are always present before any subsequent include can create a circular include dependency back to the original file.
> 
> In the example above, b.inline.hpp would become:
> 
> // b.inline.hpp
> #pragma once
> 
> #include "b.hpp"
> #include "a.inline.hpp"
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> and now both a.cpp and b.cpp compiles. The generated output now looks like this:
> 
> # 1 "b.inline.hpp" 1
> # 1 "b.hpp" 1
> 
> void b1();
> void b2();
> 
> # 4 "b.inline.hpp" 2
> # 1 "a.inline.hpp" 1
> 
> # 1 "a.hpp" 1
> 
> void a1();
> void a2();
> 
> # 4 "a.inline.hpp" 2
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> # 5 "b.inline.hpp" 2
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> # 2 "b.cpp" 2
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> The declarations come first, and the compiler is happy. 
> 
> An alternative to this, that has been proposed by other HotSpot GC devs have been to always include all .hpp files first, and then have a section of .inline.hpp includes. I've experimented with that as well, but I think it requires more maintenance and vigilance of  *users* .inline.hpp files (add .hpp include to the first section, add .inline.hpp section to second). The proposed structures only requires extra care from *writers* of .inline.hpp files. All others can include .hpp and .inline.hpp as we've been doing for a long time now.
> 
> Some notes about this patch:
> 1) Some externally-used declarations have been placed in .inline.hpp files, and not in the .hpp. Those should be moved. I have not done that.
> 2) Some .inline.hpp files don't have a corresponding .hpp file. I could either try to fix that in this patch, or we could take care of that as separate patches later.
> 3) The style I chose was to add the .hpp include and a extra blank line, separating it from the rest of the includes. I think I like that style, because it makes it easy for those writing .inline.hpp to recognize this pattern. And it removes the confusion why this include isn't sorted into the other includes.
> 4) We have a few *special* platform dependent header files. Both those that simply are included into platform independent files, and those that inject code *into* the platform independent classes. Extra care, as always, need to be taken around those files.
> 5) Mostly tested locally, but I'll test on more platforms if the idea is accepted.
> 
> What do you think?

Stefan Karlsson has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits:

 - Merge remote-tracking branch 'origin/master' into 8267464_circular-dependency_resilient_inline_headers
 - Review kbarrett
 - Review dholmes
 - Update HotSpot style guide documents
 - Merge remote-tracking branch 'origin/master' into 8267464_circular-dependency_resilient_inline_headers
 - Clean up assembler_<cpu>.inline.hpp
 - 8267464: Circular-dependency resiliant inline headers

-------------

Changes: https://git.openjdk.java.net/jdk/pull/4127/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4127&range=06
  Stats: 490 lines in 242 files changed: 349 ins; 118 del; 23 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4127.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4127/head:pull/4127

PR: https://git.openjdk.java.net/jdk/pull/4127

From neliasso at openjdk.java.net  Fri May 28 09:16:25 2021
From: neliasso at openjdk.java.net (Nils Eliasson)
Date: Fri, 28 May 2021 09:16:25 GMT
Subject: RFR: 8267726: ZGC: array_copy_requires_gc_barriers too strict
Message-ID: <t-qEjX3CnUyo5ZNfHNJ41SQA46qlOmaZ5Y9YGMC0baA=.8335064b-1e9e-4cd1-962d-04b815ba330a@github.com>

I found some cases where an arraycopy clone is eliminated with G1 but not with ZGC. This is probably something that wasn't updated fully after the transition to late gc barrier insertion.

During parse and optimizaton phases array_copy_requires_gc_barriers should return false for clones of oop-arrays. Clone of oop-arrays should be treated the same way as clones of primitive-arrays. During optimization phase - only clones of instances should return true - and that's because they can't be reduced to a raw bulk copy, Clones of instances must either become deconstructed into field copies, or be handled in a special call.

During expansion array_copy_requires_gc_barriers must return true - because we must use a copy with barriers.

To fix this I had to add an extra field to array_copy_requires_gc_barriers to be able to handle instance clones separately. I will follow up with a cleanup. The intersection of arraycopy-kinds and array_copy_requires_gc_barriers-method is the source of much unnecessary complexity. 

Please review,
Best regards,
Nils Eliasson

-------------

Commit messages:
 - Fix array_copy_requires_gc_barriers
 - adapt opt
 - Fix barrier requirement

Changes: https://git.openjdk.java.net/jdk/pull/4230/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4230&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8267726
  Stats: 21 lines in 10 files changed: 9 ins; 0 del; 12 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4230.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4230/head:pull/4230

PR: https://git.openjdk.java.net/jdk/pull/4230

From neliasso at openjdk.java.net  Fri May 28 10:02:06 2021
From: neliasso at openjdk.java.net (Nils Eliasson)
Date: Fri, 28 May 2021 10:02:06 GMT
Subject: RFR: 8267726: ZGC: array_copy_requires_gc_barriers too strict
In-Reply-To: <t-qEjX3CnUyo5ZNfHNJ41SQA46qlOmaZ5Y9YGMC0baA=.8335064b-1e9e-4cd1-962d-04b815ba330a@github.com>
References: <t-qEjX3CnUyo5ZNfHNJ41SQA46qlOmaZ5Y9YGMC0baA=.8335064b-1e9e-4cd1-962d-04b815ba330a@github.com>
Message-ID: <0uD1qfMdOxceleS5uPQQNMFzwTQijCJ9p7EJnvg7bxw=.31e2eded-a7f6-40b3-b6e3-5f8f8755cc2c@github.com>

On Thu, 27 May 2021 19:59:59 GMT, Nils Eliasson <neliasso at openjdk.org> wrote:

> I found some cases where an arraycopy clone is eliminated with G1 but not with ZGC. This is probably something that wasn't updated fully after the transition to late gc barrier insertion.
> 
> During parse and optimizaton phases array_copy_requires_gc_barriers should return false for clones of oop-arrays. Clone of oop-arrays should be treated the same way as clones of primitive-arrays. During optimization phase - only clones of instances should return true - and that's because they can't be reduced to a raw bulk copy, Clones of instances must either become deconstructed into field copies, or be handled in a special call.
> 
> During expansion array_copy_requires_gc_barriers must return true - because we must use a copy with barriers.
> 
> To fix this I had to add an extra field to array_copy_requires_gc_barriers to be able to handle instance clones separately. I will follow up with a cleanup. The intersection of arraycopy-kinds and array_copy_requires_gc_barriers-method is the source of much unnecessary complexity. 
> 
> Please review,
> Best regards,
> Nils Eliasson

Moved to hotspot-compiler

-------------

PR: https://git.openjdk.java.net/jdk/pull/4230

From kvn at openjdk.java.net  Fri May 28 18:17:57 2021
From: kvn at openjdk.java.net (Vladimir Kozlov)
Date: Fri, 28 May 2021 18:17:57 GMT
Subject: RFR: 8267464: Circular-dependency resilient inline headers [v7]
In-Reply-To: <TWlRdyfYQGw3V7VvZj79_r7HZBH_xpdIas1WlgIR2Gw=.52b9787a-4d85-4109-96ab-b18ab9c61146@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
 <TWlRdyfYQGw3V7VvZj79_r7HZBH_xpdIas1WlgIR2Gw=.52b9787a-4d85-4109-96ab-b18ab9c61146@github.com>
Message-ID: <ZzNXCG-_aGi5K69VSWVlUcjAU3piGMfgV8Id67FoSCs=.bdc435f4-075e-4657-bc8e-0fa3b5cab22a@github.com>

On Fri, 28 May 2021 07:24:37 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> I'd like to propose a small adjustment to how we write .inline.hpp files, in the hope to reduce include problems because of circular dependencies between inline headers.
>> 
>> This is a small, contrived example to show the problem:
>> 
>> // a.hpp
>> #pragma once
>> 
>> void a1();
>> void a2();
>> 
>> 
>> // a.inline.hpp
>> #pragma once
>> 
>> #include "a.hpp"
>> #include "b.inline.hpp"
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> 
>> // b.hpp
>> #pragma once
>> 
>> void b1();
>> void b2();
>> 
>> 
>> // b.inline.hpp
>> #pragma once
>> 
>> #include "a.inline.hpp"
>> #include "b.hpp"
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> The following code compiles fine:
>> 
>> // a.cpp
>> #include "a.inline.hpp"
>> 
>> int main() {
>>   a1();
>> 
>>   return 0;
>> }
>> 
>> But the following:
>> 
>> // b.cpp
>> #include "b.inline.hpp"
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> 
>> fails with the this error message:
>> 
>> In file included from b.inline.hpp:3,
>>                  from b.cpp:1:
>> a.inline.hpp: In function ?void a1()?:
>> a.inline.hpp:8:3: error: ?b1? was not declared in this scope; did you mean ?a1??
>> 
>> We can see the problem with g++ -E:
>> 
>> # 1 "a.inline.hpp" 1
>> # 1 "a.hpp" 1
>> 
>> void a1();
>> void a2();
>> 
>> # 4 "a.inline.hpp" 2
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> # 4 "b.inline.hpp" 2
>> # 1 "b.hpp" 1
>> 
>> void b1();
>> void b2();
>> 
>> # 5 "b.inline.hpp" 2
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> # 2 "b.cpp" 2
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> 
>> b1() is called before it has been declared. b.inline.hpp inlined a.inline.hpp, which uses functions declared in b.hpp. However, at that point we've not included enough of b.inline.hpp to have declared b1().
>> 
>> This is a small and easy example. In the JVM the circular dependencies usually involves more than two files, and often from different sub-systems of the JVM. We have so-far solved this by restructuring the code, making functions out-of-line, creating proxy objects, etc. However, the more we use templates, the more this is going to become a reoccurring nuisance. And in experiments with lambdas, which requires templates, this very quickly showed up as a problem.
>> 
>> I propose that we solve most (all?) of these issues by adding a rule that states that .inline.hpp files should start by including the corresponding .hpp, and that the .hpp should contain the declarations of all externally used parts of the .inline.hpp file. This should guarantee that the declarations are always present before any subsequent include can create a circular include dependency back to the original file.
>> 
>> In the example above, b.inline.hpp would become:
>> 
>> // b.inline.hpp
>> #pragma once
>> 
>> #include "b.hpp"
>> #include "a.inline.hpp"
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> and now both a.cpp and b.cpp compiles. The generated output now looks like this:
>> 
>> # 1 "b.inline.hpp" 1
>> # 1 "b.hpp" 1
>> 
>> void b1();
>> void b2();
>> 
>> # 4 "b.inline.hpp" 2
>> # 1 "a.inline.hpp" 1
>> 
>> # 1 "a.hpp" 1
>> 
>> void a1();
>> void a2();
>> 
>> # 4 "a.inline.hpp" 2
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> # 5 "b.inline.hpp" 2
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> # 2 "b.cpp" 2
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> The declarations come first, and the compiler is happy. 
>> 
>> An alternative to this, that has been proposed by other HotSpot GC devs have been to always include all .hpp files first, and then have a section of .inline.hpp includes. I've experimented with that as well, but I think it requires more maintenance and vigilance of  *users* .inline.hpp files (add .hpp include to the first section, add .inline.hpp section to second). The proposed structures only requires extra care from *writers* of .inline.hpp files. All others can include .hpp and .inline.hpp as we've been doing for a long time now.
>> 
>> Some notes about this patch:
>> 1) Some externally-used declarations have been placed in .inline.hpp files, and not in the .hpp. Those should be moved. I have not done that.
>> 2) Some .inline.hpp files don't have a corresponding .hpp file. I could either try to fix that in this patch, or we could take care of that as separate patches later.
>> 3) The style I chose was to add the .hpp include and a extra blank line, separating it from the rest of the includes. I think I like that style, because it makes it easy for those writing .inline.hpp to recognize this pattern. And it removes the confusion why this include isn't sorted into the other includes.
>> 4) We have a few *special* platform dependent header files. Both those that simply are included into platform independent files, and those that inject code *into* the platform independent classes. Extra care, as always, need to be taken around those files.
>> 5) Mostly tested locally, but I'll test on more platforms if the idea is accepted.
>> 
>> What do you think?
>
> Stefan Karlsson has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits:
> 
>  - Merge remote-tracking branch 'origin/master' into 8267464_circular-dependency_resilient_inline_headers
>  - Review kbarrett
>  - Review dholmes
>  - Update HotSpot style guide documents
>  - Merge remote-tracking branch 'origin/master' into 8267464_circular-dependency_resilient_inline_headers
>  - Clean up assembler_<cpu>.inline.hpp
>  - 8267464: Circular-dependency resiliant inline headers

Good.

-------------

Marked as reviewed by kvn (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/4127

From robilad at openjdk.java.net  Fri May 28 23:58:56 2021
From: robilad at openjdk.java.net (Dalibor Topic)
Date: Fri, 28 May 2021 23:58:56 GMT
Subject: RFR: Young refproc test
In-Reply-To: <i1Ut07L-v1U4vwWgr7EDrCXV07dsYIt2FYugnUu_9qs=.f869cf09-f41f-4081-bfdb-db95fd4884fc@github.com>
References: <i1Ut07L-v1U4vwWgr7EDrCXV07dsYIt2FYugnUu_9qs=.f869cf09-f41f-4081-bfdb-db95fd4884fc@github.com>
Message-ID: <wMmOR3e2dIXooQUsq0ih1tlk_hOAi0XOkrEwqrZSYLw=.795ac45e-cdf3-44fd-ad9a-7c744ac116cd@github.com>

On Thu, 20 May 2021 20:31:38 GMT, wadehenn <github.com+83784956+wadehenn at openjdk.org> wrote:

> Young refproc test

Hi,

Please send an e-mail to Dalibor.Topic at oracle.com so that Incan mark your account as verified.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/42

From github.com+83784956+wadehenn at openjdk.java.net  Fri May 28 23:58:56 2021
From: github.com+83784956+wadehenn at openjdk.java.net (wadehenn)
Date: Fri, 28 May 2021 23:58:56 GMT
Subject: RFR: Young refproc test
Message-ID: <i1Ut07L-v1U4vwWgr7EDrCXV07dsYIt2FYugnUu_9qs=.f869cf09-f41f-4081-bfdb-db95fd4884fc@github.com>

Young refproc test

-------------

Commit messages:
 - fix white space
 - remove duplicate young gen refproc test
 - Add young gen test to TestRefprocSanity.java

Changes: https://git.openjdk.java.net/shenandoah/pull/42/files
 Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=42&range=00
  Stats: 13 lines in 1 file changed: 11 ins; 0 del; 2 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/42.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/42/head:pull/42

PR: https://git.openjdk.java.net/shenandoah/pull/42

From github.com+83784956+wadehenn at openjdk.java.net  Fri May 28 23:58:57 2021
From: github.com+83784956+wadehenn at openjdk.java.net (wadehenn)
Date: Fri, 28 May 2021 23:58:57 GMT
Subject: RFR: Young refproc test
In-Reply-To: <i1Ut07L-v1U4vwWgr7EDrCXV07dsYIt2FYugnUu_9qs=.f869cf09-f41f-4081-bfdb-db95fd4884fc@github.com>
References: <i1Ut07L-v1U4vwWgr7EDrCXV07dsYIt2FYugnUu_9qs=.f869cf09-f41f-4081-bfdb-db95fd4884fc@github.com>
Message-ID: <DM1SR6lt5gFmxUUkATuASND7XBW4f9GajqprVgfAgfM=.e54a1f35-985a-4cb3-96ed-9335679822d5@github.com>

On Thu, 20 May 2021 20:31:38 GMT, wadehenn <github.com+83784956+wadehenn at openjdk.org> wrote:

> Young refproc test

I need to have my account verified. -wade

From: Dalibor Topic ***@***.***>
Sent: Monday, May 24, 2021 6:44 AM
To: openjdk/shenandoah ***@***.***>
Cc: Hennessey, Wade ***@***.***>; Mention ***@***.***>
Subject: Re: [openjdk/shenandoah] Young refproc test (#42)


Hi,

Please send an e-mail to ***@***.******@***.***> so that Incan mark your account as verified.

?
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<https://github.com/openjdk/shenandoah/pull/42#issuecomment-847051618>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AT7HJ7F5JSXSYW6DROM75W3TPJJYHANCNFSM45H2KXPA>.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/42

From github.com+83784956+wadehenn at openjdk.java.net  Sat May 29 00:02:42 2021
From: github.com+83784956+wadehenn at openjdk.java.net (wadehenn)
Date: Sat, 29 May 2021 00:02:42 GMT
Subject: RFR: Young refproc test
In-Reply-To: <i1Ut07L-v1U4vwWgr7EDrCXV07dsYIt2FYugnUu_9qs=.f869cf09-f41f-4081-bfdb-db95fd4884fc@github.com>
References: <i1Ut07L-v1U4vwWgr7EDrCXV07dsYIt2FYugnUu_9qs=.f869cf09-f41f-4081-bfdb-db95fd4884fc@github.com>
Message-ID: <PbwzFgONhX832y6DILXGoxRwfFSRQbs2-NWBQQdq_5M=.dcd3bae9-10cc-4019-8bec-c39144f11a03@github.com>

On Thu, 20 May 2021 20:31:38 GMT, wadehenn <github.com+83784956+wadehenn at openjdk.org> wrote:

> Young refproc test

This is my first pull request

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/42

From stefank at openjdk.java.net  Mon May 31 06:44:45 2021
From: stefank at openjdk.java.net (Stefan Karlsson)
Date: Mon, 31 May 2021 06:44:45 GMT
Subject: RFR: 8267464: Circular-dependency resilient inline headers [v8]
In-Reply-To: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
Message-ID: <h2kGJHrmJJW7MWGbxj2OBWvKfOpU3xBrLhkF3rp0ABA=.d5b055ef-8ee5-49e4-9164-2ffc26d3d2aa@github.com>

> I'd like to propose a small adjustment to how we write .inline.hpp files, in the hope to reduce include problems because of circular dependencies between inline headers.
> 
> This is a small, contrived example to show the problem:
> 
> // a.hpp
> #pragma once
> 
> void a1();
> void a2();
> 
> 
> // a.inline.hpp
> #pragma once
> 
> #include "a.hpp"
> #include "b.inline.hpp"
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> 
> // b.hpp
> #pragma once
> 
> void b1();
> void b2();
> 
> 
> // b.inline.hpp
> #pragma once
> 
> #include "a.inline.hpp"
> #include "b.hpp"
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> The following code compiles fine:
> 
> // a.cpp
> #include "a.inline.hpp"
> 
> int main() {
>   a1();
> 
>   return 0;
> }
> 
> But the following:
> 
> // b.cpp
> #include "b.inline.hpp"
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> 
> fails with the this error message:
> 
> In file included from b.inline.hpp:3,
>                  from b.cpp:1:
> a.inline.hpp: In function ?void a1()?:
> a.inline.hpp:8:3: error: ?b1? was not declared in this scope; did you mean ?a1??
> 
> We can see the problem with g++ -E:
> 
> # 1 "a.inline.hpp" 1
> # 1 "a.hpp" 1
> 
> void a1();
> void a2();
> 
> # 4 "a.inline.hpp" 2
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> # 4 "b.inline.hpp" 2
> # 1 "b.hpp" 1
> 
> void b1();
> void b2();
> 
> # 5 "b.inline.hpp" 2
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> # 2 "b.cpp" 2
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> 
> b1() is called before it has been declared. b.inline.hpp inlined a.inline.hpp, which uses functions declared in b.hpp. However, at that point we've not included enough of b.inline.hpp to have declared b1().
> 
> This is a small and easy example. In the JVM the circular dependencies usually involves more than two files, and often from different sub-systems of the JVM. We have so-far solved this by restructuring the code, making functions out-of-line, creating proxy objects, etc. However, the more we use templates, the more this is going to become a reoccurring nuisance. And in experiments with lambdas, which requires templates, this very quickly showed up as a problem.
> 
> I propose that we solve most (all?) of these issues by adding a rule that states that .inline.hpp files should start by including the corresponding .hpp, and that the .hpp should contain the declarations of all externally used parts of the .inline.hpp file. This should guarantee that the declarations are always present before any subsequent include can create a circular include dependency back to the original file.
> 
> In the example above, b.inline.hpp would become:
> 
> // b.inline.hpp
> #pragma once
> 
> #include "b.hpp"
> #include "a.inline.hpp"
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> and now both a.cpp and b.cpp compiles. The generated output now looks like this:
> 
> # 1 "b.inline.hpp" 1
> # 1 "b.hpp" 1
> 
> void b1();
> void b2();
> 
> # 4 "b.inline.hpp" 2
> # 1 "a.inline.hpp" 1
> 
> # 1 "a.hpp" 1
> 
> void a1();
> void a2();
> 
> # 4 "a.inline.hpp" 2
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> # 5 "b.inline.hpp" 2
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> # 2 "b.cpp" 2
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> The declarations come first, and the compiler is happy. 
> 
> An alternative to this, that has been proposed by other HotSpot GC devs have been to always include all .hpp files first, and then have a section of .inline.hpp includes. I've experimented with that as well, but I think it requires more maintenance and vigilance of  *users* .inline.hpp files (add .hpp include to the first section, add .inline.hpp section to second). The proposed structures only requires extra care from *writers* of .inline.hpp files. All others can include .hpp and .inline.hpp as we've been doing for a long time now.
> 
> Some notes about this patch:
> 1) Some externally-used declarations have been placed in .inline.hpp files, and not in the .hpp. Those should be moved. I have not done that.
> 2) Some .inline.hpp files don't have a corresponding .hpp file. I could either try to fix that in this patch, or we could take care of that as separate patches later.
> 3) The style I chose was to add the .hpp include and a extra blank line, separating it from the rest of the includes. I think I like that style, because it makes it easy for those writing .inline.hpp to recognize this pattern. And it removes the confusion why this include isn't sorted into the other includes.
> 4) We have a few *special* platform dependent header files. Both those that simply are included into platform independent files, and those that inject code *into* the platform independent classes. Extra care, as always, need to be taken around those files.
> 5) Mostly tested locally, but I'll test on more platforms if the idea is accepted.
> 
> What do you think?

Stefan Karlsson has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits:

 - Merge remote-tracking branch 'origin/master' into 8267464_circular-dependency_resilient_inline_headers
 - Merge remote-tracking branch 'origin/master' into 8267464_circular-dependency_resilient_inline_headers
 - Review kbarrett
 - Review dholmes
 - Update HotSpot style guide documents
 - Merge remote-tracking branch 'origin/master' into 8267464_circular-dependency_resilient_inline_headers
 - Clean up assembler_<cpu>.inline.hpp
 - 8267464: Circular-dependency resiliant inline headers

-------------

Changes: https://git.openjdk.java.net/jdk/pull/4127/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4127&range=07
  Stats: 490 lines in 242 files changed: 349 ins; 118 del; 23 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4127.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4127/head:pull/4127

PR: https://git.openjdk.java.net/jdk/pull/4127

From stefank at openjdk.java.net  Mon May 31 09:02:31 2021
From: stefank at openjdk.java.net (Stefan Karlsson)
Date: Mon, 31 May 2021 09:02:31 GMT
Subject: RFR: 8267464: Circular-dependency resilient inline headers [v8]
In-Reply-To: <h2kGJHrmJJW7MWGbxj2OBWvKfOpU3xBrLhkF3rp0ABA=.d5b055ef-8ee5-49e4-9164-2ffc26d3d2aa@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
 <h2kGJHrmJJW7MWGbxj2OBWvKfOpU3xBrLhkF3rp0ABA=.d5b055ef-8ee5-49e4-9164-2ffc26d3d2aa@github.com>
Message-ID: <ARWfm7kOPHXGZRJRYsnnJNPFSMMSlCV4Mj85tE9FpTI=.f5a7a6e3-ad08-46e5-8fa8-dbc318a2d24e@github.com>

On Mon, 31 May 2021 06:44:45 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> I'd like to propose a small adjustment to how we write .inline.hpp files, in the hope to reduce include problems because of circular dependencies between inline headers.
>> 
>> This is a small, contrived example to show the problem:
>> 
>> // a.hpp
>> #pragma once
>> 
>> void a1();
>> void a2();
>> 
>> 
>> // a.inline.hpp
>> #pragma once
>> 
>> #include "a.hpp"
>> #include "b.inline.hpp"
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> 
>> // b.hpp
>> #pragma once
>> 
>> void b1();
>> void b2();
>> 
>> 
>> // b.inline.hpp
>> #pragma once
>> 
>> #include "a.inline.hpp"
>> #include "b.hpp"
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> The following code compiles fine:
>> 
>> // a.cpp
>> #include "a.inline.hpp"
>> 
>> int main() {
>>   a1();
>> 
>>   return 0;
>> }
>> 
>> But the following:
>> 
>> // b.cpp
>> #include "b.inline.hpp"
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> 
>> fails with the this error message:
>> 
>> In file included from b.inline.hpp:3,
>>                  from b.cpp:1:
>> a.inline.hpp: In function ?void a1()?:
>> a.inline.hpp:8:3: error: ?b1? was not declared in this scope; did you mean ?a1??
>> 
>> We can see the problem with g++ -E:
>> 
>> # 1 "a.inline.hpp" 1
>> # 1 "a.hpp" 1
>> 
>> void a1();
>> void a2();
>> 
>> # 4 "a.inline.hpp" 2
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> # 4 "b.inline.hpp" 2
>> # 1 "b.hpp" 1
>> 
>> void b1();
>> void b2();
>> 
>> # 5 "b.inline.hpp" 2
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> # 2 "b.cpp" 2
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> 
>> b1() is called before it has been declared. b.inline.hpp inlined a.inline.hpp, which uses functions declared in b.hpp. However, at that point we've not included enough of b.inline.hpp to have declared b1().
>> 
>> This is a small and easy example. In the JVM the circular dependencies usually involves more than two files, and often from different sub-systems of the JVM. We have so-far solved this by restructuring the code, making functions out-of-line, creating proxy objects, etc. However, the more we use templates, the more this is going to become a reoccurring nuisance. And in experiments with lambdas, which requires templates, this very quickly showed up as a problem.
>> 
>> I propose that we solve most (all?) of these issues by adding a rule that states that .inline.hpp files should start by including the corresponding .hpp, and that the .hpp should contain the declarations of all externally used parts of the .inline.hpp file. This should guarantee that the declarations are always present before any subsequent include can create a circular include dependency back to the original file.
>> 
>> In the example above, b.inline.hpp would become:
>> 
>> // b.inline.hpp
>> #pragma once
>> 
>> #include "b.hpp"
>> #include "a.inline.hpp"
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> and now both a.cpp and b.cpp compiles. The generated output now looks like this:
>> 
>> # 1 "b.inline.hpp" 1
>> # 1 "b.hpp" 1
>> 
>> void b1();
>> void b2();
>> 
>> # 4 "b.inline.hpp" 2
>> # 1 "a.inline.hpp" 1
>> 
>> # 1 "a.hpp" 1
>> 
>> void a1();
>> void a2();
>> 
>> # 4 "a.inline.hpp" 2
>> 
>> inline void a1() {
>>   b1();
>> }
>> 
>> inline void a2() {
>> }
>> 
>> # 5 "b.inline.hpp" 2
>> 
>> inline void b1() {
>> }
>> 
>> inline void b2() {
>>   a1();
>> }
>> 
>> # 2 "b.cpp" 2
>> 
>> int main() {
>>   b1();
>> 
>>   return 0;
>> }
>> 
>> The declarations come first, and the compiler is happy. 
>> 
>> An alternative to this, that has been proposed by other HotSpot GC devs have been to always include all .hpp files first, and then have a section of .inline.hpp includes. I've experimented with that as well, but I think it requires more maintenance and vigilance of  *users* .inline.hpp files (add .hpp include to the first section, add .inline.hpp section to second). The proposed structures only requires extra care from *writers* of .inline.hpp files. All others can include .hpp and .inline.hpp as we've been doing for a long time now.
>> 
>> Some notes about this patch:
>> 1) Some externally-used declarations have been placed in .inline.hpp files, and not in the .hpp. Those should be moved. I have not done that.
>> 2) Some .inline.hpp files don't have a corresponding .hpp file. I could either try to fix that in this patch, or we could take care of that as separate patches later.
>> 3) The style I chose was to add the .hpp include and a extra blank line, separating it from the rest of the includes. I think I like that style, because it makes it easy for those writing .inline.hpp to recognize this pattern. And it removes the confusion why this include isn't sorted into the other includes.
>> 4) We have a few *special* platform dependent header files. Both those that simply are included into platform independent files, and those that inject code *into* the platform independent classes. Extra care, as always, need to be taken around those files.
>> 5) Mostly tested locally, but I'll test on more platforms if the idea is accepted.
>> 
>> What do you think?
>
> Stefan Karlsson has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits:
> 
>  - Merge remote-tracking branch 'origin/master' into 8267464_circular-dependency_resilient_inline_headers
>  - Merge remote-tracking branch 'origin/master' into 8267464_circular-dependency_resilient_inline_headers
>  - Review kbarrett
>  - Review dholmes
>  - Update HotSpot style guide documents
>  - Merge remote-tracking branch 'origin/master' into 8267464_circular-dependency_resilient_inline_headers
>  - Clean up assembler_<cpu>.inline.hpp
>  - 8267464: Circular-dependency resiliant inline headers

Thanks all for reviewing!

-------------

PR: https://git.openjdk.java.net/jdk/pull/4127

From stefank at openjdk.java.net  Mon May 31 09:02:32 2021
From: stefank at openjdk.java.net (Stefan Karlsson)
Date: Mon, 31 May 2021 09:02:32 GMT
Subject: Integrated: 8267464: Circular-dependency resilient inline headers
In-Reply-To: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
References: <j667r5WDHRQS1CakElSEwxsn8VqiByWWz___8s9OQeE=.44ad5587-2502-45cb-b329-d2c24330799f@github.com>
Message-ID: <H9v57pFhdfqtfzMpdJfBS5Ed3hNBQg6hSM3XuNYmGcY=.3f812158-2b34-418e-8172-5d47af7fa8cc@github.com>

On Thu, 20 May 2021 11:55:05 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

> I'd like to propose a small adjustment to how we write .inline.hpp files, in the hope to reduce include problems because of circular dependencies between inline headers.
> 
> This is a small, contrived example to show the problem:
> 
> // a.hpp
> #pragma once
> 
> void a1();
> void a2();
> 
> 
> // a.inline.hpp
> #pragma once
> 
> #include "a.hpp"
> #include "b.inline.hpp"
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> 
> // b.hpp
> #pragma once
> 
> void b1();
> void b2();
> 
> 
> // b.inline.hpp
> #pragma once
> 
> #include "a.inline.hpp"
> #include "b.hpp"
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> The following code compiles fine:
> 
> // a.cpp
> #include "a.inline.hpp"
> 
> int main() {
>   a1();
> 
>   return 0;
> }
> 
> But the following:
> 
> // b.cpp
> #include "b.inline.hpp"
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> 
> fails with the this error message:
> 
> In file included from b.inline.hpp:3,
>                  from b.cpp:1:
> a.inline.hpp: In function ?void a1()?:
> a.inline.hpp:8:3: error: ?b1? was not declared in this scope; did you mean ?a1??
> 
> We can see the problem with g++ -E:
> 
> # 1 "a.inline.hpp" 1
> # 1 "a.hpp" 1
> 
> void a1();
> void a2();
> 
> # 4 "a.inline.hpp" 2
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> # 4 "b.inline.hpp" 2
> # 1 "b.hpp" 1
> 
> void b1();
> void b2();
> 
> # 5 "b.inline.hpp" 2
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> # 2 "b.cpp" 2
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> 
> b1() is called before it has been declared. b.inline.hpp inlined a.inline.hpp, which uses functions declared in b.hpp. However, at that point we've not included enough of b.inline.hpp to have declared b1().
> 
> This is a small and easy example. In the JVM the circular dependencies usually involves more than two files, and often from different sub-systems of the JVM. We have so-far solved this by restructuring the code, making functions out-of-line, creating proxy objects, etc. However, the more we use templates, the more this is going to become a reoccurring nuisance. And in experiments with lambdas, which requires templates, this very quickly showed up as a problem.
> 
> I propose that we solve most (all?) of these issues by adding a rule that states that .inline.hpp files should start by including the corresponding .hpp, and that the .hpp should contain the declarations of all externally used parts of the .inline.hpp file. This should guarantee that the declarations are always present before any subsequent include can create a circular include dependency back to the original file.
> 
> In the example above, b.inline.hpp would become:
> 
> // b.inline.hpp
> #pragma once
> 
> #include "b.hpp"
> #include "a.inline.hpp"
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> and now both a.cpp and b.cpp compiles. The generated output now looks like this:
> 
> # 1 "b.inline.hpp" 1
> # 1 "b.hpp" 1
> 
> void b1();
> void b2();
> 
> # 4 "b.inline.hpp" 2
> # 1 "a.inline.hpp" 1
> 
> # 1 "a.hpp" 1
> 
> void a1();
> void a2();
> 
> # 4 "a.inline.hpp" 2
> 
> inline void a1() {
>   b1();
> }
> 
> inline void a2() {
> }
> 
> # 5 "b.inline.hpp" 2
> 
> inline void b1() {
> }
> 
> inline void b2() {
>   a1();
> }
> 
> # 2 "b.cpp" 2
> 
> int main() {
>   b1();
> 
>   return 0;
> }
> 
> The declarations come first, and the compiler is happy. 
> 
> An alternative to this, that has been proposed by other HotSpot GC devs have been to always include all .hpp files first, and then have a section of .inline.hpp includes. I've experimented with that as well, but I think it requires more maintenance and vigilance of  *users* .inline.hpp files (add .hpp include to the first section, add .inline.hpp section to second). The proposed structures only requires extra care from *writers* of .inline.hpp files. All others can include .hpp and .inline.hpp as we've been doing for a long time now.
> 
> Some notes about this patch:
> 1) Some externally-used declarations have been placed in .inline.hpp files, and not in the .hpp. Those should be moved. I have not done that.
> 2) Some .inline.hpp files don't have a corresponding .hpp file. I could either try to fix that in this patch, or we could take care of that as separate patches later.
> 3) The style I chose was to add the .hpp include and a extra blank line, separating it from the rest of the includes. I think I like that style, because it makes it easy for those writing .inline.hpp to recognize this pattern. And it removes the confusion why this include isn't sorted into the other includes.
> 4) We have a few *special* platform dependent header files. Both those that simply are included into platform independent files, and those that inject code *into* the platform independent classes. Extra care, as always, need to be taken around those files.
> 5) Mostly tested locally, but I'll test on more platforms if the idea is accepted.
> 
> What do you think?

This pull request has now been integrated.

Changeset: 64f0f689
Author:    Stefan Karlsson <stefank at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/64f0f68958a74d4ee34c4b738759fc2278fa8b47
Stats:     490 lines in 242 files changed: 349 ins; 118 del; 23 mod

8267464: Circular-dependency resilient inline headers

Reviewed-by: kbarrett, eosterlund, dholmes, kvn

-------------

PR: https://git.openjdk.java.net/jdk/pull/4127

From shade at redhat.com  Mon May 31 09:29:44 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Mon, 31 May 2021 11:29:44 +0200
Subject: [sh/jdk8u] Bulk backports to sh/jdk8
Message-ID: <c4fa39a6-6636-9643-e909-1ea33b2f75ca@redhat.com>

Hi,

I would like to backport these simple things to reduce the difference between codebases:
   [backport] 8261413: Shenandoah: Disable class-unloading in I-U mode
   [backport] 8261251: Shenandoah: Use object size for full GC humongous
   [backport] 8259954: gc/shenandoah/mxbeans tests fail with -Xcomp
   [backport] 8259580: Shenandoah: uninitialized label in VerifyThreadGCState

Webrev:
   https://cr.openjdk.java.net/~shade/shenandoah/backports/jdk8u-20210531/webrev.01/

Testing: hotspot_gc_shenandoah {fastdebug,release}

-- 
Thanks,
-Aleksey